Advanced Example CaTS

 

Responsible Geant4 Collaborator: Hans-Joachim Wenzel, Fermilab, US.

Developers: Hans Wenzel, Soon Yung Jun and Krzysztof Genser (Fermilab, US).

CaTS logo

Short description

CaTS (Calorimeter and Tracker Simulation) is a flexible and extendable Geant4 based simulation framework.

CaTS is used to demonstrate the G4Opticks hybrid workflow calling Opticks from within Geant4 for the creation and propagation of optical photons on GPU’s. Opticks is an open source project. It translates Geant4 optical physics to NVIDIA® CUDA, OptiXTM to run on GPU’s and the Geant4 geometry into a geometry optimized for ray tracing on GPU’s. Opticks only handles optical photons while Geant4 (running on the CPU/host) handles all other particles. G4Opticks is part of Opticks and provides the interface between Geant4 and Opticks. Figure 1 shows the visualisation of the CaTS example simulation.

CaTS simulation set-up
Figure 1. CaTS simulation set-up: 2 GeV electron shower in liquid Argon.

In many ways the generation and tracing of optical photons is the ideal application to be ported to GPU’s:

  • there is only one particle involved (optical photon), but depending on the use case there can be many of them (10s of millions in case or a liquid Argon TPC for a typical neutrino events). This allows for massive parallelism (low latency, no big fluctuations in computing time).
  • Only a few simple physics processes need to be implemented on the GPU. The processes are:
    • G4Cerenkov (generate),
    • G4Scintillation (generate),
    • G4OpAbsorption,
    • G4OpRayleigh,
    • G4OpBoundaryProcess (only a few surface types),
    • G4OpWLS (not yet implemented),
    • G4OpMieHG (not needed),
    • Reemission (Opticks specific, no equivalent in Geant4).
  • These processes don’t need a lot of input data (the so called Gensteps for the Cerenkov (C) and Scintillation (S) processes) and don’t produce a lot of output. Only few photons reach the photodetectors and produce a Hit. So the amount of data to be transferred between host (CPU) and device (GPU) (Gensteps –> Device, PhotonHits –> Host) is small.
  • Optical ray tracing is a well established field so we benefit from available efficient algorithms (e.g., OptiXTM).
  • Use NVIDIA® hardware (some with RTX: raytracing hardware acceleration) and software (CUDA, OptiX).
  • G4Opticks is a runtime/build time option. One can switch between Opticks and the use of the Geant4 native optical processes for one to one comparison.
CaTS Opticks

CaTS was developed to support R&D projects where it is necessary to easily change the geometry, physics configurations, etc. at run-time without recompiling. CaTS allows to simulate Detector setups ranging from a single Detector (e.g. single crystal) to complex setups like test beam experiments consisting of many different detector elements.

The components of CaTS are:

  • Detector Description: described in gdml input file (e.g. crystalcal.gdml) (geometry, materials, optical properties, sensitive detector). We use extensions to the gdml schema to attach sensitive detectors to logical volumes, assign step limits to specific volumes and to apply visualization attributes (e.g. color, transparency, etc.) to volumes.
  • (needs work) plug and play. Making use of predefined components. Automated just to attach a sensitive detector to a logical volume in the gdml file. Hits are added to the ROOT based output and properly named.
  • Input modules:
    • GPS
    • Particle Gun
  • Physics: CaTS uses G4PhysListFactoryAlt to define and configure physics. One can choose between all Geant4 reference Physics Lists and can extend it by adding various physics constructors like the optical physics processes (Cerenkov, Rayleigh, Scintillation etc.), step limiter, neutron limiters etc. The physics configuration can be specified via a command line option.
./CaTS -g simpleLArTPC.gdml -pl 'FTFP_BERT+OPTICAL+STEPLIMIT' -m time.mac -t 4
  • where:
    • -g specifies the gdml file to use (mandatory argument)
    • -pl specifies the physics configuration in this case we select the FTFP_BERT reference physics list and extend it with the OPTICAL and STEPLIMIT and physics constructors. The default is FTFP_BERT+OPTICAL+STEPLIMIT
    • -m specifies the Geant4 macro to use. If no macro is provided interactive processing is assumed.
    • -t specifies the number of threads. If the number of threads is not given it’s set to the number of CPU cores available.
  • Sensitive Detectors: (+ corresponding Hit classes).
  • TrackerSD(Hit) registers the step points of charged particles.
  • CalorimeterSD(Hit) registering energy deposit.
  • DRCalorimeterSD(Hit) besides registering energy deposit counts produced Cerenkov photons.
  • MsCSD(Hit) used to study multiple scattering on a thin layer.
  • PhotonSD(Hit) sensitive detector that registers optical photons.
  • lArTPCSD(Hit) sensitive detector that registers ionization and collects Gensteps (Scintillation and Cerenkov) to be processed by Opticks.
  • RadiatorSD is special since no Hits are created. This sensitive detector just collects the Gensteps (Scintillation and Cerenkov) to be processed by Opticks.
  • CaTS uses ROOT IO to provide persistency for Hits.
  • Using of G4Opticks doesn’t require changes to Geant4! CaTS only makes use of provided Geant4 user interfaces: UserActions, Sensitive Detectors, etc.
  • When G4Opticks is used for the generation and tracing of optical photons we still use the Geant4 Cerenkov and Scintillation (C/S) processes to calculate the number of optical photons to be generated at a given step and to provide all necessary quantities to generate the photons on the GPU. The information collected is the so called GenStep which is different for Cerenkov and Scintillation (C/S). The harvesting is done in sensitive Detectors(SD) (RadiatorSD/lArTPCSD). The photon Hits are collected in PhotonSD (see the CaTS workflow below).
  • Note: even when G4Opticks is used the Geant4 Cerenkov and Scintillation processes are still required to calculate the number of photons to be generated at each given step. The number of photons generated by Opticks is constrained to the number calculated by the equivalent Geant4 Cerenkov/Scintillation process.
CaTS Opticks

Opticks resources

CaTS resources

Examples

Here we provide examples for running CaTS (without Opticks). The examples consist of a gdml file to define the detector configuration and a macro to control CaTS.

Homogeneous Crystal Calorimeter

Here we just read out the energy deposited in a huge crystal.

time ./CaTS -g homogeneous_pbwo_crystalcal.gdml  -pl 'FTFP_BERT'  -m pip_IO_Calo.mac
time ./readCalorimeterHits CaloHits_Run0.root CaloHistos.root CalorimeterVolume

The created histograms can then be looked at using ROOT.

CaTS Opticks example 1 
time ./CaTS -g homogeneous_pbwo_crystalcal.gdml  -pl 'FTFP_BERT'  -m pip_IO_Calo.mac
time ./readCalorimeterHits CaloHits_Run0.root CaloHistos.root CalorimeterVolume


Dual Read out Crystal Calorimeter

Here in addition to the deposited energy in the calorimeter cell, we read out the number of Cerenkov photons that have been produced.

./CaTS -g crystalcal_pbwo.gdml -pl 'FTFP_BERT+OPTICAL' -m pip_IO_DR.mac 
./readDRCalorimeterHits DRCaloHits_Run0.root DRCaloHhstos.root CalorimeterVolume


Multiple scattering at a thin Diamond layer

In this example, we register the momentum of a particle as it undergoes multiple scattering in a thin Diamond layer.

./CaTS -g DiamondTarget.gdml -pl 'FTFP_BERT' -m msc.mac 
./readMscHits  Msc_Run0.root ms_histos.root 'volTarget'

The created histogram can then be looked at using ROOT.

CaTS Opticks example 2

Publications based on CaTS-simulation

  • Space-Time Development of Electromagnetic and Hadronic Showers and Perspectives for Novel Calorimetric Techniques. Andrea Benaglia(CERN), Etiennette Auffray, Paul Lecoq, Hans Wenzel, Adam Para(Fermilab) (Apr, 2016), Published in: IEEE Trans.Nucl.Sci. 63 (2016) 2, 574-579 , Contribution to: SCINT 2015, 574-579.

  • Physics of hadron shower development and the implications for calorimetric resolution. Hans Wenzel(Fermilab), Adam Para(Fermilab), Andrea Delgado(Texas A-M) (2013), Contribution to: CHEF 2013, 265-271.

Last updated: 03/02/2022 by S. Guatelli