Page History

Take a look at: https://www.cs.rice.edu/~vo9/deeplearning/2019/slides/Rivanna.pdf

Updated Instructions for Generating Monte Carlo Simulations in RUS format for SpinQuest (Last updated on Feb 23, 2025, by Forhad)

If you would like to generate simulated events for SpinQuest experiments and save the output file in RUS format, you should follow these steps.

1. Access the Rivanna computer(https://ood.hpc.virginia.edu/pun/sys/dashboard^{Image Added})

2. Clone the repository:git clone https://github.com/uva-spin/UVA_RUS_Basic^{Image Added}

3. Go to the repository:
cd UVA_RUS_Basic

4. Set up the environment:
source /project/ptgroup/spinquest/this-e1039.sh

5. Run the simulation macro locally for testing. In the Fun4Sim.C macro, we have used: 

const bool count_only_good_events = true;
se->run(nevent, count_only_good_events);

This means that the Fun4All macro will keep running until we get accepted events, as required by the SQGeomAcc condition.The legacy generator has been used as the default for J/Psi productions because it is faster compared to Pythia8. You can use the package based on your interest. For example, let's use Drell-Yan events, where the beam interaction point is at the target location:

cd DY_Target
root -b 'Fun4Sim.C(10)'

6. Once the job runs locally and looks alright, you can submit a few jobs on the grid before submitting large jobs:
./jobscript.sh test 2 10

• "test" is the name of this job, used as the name of a new directory to store job outputs.
• "2" is the number of jobs.
• "10" is the number of events events per job.
• Job outputs will appear under /sfs/weka/scratch/<user_name>/MC.
• You can use squeue -u user  to check the status of your jobs (or use this link: https://ood.hpc.virginia.edu/pun/sys/dashboard/activejobs?jobcluster=all&jobfilter=user^{Image Added}).

7. Once everything looks alright, submit large jobs. For example:
./jobscript.sh DY_Target 100 1000

8.   Once all the jobs are completed, you can use the hadd or TChain command to merge the ROOT files. Here is an example using the hadd command:

1. Basic Usage: To merge multiple ROOT files into a single output file, use: hadd output_file.root input_file1.root input_file2.root input_file3.root
2. Merging Large File Lists with a Minimum File Size Constraint: To merge only root.root files larger than 20 KB from a directory, use:   hadd output_file.root $(find output_file_path -path "*/RUS.root" -size +20k)

Configurations of Event Generation Properly  (Last updated on Jan 30, 2025, by Forhad)

Wall time when dimuons are accepted in a top-bottom plane-wise (default).

1038

Wall time when dimuons are accepted in SpinQuest spectrometer

593

Here are the updated instruction of generating Monte Carlo for SpinQuest simulations and analysis

• Login to Rivanna (https://ood.hpc.virginia.edu/pun/sys/dashboard)
• Create a folder to submit your jobs from on /project/ptgroup/ or /project/UVA-Spin
• The scripts to generate Monte Carlo (MC) events on Rivanna are in: 

• /project/ptgroup/work/MC_Generation

  You can make more and store the generating scripts here but label them in a way that is easy to understand.  The present naming convention is <Exp>_<channel>_<vertex origin>
  • Option for <Exp> are 906 or 1039.  The Geant4 geometry is different for the setup of E906 and E1039, so for SpinQuest make sure you are using the E1039 geometry.
  • Option for <channel> are : Drell-Yan (DY), Jpsi (JPsi), Psi' (Psip) Pion background (Pion), Kaon background (Kaon), Random-multi muons (MultiMuon), Single Muons + or- (SingMup/m)
  • Option for <Vertex origin> : target (Target), Beam Dump (Dump), Everything that seen by the beam (All), The gap between the target and dump (TargetDumpGap), and arbitrary vertex origin (Manual, should specify numbers such as x,y,z→25,25,100 would be a target size with diameter 25cm and 100cm long.
• You can copy any MC scripts to your directory to submit.  The scripts to generate Monte Carlo (MC) events on Rivanna are in: 

  /project/ptgroup/work/MC_Generation

  Example of copying: cd /project/ptgroup/YourFolder

  rsync -av --exclude scratch /project/ptgroup/work/MC_Generation/DY_Target_script .

  
  

  We will keep track of all MC generation scripts so we can all see how a particular MC batch was produced.  Details of each one should be kept in ReadMe in that directory.
• Other MC configurations you make that are not included in this directory you should copy over and make a note about so other people can use it as well.
• To summit you jobs to produce the MC navigate to the directory where you want to run the script.
• Setup the environment using: source /project/ptgroup/spinquest/this-e1039.sh  command
  

• Submit your job using the following command.
  $./jobscript.sh<Outpu{t}_{f}olde{r}_{n}ame>\; <\mathrm{Number\; of}{}_{j}obs>\; <\mathrm{Number\; of\; Events\; per\; Job}>\; For\; example:$./jobscript.sh DY_Target_1M  100  10000       (Here we need to generate ~1M events. The accepted events depend on the channel). Strongly recommended not to exceed 10K events per job.

...

• Beam profile
  • The profile here means the distribution shape of beam protons in X and Y (= R). 
  • When "legacyVtxGen SQ_vtx_gen = true" (or the variable name is "legacyVtxGen" in old versions)
    • The shape is "Gaussian" at "R < 5*sigma" and "1 / R" at "R >= 5*sigma" (cf. https://github.com/E1039-Collaboration/e1039-core/blob/master/generators/E906LegacyVtxGen/SQPrimaryVertexGen.C#L290 )
    • The Gaussian width ("sigma" in cm) is defined by "SIGX_BEAM" and "SIGY_BEAM" in Fun4Sim.C.
    • The Gaussian center is X = Y = 0 cm by default, and can be changed by setting "SIGXX_BEAM" and "SIGYY_BEAM";

      rc->set_DoubleFlag("X_BEAM"

      in Fun4Sim.C.

      , 0.1); rc->set_DoubleFlag("Y_BEAM", 0.1);

  • When "SQ_vtx_gen = false" (or the variable name is "legacyVtxGen" in old versions)When "legacyVtxGen = false"
    
    • The shape is defined by the following functions of event generators.
    • "set_vertex_distribution_function()" sets the shape to "Uniform" or "Gaussian".
    • "set_vertex_distribution_mean()" sets the mean of the distribution.
    • "set_vertex_distribution_width()" sets the half width in case of "Uniform" or "sigma" in case of "Gaussian".
• Distribution of z-vertex
  • The z-vertex means the position of each generated event in Z.
  • When "legacyVtxGen = true"
    • The distribution shape follows the probability of beam-material interaction, namely the product of beam intensity and material amount at Z, where the beam intensity is attenuated over Z.

    • When "VTX_GEN_MATERIAL_MODE" is set to "Target" (via rc->set_CharFlag("VTX_GEN_MATERIAL_MODE", "Target")), events are generated only on the target material.

    • "VTX_GEN_MATERIAL_MODE" can be set to "Dump" or "TargetDumpGap" to enable only the dump material or the material (i.e. air) in between the target and the dump.
  • When "legacyVtxGen = false"
    • The distribution shape is defined by "set_vertex_distribution_function()" etc. together with the beam profile.
• Target position
  • The z-center and the length of the target is defined by "double target_coil_pos_z" and "double target_l" in Fun4Sim.C (-300 and -7.9 cm by default).
  • "target_z" is the position of the target material relative to the target coils.  It is 0 cm by default.  If you change "target_l", you have to change "target_z" to "0" together, unless you intend to make the relative position non-zero.
  • The x,y-center and the angle of the target are 0 by default.  If you need to change them,

    • Copy "G4_Target.C" to your working directory;

      cp /project/ptgroup/spinquest/core/default/macros/top/G4_Target.C .

      
      

    • Edit your "G4_Target.C" file.
    • Replace #include <top/G4_Target.C> with #include "G4_Target.C" in "Fun4Sim.C".

...

  /// Save only events that are in the geometric acceptance.
  SQGeomAcc* geom_acc = new SQGeomAcc();
  geom_acc->SetMuonMode(SQGeomAcc::PAIR); // PAIR, PAIR_TBBT, SINGLE, SINGLE_T, etc.
  geom_acc->SetPlaneMode(SQGeomAcc::HODO_CHAM); // HODO, CHAM or HODO_CHAM
  geom_acc->SetNumOfH1EdgeElementsExcluded(4); // Exclude 4 elements at H1 edges
  se->registerSubsystem(geom_acc);Fun4Sim.C scripts