Pipelines & configs generation

Prod3 & Prod5 pipelines

Here is the typical MC pipeline for the prod3 and prod5 productions

To generate a config for that pipeline, you may run:

lstmcpipe_generate_config PathConfigProd5Trans80 --prod_id whatagreatprod

IMPORTANT NOTE: prod5 MC files need the config to set “focal_length_choice”: “EQUIVALENT” to be analyzed with lstchain >= v0.9

In the lstchain config, please set:

"source_config": {
    "EventSource": {
        "focal_length_choice": "EQUIVALENT"
    }
}

One can also start back from DL1, applying the dl1ab stage:

AllSky production pipeline

R0 to IRFs

Standard AllSky production pipeline for one training declination

To produce a config to run such a pipeline, typically run on the cluster:

lstmcpipe_generate_config PathConfigAllSkyFull --prod_id whatagreatprod --dec_list dec_2276

This will generate a lstmcpipe config file and a lstchain config file.

Please:

• check thoroughly the lstmcpipe config

• modify the lstchain config as you wish

DL1ab

The DL1ab workflow is very similar, only starting from an existing DL1 dataset.

This workflow is typically used when a tuned production is needed for a given dataset. For example, a production specifically tuned to match the Crab FoV NSB level and analyse Crab data. In this case, you should produce your own lstchain config file using lstchain tools (see lstchain documentation).

The workflow then starts from a base production (the source_prod_id), produces new tuned DL1 (dl1ab steps) and trains a new set of models.

To prepare the lstmcpipe config, you want to:

• find a base production to start with (see the list of productions in the documentation <productions> and look for the latest “base” or “unuted” one)

• run on the cluster:

lstmcpipe_generate_config PathConfigAllSkyFullDL1ab --dec_list dec_2276 --prod_id anothergreatprod --kwargs source_prod_id=whatagreatprod

Retrain and apply a model

The workflow starts from an existing PROD A with merged DL1b datasets, trains a new set of models and applies them to create a new set of DL2. Note: In case of source-dependent analysis, the missing parameters are computed on the fly by lstchain, allowing the use of this PathConfig to not recreate DL1 files.

Example of command to generate such a config:

lstmcpipe_generate_config PathConfigAllTrainTestDL1b --dec_list dec_2276 dec_931 --prod_id MY_NEW_PROD --kwargs source_prod_id=PROD-A

Using GammaDiffuse to produce full-enclosure IRFs

The configs PathConfigAllSkyTrainingWithSplit, PathConfigAllSkyTestingGammaDiffuse and PathConfigAllSkyFullSplitDiffuse introduce the possibility to divide the GammaDiffuse dataset into training and testing datasets:

To use, you may run:

lstmcpipe_generate_config PathConfigAllSkyFullSplitDiffuse --dec_list dec_2276 --prod_id MY_NEW_PROD