ctapipe Bi-Weekly Digest — Activity since 2025-09-03

» cta-observatory/ctapipe | cta-observatory/ctapipe

• cta-observatory/ctapipe repo issues [back to top]

  • Document HDF5 format produced by ctapipe #2080 [maxnoe] enhancement documentation

    Please describe the use case that requires this feature. For the planned public release of a machine learning...

    • maxnoe: Mostly done: https://ctapipe.readthedocs.io/en/latest/user-guide/data_format/index.html

  • position_to_pix_index wrong behaviour for x,y outside camera #2795 [moralejo]

    ctapipe/src/ctapipe/instrument/camera/geometry.py Line 941 in 2bf1942 ...

  • Handling of random seeds #2830 [kosack]

    in #2819 (and before), we have a few places where the random seed is hard-coded to 0, which means the random...

    • maxnoe: We do allow that for the rngs where this might be important, e.g. the seeds to sample events for...

    • kosack: Ok, if the imporant cases are already handled (I didn't check I admit), then I think it's ok....

  • Should we deprecate changed config options? #2521 [kosack]

    In #2520, I changed a bunch of config option names, which will break existing config files. Maybe before release...

    • maxnoe: Duplicat of #1879

  • Segmentation fault when calling FlashCamExtrator #2714 [gschwefer] bug

    I get segmentation faults when calling the FlashCamExtractor on simulated events. This had happened before, and...

    • jsitarek: Hi, I am not sure if it is related, but I'm also struggling now with a similar crash on...

    • maxnoe: I don't really expect this is directly related, as @gschwefer seems to have traced it to the...

    • jsitarek: thx @maxnoe for quick feedback. in CNAF I have magicctapipe installed in a conda environment...

    • maxnoe: Thanks, can you also provide the output of conda list and pip freeze?

    • jsitarek: Hi @maxnoe here it is conda list # packages in environment at...

    • maxnoe: @jsitarek I see numpy and scipy were installed by pip over the ones from conda and I suspect...

    • jsitarek: Hi @maxnoe thanks a lot! I checked my environments and all three with ctapipe0.25 indeed have...

    • Elisa-Visentin: I have it from pypy in the env I am using with slurm for mcs. But I am checking if I can fix...

    • maxnoe: I will investigate at which step they get downloaded from pip. I would assume that the conda...

    • Elisa-Visentin: In conda we are not setting versions (well, for these two packages), so the conda version is...

    • jsitarek: Hi, pip is doing something strange during installation of magicctapipe, removing and installing...

    • maxnoe: @jsitarek It down-graded numpy and scipy actually: from 2.2.6 to 1.26.4 (numpy) and from 1.13.1...

    • maxnoe: I suggest to move this somewhere else, I think it's pretty clear that these are two unrelated issues

  • Confusing error message when tool input path does not exist #2825 [gschwefer]

    I was trying to run ctapipe-apply-models and got the the error message CRITICAL [ctapipe.ctapipe-apply-models]...

  • Missing dl2 tables metadata #2761 [Hckjs] bug

    Reading in data from the DL2_TELESCOPE_GROUP with the HDF5EventSource leads to default filled/empty dl2 telescope...

  • Add tool to export astropy cache to specific location #2748 [maxnoe] enhancement

    Please describe the use case that requires this feature. For working offline and on clusters, we should have a tool...

  • star catalog does not contain human readable names anymore #2695 [maxnoe]

    Before: >>> get_bright_stars(magnitude_cut=2) Downloading yale_bright_star_catalog5.fits.gz:...

    • kosack: Just out of curiosity: what changed that caused this? The catalog on Visier?

    • maxnoe: The PR adding the Hipparcos catalog rebuild the catalog file and didn't include it: #2625

  • SigmaClippingAggregator returns wrong sample size (n_events) value #2834 [mexanick] bug

    SigmaClippingAggregator returns n_events corresponding to the sample it was provided with, but not the "clipped"...

  • Numba caching race condition might lead to segfaults when importing ctapipe #2837 [maxnoe] bug

    Describe the bug A clear and concise description of what the bug is. @jsitarek and @Elisa-Visentin report segfaults...

    • maxnoe: See initial discussion here: #2714 (comment) and more details here: cta-observatory/magic-cta-pipe#306

    • maxnoe: @gschwefer Could this be the same issue as #2714 after all? E.g. there is not an actual problem...

    • maxnoe: To check, you could run with an individual cache directory per job (I.e. set...

  • Update trigger event types to match R1 data model #2094 [maxnoe] bug

    Describe the bug The trigger event types definition in ctapipe.containers currently does not match the released R1...

• cta-observatory/ctapipe repo pull_requests [back to top]

  • EventViewer as part of ctapipe-process tool #2556 [maxnoe]

    This adds a QT based GUI EventViewer that can be started and used to browse through event data using ctapipe-process...

  • Allow cut optimization to be stored in compressed fits files #2829 [maxnoe]

    Fixes #2693

    • kosack

  • Use scipy sparse array instead of matrix interface #2821 [maxnoe]

    Fixes #2678

    • kosack

  • Improve docs of position_to_pixel_index #2822 [maxnoe] no-changelog-needed

    fixes #2795

    • kosack

    • moralejo

  • Use modern random interface where possible #2819 [maxnoe] maintenance no-changelog-needed

    Performance of the lts fit is about the same: ❯ git switch main Switched to branch 'main' 170 μs ± 235 ns per loop...

    • mexanick: why do we need a custom function and not rng.choice(len(y), 2, replace=False)?

      • maxnoe: also, this specific implementation for the 2 out of 0 to N integers will always be faster I...

    • kosack

    • kosack: this means that where this is used (e.g. in impact.py or image/timing.py) the RNG will always be...

      • maxnoe: The seed is the same for each start of ctapipe. So we also process a real data file...

      • maxnoe: Whether this seed should be configurable... I don't know. I don't really see a reason why and I...

  • Add averaging #2811 [burmist-git]

    Phi dist averaging

    • kosack: needs better help text. What does the value mean?

  • Improve error messages for missing required Path config values #2826 [maxnoe]

    Fixes #2825

    • kosack

  • Implement reading and writing column metadata for tables. Fix reading of dl2 ml tel output in HDF5EventSource #2824 [maxnoe]

    Fixes #2761

    • maxnoe: Columns that are stored without a prefix do not need the additional NAME

    • Hckjs

    • kosack

  • Add tool to store astropy cache data #2823 [maxnoe]

    Fixes #2748

    • kosack

  • Unpin 3.13 in CI #2820 [maxnoe] maintenance no-changelog-needed

    Github runner images now have 3.13.7, no need to pin anymore due to the numba breakage in 3.13.4. See #2767

    • kosack

  • Test new sonar config #2817 [maxnoe] no-changelog-needed

    Dummy PR with small docs change to test if new sonar config is working.

    • kosack

    • mdebony

  • Add star names back to Yale, add for Hipparcos #2827 [maxnoe]

    Fixes #2695 In [1]: from ctapipe.utils.astro import get_bright_stars In [2]: from astropy.time import Time In [3]:...

    • kosack

  • Unpin python 3.13 in ci #2772 [maxnoe] no-changelog-needed

    python 3.13.5 was released fixing the regression that caused numba compilation failures in 3.13.4

  • Make StereoCombiner configurable in ApplyModels #2724 [Hckjs] bug

    Since the loaded joblib-pickled Reconstructors in ApplyModels ...

    • Hckjs: Superseded by #2773

  • Rename 'classification' field from ReconstructedContainer to 'particle_type' #2763 [Hckjs]

    Update DataModel Version to v7.2.0 (DataWriter writes dl2 'particle_type' group instead of...

    • maxnoe: Changelog missing and some sonarqube issues.

    • maxnoe: For the uncovered lines, you can add a test like this: def test_deprecated_alias(): ...

    • maxnoe

  • Pass project version to sonar #2831 [maxnoe] no-changelog-needed

    Fixes the "Version: not provided" in sonarqube

    • kosack

  • Generalise table preprocessing #2791 [Voutsi] module:irf

    Closes #2790 Refactoring the DL2 processing for producing IRFs. The motivation is to use the same code for more...

    • mexanick: Do you see something that might be an obstacle for the production of DL3? My point remain, if...

    • maxnoe: I think since this is the smaller pull request, we should include this first and then the DL3 PR...

  • Test with iter_all_events branch of pyeventio #2835 [maxnoe]

    More tests what does and does not work in case we include all shower events Include more information than just...

    • maxnoe: note: these are minimal changes to get most things working, we don't yet take full advantage of...

  • Add env variable to disable numba cache #2836 [maxnoe]

    ❯ time python -c 'import ctapipe.image.extractor' python -c 'import ctapipe.image.extractor' 2.14s user 0.16s...

    • jsitarek: looks good to me, and I also tested that this solves the original problem that I saw in...

    • kosack

  • add impactpoint_intensity_fitter.py #2814 [burmist-git]

    An important error has been found and fixed (in the Chord calculation). It shows an improvement in the resolution...

  • Add DL0 telescope event type, add missing types #2769 [maxnoe]

    Marking as draft for now, I will prepare a change request to the R1 model so that the additional types we define...

    • mdpunch: Looks fine to me (though I haven't tried testing it), and nice to see it implemented for use...

    • kosack

    • maxnoe: The change request was approved today, undrafting

    • mexanick: is this needed to have correct bitmask representation? If so, perhaps it would be more obvious...

      • maxnoe: I'm not sure I understand the question. These bits are explicitly listed as "reserved" in the...

      • mexanick: but in this case, aren't the values should be powers of two? from enum import IntFlag,...

      • maxnoe: Yes, but that's taken care of by using enum.IntFlag, no additional code needed: In [1]: from...

    • kosack

    • mexanick

  • add MonitoringSource #2802 [TjarkMiener] API change Breaking change hdf5 data format

    This PR prepares ctapipe to interface with camera calibration data products from calibpipe via a new...

    • maxnoe: Word order reads weird here for me. Doesn't CameraMonitoringContainer and...

      • kosack: However, elsewhere we use e.g. TelescopePointingContainer, not PointingTelescopeContainer, so I...

    • maxnoe: Related to the comment below about the subarray=None case.

      • kosack: Rename to "force_file_subarray_description" or maybe better "use_subarray_from_file" , since...

    • mexanick: Looks good to me. I'm not sure the way how the potential monitoring overriding in multiple files...

    • maxnoe: It's not a false positive, the complexity score is a well-defined number ;) I agree though that...

    • mexanick: I think sonar issue is false-positive, I would not refactor that function further

    • kosack: Very nice, and lots of work! I did a preliminary review with some quick comments here, but I...

      • kosack: This should be a ToolConfigurationError, not an IOError, which is more for real hardware or...

        • maxnoe: This is not a Tool, so why should it raise a ToolConfigurationError?

        • TjarkMiener: Yes, in the component I used IOError, while inside the Tool it should raise a ToolConfigurationError.

      • kosack: Similar to my previous comment, I'm not quite following why the user would want to do this.

        • TjarkMiener: This flag is a placeholder at the moment because we always have a subarray required (for...

        • maxnoe: The subarray provided to the monitoring source should always be the one of the event source that...

        • TjarkMiener: The subarray provided to the monitoring source should always be the one of the event source that...

        • TjarkMiener: @mexanick what about removing this flag and shift discussion of this to the FRAM integration...

        • maxnoe: . I think QualPipe should be able to use (and display) the MonitoringSource without any...

        • TjarkMiener: But I think you misunderstood my point above. I wasn't saying you must pass a...

        • TjarkMiener: I pushed a suggestion to resolve this discussion with 1f3a820. @maxnoe @kosack @mexanick please...

      • kosack: you only need this if the input file is missing the effective focal length, which should not be...

        • TjarkMiener: We currently only have lst_prod3_calibration_and_mcphotons.simtel.zst at the public test data (2...

      • kosack: Suggested change assert...

        • TjarkMiener: Done

      • kosack: You should give more info here, as this is the only message the user will see without reading...

        • TjarkMiener: Done

      • kosack: please change all to use relative imports (from ..containers import ...). In ctapipe, we always...

        • TjarkMiener: Done

    • maxnoe: I haven't checked, but what if the monitoring data tables are inside the same file as the...

    • kosack: this prints out thousands of lines, since {h5file} is not a string, but an object and it's...

      • TjarkMiener: Yes, I changed to the {h5file.filename}

    • kosack: I don't think you need to start with gamma_diffuse_full_reco_file here, since all you want to...

      • TjarkMiener: Ok, I haven't had in mind the creation when only running this test isolated.

    • mexanick

    • TjarkMiener: I think we also need to modify the example configs produced by ctapipe-quickstart to include...

    • maxnoe: Would that be something to support? Yes, not for camera calibrations, but for other things....

    • mexanick: I personally would keep calibrations and monitoring in the separate stream. I understand the...

    • maxnoe: Suggested change supported by simply...

    • maxnoe: input_files here is a single string? That seems weird, shouldn't it be a sequence? Does this...

      • TjarkMiener: Yes, it's allowed to provide str or Path, but as the trait name say input_files, we should...

    • TjarkMiener: The merger tool let you do this and as Max said you should be able to read it afterwards. I...

    • mexanick: I would not mask any exception as IOError, remove this.

created at 2025-09-17 06:33 by dinghy 1.4.1.