ctapipe Bi-Weekly Digest — Activity since 2025-10-08

» cta-observatory/ctapipe | cta-observatory/ctapipe

• cta-observatory/ctapipe repo issues [back to top]

  • Tool.get_current_config() misses Components that are in lists #2850 [kosack] bug

    Describe the bug e.g. in #2802 , there is a property of the Component/Tool that is a list of Components. These do...

  • Event IDs duplicated/mismatched between tables when writing simtel to HDF5 with ctapipe-processor #2839 [jbuces] bug

    Describe the bug When writing an NSB simtel file to hdf5 format using the ctapipe-processor (v 0.23.2), the...

  • Generalise DL2 tables loading and filtering #2790 [Voutsi] enhancement

    Please describe the use case that requires this feature. Telescope cross-calibration. Describe the solution you'd...

  • Remove support for backseekable in SimTelEventSource #2858 [maxnoe]

    This was always a hacky solution requiring modifying private members of SimTelFile. We should remove support for...

  • High level monitoring data interface class #2862 [maxnoe] enhancement

    Please describe the use case that requires this feature. A clear and concise description of why this feature is...

  • Service Data Source API #2863 [mexanick] enhancement

    Please describe the use case that requires this feature. ctapipe needs I/O capabilities to read DL0/Service data:...

  • optimize event selection - masked events in astropy tables #2861 [BastienLacave]

    Hi, I am currently ptrying to produce the IRFs with ctapipe tools. Notably I use the...

    • BastienLacave: In cuts.py in pyirf (lines 85 and after) the cut_table["cut"] is set but then calling it leads...

    • BastienLacave: For both files, I print np.asanyarray(fill_value, values.dtype) and it gives 1.0 (as expected),...

    • Hckjs: Hey Bastien, what DL2 data did you use as input for ctapipe-optimize-event-selection exactly?...

    • TjarkMiener: Might be related to #2839, i.e. corruption on DL1 files (rather than DL2, which was produced...

    • BastienLacave: The files (good* and bad*) can be found in /fefs/aswg/workspace/bastien.lacave/IRF_test and the...

    • maxnoe: @BastienLacave maximilian.linhoff@tcs01 ~ $ ll /fefs/aswg/workspace/bastien.lacave/IRF_test/ total 0

    • BastienLacave: Sorry, they are there now

  • Add tool to check data model adherence / consistency / integrity #2864 [maxnoe] enhancement

    Please describe the use case that requires this feature. Other projects create files using ctapipe and their...

  • Shape mismatch for gain selected data in GlobalPeakWindowSum with new broken pixels from camera monitoring #2866 [burmist-git] bug

    Describe the bug Caught unexpected exception: operands could not be broadcast together with remapped shapes...

    • burmist-git

    • burmist-git: processor_tool_muon_configuration_MuonIntensityFitter.yaml

    • maxnoe: I think the reason is that with this new production, @orelgueta included the settings for the...

    • burmist-git: My files "old" have same problem (muons as well).

    • burmist-git: Just in case this commit make it working just fine 722112b

    • maxnoe: Test showing the same behavior on our gamma_prod5 simtel file here: #2867 The changes in #2802...

    • mexanick: Issue found, we need to gain-select the broken pixels mask. This is the commit in question: d9e5eee

    • TjarkMiener: I think the filling in the event source is the problem, i.e. see...

    • maxnoe: Before, this information was taken from another container, that was removed in your...

    • maxnoe: Fixes in #2867

• cta-observatory/ctapipe repo pull_requests [back to top]

  • Add support for list of components in get_current_config #2852 [maxnoe]

    Fixes #2850

  • Waveform noise adder #2847 [moralejo]

    First version of the component to add noise in MC waveforms, for adapting it to different observation conditions. It...

    • maxnoe: This looks good in general. However, at the moment, it's not usable via ctapipe-process. I think...

    • maxnoe: This leaves the question when and how to initialize it. Simplest would be adding a new boolean...

    • maxnoe: In the case of using observed pedestal events, it would be the other way around, right? We have...

      • moralejo: Yes, one can think of other possibilities, and one certainly is producing signal-only MC with no...

      • moralejo: However one should be careful, if real noise events are used, no "pixel shuffling" should be...

    • maxnoe: Suggested change ...

    • maxnoe: Suggested change rng_seed =...

    • maxnoe: I think this __init__ would benefit by being broken down into several smaller functions,...

    • maxnoe: This if else is a bit hard to read and see the differences. Maybe it would be simpler like...

      • maxnoe: Or even smaller: noise =...

  • treat list[Component] in get_current_config() #2851 [kosack]

    Fixes #2850 also supports tools containing tools, which is something that is being used by camera teams

    • maxnoe: Depending on the content of the tool, this could be rather expensive (imagine a tool stores a...

      • kosack: Hah, yeah, I was trying to be more general than list, but maybe it's best to just check for...

    • maxnoe: A Tool should never appear nested, so this check shouldn't be necessary.

      • kosack: Actually, this came up recently: in NectarCam, they have a bunch of Tools, and then wanted to...

      • maxnoe: If this is something we are willing to support, we should add dedicated tests for the...

      • kosack: I added a test case for this

      • kosack: I mean the ability to run Tools independently of a CLI in a script was part of the original...

      • kosack: Even the provenance log works ok in that case: it gets an entry for each internal tool, and then...

      • maxnoe: Ok, if it's that simple, great!

    • maxnoe: need to perform a length check, this raises an exception in case of empty list / tuple

    • maxnoe: Sonarqube complains about code duplication. There is a big block exactly copy/pasted in the...

    • maxnoe

  • Remove write_index_tables option of DataWriter, seems to corrupt files #2853 [maxnoe]

    Fixes #2839

    • TjarkMiener

    • kosack: That is strange, since that is part of pytables, so shouldn't cause any corruption - I wonder if...

    • maxnoe: It's already turned off by default, and it wasn't tested at all. You can see here that I...

    • maxnoe: I also don't see how it would speed up merges using tables, as that doesn't perform any joins or...

    • maxnoe: And yes, I suspect an upstream bug and will try to produce a minimal example for a report

    • kosack: Ok, fine to remove it then -I did check and there is no real need since we do the merging...

    • kosack

    • maxnoe: I prefer to remove now and eventually add it back later once we understood why the corruption is...

  • Fix sonar complaints about float comparisons #2854 [maxnoe] no-changelog-needed

    • mexanick

    • kosack

  • Generalise table preprocessing #2791 [Voutsi] module:irf

    Closes #2790 Refactoring the DL2 processing for producing IRFs. The motivation is to use the same code for more...

    • maxnoe

    • mexanick

    • mexanick: Looks like sonar couldn't find the coverage report: An error occurred while trying to import...

    • maxnoe: I restarted the sonar job, the artifact exists

    • maxnoe

    • maxnoe: The error is that a merge happened between the CI run here creating the coverage report and the...

    • kosack: Looks good now, thanks for addressing my comments.

  • Prepare 0.27 #2856 [maxnoe] no-changelog-needed

    • maxnoe: @Voutsi Do you want to appear with a proper email / name here?

      • Voutsi: Suggested change Georgios.Voutsinas...

      • maxnoe: Done via mailmap, thanks

    • kosack

    • mexanick: Ups, I have a clone now

    • mexanick

    • kosack

  • Test with iter_all_events branch of pyeventio #2835 [maxnoe]

    More tests what does and does not work in case we include all shower events Include more information than just...

  • Add CI with python 3.14, drop python 3.10 #2855 [maxnoe]

    Tests are passing with numba beta version. No changes needed in ctapipe itself. I suggest to wait for the numba...

  • Fix PendingDeprecationWarning for docutils arg #2860 [maxnoe] no-changelog-needed

    Fixes: src/ctapipe/core/tests/test_component.py::test_component_html_repr ...

  • Turn UnclosedFileWarning into error, fix #2859 [maxnoe] maintenance no-changelog-needed

  • Allow eventio 2.0 #2857 [maxnoe]

    New start for updating to eventio 2.0, easier than resolving merge conflicts. I'll make the minimal changes here to...

  • Fix the normalization for muon analysis to account for different PixelShapes. #2845 [burmist-git]

    issue : #2844 (comment) Parent issue : #2815 The goal is to improve the existing intensity fitter and prepare it for...

    • maxnoe: What is the motivation for this redefinition? It seems like an arbitrary choice, so why change...

      • maxnoe: Then I'd expect this transformation as part of the likelihood definition, not in the low-level...

      • burmist-git: What should we do if this value is negative? Raise an error? If the value is negative, it still...

      • maxnoe: Yes, raising a ValueError(f"r must be > 0, got {r}") seems appropriate here.

    • mexanick: LGTM

    • maxnoe: point implies 2d, but we pass radial coordinates r, phi0. So I'd call it impact_distance (as...

      • burmist-git: ok

      • burmist-git: Dear @maxnoe, this is a friendly reminder about this MR — let’s merge it :-)

    • maxnoe: Please explain this change. Why do you change the order of this array?

      • burmist-git: https://github.com/cta-observatory/ctapipe/pull/2845/files/397903ad39453ed0227fb8928d124e4b95c26e...

      • maxnoe: But phi0 is subtracted from phi. Now you again subtract phi from the array of all angles.

      • maxnoe: And: if you are changing the coordinate system of the muon fit, don't you think this is a major...

      • burmist-git: Please note that phi0 is set to 0. As we agree do not change the current fitter.

      • maxnoe: Even more misleading. Please... do not mix bug fixes with such changes. I would propose to close...

      • burmist-git: Well reuse this one for Fix the normalization for muon analysis to account for different...

    • maxnoe: Why do you change here + to -?

      • burmist-git: The answer is here :...

  • Add averaging #2811 [burmist-git]

    Phi dist averaging

  • Fix muon normalization pixel shapes #2865 [burmist-git]

    related to : #2845

    • maxnoe: You also have the phi0 related changes here.

    • burmist-git: @maxnoe please have a look now.

    • mexanick: I guess this also should be reverted? same for other invocations of rho

      • burmist-git: Yes, but in principle, in this particular case, one can have the same values. Anyway, it has...

    • maxnoe

    • mexanick

  • Fix gain selection of broken pixels, add tests for all extractors with CameraCalibrator #2867 [maxnoe]

    • burmist-git: Looks good for me.

    • burmist-git

    • burmist-git

    • burmist-git

    • TjarkMiener: Thanks! LGTM!

    • mexanick

      • mexanick: why is it 7? do we have 7 events in the test file? Shall we control it with max_events just in...

      • mexanick: please do not call the variable "array", I almost broke my head confusing it with a python...

  • Phi0 should be subtracted from phi not added. #2869 [burmist-git]

    Phi0 gives us the azimuth of the impact point, whereas before it was giving us the complementary angle of the impact point.

  • Visualizing schematic representation of Hillas parameters #2868 [vuillaut]

    This is an addition to the documentation to display as a schema what are the Hillas parameters:

    • maxnoe: Nice! Does it also work in TelescopeFrame? As we are about to drop support for image parameters...

    • sfegan: The way you have drawn the length and width is confusing since it shows them as a "diameter"...

  • Improve statistics aggregators #2848 [mexanick]

    Relax requirements on the data shape (aggregate arbitrary arrays on axis 0) Add option to aggregate statistics by...

    • mexanick: @maxnoe @TjarkMiener @matteobalbo the code has been updated implementing the chunking strategies...

    • maxnoe: Suggested change ...

    • maxnoe: Do we want to return the table here? Or just start index / stop index for indexing into the...

      • maxnoe: Especially the copy vs. view part should be part of the interface I think.

      • mexanick: good catch, you got me head scratching. SizeChunking returns views, but TimeChunking returns...

    • maxnoe: Do we only plan to use this with events? Or might it also be used for a monitoring table of...

      • mexanick: So far it is geared towards events, it requires event_id columns. While it is of course possible...

      • mexanick: I think, for generic table with other columns, the aggregation can be accommodated using the...

      • maxnoe: Why does it need the event_id column? It just splits by len, right?

    • maxnoe: Why do we raise an error here instead of yielding a single chunk?

      • maxnoe: I feel like we should add some configuration options here on the behavior for the case of a)...

      • mexanick: If the chunk size is set to None, a complete table will be used (i.e. user doesn't require...

    • maxnoe: Suggested change if...

      • mexanick: Is it needed? Does astropy allow negative time? It is AstroQuantity with the physical type of time.

      • maxnoe: Sure: In [3]: -5 * u.s Out[3]: <Quantity -5. s>

    • maxnoe: I think this loop here can be much simplified by computing the chunk once by doing: chunk_index...

      • mexanick: oh, I didn't think that python supports integer division for floats!

created at 2025-10-22 06:35 by dinghy 1.4.1.