Verification and scientific validation

The PyAPS validation programme has four parts:

1. cross-mode APSOB contract verification;

2. external and contributed-adapter contract and failure tests;

3. global versus spaxel-weighted and bucketed LSF comparisons;

4. target-class and scientific-module performance together with Level-2 conformance.

Validation includes unit tests, interface tests with real data, results from earlier operational rehearsals and the single-field LIFU seg2d/seg3d example. The interface checks below exercise specific data paths and failure cases. Numerical scientific results are reported together with their measurement samples.

In that LIFU stackcube, seg2d returned 23 continuum apertures and seg3d reduced 512 initial
$S/N\ge 6$

candidates to 127 after connected-voxel and spatial-edge vetoes. All 127 lacked a continuum counterpart. The single-line sign-flipped- cube gate remained below its 0.9 target and promoted none. The separate multi-line gate found four groups above its field-calibrated combined
$S/N\ge 12$

threshold; one was rejected as a duplicate, leaving three. One additional singleton passed the extreme-event threshold, producing four new patch rows in total. This shows the candidate selection, grouping and duplicate-removal steps on one real field. Because the field has no truth catalogue, independent line confirmation or injection-recovery test, it cannot yet provide a completeness or false-positive rate for the wider survey.

For a real BLUEL11/LIFU calibration product, the LSF reader constructed 601 fibre-level interpolators, and the associated LIFU L1 header resolved the same CALDATE/setup/FPMODE path through l1_fileinfo.

The complete single-arm L1-to-APSOB path was also exercised on a real 2-D LIFU-form file. The header was normalised to MOSLIFU/BLUEL11; with the pre-computed LSF branch, APSOB produced 600 APSTARG objects, each carrying a 4710-pixel APSSPEC, and attached finite FWHM metadata after applying the missing-NSPEC-601 fallback (NSPEC is the fibre-spectrum index). The run took approximately 2.03 s locally. This test covers the complete path from the L1 layout to a calibrated APSOB for this single-arm, fibre-level product.

Deterministic tests additionally pass the limiting cases for fibre overlap, normalized distance weighting, repeated-dither collapse, flux-weighted bin aggregation and resolution-bucket assignment. These tests guard the geometry and bookkeeping invariants used by the LSF path.

WEAVE Operational Rehearsal 3b sent 180 simulated MOS/LIFU observation blocks through the wider CPS-APS-archive flow. For a simulated WEAVE-LOFAR field, the survey paper reports that 93.3% of APS Redrock redshifts had absolute error below 0.001 for objects with redshift at most one. The same rehearsal exposed detector-position dependent residuals associated with spectral-resolution variation, motivating resolution-aware processing.

Historical capacity planning and local loading tests

This section records historical capacity-planning assumptions, a small local test of how repeatably PyAPS loads a Level-1 product, and historical development optimisation measurements. None of them is a measured end-to-end processing throughput of PyAPS.

Historical capacity planning

The WEAVE APS requirements ask for one year of accumulated data to be processed in fewer than 10 days, assuming up to 10,000 spectra per night and about 2 million spectra per year. A separate data-rate discussion assumes 2.1 million spectra per year and a one-week reprocessing campaign, which corresponds to a required processing rate of 300,000 spectra per day. These are bulk-reprocessing requirements. They are not measured PyAPS throughput and they are not the nightly acquisition rate.

An APS hardware-sizing exercise estimated approximately 55 serial compute-hours per night for about 110 extragalactic observing nights per year, or roughly 6100 serial hours. Completing this workload in 240 hours requires a speed-up of about 25. Under ideal 64-way parallelisation, 6100/64 is approximately 95 hours, or four days. These are historical planning assumptions, not measured parallel-scaling results.

The two planning scenarios are different: the requirement to process a year of data in fewer than 10 days and the one-week data-rate calculation do not share one assumption.

Local loading repeatability test

The test ran on macOS arm64 with ten logical CPUs, Python 3.12.12, NumPy 2.3.4, SciPy 1.16.2 and Astropy 7.1.1. After one cache-building warm-up, each condition was measured in five fresh processes, using a real 600-row LIFU-form input.

Each process preserved the expected 600-target state and the corresponding 4710/7340-sample state, and the hashes of the selected state were identical between processes.

These are local, warm-cache repeatability results for one input and one loading path. They are not the full end-to-end processing time, not a cold-cache benchmark, not evidence covering every observing mode, and not a replacement for the larger ingestion benchmark in the accompanying paper. Peak process RSS is the largest resident memory of the whole process, not the incremental memory used by loading.

This test is different from the single-arm L1-to-APSOB integration test above, which took approximately 2.03 s and also involved building the 601 LSF interpolators and applying the missing-NSPEC-601 fallback. The two timings answer different questions and should not be compared.

Historical development optimisation measurements

PyAPS development included targeted tests of LIFU loading, LSF interpolation and repeated array operations. The table records changes measured for these individual operations under their respective test conditions. These are historical development measurements, not end-to-end pipeline timings or measurements of sustained processing throughput.

The first row describes successive changes to one load: the times are not to be added together and are not separate observing modes. The last row gives speed-up factors, not elapsed times or memory savings. These tests are separate from the 600-row repeatability test above, from the integration test of approximately 2.03 s and from the ingestion benchmark in the accompanying paper. They were not rerun on the final release.