What is WEAVE?

WEAVE (Wide-field fibre-optic spectrograph) is the next-generation multi-object spectrograph deployed at the prime focus of the 4.2-metre William Herschel Telescope (WHT) at the Roque de los Muchachos Observatory, La Palma, Canary Islands. Delivered by an international consortium spanning the UK, Netherlands, Spain, France, Italy, Germany, Sweden, Mexico, Hungary and the United States, WEAVE began science operations in 2023 and is transforming wide-field spectroscopic astronomy.

The instrument deploys up to ~1000 optical fibres across a two-degree field of view, feeding a dual-arm spectrograph that simultaneously covers the full optical window from 3660 to 9590 Å. A dichroic at 5900 Å splits light into blue and red arms, each recorded on two 6k × 6k low-fringing CCDs. Two resolution settings — low (R~5 000) and high (R~20 000) — are available, making WEAVE suitable for a vast range of science cases from massive stellar surveys to detailed galaxy kinematics.

Three observing modes

CAMCEAD's role in WEAVE

CAMCEAD at the Institute of Astronomy, University of Cambridge, is responsible for the full WEAVE data-flow infrastructure — from survey preparation at the telescope through to archive delivery. This covers the WEAVE Automatic Submission Platform (WASP), the Operational Repository and its data-access protocol (WOR/WDAP), the Core Processing System (CPS, Level 1), and the Advanced Processing System (APS, Level 2). The CAMCEAD team also contributes to the WEAVE Archive System (WAS) and hosts contributed software.

Pipeline execution is split by infrastructure type: WASP, WOR/WDAP and CPS run on CAMCEAD in-house computational and storage resources at the Institute of Astronomy; APS (L2) runs on the IRIS National Cloud Facility using an OpenStack Kubernetes cluster managed by CAMCEAD.

Overall dataflow architecture

The Operational Repository (OR) sits at the centre of all WEAVE data operations — connecting survey preparation, telescope operations, pipeline execution, provenance tracking and archive publication through a single orchestration layer.

WASP — WEAVE Automatic Submission Platform

WASP is the gateway between survey science teams and WEAVE operations, running on CAMCEAD in-house computational and storage resources. It manages the full chain from survey design to executable observing blocks: catalogue submission, validation against the WEAVE data model, trimester planning, OB generation, and delivery of FITS target catalogues and XML observing blocks to the Observation Control System (OCS) at the WHT.

WOR & WDAP — Operational orchestration

The WEAVE Operational Repository (WOR) is the central state, provenance and workflow hub for all WEAVE data operations, hosted on CAMCEAD in-house computational and storage resources at the Institute of Astronomy. WDAP (WEAVE Data Access Protocol) provides the recipe-driven, secure bidirectional interface through which every component — WASP, WHT, CPS, APS and WAS — communicates with the OR. All product transfers, state transitions, data-model validation and provenance records pass through this layer.

CPS — Core Processing System (Level 1)

The Core Processing System transforms raw WHT observations into calibrated science-ready spectra and data cubes. CPS runs on CAMCEAD in-house computational and storage resources and handles all three WEAVE observing modes — MOS, LIFU and mIFU — through a highly parallelised, recipe-driven pipeline. Outputs include fully calibrated 1D spectra for all fibres, IFU datacubes (LIFU and mIFU), OB-level and target-level stacked products, quality control reports, and calibration datasets distributed to APS.

APS — Advanced Processing System (Level 2)

APS transforms calibrated L1 data into science-ready measurements, maps and catalogues. Unlike CPS, APS runs on the IRIS National Cloud Facility — an OpenStack Kubernetes cluster managed by CAMCEAD with ~1400 vCPUs, 8 GPUs, 5 TB RAM and 2 PB Ceph storage — providing the elastic compute needed for large-scale redshift fitting, spectral analysis and classification across all WEAVE surveys.

The APSOB common reader preprocesses L1 FITS files from all observing modes. Science modules include Redrock (redshifts and object classification), FERRE (stellar parameters and chemical abundances), RVS (radial velocities), pPXF and EMI-pPXF (stellar kinematics and emission-line analysis), and LS (emission-line measurements). The PyAPS pipeline provides the core framework orchestrating these modules within APS. Galactic and extragalactic processing branches run independently, with outputs covering redshifts, stellar parameters, kinematics, IFU emission-line maps, and science catalogues.

End-to-end data journey

Each night, raw data from the WHT at La Palma is transferred to Cambridge where CAMCEAD takes over. The diagram below shows the full path — from raw frames through CPS, the OR validation layer, APS cloud processing, and finally to the WEAVE Archive System (WAS) in the Canary Islands — together with the typical data volumes at each node.

Infrastructure