SDPE — SAR processing pipeline orchestration
Lumir LumirX multi-stage SAR pipeline · NestJS with 5 subsystems + DAG
From the console (a React Flow canvas) an operator composes and deploys a processing DAG; the Pipeline Workflow subsystem's orchestrator (CSC-08) acts as a control tower, assigning per-stage jobs over pgmq (SI-04) and tracking completion (SI-03) to advance L0→L3. The DAG itself does not listen to events — it only defines deployment rules ('which condition → which pipeline'; a pipeline is data, not code). Nine CSCs map onto collection (DCS), signal processing (SPS), post-processing (PPS) and service (DSS); files stay on NAS while queues carry only paths/metadata. Products are registered into a PostGIS STAC catalog and the Data Service (CSC-09) serves them over REST·OGC·STAC. A new satellite is added by registering a DAG + profile (near-zero code change). Teal = control (pgmq) / amber = data.
There was no system letting an operator configure, run, trace, and recover the L0–L3 LumirX raw-SAR pipeline, and adding new satellites or algorithms needed to cost as little code change as possible. From my side, I was dropped into the pipeline domain blank-slate.
On top of an inherited NestJS 5-subsystem monorepo, I designed and built the DAG planning UI (no Figma — the UI code plus Playwright e2e is the planning document). Built the GitLab CI/CD from zero, did the detailed design for interfaces/csc-8, fed the 80–100-page ICD/SAD docx directly into AI, ran a tight loop of small 'does this break the rule?' reviews, and wired up an auto-redeploy hook that fires when ops-console work completes.
Despite entering blank-slate, I produced detailed design and implementation on top of my senior's base. The architecture has settled into a shape where adding Sentinel support or a Snappy-based DAG step takes near-zero time.
Designed and built the DAG-pipeline planning UI; built the GitLab CI/CD from zero; produced the detailed design for interfaces/csc-8 (csc-7/9 to follow).
Apps folder structure + natives/csc03 · csc04 + pgmq + ICD/SAD (base design finished by my senior).
pgmq and the tech-stack decisions came from the inherited base design. I read the ICD/SAD myself and extracted the detailed design from them. My senior only dropped hints — but the base design was solid enough that the work went smoothly, which I should acknowledge too.