Overview
Conformly evaluates automotive engineering documents against compliance standards (ASPICE, ISO 26262, ISO/SAE 21434). The pipeline runs in stages: document parsing (Docling plus a vision pass for figures), retrieval of standard/clause context, and a LangGraph-orchestrated, per-process LLM evaluation that produces structured, evidence-anchored findings. For an HPC allocation the pipeline is reconfigured to call open-weight models served on the allocation in place of external APIs — a single provider-interface change. The workload is batch and episodic — open-weight model benchmark sweeps and LoRA fine-tune runs scored against a frozen evaluation corpus. It is not a latency-sensitive online service and maps cleanly to Slurm batch scheduling.Software requirements
| Component | Version / detail |
|---|---|
| OS / runtime | Linux; executed as an Apptainer/Singularity container (built from a definition file converted from our Docker image; rootless, HPC-standard) |
| Python | 3.11 |
| GPU compute | CUDA (NVIDIA); cuDNN, NCCL for intra-node GPU collectives |
| Framework | PyTorch 2.x (CUDA build) |
| Model loading / fine-tuning | Hugging Face Transformers 4.x + PEFT (LoRA) |
| Inference server | vLLM (CUDA; tensor-parallel for large models; FP8 on Hopper) |
| Document parsing | Docling ≥ 2.80 (CPU-bound; optional GPU for layout models) |
| Orchestration | LangGraph 0.2.60 (CPU-side control flow; negligible GPU) |
| Models | Pulled from Hugging Face Hub: Qwen2.5-VL 7B (vision); Llama-3.1-8B / Qwen2.5-7B–14B (evaluator); optional ~70B for the FP8 benchmark |
| Scheduling | Slurm / SBATCH |
| Reproducibility | Python dependencies pinned in a lockfile inside the container; model versions pinned by Hugging Face Hub revision; no network access required at run time once weights are staged |
Hardware compatibility
| Requirement | Detail |
|---|---|
| GPU vendor / architecture | NVIDIA, CUDA compute capability ≥ 8.0 (Ampere/A100); ≥ 9.0 (Hopper/H100) for native FP8. CUDA-native; does not target AMD ROCm. |
| GPU memory | 7–8B LoRA fine-tune fits on a single 64 GB H100 (base model bf16 ≈ 14–16 GB + adapters + optimizer state). ~70B evaluator in FP8 ≈ 70 GB → 2× H100 tensor-parallel (TP=2). |
| Multi-GPU | Intra-node only at this scale. Tensor parallelism (vLLM) for large-model serving; data-parallel for LoRA. NCCL over NVLink/NVSwitch. |
| Nodes / interconnect | Single node (≤ 4 GPUs) for this scope — no multi-node MPI / InfiniBand required. Scales to multi-node for larger allocations but does not need it here. |
| CPU / RAM | Standard accelerated-node CPU/RAM; used for parsing and orchestration. No special requirement. |
| Storage | ~500 GB project/scratch for model weights, evaluation corpus, and checkpoints. I/O is bursty (model load, checkpoint write), not sustained streaming. |
| Run-time network | None required once weights are staged; jobs can run on compute nodes without outbound internet. |
Performance characteristics
- Profile. GPU-bound, batch-shaped: inference sweeps and fine-tune steps. Per-process compliance evaluation (8 ASPICE/ISO processes per document) is embarrassingly parallel across processes and documents.
- Inference. Served through vLLM with continuous batching; large evaluators use tensor parallelism within a node. GPU utilisation is high during sweeps.
- Fine-tuning. LoRA (PEFT) updates a small adapter rather than the full model, so memory and compute per run are modest relative to full fine-tuning, and it scales data-parallel across GPUs.
- Scaling. Strong scaling within a node for tensor-parallel serving; near-linear data-parallel scaling for LoRA replicas. No multi-node communication at this scale.
- Indicative resource use (to be confirmed empirically — measuring these is a project objective): benchmark sweep ≈ tens of GPU-hours; a 7–8B LoRA iteration ≈ low-hundreds of GPU-hours including hyper-parameter variation; ~70B FP8 inference benchmark ≈ hundreds of GPU-hours. H100 throughput means each GPU-hour does more work than on A100.
- Determinism / evaluation. Quality is measured with a frozen, labelled corpus and a reproducible detection-metric harness (precision/recall/F1), with low-temperature decoding and inter-run stability tracked, so runs are comparable and the allocation’s output is auditable.
Execution model
- Build the Apptainer image once from the pinned definition file.
- Stage open-weight model weights from Hugging Face Hub to project storage.
- Submit SBATCH jobs requesting up to one accelerated node (4× H100): benchmark-sweep jobs and LoRA fine-tune jobs.
- Score outputs against the frozen evaluation corpus inside the same container.
- Push the resulting fine-tuned adapter/model artifact (our IP, not customer data) to a private Hugging Face Hub repository.