> ## Documentation Index > Fetch the complete documentation index at: https://docs.conformly.ai/llms.txt > Use this file to discover all available pages before exploring further. # HPC Technical Reference > Software requirements, hardware compatibility, and performance characteristics of the Conformly analysis & evaluation pipeline for high-performance computing allocations. The Conformly analysis & evaluation pipeline is proprietary and not publicly released. This reference provides the software requirements, hardware compatibility, and performance characteristics needed to evaluate its suitability for a high-performance computing allocation, without disclosing source. Reference target: NVIDIA H100 accelerated partition (e.g. MareNostrum 5, BSC). ## Overview Conformly evaluates automotive engineering documents against compliance standards (ASPICE, ISO 26262, ISO/SAE 21434). The pipeline runs in stages: document parsing (Docling plus a vision pass for figures), retrieval of standard/clause context, and a LangGraph-orchestrated, per-process LLM evaluation that produces structured, evidence-anchored findings. For an HPC allocation the pipeline is reconfigured to call open-weight models served on the allocation in place of external APIs — a single provider-interface change. The workload is **batch and episodic** — open-weight model benchmark sweeps and LoRA fine-tune runs scored against a frozen evaluation corpus. It is not a latency-sensitive online service and maps cleanly to Slurm batch scheduling. ## Software requirements | Component | Version / detail | | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | OS / runtime | Linux; executed as an **Apptainer/Singularity** container (built from a definition file converted from our Docker image; rootless, HPC-standard) | | Python | 3.11 | | GPU compute | **CUDA** (NVIDIA); cuDNN, NCCL for intra-node GPU collectives | | Framework | **PyTorch 2.x** (CUDA build) | | Model loading / fine-tuning | **Hugging Face Transformers 4.x** + **PEFT** (LoRA) | | Inference server | **vLLM** (CUDA; tensor-parallel for large models; FP8 on Hopper) | | Document parsing | **Docling ≥ 2.80** (CPU-bound; optional GPU for layout models) | | Orchestration | **LangGraph 0.2.60** (CPU-side control flow; negligible GPU) | | Models | Pulled from Hugging Face Hub: Qwen2.5-VL 7B (vision); Llama-3.1-8B / Qwen2.5-7B–14B (evaluator); optional \~70B for the FP8 benchmark | | Scheduling | Slurm / SBATCH | | Reproducibility | Python dependencies pinned in a lockfile inside the container; model versions pinned by Hugging Face Hub revision; no network access required at run time once weights are staged | A single Apptainer image carries the full stack; SBATCH scripts request GPUs and launch container jobs. No compilation on the login node. ## Hardware compatibility | Requirement | Detail | | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | GPU vendor / architecture | **NVIDIA**, CUDA compute capability ≥ 8.0 (Ampere/A100); **≥ 9.0 (Hopper/H100) for native FP8**. CUDA-native; does **not** target AMD ROCm. | | GPU memory | 7–8B LoRA fine-tune fits on a single 64 GB H100 (base model bf16 ≈ 14–16 GB + adapters + optimizer state). \~70B evaluator in FP8 ≈ 70 GB → **2× H100 tensor-parallel** (TP=2). | | Multi-GPU | Intra-node only at this scale. Tensor parallelism (vLLM) for large-model serving; data-parallel for LoRA. **NCCL** over NVLink/NVSwitch. | | Nodes / interconnect | **Single node** (≤ 4 GPUs) for this scope — no multi-node MPI / InfiniBand required. Scales to multi-node for larger allocations but does not need it here. | | CPU / RAM | Standard accelerated-node CPU/RAM; used for parsing and orchestration. No special requirement. | | Storage | \~500 GB project/scratch for model weights, evaluation corpus, and checkpoints. I/O is **bursty** (model load, checkpoint write), not sustained streaming. | | Run-time network | None required once weights are staged; jobs can run on compute nodes without outbound internet. | Fallback to an A100 64 GB system is supported, with the FP8 path replaced by bf16/INT8 (more GPU memory / GPUs for the 70B case). ## Performance characteristics * **Profile.** GPU-bound, batch-shaped: inference sweeps and fine-tune steps. Per-process compliance evaluation (8 ASPICE/ISO processes per document) is embarrassingly parallel across processes and documents. * **Inference.** Served through vLLM with continuous batching; large evaluators use tensor parallelism within a node. GPU utilisation is high during sweeps. * **Fine-tuning.** LoRA (PEFT) updates a small adapter rather than the full model, so memory and compute per run are modest relative to full fine-tuning, and it scales data-parallel across GPUs. * **Scaling.** Strong scaling within a node for tensor-parallel serving; near-linear data-parallel scaling for LoRA replicas. No multi-node communication at this scale. * **Indicative resource use** (to be confirmed empirically — measuring these is a project objective): benchmark sweep ≈ tens of GPU-hours; a 7–8B LoRA iteration ≈ low-hundreds of GPU-hours including hyper-parameter variation; \~70B FP8 inference benchmark ≈ hundreds of GPU-hours. H100 throughput means each GPU-hour does more work than on A100. * **Determinism / evaluation.** Quality is measured with a frozen, labelled corpus and a reproducible detection-metric harness (precision/recall/F1), with low-temperature decoding and inter-run stability tracked, so runs are comparable and the allocation's output is auditable. ## Execution model 1. Build the Apptainer image once from the pinned definition file. 2. Stage open-weight model weights from Hugging Face Hub to project storage. 3. Submit SBATCH jobs requesting up to one accelerated node (4× H100): benchmark-sweep jobs and LoRA fine-tune jobs. 4. Score outputs against the frozen evaluation corpus inside the same container. 5. Push the resulting fine-tuned adapter/model artifact (our IP, not customer data) to a private Hugging Face Hub repository. No customer data is processed on the allocation — the corpus is synthetic and self-authored.