English 
Русский 
中文 
Español 
Deutsch 
Français IB cabling by generation: DAC, AOC, transceivers, split cables
An InfiniBand cluster is cabled by reach class: copper where the switch is within arm's length, active copper a little further, active optical cables to the next row, transceivers with structured fibre for the rest. Each generation moved the boundaries inward and changed the connector, and NDR added twin-port modules and split cables that confuse first-time installers. This page is the cabling reference — reaches, connectors, split configurations, mechanical and identity rules.
Connectors by generation
| Generation | Per-lane rate | 4x port | Connector | Notes |
|---|---|---|---|---|
| SDR / DDR | 2.5 / 5 Gb/s | 10 / 20 Gb/s | CX4 (copper), later QSFP | legacy |
| QDR | 10 Gb/s | 40 Gb/s | QSFP | SFF-8436 management |
| FDR | 14.06 Gb/s | 56 Gb/s | QSFP+ (FDR-rated) | 64B/66B; FEC optional |
| EDR | 25.78 Gb/s | 100 Gb/s | QSFP28 | same optics as 100G Ethernet, different coding |
| HDR | 53.1 Gb/s PAM4 | 200 Gb/s | QSFP56 | 1:2 splitters to HDR100 |
| NDR | 106 Gb/s PAM4 | 400 Gb/s | OSFP twin-port (2 × 400G per module) on switches; OSFP or QSFP112 on HCAs | 800G per switch cage |
| XDR | 212 Gb/s PAM4 | 800 Gb/s | OSFP / OSFP-XD | 1.6T per switch cage |
Form-factor details: QSFP28, QSFP56, OSFP, QSFP112.
Reach classes
| Medium | QDR | FDR | EDR | HDR | NDR | Notes |
|---|---|---|---|---|---|---|
| Passive DAC | ≤ 7 m | ≤ 3–5 m | ≤ 3 m | ≤ 2 m | ≤ 1.5–2 m | reach shrinks each generation; thicker gauge for longer runs |
| Active copper (ACC/AEC) | — | ≤ 5 m | ≤ 5 m | ≤ 3–4 m | ≤ 3 m | powered plugs; AEC with retimers for NDR |
| AOC | ≤ 100 m (some 300) | ≤ 100 m | ≤ 100 m | ≤ 100 m | ≤ 30–50 m typical | 850 nm VCSEL, MMF ribbon inside |
| Transceiver SR-class (MMF) | ≤ 300 m | ≤ 100–150 m | ≤ 70–100 m | ≤ 70–100 m | ≤ 50 m (VR) / 100 m (SR) | OM3/OM4, MPO |
| Transceiver DR/FR-class (SMF) | — | — | 500 m – 2 km (PSM4, CWDM4) | 500 m – 2 km | 500 m (DR4) / 2 km (FR4) | MPO-12 APC (DR4), duplex LC (FR4) |
What is inside each cable type: Cable internals.
NDR twin-port and split cables
An NDR switch cage holds one twin-port OSFP module = two independent 400G ports behind two MPO-12/APC (or two LC-duplex) receptacles. Cabling options from one module:
| Configuration | Cable | Far end |
|---|---|---|
| 2 × 400G straight | two MPO-12 APC trunks | two 400G NDR HCA ports (OSFP or QSFP112) or another switch |
| 1:2 split (800G → 2 × 400G) | Y-cable, one OSFP end, two OSFP/QSFP112 ends | two HCAs — the standard server connection |
| 1:4 split (800G → 4 × 200G) | one OSFP end, four QSFP56/QSFP112 ends | HDR-speed HCAs (NDR200) |
| AOC twin-port | one OSFP twin-port end, two OSFP/QSFP112 ends | up to ~30–50 m |
| DAC twin-port | one OSFP end, two ends | ≤ 1.5–2 m; heavy — in-rack only |
Each split leg is a separate link with its own negotiation; the switch sees two (or four) ports. Identity coding must describe the whole assembly consistently at every end (Cable internals).
Finned vs flat OSFP
- Finned-top OSFP — integrated heat sink for switch cages with open airflow above the module. Does not fit a flat cage.
- Flat-top OSFP — for HCAs and cages with a riding heat sink. Fits either cage, but in an open switch cage it may run hot.
- QSFP112 on HCAs shares the NDR electrical interface at 4 × 106 Gb/s in a QSFP footprint; a QSFP112 HCA port pairs with a split cable from an OSFP switch.
Thermal consequences: Power & thermal.
Mechanical rules
| Rule | Why |
|---|---|
| Respect bend radius (DAC ≥ 10× diameter; AOC/fibre ≥ 30 mm) | crushed twinax → lane errors; bent fibre → Rx loss |
| Support cable weight — twin-port DACs are heavy | pulls on the cage, intermittent contact, cracked paddle cards |
| Pinned/unpinned and APC on every MPO | Breakout & MPO cabling |
| Clean and inspect every MPO before insertion | one particle costs a lane 1–3 dB |
| Keep dust caps on unused cages and connectors | contamination is the top cause of lane failures |
| Route away from hot exhaust | AOC engines and AEC plugs age with temperature |
Identity and vendor policy
InfiniBand switches and HCAs from the dominant vendor validate cables and transceivers against a supported list and read their cable info (vendor, PN, SN, length, technology) into the fabric manager. An unvalidated cable may link at reduced speed or be flagged; a cable coded with the wrong length or rate class may fail negotiation. The identity bytes are the standard SFF-8636/CMIS fields plus the InfiniBand rate declarations (Compliance codes, Vendor lock).
Inventory practice
- Label both ends with the link (switch/port ↔ host/port); split cables need per-leg labels.
- Record vendor, PN, SN and length per cable in the fabric manager or CMDB; pull the data from
mlxcablesor from CodingBox at incoming inspection. - Keep spares by reach class and connector; NDR twin-port and split variants are not interchangeable with plain 400G.
- Burn in new optics before deployment (Link reliability & monitoring).
In CodingBox
Cable ends and transceivers are read like any module: identity with cable length, technology and rate class, per-lane DDM for AOC and optics, CMIS state for NDR modules (Check transceiver, DDM). Reading every leg of a split cable before installation catches the mismatched end that would otherwise link at the wrong speed.