CodingBox Documentation

NICs with optical ports: an overview

The other end of every server link is a network interface card (NIC), and it plays the same game as the switch — cages, SerDes lanes, a management bus to the module, a policy about which modules it accepts — with two differences: the policy lives in a driver, not a network OS, and the bottleneck is often the PCIe bus, not the port. This section covers NICs with SFP/QSFP-class cages: the families and vendors, how ports relate to PCIe, transceiver compatibility and its driver-level overrides, the tools that read a module from the server side, and the programmable adapters (SmartNICs, DPUs) that blur the line between card and switch.

What a NIC is, in optics terms

ElementOn a switchOn a NIC
Cages24–64 per box1–4 per card (1–2 typical)
SerDesforwarding ASICNIC controller ASIC (Intel E810, Broadcom BCM57508, Nvidia ConnectX-7…)
Module managementNOS over I²C mux and CPLDdriver/firmware over the controller's I²C
PolicyNOS whitelist (How each NOS validates a module)driver/firmware whitelist (Optics compatibility on NICs)
Speed/FEC configCLIethtool, vendor tools, BIOS/UEFI, OS device manager (Tools)
Backhaulfabric ASICPCIe lanes to the CPU (Ports & PCIe)
Power3.3 V per cage from the chassisPCIe slot (25–75 W) plus auxiliary connector on high-end cards

Form factors of the card

Card typeWherePorts
PCIe add-in card (half-height half-length, full-height)any server slot1–2 × SFP28/QSFP28, 1–2 × QSFP56/QSFP112/QSFP-DD/OSFP
OCP NIC 3.0 (SFF / LFF mezzanine)OCP slot at the rear of the serversame cages, front-accessible, hot-swap on some chassis
LOM / embeddedon the motherboardusually RJ45; SFP+ on some
Mezzanine / bladeblade serversconnects to blade chassis switch, no cages
SmartNIC / DPUPCIe or OCP2 × 100/200/400G, own Arm cores (SmartNICs & DPUs)

Families and vendors

VendorFamilies (optical-port models)SpeedsCagesDriver (Linux)
IntelX520 / X710 / XXV710 / E810 / E83010G / 10G / 25G / 25–100G / 200GSFP+, SFP28, QSFP28, QSFP56ixgbe, i40e, ice, idpf
Nvidia (Mellanox)ConnectX-4 Lx / -5 / -6 Dx / -6 Lx / -7 / -8 SuperNIC25G / 100G / 100–200G / 25–50G / 200–400G / 800GSFP28, QSFP28, QSFP56, QSFP112, OSFPmlx5_core
BroadcomNetXtreme-E BCM5741x / 5750x (P225, P425, P2100, P1400GD)25G / 100–200G / 400GSFP28, QSFP56, QSFP-DDbnxt_en
Marvell (QLogic)FastLinQ 41000 / 4500010–25G / 100GSFP28, QSFP28qede
AMD (Solarflare, Pensando)X2522 / X3522; Pensando DSC10–25G; 100–200GSFP28; QSFP56sfc; ionic
ChelsioT610–100GSFP28, QSFP28cxgb4
OthersSilicom, Napatech (FPGA), Xilinx/AMD Alveo, Netronome10–100Gmixed

Speeds per card model are fixed by controller and cages; "25G capable" means 10G and 25G, rarely 1G (Ports & PCIe).

What the NIC needs from a module

  • Compliance codes for a speed the controller supports at that cage.
  • Identity the driver accepts — Intel families are the strict ones (Optics compatibility on NICs).
  • FEC/AN behaviour the driver can match: DAC at 25G+ requires Clause 73 AN on both ends; optics need forced speed and matching FEC (FEC & AN).
  • Power class within the card's cage budget — most NICs allow QSFP28 class 4 (3.5 W) and QSFP-DD up to 12–15 W; check for ZR/tunable modules (Power & consumption).
  • Cooling: server airflow is front-to-back through the card; modules on rear-facing cards sit in exhaust air (Temperature grades).

Typical roles and their optics

RoleNICLink
General virtualisation host2 × 25G SFP28DAC to ToR (Roles & topologies)
Storage / NVMe-oF2 × 100G QSFP28DAC/AOC, RoCE with PFC (Lossless Ethernet)
GPU server4–8 × 400G OSFP/QSFP112 (one per GPU)AOC/DR4 to rail leaves (Cabling an AI cluster)
HPCInfiniBand HCA 1–2 × NDR/XDRDAC/AOC (IB cables)
Edge / branch2 × 10G SFP+LR to a campus core
Management / OOB1G RJ45 or SFP

Further reading

In CodingBox

A module that a NIC rejects is diagnosed the same way as one a switch rejects: read the identity and compliance codes on the bench, compare with the driver's list, code if the policy demands it (Check transceiver, Vendor lock).