CodingBox Documentation

Inside a switch: ASIC, SerDes, port groups, control plane

A switch port is the last stop before the transceiver, and most of what a module is allowed to do — which speeds, whether breakout works, how many ports can run 25G at once, how fast DDM is polled — is decided by the silicon and firmware behind that port. This page opens the box: the forwarding ASIC and its SerDes, how lanes are grouped into ports, what the control-plane CPU does with the two-wire bus, and where the power and heat for optics come from.

The three planes

PlaneHardwareWhat it does for optics
Data planeforwarding ASIC (merchant silicon such as Broadcom Trident/Tomahawk, Marvell, Nvidia Spectrum, or vendor-custom), its SerDes, PHYs/retimersdrives the electrical lanes into the module, runs PCS/FEC, MACs, forwarding
Control planea general-purpose CPU running the network OSreads module EEPROMs over I²C, validates identity, applies policy, configures lanes, polls DDM, raises alarms
Managementout-of-band Ethernet/console, sometimes a BMCwhere you see show interface transceiver and syslog

The data plane is fast and dumb; the control plane is slow and makes the decisions. That is why a module can be electrically perfect yet "unsupported": the CPU said no (What the host does).

SerDes and lanes

Every port is a bundle of SerDes lanes from the ASIC. Their per-lane rate is a property of the chip generation:

ASIC generation (typical)SerDes lanePort options
1G-era1.25 GBdSFP 1G, RJ45
10G10.3 GBdSFP+; 4 lanes → QSFP+ 40G
25G25.78 GBd NRZSFP28; 4 lanes → QSFP28 100G; often 10G fallback
50G26.5 GBd PAM4SFP56; QSFP56 200G; 8 lanes → 400G (QSFP-DD/OSFP)
100G53.1 GBd PAM4QSFP112; 8 lanes → 800G
200G106 GBd PAM48 lanes → 1.6T (OSFP224, OSFP-XD at 16 lanes)

A module needs a lane rate the SerDes can produce. A 25G-SerDes port drives a 10G module only if the SerDes supports the lower rate too (most do); a 50G-PAM4 lane cannot be slowed into a 25G-NRZ lane on some chips — hence "this port does not support 25G" (Ethernet generations).

Port groups and speed constraints

SerDes are organised in quads (groups of four lanes sharing a clock/PLL). Consequences:

  • Speed groups — on many 25G platforms the four SFP28 ports of a quad must all run 10G or all run 25G; mixing is refused or silently downgrades (Speed & rate).
  • Breakout — a QSFP port's four lanes can become four ports only if the ASIC and the port's PHY mapping allow it; often only certain ports (or every other port) can break out, and enabling breakout disables a neighbour (Breakout & MPO cabling).
  • FEC per port group — FEC mode is sometimes shared across a group.
  • Oversubscribed uplinks — some designs multiplex more front-panel lanes than the ASIC has, so not all ports can run full rate simultaneously.

The platform's hardware guide lists these rules per port; they are not visible in the module.

PHYs, retimers and gearboxes on the board

Between ASIC and cage there may be a PHY (for RJ45 ports), a retimer (to clean long PCB traces on high-radix boxes), a gearbox (e.g. 2 × 50G ASIC lanes ↔ 4 × 25G module lanes on some 100G designs) or a MACsec engine. Each adds latency and its own speed/FEC limitations, and each is another place where a lane can be mis-configured. Whether a port is "direct-attached to the ASIC" matters for LPO optics, which need the ASIC's own equaliser to reach the module (Modulation & DSP).

The control plane and the two-wire bus

  • The CPU reaches module EEPROMs through I²C multiplexers (one bus per port or per group), often via a CPLD/FPGA that also handles presence, LPMode, reset and interrupt pins (Two-wire interface).
  • Bring-up is a software sequence: detect → read identity → validate → power up → configure lanes → enable Tx → poll DDM; NOS releases differ in how thoroughly they implement CMIS (CMIS issues).
  • DDM polling is scheduled by the CPU — typically every few seconds to a minute; hundreds of ports share one management CPU, so heavy SNMP polling can slow it (Reading DDM with tools).
  • Policy (whitelists, errdisable, third-party overrides) is pure control-plane code (How each NOS validates a module).

Power and cooling for optics

ItemTypicalNotes
3.3 V per cageclass-limited: 1.5 W (SFP+), 3.5–5 W (QSFP28), 12–15 W (QSFP-DD), 15–25+ W (OSFP)exceeding it → module held in low power (Power & thermal)
Faceplate airflowfront-to-back or back-to-frontmismatched module heat-sink orientation or blocked intakes overheat optics
Total optics budget32 × 12 W ≈ 0.4 kW on a 400G box; 64 × 16 W ≈ 1 kW on 800Ga meaningful share of the switch's PSU rating

Buffers, forwarding and what they have to do with optics

Almost nothing — except that congestion features (PFC, ECN) interact with link errors: a port dropping frames on CRC looks like congestion to storage and RDMA (Lossless Ethernet). Forwarding capacity (Tb/s), buffer size and table sizes decide the switch's role (Roles & topologies), not its optics.

Merchant vs custom silicon, open vs closed NOS

ModelExamplesConsequence for optics
Merchant ASIC + vendor NOSmost enterprise and DC switchesvendor policy decides optics support; hardware limits from the ASIC
Merchant ASIC + open NOSSONiC, Cumulus on white-box hardwarepolicy is minimal; you own the compatibility testing
Custom ASIC + vendor NOSsome carrier and high-end DC platformstight integration; often the strictest validation

The NOS landscape and its optics behaviours: Network operating systems.

In CodingBox

The switch's decisions are made on the module's identity bytes, and those are what CodingBox shows and edits: rate codes and applications the SerDes must support, power class against the cage, connector and technology for the port's PHY path (Check transceiver, EEPROM editor).

The lane bundles the SerDes hands to the cage — SFI, 25GAUI, CAUI-4, 400GAUI-8, 800GAUI-8 — with C2M compliance points and lane mapping: XGMII, XAUI and the AUI family; the whole ladder from MII to AUI: Media-independent interfaces.

How these parts are packaged — fixed, chassis with line cards and fabric, stackable, industrial, white-box — with airflow direction and redundancy: Switch classes & form factors; what the ports are called once the box is racked: Port naming & LEDs.