The Coming Shift in AI Infrastructure: Programmability Beyond Silicon

While the whole world is more and more enamored of AI and all its applications, there exist some very real barriers impeding its full success. Take, for example, AI data center infrastructure, which faces significant reliability challenges, performance bottlenecks, and increasingly power-consumption constraints that limit how far AI systems can scale in practice. Indeed, AI’s ever-changing workloads demand a move into the next stage of OCS development — programmable silicon photonic-based OCSs — which enables levels of network flexibility never seen before.

How we got here: The history behind OCS development

Optical Circuit Switches (OCSs) are rooted in a long history of telephony, beginning in the late 19th/early 20th century when voice communication relied on circuit switching — physically switching cables to establish a telephone connection between two parties. Packet switching was introduced in the 1960s as a way to make better use of shared infrastructure. It involved breaking data into small “packets” to allow multiple transmissions to travel across a network on any route. In the 1970s, these packets were further defined in how they were addressed, routed and delivered across heterogeneous systems, and in the 1980s, this definition — Transmission Control Protocol/Internet Protocol, or TCP/IP — became the Internet standard to allow previously incompatible networks to communicate under a common framework. As network and scalability demands grew in the 1990s, Electrical Packet Switches (EPSs) were introduced. Combined with TCP/IP, EPSs underpinned the Internet’s growth and connected millions of users globally. At the same time, fiber began replacing copper in global networks, offering higher capacity and longer reach and an ability to support multi-terabit speeds.

The dynamic AI environment

But by the early part of the 21st century, AI workloads put tremendous strain on current electronic-based networks, prompting the development of the first commercial MEMS-based Optical Circuit Switch (OCS) data center architectures. Optical MEMS switches are all-optical switching devices that use microscopic movable mirrors to redirect light between input and output fibers without converting the signal to electricity. These MEMS-based OCSs support large port counts, which are ideal for optically connecting distant servers overcoming copper limitations in data centers. However, limits in reconfiguration speed, cost-per-port and form factor have become evident. These limits prevent MEMS-based OCSs from addressing the need for network real-time reconfiguration in the heart of the data center computing engine, the scale-up network — especially in the face of AI workloads.

Indeed, today, the limits to MEMS-based OCSs and the demands on the AI data center are only becoming more pronounced, thanks to the massive, non-linear, unpredictable changes introduced by AI every year or every six months — if not every quarter. AI data center ecosystem actors are now being asked to adapt rapidly and respond to the ever-changing AI landscape. And network designers are pressured to reconfigure or reprogram their AI data center networks as needed to circumvent problems within the network, or manage the new level of AI workloads needing optimized performance.

Programmable silicon photonics: Moving beyond a ‘frozen’ network

Programmable silicon photonic (SiPh) OCSs are the next step in OCS development. Low cost, very compact and driven by software, these photonic chips can be instantaneously reprogrammed to adapt in real time the way of light and therefore reconfigure the network. As compared with MEMS, the programmable SiPh OCS is solid-state technology, which removes a lot of reliability risk because there are no moving parts. Solid-state, CMOS compatible technology also implies that it can match the optimum GPU cluster target cost of $100 per radix.

Programmable SiPh OCSs further strengthen AI data center architectures in two critical ways. First, they enable rapid reconfiguration of GPU interconnects so workloads can be executed more efficiently and complete faster. As AI training evolves, communication topologies must change dynamically—even within the training job—without packet loss. This requires extremely fast reconfiguration times, an area where SiPh OCS scalability is fundamentally superior to MEMS-based approaches, supporting reconfiguration and transduction times orders of magnitude faster than MEMS technologies.

Second, SiPh OCS programmability allows additional functions to be integrated directly into the switching fabric without scaling on form factor. Capabilities such as real-time telemetry through SiGe-integrated photodetectors and link amplification can be incorporated to improve observability and enhance failure resilience. While MEMS-based OCSs typically introduce 2–3 dB of optical loss, SiPh OCS implementations can be designed to be effectively lossless, improving overall system flexibility and efficiency.

Looking ahead

Because historical data center networks are rigid and can’t keep pace with the changing needs of AI data centers, the market for programmable SiPh technology presents a multi-billion-dollar opportunity. Along with this huge boom comes the need for collaboration and cooperation among businesses that are at the heart of this new technology. To that end, there exists an OCP standardization body — which includes Google, Microsoft, Lumentum and other innovators — that aims to make the software interface for the network manager using OCS as standard and easy to use as possible. Together, these companies like to share their perspectives and create standards to move the technology forward and accelerate adoption.

As AI drives evolution in our world, AI data center networks must likewise evolve and be future-proof to support it. Programmable SiPh OCSs enable companies to create at the peak of innovation and realize new and exciting opportunities for all.