AI infrastructure is a systems problem, and systems are what I do.
Enterprise AI succeeds or fails on the infrastructure underneath it: the networks that move data between GPUs, the storage that feeds them, and the team that keeps all of it running. I’ve spent 25 years designing, deploying, and supporting systems where the network has to simply work, and I’ve built two interactive demonstrations of how enterprise AI is sold and run. I’m looking for an AI infrastructure engineer or architect role.
Where an AI cluster actually succeeds or fails
The same problems at a different scale
AV moved onto the network years ago, and it brought the same constraints an AI cluster has: traffic that cannot drop, bandwidth that has to be planned before anything is ordered, and racks that run hot. This is how the work I have done maps to the work an AI infrastructure team does.
| What I have done | The AI infrastructure version | What carries over |
|---|---|---|
| Dante audio networks | GPU traffic that cannot be lost. Remote direct memory access over Converged Ethernet (RoCE) gets its own protected traffic class, with priority flow control and congestion notification on every switch. | Dante gives clock sync (Precision Time Protocol) the highest network priority and audio the next, because a late clock packet means a dropout.1 Marking traffic, protecting one class from loss, and proving every switch honors it is the same discipline. |
| Video over IP with Crestron NVX | A GPU network sized so it does not choke: one high-speed link per GPU, and switch uplinks sized for the traffic the cluster will actually generate. | Each NVX endpoint needs its own 1 Gb/s port, multicast control through Internet Group Management Protocol (IGMP) snooping, and uplinks budgeted at about 1 Gb/s per device.2 Doing the bandwidth math before the hardware order is the habit that matters. |
| Equipment racks | GPU racks, where one 8-GPU server can draw more power than the average data center rack does today. | Amplifier heat, uninterruptible power supply (UPS) loads, and rack power budgets taught me to plan power and cooling first, rather than discover the limit on install day. |
| Standard room designs | A reference cluster design repeated across racks and sites. | A design built once and deployed many times is cheaper to support and fails in ways the team already knows how to fix. |
| Control programming and automation | Servers provisioned, updated, and monitored through their management interfaces, such as Redfish on a server’s remote management controller. | Crestron SIMPL# (C#), PowerShell, and n8n workflows against device interfaces. Hundreds of devices are managed with code, not by hand. |
| Final escalation tier | Day-two operations: GPU health, link errors, firmware drift, and the escalations nobody else can close. | Monitoring 150+ IPsec VPN tunnels and owning the problems that reach the top tier. Find the fault before users do, and own it until it closes. |
| Director of Integration Services | Cost per user, power cost, and on-premises versus cloud, presented to the people who approve the spend. | Cost comparisons, technology vetting, and proposals. The architecture has to turn into a decision the buyer can sign. |
What I would bring to an AI infrastructure team
Networks that cannot drop
- Years of AV over IP, where Dante audio and NVX video depend on multicast, quality of service, and low latency
- 150+ IPsec VPN tunnels on FortiGate, managed and monitored as the final escalation tier
- AI fabrics raise the stakes on the same discipline: bandwidth, latency, and congestion
Deployment at scale
- Director of Integration Services: deployment strategy, technology vetting, and cost comparisons
- Owned a service-provider practice responsible for the work my team installed
- Standard designs repeated across many sites instead of a custom build every time
Operations and escalation
- Final escalation tier for problems nobody else could close
- Proactive monitoring, so problems are found before users find them
- Documentation that lets the next engineer pick up where I left off
Translation for the business
- Turning architecture into cost, risk, and return for the people who sign
- Explaining technical problems in plain language to executives and end users
- An ROI calculator and use-case explorer built into my AI Factory demo
Two enterprise AI demonstrations I built
AI Factory presentation
A ten-slide talk on moving enterprise AI from pilot to production: what an AI factory is, how the networks, storage, and GPUs connect, and what it takes to run it. Three slides are live, simulated tools: an operations console, a deploy-a-model wizard, and an illustrative ROI model. Not affiliated with, authorized by, or endorsed by Dell Technologies.
Open the presentationAgent Workflows console
An interactive console that walks through four enterprise AI agent workflows designed for n8n: support ticket triage, a retrieval-augmented knowledge agent, a multi-agent workflow with a gatekeeper, and a document-to-action pipeline. Each one plays a simulated run step by step, with governance checks along the way.
Open the Agent Workflows consoleThe principle behind the agent workflows is the same one I would bring to AI infrastructure: the language model owns exactly one fuzzy decision per agent, and everything before and after it is deterministic, auditable workflow.
How I would approach the engineer or architect role
- Start from the workload: which models, how many users, and what response time, before choosing any hardware.
- Design the networks as carefully as the compute, because idle GPUs are usually waiting on the network or the storage.
- Plan for day two: monitoring, firmware, spares, escalation paths, and documentation.
- Translate the design into cost and business outcome for the people who approve it.
- Standardize on a reference design that can be repeated, rather than a custom build for every customer.
Sizing an inference cluster for 5,000 employees
This is how I would size a cluster for a real request, step by step. The figures marked as assumptions or estimates would be replaced with the client’s numbers and a real benchmark. The method is the point.
- A private AI assistant for 5,000 employees, built on an open model with about 70 billion parameters.
- Company documents searchable through the assistant (retrieval-augmented generation), and no data leaves the building.
- Answers should start appearing within about a second.
- 01
Start from the workload
Not everyone uses it at once. If about 10% of employees use it in the busiest hour and a few percent have a request in progress at the same moment, the cluster has to carry about 150 simultaneous requests at peak (assumption). A pilot with real users would replace this number before anything is ordered.
- 02
Fit the model in GPU memory
70 billion parameters at 2 bytes each is about 140 GB, and about 70 GB in 8-bit form. An H100 GPU has 80 GB of memory.3 Running each copy of the model across 4 GPUs leaves about 250 GB for the memory each conversation uses, which grows with the number of users and the length of their documents. That gives two copies of the model per 8-GPU server.
- 03
Count servers from throughput, not memory
Memory says the model fits. A load test says how many requests each copy can serve at the target response time. If one copy handles about 25 simultaneous requests (assumption), the peak needs 6 copies, which is 3 servers. A fourth server covers a failure or maintenance without an outage. Four 8-GPU servers, 32 GPUs, in a class such as the Dell PowerEdge XE9680.
- 04
Check power and cooling before anything else
Eight H100 GPUs alone are rated at up to 5.6 kW.3 The whole server is about 8 to 10 kW under load (estimate)4, so the cluster with its switches and storage is about 35 to 45 kW (estimate). The average data center rack runs below 8 kW.5 Either the servers spread across several racks, or the room needs high-density racks with liquid or rear-door cooling. That conversation happens in week one, not on install day.
- 05
Design only the networks the workload needs
Inside each server, the 8 GPUs talk over NVLink, so a model copy split across 4 GPUs never touches the network.3 That means this cluster does not need a GPU-to-GPU fabric between servers. Buying a training fabric for an inference workload is an expensive mistake, so I would leave room to add one only if the client plans to fine-tune models later. What it does need: a redundant front-end network for user requests and document lookups, a storage network, and a separate out-of-band management network.
- 06
Size storage for models, documents, and audit
Each model version is 70 to 140 GB, and the team will keep the current one, the previous one, and one under test. Add the company documents, the search index built from them, and the logs an audit will ask for. That is tens of terabytes, not petabytes (estimate), on shared file storage such as Dell PowerScale so any server can load any model.
- 07
Plan day two
Monitor GPU use, memory, temperature, and error counts. Keep drivers and firmware identical across servers. Roll out a new model one server at a time using the spare capacity, so users never see an outage. Keep spares on site and a clear escalation path to vendor support.
- 08
Give the buyer one page
What it supports, what it costs to buy and to power, what happens when a server fails, and how it grows: add a server for each additional 50 simultaneous requests (assumption). Then the list of what to confirm before ordering.
Sources and notes
- Q-SYS Help. Design Properties: Audinate QoS preset. The preset marks clock sync above audio and video, matching Dante’s default priorities.
- Crestron. AV-over-IP Network Design. One 1 Gb/s port per endpoint, IGMP snooping and querier, and uplinks budgeted at about 1 Gb/s per device.
- NVIDIA. H100 Tensor Core GPU. The H100 SXM has 80 GB of memory, is rated at up to 700 W, and connects to the other GPUs in the server over NVLink.
- Server Parts. Dell PowerEdge XE9680 with NVIDIA H100. A reseller’s estimate of 8 to 10 kW under load. Dell does not publish a system-level figure, so a real design would confirm it with Dell’s power planning tool.
- Uptime Institute. Global Data Center Survey 2024. Average server rack density is rising but remains below 8 kW.