InferX

About InferX

Most inference platforms sell you compute. InferX gives you control of it. We built InferX because most GPU spend is wasted — models sitting loaded in VRAM waiting for traffic that isn’t there, and painfully slow cold starts when it finally shows up. InferX is an inference runtime that virtualizes GPU execution — CUDA and NCCL — at the boundary, not just the scheduler. That’s what makes sub-second cold starts possible: instead of rebuilding CUDA state, loading weights, and re-establishing multi-GPU topology from scratch, InferX snapshots an initialized inference system and restores it in milliseconds. In production, we’ve measured 494ms cold-start-to-first-token on a 27B model at full precision — a metric we’ve submitted to NeurIPS as a standardized benchmark. One snapshot becomes four production primitives: cold start, scale-out, GPU migration, and failure recovery — all from the same mechanism. Because the runtime controls the boundary all the way through GPU communication, isolation is built into the architecture, not bolted on top. InferX deploys inside your existing Kubernetes clusters — you keep your infrastructure, we handle inference execution underneath it. Inference is a systems problem, not a chip problem. Learn more: inferx.net