Red Hat is rethinking the CPU-GPU split, offloading prefill and memory-bound KV cache operations to modern CPUs to focus GPUs on compute-heavy token generation.
Published
Signal category
Research & Knowledge
Quote
“Red Hat is rethinking the CPU-GPU split, offloading prefill and memory-bound KV cache operations to modern CPUs so GPUs can focus entirely on compute-heavy token generation.”
— Eduardo Alonzo|Red Hat team
Company
Red Hat
The leading provider of enterprise open source solutions.
- Industry
- Software Development
- Location
- Raleigh, US
- Company size
- 19,494 employees
Red Hat is the world’s leading provider of enterprise open source solutions, using a community-powered approach to deliver high-performing Linux, hybrid cloud, edge, and Kubernetes technologies. We hire creative, passionate people who are ready to contribute their ideas, help solve complex problems, and make an impact. Opportunities are open. Join us.
Founded 1993