Magic developed a new pretraining recipe for Frontier that matches DeepSeek V4 Pro's pretrain using 50x less compute, roughly half the FLOPs used for GPT3, or ~$0.5M on GB200.
Public source
Publisher name
Public post
Frontier pretraining is said to be a big-lab-only game. We don’t have 100k chips yet, so there’s only one way: algorithmic efficiency. Our new recipe matches DeepSeek V4…
Company
Magic
Build aligned and more complete AI to accelerate humanity’s progress on the world’s most important problems
- Industry
- Software Development
- Location
- San Francisco, US
- Company size
- 51–200 employees
About Magic
Magic is working on frontier-scale code models to build a coworker, not just a copilot. Come join us: http://magic.dev
See moreLatest activity
Latest activity from Magic
3 signals
Discover more
Similar signals
Similar public activity from other companies.
Research & Knowledge
Together AI
Together AI published research on QLoRA to compress the base model to 4 bits and reduce memory usage to 1/4, allowing the base model plus 16-bit adapters to fit on a smaller GPU.
Research & Knowledge
Applied Compute
Applied Compute developed a 35B open-weight model trained to search a precomputed index answering repo search questions at 100x lower cost than a frontier model.
Research & Knowledge
Infinity Artificial Intelligence Institute
Infinity Artificial Intelligence Institute recently beat vLLM on a single H100.
Research & Knowledge
Runway
Runway published new research on Solaris, our first Interface World Model, which finds that Solaris outperforms frontier LLMs when generating new interfaces across structural similarity and information retention.
Research & Knowledge
GitHub