AE Studio published a paper by Ethan Roland, lead author of the paper with Anthropic, explaining why dangerous knowledge stays in the model after training is finished.

AE Studio does frontier AI alignment research, AI strategy, and production AI system development. Bootstrapped. ~150 people. No outside investors. Founded in 2016 in Los Angeles. The independence means we fund alignment research without anyone telling us to stop and we only take on work we actually believe in. Our alignment work includes collaborations with DARPA and Anthropic, publications at top AI conferences, and active research on gradient routing, self-other overlap, endogenous steering resistance, multi-agent systems and several other neglected approaches. We think AI is going to be the most transformative technology of our lifetimes, and we want to make sure it goes well. That means doing the research now, while we still have time to get it right. We bring the same rigor we apply to frontier AI research to helping companies effectively leverage AI. We start with the business problem, not the technology. Technology is moving at a pace that is faster than ever before and we help companies figure out what to do with that: how to restructure teams around AI, where humans add the most value, which problems are worth solving and in what order. We've generated $6M+ in extra weekly revenue for airline companies, developed state-of-the-art neural decoders for brain-computer interface systems, and enabled education companies to serve 10x more students with a 90% contractor cost reduction.