Onyx Government Services published a blog post on continuous batching in LLM serving.

Onyx Government Services, a Service-Disabled Veteran-Owned Small Business (SDVOSB), is headquartered in Chantilly, Virginia, with an additional office in Huntsville, Alabama. The company comprises a highly specialized team of software engineers, technical analysts, and data scientists who excel in adapting to emerging challenges, fostering innovative thinking, and delivering exceptional results for Federal Civilian Agencies, the Intelligence Community, and Department of Defense service branches in support of warfighters. Established in 2006 by Ken Jensen, a U.S. Army veteran and business executive with over three decades of experience in government consulting services, Onyx GS offers expertise in: Extracting actionable insights from data Securing critical national networks against cyber threats Defending financial systems from fraud and terrorist activities Facilitating Open-Source Intelligence (OSINT) collection Designing and developing software to enhance operational efficiencies and workflows Onyx GS distinguishes itself through its commitment to delivering innovative solutions in data analytics, enterprise technology, cybersecurity, and software development. The company's team of strategists, cloud architects, data scientists, statisticians, and cybersecurity engineers are united by a shared mission: leveraging insights to advance operational objectives. The Onyx team challenges conventional approaches, providing tailored solutions that address the complex needs of government and defense sectors. Their collective expertise enables them to develop and implement cutting-edge technologies that enhance mission capabilities and operational effectiveness. For further information or to discuss how Onyx Government Services can assist with your organization's needs, please contact us at 571-340-3900 or visit our website at https://www.onyxgs.com/contact-us.

Founded 2006