Aegis | AI Evaluation Platform reported that 640 runs of the same input, same output, and same metric configuration resulted in a mean bias score sliding from 28 to 72 after the judge changed.
Public source
Publisher name
Public post
640 runs. Same input, same output, same metric configuration. The only thing that changed was the judge. This got the mean bias score to slide from 28 to 72. The spread…
Company
Aegis | AI Evaluation Platform
Evaluate and monitor AI systems with confidence.
- Location
- Iasi, RO
- Company size
- 2–10 employees
About Aegis | AI Evaluation Platform
Aegis is an AI evaluation and monitoring platform for teams building GenAI-powered products. We help companies measure model behaviour, detect regressions, benchmark prompts, and make release decisions based on evidence - not intuition. From pre-release testing to production monitoring, Aegis gives product and engineering teams the confidence to ship AI systems at scale. Aegis was created by the team behind Strongbytes.
See moreLatest activity
Latest activity from Aegis | AI Evaluation Platform
2 signals
Discover more
Similar signals
Similar public activity from other companies.
Research & Knowledge
Eticas.ai
Eticas.ai published an independent stage-by-stage evaluation of Barcelona Activa's AI-assisted shortlisting pipeline findings and methodology.
Research & Knowledge
Geogen
Geogen measured 16,979 cases where a single engine answered an identical question seven separate times across seven AI engines, compared answers against each other rather than against other engines.
Research & Knowledge
Decision Systems AI
Decision Systems AI published a research paper titled 'A human in the loop is not the same as human judgment in the loop' citing a field experiment involving 228 experienced evaluators.
Research & Knowledge
Humane Intelligence
Humane Intelligence published a methodology paper detailing its ontology-based contextual AI evaluation mathematical modeling and software.
Research & Knowledge
LangGuard.AI