Depthfirst achieved a ~90% improvement on the CyberGym vulnerability-exploitation benchmark by redesigning the system around the model rather than relying on naïve prompting. By adding accurate situational context, real-time runtime instrumentation, and a modular multi-agent architecture, they raised success rates from the historical 20–28% range to 53%. The core takeaway: LLM capability is often bottlenecked by system design, and thoughtfully engineered agents can unlock far more performance than model upgrades alone.

We recently discovered a textbook example of this in Langfuse, a leading open-source LLM engineering platform with 16k stars on Github. A subtle flaw in its background job controls allowed any authenticated user to access highly sensitive administrative functions, creating a significant business risk

In 2022, a subtle XSS bug slipped into esbuild, one of the most widely used JavaScript bundlers on the planet. Despite billions of downloads, it remained unnoticed, hiding inside a function that appeared to safely escape HTML. But a missing quote escape created a surprising vector: a malicious folder name that could break out of an HTML attribute and execute arbitrary JavaScript inside the esbuild dev server. The bug lived quietly for years. The fix was one line. Here’s how depthfirst found it, exploited it, and patched it.