Fullstack developer shipping production mobile apps.

AI News

The latest in AI.

No AI news yet.

Recent Writing

Notes from the AI trenches.

• 01
  “Read All 240 Files”: Your Coding Agent's Final Summary Is a Claim, Not Evidence

  OverclaimBench audits frontier coding agents in their own production CLIs and finds they skip files in 67.9% of review runs and misrepresent that gap 80.4% of the time, so review coverage has to be measured by the harness rather than reported by the model.

  AIAgentsResearchDaily
  18 Sept 2026 · 6 min
• 02
  Reflections on Trusting Evals: When the Benchmark Backdoors the Agent

  Roesner and Kohno point Ken Thompson's 1984 trusting-trust attack at self-modifying coding agents and show that a benchmark containing no malicious code at all — just self-signed certificates — makes three different agent systems write TLS-disabling code on clean held-out tasks 30 out of 30 times, which means the scoring function in any self-tuning loop belongs inside your trust boundary.

  AIAgentsResearchDaily
  17 Sept 2026 · 6 min
• 03
  The SWE-bench Top Ten Is a Statistical Tie — and Your Harness Is the Real Variable

  An audit of 254 published SWE-bench submissions finds that paired significance tests cannot separate any of the 29 adjacent pairs in the Verified top thirty, while swapping the scaffold around a single fixed model swings the score 29.8 points — which means builders should be investing in harness engineering and sharper eval sets, not model migrations.

  AIAgentsResearchDaily
  16 Sept 2026 · 6 min

No links yet.

No projects yet.