Self-Verification Pushes DeepSeek Past Claude Fable 5 on Terminal-Bench at 11x Lower Cost
The day's loudest signal is that harness work, not new weights, is closing the open-model gap, with DeepSeek V4 Flash and V4 Pro reportedly beating Claude Fable 5 via self-verification and a runtime fix. Meanwhile practitioners warn that agent-written code is bloating codebases, and vendors from Claude Code to OpenCode shipped agent-tooling updates.
Quick Hits
- @jackyk02 reports that sampling just 5 solutions with DeepSeek V4 Flash and ranking them with the same model as an LLM-as-a-Verifier lifted accuracy from 79% to 88%, beating Claude Fable 5 on Terminal-Bench 2.1 at 11x lower cost; @jun_song flagged the result as evidence Stanford is "cooking something crazy."
- @WesRoth claims DeepSeek V4 Pro's benchmark scores jumped double digits with the same weights and no retraining, purely from a fixed runtime/reasoning protocol, passing Fable 5's published scores on several tests.
- Cursor has completed its migration from Solid to React, with @poteto explaining that agents write poor Solid code and that Cursor's in-house framework bans useEffect outright.
- @abhijaymrana says Google is buying Spirit Airlines' enterprise data for $10m after outbidding Mercor at $7.5m, which @levie reads as evidence that data now belongs on the balance sheet.
- @TheAhmadOsman notes Qwen 3.8 27B scored 51 on the Artificial Analysis Agentic Index, ahead of GLM 5.2 and DeepSeek V4 Pro 0813, while running on roughly $2-3k of hardware.
Open Models Pass Frontier Benchmarks With Harness Work, Not New Weights
The clearest pattern in today's posts: the open-versus-closed gap is being closed at the harness layer. @jun_song's post quotes @jackyk02's method in detail: as open models get cheaper, they can generate many high-quality candidate solutions and verify their own outputs at very low cost, so ranking five DeepSeek V4 Flash samples with LLM-as-a-Verifier beats closed frontier models on Terminal-Bench while staying 11x cheaper.
@WesRoth adds the sharper version of the argument for DeepSeek V4 Pro: same weights, no retraining, only a repaired runtime/reasoning protocol, and the scores move (DeepSWE 62.7 → 72.0, NL2Repo 61.5 → 73.4, HLE with tools 60.0 → 67.7, Toolathlon 74.1 → 79.5). His takeaway: the model "wasn't necessarily lacking intelligence. It was leaking it through bad runtime control." His post quotes @jun_song crediting a simple harness for unlocking Deepseek-V4-Pro-0813.
The local-AI implication follows quickly. @jun_song argues that if LLM-as-a-verifier holds up, pairing a GLM-5.3 API with a locally hosted DeepSeek V4 Flash on DGX Spark, or multi-batching both on a 4x Spark node, could beat Fable entirely offline. Supporting that, @jackyk02 replies that the underlying paper previously used Gemini 2.5 Flash to verify GPT-5.5 trajectories and reached SOTA on Terminal-Bench 2.0, so cross-model verification is an established pattern rather than a one-off.
@TheAhmadOsman rounds out the open-model picture with Qwen 3.8 27B hitting 51 on the Artificial Analysis Agentic Index, trailing only a few state-of-the-art models "several to tens of times its size." Standard caveat: these are social posts pointing at GitHub repos, not peer-reviewed results, so treat the numbers as leads to reproduce.
Agent Codebases Are Getting Worse, and the Countermeasures Are Data Work, Evals, and Routing
@GeorgeMayer's thread is the day's main dissent. His claims: agents have become slower, more verbose, and more paranoid yet still blunder badly; Opus "will take a day to build something totally incoherent," while "sol" stays focused but cannot regain its original trajectory. Excess tests and guardrails make codebases slower and costlier, and agents respond with even more complexity. His verdict: "many (most!) codebases at this point are worse off than they were 18 months ago," and the only way out is through.
@addyosmani offers the factory-discipline counterweight: keep humans in the loop for product intent, system design, and the quality bar; review code intentionally, especially where automated back-pressure breaks; run type systems, tests, mutation testing, security scanners, and architecture linting as early and continuously as possible; and remember that "Number of checks != quality." The goal is encoding human taste into the environment so agents show evidence of correctness while a human still owns what ships.
@alex_prompter calls @mardehaym's healthcare-billing writeup the best engineering team he has come across, and it is a concrete blueprint: four months to seven production agents handling claims under HIPAA, with the first month spent mapping data rather than touching a model, a 34-variable enrichment layer per claim, patient data stripped before any LLM sees it, strict schema validation with deterministic fail-closed code, and an eval harness that catches regressions before deployment. The flagship agent reconciles denials at 59 out of 60 on the eval set, and swapping model providers changes nothing in output. His kicker: "Model integration was the shortest line item in the four-month build."
@lukepierceops gives away the playbook behind his $25-60k operating-system builds: map every workflow and time the manual hours first, sort the stack into absorb/keep/kill, give every entity exactly one write path, consolidate before automating, layer four agents in a specific order with intake first, and route simple tasks to cheap models for an 80%+ token-spend cut.
Agent-Friendly Foundations: React Over Solid, VMs Over Worktrees, Specs as Contracts
@theo spotlights @poteto's notes on Cursor's finished Solid-to-React migration (plus the move from scss and Tailwind to StyleX). The reasoning is agent-centric: signal-based reactivity caused accidental fan-out in a large app, and "agents were quite bad at writing good Solid code." React is "the most known quantity," React Compiler handles over-rendering, useEffect reads like Rust's unsafe block, and Cursor's internal Dune framework bans useEffect outright, exposing it only through framework-provided hooks.
@nateberkopec is evaluating client-layer tooling for agent environments and argues bluntly that "The solution is VMs. Worktrees and port registries are lipstick on a pig," while noting VM/sandbox provisioning is currently the more mature part of the ecosystem compared with the human-facing frontend.
For practice, three resources: @simas_ch's book "Spec-Driven Development: From Specs to Code with AI Agents" (Apress Pocket Guides, ~150 pages) arrived early and frames specs as the contract between you and your agents via the AI Unified Process. @pvncher resurfaced his guide to multi-agent orchestration in Codex, now more relevant for scoping bounded work to Luna sub-agents. And @trevin's Compound Engineering (August 17) collects agent skills for Claude Code, Cursor, Codex, and Grok under the philosophy that each unit of engineering work should make the next easier.
Ships of the Day: /design in Claude Code, Paperclip, OpenCode Usage Boosts, Whop in Grok
@nateparrott announced an early preview of the /design command in Claude Code: try "/design a few options for {feature}" before building, then pick an artboard, edit, and implement. @trq212's endorsement is simply "do it rn."
@papercliping shipped Paperclip v2026.817.0, featuring decisions where agents propose and you decide, improved company import and export, a unified Activity page, self-updating managed installs, and 311 commits from 17 contributors.
On pricing, @opencode declared "Operation Cheepseek: Phase 1 Complete," giving OpenCode Go subscribers $30 of usage for $10. @thdxr cautions that the team is "not in control of the original price increase" and is scrambling to find capacity at lower prices.
In agent-native distribution, @whop is now a native connector in Grok via a @spacexai partnership, and @vasuman argues companies without agent-native interfaces and deep CLI integration "are going to lose." Rivalry watch: @morganlinton predicts "Github's days are numbered" with Graphite founder @TomasReimers driving Origin.
Around the Community: LEDs That Leak Keys, Drones That Recharge, Personal Agent Stacks
@DoingFedTime resurfaces Ben-Gurion University's 2023 side-channel attack: researchers filmed a smart-card reader's power LED from 16 meters through a window using a hijacked security camera and recovered a 256-bit ECDSA key, with rolling shutter turning 60fps into roughly 60,000 brightness samples per second.
@karpovlabs' electromagnet landing pads enable a full auto land/recharge cycle, meaning 24/7 flight with a few rotating drones; @10_X_eng points to its 824 followers as proof people prefer real builds over timeline slop. On personal stacks, @KSimback describes a Hermes VPS agent (running 5.6 Sol and GLM-5.3) paired with Hermes Desktop (Grok 4.6, Bot Mode) linked over SSH, like an office assistant and a remote chief of staff, which @Shaughnessy119 calls a north star for owning your AI stack. And @0xSero boosts @_cpatonn (Ton), who has posted more model quants than anyone on HuggingFace for Ampere-generation cards and "makes nothing doing this."
Practical Takeaway
The most testable signal today is verification scaling. Before paying frontier API prices for agentic or terminal work, try @jackyk02's setup: sample around five candidates with a cheap model and rank them using the same model as verifier; his numbers show 79% → 88% accuracy and a win over closed models at 11x lower cost. If a model underperforms, suspect the harness before the weights, since @WesRoth's V4 Pro gains came from a runtime fix alone. Just reproduce these results on your own evals first, because these are social posts with GitHub links, not published papers.
Sources
Practical multi-agent orchestration in Codex
https://t.co/WPSWyw57q3
The Full Guide to Building an AI-Native Operating System
Okay electromagnet landing pads work actually pretty well. Just gotta train a landing routine and we have a the full auto land/recharge cycle active, aka 24/7 flying if rotate a few drones https://t.co/U60y5KFYsH
Operation Cheepseek: Phase 1 Complete OpenCode Go subscribers now get $30 of usage for $10 Phase 2 initiating... https://t.co/zWbdeiZeVj
Much love to the Solid team and congrats on launching Solid 2! Cursor has largely completed this migration from Solid to React. We also have since converted all our scss and Tailwind to @stylexjs, in both Cursor and in @bot. The new agents window is 99% React, with a few stragglers left that we have yet to migrate. The big motivation for this migration was for perf and maintainability reasons. We have seen that in practice, it is actually quite easy to create perf footguns for yourself in signal based approaches when reactivity causes a large amount of accidental fan out in large and complex apps. It probably was a skill issue. But in the age of agents, choosing what pitfalls you're okay with is an important decision. I think it's fair to say that every UI framework has its pitfalls and that no perfect one exists yet. For us, React was the most known quantity, and we found that solving its problems is actually quite straightforward and boring: exactly what you want when you're trying to ship! Something interesting I also observed is the uncanny valley effect of non-React frameworks looking like React. I think it confuses agents, because in my experience, agents were quite bad at writing good Solid code. Everything ended up being accidentally tracked which caused so many perf problems. React Compiler solves most of your perf problems with over-rendering. The remaining cases just take some manual memoization or refactoring/composition, which an agent can easily do. The nice thing about useEffect is that it's like an unsafe {} block in Rust: it shows me at a glance where the problems are. @bot (which is 100% React) is built on Dune, our in-house, agent friendly React framework for desktop apps, which actually bans useEffect outright and only exposes it indirectly through framework-provided hooks. Highly recommend setting something like that up for yourself!
Guys, I don’t think ai is good at coding…
Thrilled to partner with @spacexai to help people turn tokens into economic value. Whop is now live as a native connector in @grok https://t.co/GLy0nq0vCp
As the backlash against AI was growing stronger (and as my friends were becoming more fervently anti-AI), I decided to return to an increasingly hated field. Why? I agree there is a lot that is terrible about AI. 1/ https://t.co/e1kXkzH2eF
Here is my new setup with Hermes My original Hermes Agent on VPS connected via TG remains my workhorse - currently using 5.6 Sol and GLM-5.3 via sub plans I've been loving Hermes Desktop so I have that running separate to my VPS with Grok 4.6 and playing with Bot Mode, loving it so far My Desktop agent interacts with my VPS agent via custom skill using ssh so I don't need to leave the desktop app to work with my VPS agent The way I think about it is like having a personal assistant sitting in my office (my Desktop agent) and a remote Chief of Staff (my VPS agent) There is benefit in the proximity - my Desktop agent can do things locally on my PC that my VPS can't easily do And there's benefit in the separation - my VPS agent is always on, can spin up sandboxes and has more independent functional capabilities But now I can work with all of them in one surface area via Hermes Desktop and the Bot Mode plugins make my PA agent infinitely powerful There are now many ways to design your setup across Hermes Desktop, Hermes Cloud agents, and of course the original VPS agents Curious to hear how others design their setups if using Hermes Desktop + VPS agents
Compound Engineering - August 17, 2026
Compound Engineering is a plugin of skills for Claude Code, Cursor, Codex, Grok and other agents that keep getting added. Our core philosophy is that ...
Wow, seems like Google is buying Spirit Airlines' enterprise data for $10m (outbidding Mercor at $7.5m). Basically includes every internal document, email, workflow, and codebase for a once $6B company. Honestly, $10m for 34 years of operational data really seems like a steal. https://t.co/9qin2ef6Az
The solution is VMs. Worktrees and port registries are lipstick on a pig.
Scaling self-verification with DeepSeek V4 Flash beats Claude Fable 5 on Terminal-Bench 2.1, while being 11x cheaper 💰 As open-source models become more capable, they can now generate large numbers of high-quality candidate solutions and verify their own outputs at very low cost. For example, we find that sampling just 5 solutions with DeepSeek V4 Flash and ranking them using the same model with LLM-as-a-Verifier can lead to a significant boost in accuracy (79% → 88%), outperforming closed frontier models on Terminal-Bench. Try it out today: https://t.co/UhudjKDaYI More on verification scaling in my previous post.
Today we’re releasing an early preview of the /design command in Claude Code! from CC Desktop or CLI, try something like "/design a few options for {feature}" before you build — pick your fave artboard, edit it and implement. https://t.co/2JKKBnU60Z
Someone figured out how to unlock the true power of Deepseek-V4-Pro-0813. They fixed the thinking process errors with just a simple harness, and it completely outperforms Fable across every task. The benchmark scores are insane. GitHub ⬇️ https://t.co/NuzMSrlYtt
@jun_song Of course! In our paper (https://t.co/6cnzxcgiP7), we actually used Gemini 2.5 Flash to verify GPT-5.5 trajectories, and it got SOTA on Terminal-Bench 2.0 at the time. So local V4 Flash verifying GLM 5.3 trajectories should definitely work 👀
How will people in the next thousands of years view our work? Will our technology be imprinted for thousand of years, under every circumstances, e.g., natual disasters, nuclear war, etc? I hope to build things that last and benefit humanity in generations. https://t.co/gBYELlpKMW
A PE operating partner asked us to build production AI agents inside a portfolio company's billing system, processing real healthcare claims under HIPAA. Two people hand-wrote every rule in their claims engine across 300+ denial codes and payer logic that changes quarterly. Four months later, seven production agents handle it with zero patient data exposure. First month, we didn't touch a model. We mapped their data: where it sits and what's missing, so agents reason from structured facts instead of guessing. I've watched teams skip this step across dozens of engagements. They bolt a model onto the product, watch it hallucinate over unstructured inputs, and decide AI isn't ready for their industry. The data work is what makes it ready. We built an enrichment layer that assembles 34 dynamic variables per claim before any LLM sees it, pre-computed and versioned so the agent receives ranked facts instead of searching for context. Every agent follows one pattern: pre-compute context, strip all patient data before the model sees it, validate output against a strict schema, let deterministic code accept or reject the action. If the output falls outside the allowlist, the system fails closed. Seven agents, each locked to a single workflow like denied claim follow-up or billing reconciliation, each running its own enrichment payload. Then we built the eval harness. Every agent runs against a curated test suite before any update reaches production. When a model provider ships a new version or payer logic changes, the harness catches regression before a single live claim is affected. The flagship agent reconciles denials to the penny: 59 out of 60 on the eval set. Most teams launch an agent and hope it keeps working. We launch one and prove it does on every deployment. We route calls across two model providers. Swapping one changes nothing in the output because the eval harness verifies it. Model integration was the shortest line item in the four-month build. The operating partner now benchmarks the rest of the portfolio against this system. That's the line between a portfolio company running AI and one still running demos.