AI Digest.

Grok Bot Learns to Delegate: HarnessRouter Routes Tasks to Claude Code, Codex, and 20 More Harnesses

The open-source HarnessRouter reportedly lets Grok Bot coordinate work across rival agent harnesses through one unified protocol, while Microsoft's Skill Recorder teaches agents by watching you complete a task once. On the cautionary side, research circulating on the feed found coding agents follow stale CLAUDE.md instructions over actual code roughly nine times in ten, and @jack unveiled Dessa, a full-duplex speech model.

Quick Hits

  • @_avichawla reports that newly open-sourced HarnessRouter lets Grok Bot delegate tasks to external harnesses including Claude Code, Codex, Hermes, DeepSeek Harness, and Pi, with a Unified Harness Protocol (UHP) standardizing sessions, streaming, files, cancellation, and failure handling across all of them.
  • Research summarized by @ai_database (from Georgia Tech and other institutions) found Claude Code followed outdated CLAUDE.md and AGENTS.md instructions over the real code in 79 of 80 trials, even while noting the conflict in its own output. One added sentence largely fixes it.
  • @RoundtableSpace says Microsoft open-sourced Skill Recorder (4.3K stars): record yourself doing a task once, and Copilot turns the capture into an editable skill your agent can run on a schedule.
  • Security had a rough morning: @IntCyberDigest reports researcher Erica Windisch published 11 unpatched Tailscale flaws without coordinated disclosure, while @ryaneshea (boosted by @jaredctate) warns Apple's macOS certificate bundle still trusts certificates forgeable with compromised eMudhra keys.
  • @jack introduced Dessa, calling it "the most authentic full-duplex speech model ever built," with emotion, pauses, and natural cadence.

Grok delegates, Pi gets durability, and Hermes keeps spreading

The day's densest cluster was plumbing for multi-agent setups: routing work between harnesses, surviving crashes, and feeding skills in.

The routing story is @_avichawla's: Grok Bot stays a coordinator while delegated tasks run in separate HarnessRouter sessions, each using its own harness, model, and tools. In his test, Grok Bot assigned support-ticket CSV analysis to DeepSeek Harness running Claude Sonnet 5.5, passed output to Codex on GPT-5.5, then checked the finished dashboard in its browser. He stresses these workers are not extra Grok Bots; UHP just makes sessions, streaming, and files uniform regardless of which harness executes.

Durability showed up too: @onusoz relayed @nicopreme's news that pi-subagents hit 4K GitHub stars and no longer lose work when a session crashes. @hayatomaruu passed along a walkthrough attributed to an Anthropic engineer on harness engineering (loops, sandboxing, resuming from logs, converting yesterday's logs into new skills, which Anthropic allegedly calls "dreaming"). The salary figures and attributions are unverified and the format reads like engagement bait, but the logs-to-skills idea rhymes with the durability work above. The full guide he links is @beamnxw's "Make Opus 5.5 Finish Long Tasks."

The Hermes ecosystem kept compounding: @tonysimons shared his agent profile stack across four machines, and @Teknium pointed to @witcheer's map of Hermes Agent videos sorted by use case, from running a business on it to Kanban teams, Bot Mode, and cron. @HamelHusain, riffing on @hwchase17's five components of a "company harness," noted two of his three agents are marketing-related: "It feels right."

For skills themselves, Skill Recorder needs a GitHub account with Copilot access and runs on Mac, Windows 11, and Ubuntu; it captures clicks, app switches, and terminal commands locally, then generalizes from a single example, per @RoundtableSpace. @nurijanian offers the hand-written alternative: a /mece-tree skill that checks whether a list is mutually exclusive and collectively exhaustive, useful for PM issue trees. And @backnotprop endorsed @plannotator's Inbox, a free, local, open-source surface for decisions, images, and video from background agents, arguing decisions are the new bottleneck; he calls it the best remote mobile UX he's used.

At the organizational level, @vasuman argues most "AI transformation" firms just sell skill files, Claude jobs, and prompting workshops, while real transformation means redesigning processes around background agents with human-in-the-loop review, auditability, and evals. He says Varick is hiring in SF for what he calls a $30T market.

Stale CLAUDE.md beats real code, and one sentence fights back

The most actionable research of the day, summarized in Japanese by @ai_database: researchers planted conflicts between config files and code across 40 projects. Claude Code followed the stale documentation in 79 of 80 runs, even while writing that it noticed the mismatch. Codex rewrote working code to match old docs in 20 of 80 runs, breaking tests in 17 of those. Their explanation: models are trained to prioritize instructions, and config files arrive as instructions while code arrives as observations.

The recommended fix is one sentence in CLAUDE.md or AGENTS.md: "When a repository's documentation and its existing code disagree, follow the conventions used in the existing code, and in your final reply point out each inconsistency you found between the documentation and the code."

Related config-file wisdom from @vikingmute, quoting @davekiss on OpenCode's AGENTS.md: before adding complexity for a speculative race or security edge case, explain the concrete failure mode, likelihood, and cost, and get buy-in. He now uses OpenCode's style guide as the default prompt for most of his agents. Meanwhile @amQnese offers an emphatic, link-only plea for coding-agent users to adopt an unnamed practice "at least once" in every app they build, regret avoidance without specifics.

Jev-style models are bottlenecked on training data

@vanstriendaniel predicts that whoever releases a high-quality dataset or recipe for training Jev-style models would land "among the top 10 open contributions in 2026," with something of his own to share next week. His evidence, drawn from the "Releasing the Inner Jev" paper: a letter-readout approach nearly matches Jev zero-shot (Gemma 4 31B scored 79.6 on OpenDecisions versus Jev's 80.1); LoRA-only training hit 84.9 versus 84.4 for the full recipe and 78.9 for head-only; short runs of 400 to 800 steps won; and 4B models gained nothing without extra tricks. His survey of roughly 100 Hub datasets found most are recast classification/NLI data or LLM-labeled, eval sets were tiny until OpenDecisions (2,352 questions), and relabeling flipped 52% of contested items. He reproduced the paper's 4B zero-shot result for about $0.20.

On the applied side, @dani_avila7's weekend reading list leads with an article on using Jev to improve skill selection in Claude Code.

Agents join the defense: Tailscale flaws, Apple's root CA, and the software factory

The security posts intersect with AI in one recurring way: agents as first responders. @IntCyberDigest reports Windisch's 11 unpatched Tailscale flaws include SSH race conditions that let one compromised device gain root on other servers and add a rogue node. @somewheresy is migrating his personal setup off Tailscale, and @alexhillman shared an agent prompt that walks you through locking down your tailnet ACL.

On Apple, @ryaneshea confirmed on macOS 26.7.1 and 27.0.1 that /etc/ssl/cert.pem still trusts chains forgeable with compromised eMudhra keys, a real risk for curl-pipe-to-shell installs; removing the roots works, but OS updates restore them. His suggested mitigation is literally a prompt for ChatGPT/Codex or Claude Code to verify and patch the bundle, and @jaredctate is asking why a $5T company hasn't fixed it.

The broadest argument comes from @alanwells via @angjiang: after seven days fighting an active attack, he says software factories are now "required equipment," describing a pre-provisioned cloud coding server with read-only log access, shared sessions for multiplayer incident response, and a six-week pause on product work to enable shipping PRs without human review. His team is three people, built on an OpenCode server. He urges filing KYC for Anthropic's and OpenAI's cyber programs before you need them. Also on hygiene, @my_knn_totoro reshared @karpathy's new "Digital Hygiene" post on no-brainer protective decisions.

A voice bet, a departure, and the weekend pile

@jack's Dessa post (from October 23) argues voice is "the most natural way to interact with intelligence"; the authenticity claim is his, with no independent benchmarks in the post. @amorriscode announced his last day at Anthropic after building Claude Code on desktop, promising news on what's next. @infinterenders boosts @brandon_galang's career advice: stop self-rejecting as "not technical enough" and ask people doing the job what it actually requires. For reading: @Saboo_Shubham_ recommends a 45-minute @leerob masterclass on always-on agents, @101xanshu opens a system design series, @kymatstudio reminds reviewers to check the longest button label before approving a component, and @timClicks flags an unexplained "proof of discovery" hash posted against Kubernetes by @GrahamHelton3.

Practical Takeaway

If you run Claude Code or Codex in any repo, add the researchers' documentation-conflict sentence to your CLAUDE.md or AGENTS.md today, then audit those files for drift, since the evidence says your agent will obey them over your code. If you operate multiple harnesses, HarnessRouter and UHP are worth reading even if you never touch Grok. And if macOS or Tailscale applies to you, apply the published mitigations now rather than mid-incident.

Sources

J
jack @jack ·
We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence. https://t.co/r0qFLdvd6c
A
arhen @amQnese ·
PLEASE. Siapapun kalian yang sudah pakai coding agent jadi bagian tak terpisahkan dair kehidupan, pakai lah ini. SEKALI AJA di app apapun yg kalian buat. Daripada menyesal di masa depan. https://t.co/6NF8A8zzjS
H
hayato @hayatomaruu ·
Anthropic engineer: “Anthropic hired me at $350K–$950K because I know how to build harnesses. 99% of people still hand-roll multi-agent systems and wonder why they break. I build systems that run loops + sandbox tools + turn yesterday’s logs into new skills — same model, 10x more reliable agents that keep improving overnight.” Opus 5.5 → Agents → Harness → Loops → Graphs In a short workshop, he shows exactly how to build one from scratch. step 1 → harness handles loops, context, and sandboxing step 2 → brain and hands separated — 60% faster to first token step 3 → runs server-side, resumes from the log after crashes step 4 → failure is cheap — retry and replay instead of starting over step 5 → yesterday’s logs become new memory — the system wakes up smarter Anthropic calls this “dreaming”. Most people spend weeks building this by hand. You don’t have to. Bookmark and watch it. Then read the full Opus 5.5 harness engineering guide below ↓
B beamnxw @beamnxw

Make Opus 5.5 Finish Long Tasks: A Reusable Harness Engineering Setup

0
0xMarioNawfal - The Roundtable Show @RoundtableSpace ·
Microsoft open-sourced an app that watches you do a task once, then teaches your AI agent to do it for you. 4.3K stars. How Skill Recorder works: 1. Hit record (⌘⇧R or Ctrl+Shift+R) and do the task normally. You can talk through it out loud too. 2. It captures your clicks, app switches, pages and terminal commands, all on your machine. 3. GitHub Copilot turns the recording into a list of steps you can edit. 4. One click turns it into a skill your agent can run, or an automation on a schedule. It generalizes from one example, so recording yourself submitting one form teaches the agent to submit all of them. Runs on Mac, Windows 11 and Ubuntu. Needs a GitHub account with Copilot access. Install: copy the one-line command from the latest release: https://t.co/NmDteJ1WgB
A
Avi Chawla @_avichawla ·
Insane Grok @bot upgrade! Finally, you can connect Grok Bot to different agent harnesses and run tasks through their native runtimes. Devs just open-sourced HarnessRouter, the infrastructure layer that lets Grok Bot run tasks across: - Codex - Hermes - Claude Code - DeepSeek Harness - System One, powered by Jev - Pi, and 20 more GitHub repo: https://t.co/4y6zJShd2W (don’t forget to star it ⭐) With HarnessRouter, each external specialist gets to use its own harness, model, tools, skills, and agent loop while Grok coordinates the project from one conversation. I tested this by asking Grok Bot to turn a support-ticket CSV into an interactive dashboard. Grok Bot assigned the analysis to DeepSeek Harness with Claude Sonnet 5.5, passed the output to Codex with GPT-5.5, and checked the finished dashboard in its browser. This differs from simply creating more Bots. These workers are not additional Grok Bots, and Grok Bot itself does not switch harnesses. Instead, it remains the coordinator while the delegated tasks run inside separate HarnessRouter sessions with the harness of your choice. So for another task, you can use Claude Code, Hermes, Pi, etc., without requiring a separate integration for task execution, sessions, streaming, and generated files. HarnessRouter standardizes all those operations through the Unified Harness Protocol (UHP). UHP handles sessions, streaming, files, cancellation, and failures through the same interface, regardless of which harness runs the task. I shared the GitHub repo above. And if you want to dive deeper into Grok Bot, I wrote about it in full in a masterclass, including the shared computer, memory, skills, routines, and the macOS setup. Read it below.
_ _avichawla @_avichawla

Grok Bot Masterclass

A
Anthony Morris ツ @amorriscode ·
Yesterday was my last day at Anthropic. I created Claude Code on desktop because I believe everyone can be a builder. Every feature request and bug report we got from this community was a gift that impacted millions of lives. I’m eternally grateful for those gifts, the product wouldn’t be what it is without you. Anthropic is doing some of the most important work of our time. Contributing to that mission will forever be a career highlight for me. I never imagined I’d get to contribute to something so special in my lifetime. Walking away from my best friends, a product I poured my heart into, and the mission was the hardest thing I’ve done in my career. But now I have some free time for GTA VI. I'm already itching to get back to addressing all your feedback. I'll share what's next soon!
S
Somewhere @somewheresy ·
Moving off of tailscale. Only affects my personals but whew
V
Viking @vikingmute ·
OpenCode 的 AGENTS.md 值得拥有,专治 Codex 的过度设计,茫茫多的防御性代码,一坨坨几乎根本不会发生的极端边界。 不仅仅是这一点,整个文件的内容都值得一看 ,style guide 一节写的很好,我已经用于我大部分 agent 的默认提示词了 https://t.co/d6gUuw7AYG “Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness.” 让 AI 翻译一下: “在为推测性的、或者几乎不可能发生的竞态条件/安全边界情况增加复杂度之前,先向用户讲清楚:具体会出什么问题、发生的概率有多大、以及增加这些复杂度的成本,并得到用户的明确同意。不要为了理论上的健壮性,就悄悄把范围扩大。” 非常贴切
D davekiss @davekiss

This is such a banger rule in @opencode's AGENTS.md "Before adding complexity for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode, likelihood, and complexity cost to the user and get their buy-in. Do not silently expand scope for theoretical robustness."

A
Anshu @101xanshu ·
System Design 01 - Before You Draw a Single Box in System Design
T
Tony Simons @tonysimons ·
FOR EVERYONE IN MY DMs… So many folks have been asking, so here it is: The complete stack of Hermes Agent profiles I run across 4 machines. Save this one. Steal these profile ideas and tweak them to fit YOUR stacks. Thank me later! 🫡
T tonysimons @tonysimons

Here is the Hermes Agent Stack I'm Running RIGHT NOW

S
Shubham Saboo @Saboo_Shubham_ ·
How always-on AI Agents work is a 45 mins masterclass from @leerob. Must watch! https://t.co/ktM9bbq2L9
D
Daniel San @dani_avila7 ·
3 articles for your weekend reading: 1- Jev Skill Suggestion In this article, I share how I’ve been experimenting with Jev to improve skill selection in Claude Code https://t.co/YD5rjfHqwL
R
render @infinterenders ·
if you’re deep in the trenches of applied AI, this is the guy you need to be following
B brandon_galang @brandon_galang

you want to work in applied AI, but you keep telling yourself you’re not technical enough. you only “vibe code.” you need another few months. says who? have you talked to a team doing the work you want to do? i used to be that guy. thousands of github commits, nights up until 1am, another project to prove i was ready. some of that skill development mattered. but i was also rejecting myself before finding out what the role actually required. talk to someone doing it. ask what their day-to-day work involves, how much technical depth they need, and what their interviews test. the bar might be higher than you think. it might be in a completely different place. find out what you actually need to get better at before spending another month trying to become “technical enough.”

G
George from 🕹prodmgmt.world @nurijanian ·
give your agent this skill asap, if you're a PM, issue trees are a key technique for breaking down a problem use this when you already have a list of items (options, causes, segments, etc.) and you need to check if it is MECE MECE means mutually exclusive (no two items overlap) and collectively exhaustive (no real case is missing). it does some magic behind the scenes to achieve this. in addition, items in the same list must belong to the same ontology and represent correct parent-child relationships with the previous and next nodes.
N nurijanian @nurijanian

/mece-tree: a simple AI skill for a messy world

T
Teknium 🪽 @Teknium ·
Looking to get deeper into Hermes Agent? Check this thread
W witcheer @witcheer

here is a map of the Hermes Agent videos worth watching, sorted by what you want to do with it: (1) start here (2) running a business on it (3) real setups and use cases (4) skills (5) teams of agents: Kanban, Bot Mode, cron (6) by job: marketing, sales, support, research see thread below

V
vas @vasuman ·
No one is doing AI Transformation. The gap between what is possible and what is actually happening has never been wider. When we started Varick we weren't sure this industry would exist in 2 years. Now we're sure it will be here for decades. There are many "Applied AI" and "AI Transformation" firms, but most of them sell: - skill files - some Claude jobs - prompting workshops That isn't AI Transformation. AI Transformation is: - redesigning an organization based on AI's capabilities - agent-ifying each workflow in a department, primarily running in the background - HITL so agents improve themselves, with auditability, governance and evals built in I don't blame the first group. It's quick cash, easier work, and doesn't need the best people in the industry. But it also barely scratches the surface, and most companies are none the wiser. Varick is doing the second. We're not here to "apply AI" for a week and take short-term wins. We're here to drive billions in value. That is only possible by going deep, redesigning processes, and building custom agents on a company's own systems of record and workflows. We're hiring in SF. You should: - have 1+ years in startups / PE / banking / consulting (long hours, real stakeholder management) - build with AI every day (GitHub is green, with several tasteful projects, not just wrappers) - want to be in this market (a $30T market that we're taking the lead in) DMs are open.
J
Jared Tate ©️ @jaredctate ·
How does a $5 trillion company @Apple not have a fix for the compromised Indian root CA yet? Every macOS user is still a sitting duck. Absolutely mind blowing.
R ryaneshea @ryaneshea

PSA for macOS users: 1. Apple’s supplied SSL certificate bundle at /etc/ssl/cert.pem still trusts certificates attackers can forge using compromised eMudhra keys. 2. I confirmed this on macOS 26.7.1 and 27.0.1 using Apple’s built-in curl and Git. 3. An attacker intercepting connections from tools using this bundle could replace downloads with malware. With commands like curl … | sh, that malware would execute immediately. 4. Removing the roots that authorize those certificates blocks the tested chains. Both OS updates restored roots I’d previously removed, requiring me to remove them again. 4. I’ve submitted the findings to Apple’s security team and am awaiting their response. In the meantime, you can apply a temporary fix as outlined below. 5. To check and fix your machine, give an agent like ChatGPT/Codex or Claude Code the following prompt: “Independently verify whether Apple’s /etc/ssl/cert.pem still permits trust in certificates signed with compromised eMudhra keys. If so, back it up and remove the roots enabling that trust, identified by verified SPKI fingerprints. Confirm Apple’s built-in curl and Git reject those chains while ordinary HTTPS still works. Preserve rollback and never disable TLS verification.” Note that removing these roots can also block legitimate sites. This is a temporary mitigation, not a substitute for a fix from Apple. I’ll share the script I used in a follow-up post if you’d prefer to use that directly.

M
Michael Ramos @backnotprop ·
This has become, and feels like, a local version of Claude Tag, and it has also become the best remote mobile UX I’ve used. bc you are no longer operating in session threads and instead where decisions & images/html/video get surfaced right to you in an organized way.
P plannotator @plannotator

Introducing Plannotator Inbox. Making decisions is the new bottleneck, while your agents work on many things across many threads over longer periods of time. Inbox is how you stay in-the-loop, organized, alongside the agent harnesses you already prefer to use. Try today, free, local, open source.

H
Hamel Husain @HamelHusain ·
2 of 3 agents for this Company harness are marketing related. It feels right
H hwchase17 @hwchase17

The five components of building a "company harness"

📙
📙 Alex Hillman @alexhillman ·
Hey fellow Tailscale loving friends, with all the security shenanigans going on here’s a prompt I recommend for making sure the weakest link on your talilnet can’t wreck everything else over ssh or otherwise. Give your agent this and let it walk you thru locking down your ACL: https://t.co/xFy0M1Adtx
A
AIDB @ai_database ·
研究者らによると、CLAUDE.mdやAGENTS.mdにあらかじめ次の一文を足すことが推奨されます。 「ドキュメントと既存のコードが食い違うときはコードの書き方に従い、見つけた食い違いを最後の返答ですべて挙げること」 なぜか。 Claude CodeやCodexは、CLAUDE.mdやAGENTS.mdの記述が今のコードと食い違っていると、9割がた古い記述のほうに従うそうです。"指示が古い"と気づいていても抗えない仕組みです。 この発見に至るまでに、ジョージア工科大などの研究者らは、設定ファイルとコードがわざと食い違う40のプロジェクトでClaude CodeとCodexに機能を足させて調べました。 Claude Codeは80回中79回、食い違いに気づいたと"自分で書きながら"、古い情報に流されました。 Codexのほうは、80回中20回で、動いていた既存のコードを古い記述に合わせて書き換え、うち17回でテストが通らなくなりました。 研究者らの見立てとしては、AIは、指示を優先するよう訓練されています。CLAUDE.mdやAGENTS.mdは「指示」としてAIに渡り、コードはファイルを読んだ「結果」として渡るのが原因だと言います。その仕組みが、古いものに従わせるのだとか。 そして設定ファイルは一度書いたきりになりやすい傾向があります。更新不精の人は上の一文をあらかじめ追加しておくのも一つです。 指示の原文→"When a repository's documentation and its existing code disagree, follow the conventions used in the existing code, and in your final reply point out each inconsistency you found between the documentation and the code."
A
Angela Jiang @angjiang ·
A very clear eyed picture of what’s coming. You will need a software factory for defense, not just internal productivity.
A alanwells @alanwells

Just spent the last 7 days on the front lines of the vulnpocalypse fending off an active cyber attack on one of our products. Some thoughts: - Software factories are going to be required equipment for everyone. There’s just no way to keep up with the pace of highly automated adversaries without one. - We were lucky because we had spent the last few months getting our software factory running well. It is nice to have for product development but proved to be absolutely essential this week. - At the minimum you need to have a coding server in the cloud pre-provisioned with read only access to all your logs and all the services you depend on for observability. With that in place you are a prompt away from reasonably good situational awareness, ongoing monitoring with loops, and ability for coding agents to validate fixes against your actual, observed attack patterns. - Shared coding server also unlocks multiplayer incident response where everyone can see every session and jump in as needed. Sharing session links that can be forked is much higher bandwidth than re-explaining things. - You don’t need to be Ramp/Stripe/Uber scale to have a working software factory in the cloud. We are a team of 3. Ours is built with an @opencode server (thanks @thdxr!), a forked version of compound engineering optimized for automated workflows (thanks @kieranklaassen!) and a multiplayer agent in slack that handles crons (we have our own harness but pick your favorite here). - To get our software factory up and running well, we had to stop all new product development for ~6 weeks and just focus on getting the validation systems working that enabled us to reliably shipping PR without human code review. This felt very painful in the moment but was totally worth the eventual speedup in velocity. - KYC for @AnthropicAI & @OpenAI cyber programs is now easy and quick (at least for defensive access), but a day felt like a lifetime this week. Submit your app now before you really need it. - Getting Mythos access unlocked a deeper level of analysis that helped us shift from reactive mitigation to proactive patching. Wish we had done this a few weeks earlier. Get started now!

O
Onur Solmaz @onusoz ·
RT @nicopreme: pi-subagents just hit 4K stars on GitHub! 😇 Just shipped: it's now durable, so Pi no longer loses work when the session cra…
D
Data Scientologist @my_knn_totoro ·
RT @karpathy: I wrote a quick new post on "Digital Hygiene". Basically there are some no-brainer decisions you can make in your life to dr…
T
Tim McNamara @timClicks ·
A proof of discovery hash appears against Kubernetes. How's your weekend going?
G GrahamHelton3 @GrahamHelton3

Good morning 045fa83b7c7a42dba1acb2c19794c1bf1091c448eb9c483503e8f8e110fcec42 https://t.co/B6vooLkfWC

K
KYMAT Studio @kymatstudio ·
A long button label can wrap onto two lines on a small screen. Check the real wording, including the longest version, before approving the component.
I
International Cyber Digest @IntCyberDigest ·
‼️ Researcher Erica Windisch skipped coordinated vulnerability disclosure and published 11 unpatched Tailscale flaws. In her lab, Tailscale SSH race conditions let one hacked device get root on other servers and add a rogue node. https://t.co/pK4NPIOd9g
D
Daniel van Strien @vanstriendaniel ·
Having done more work on this (aiming to have something to share next week), I am pretty confident that someone who releases a very high-quality dataset/recipe for generating training data for Jev-style models would be among the top 10 open contributions in 2026.
V vanstriendaniel @vanstriendaniel

After a month or so of open Jev-style decision models, here's what I found on the question of data. Most of the findings come from one paper, "Releasing the Inner Jev" (@JonathanGeuter et al.). You may need less training than you think. Show the options as letters, run one forward pass, and read the scores for "A", "B", "C"… An instruct model done this way already gets close to Jev: Gemma 4 31B scores 79.6 on OpenDecisions vs Jev's 80.1. LoRA vs the head (31B ablations): → Train only the head, backbone frozen: 78.9 (slightly below zero-shot) → Train only LoRA, head frozen at the letter rows: 84.9 → Full recipe, LoRA + head: 84.4 So the gain comes from adapting the representations. The letter readout is already a good head. Also: → Short runs win (400–800 steps); longer runs mostly lose calibration → Smaller models gain less: no gain at 4B without extra tricks → A validation split from your own training data predicted the leaderboard badly I also checked ~100 decision-related datasets on the Hub: → Most training data is classic classification/NLI datasets recast as choice/yes-no/score questions → Much of the rest is labelled by an LLM teacher → Eval sets were tiny (100–350 questions) until OpenDecisions (2,352) → Labels need re-checking: when Contested Fast Decisions relabelled the items 8 models disagreed on, 52% changed I'm working on this a bit more to understand what data actually helps when you train one for your own task. So far I've reproduced the paper's zero-shot 4B result on @huggingface Jobs (64.7 vs 63.6, ~$0.20). Paper: https://t.co/y0e5RTNGAe Datasets: https://t.co/fqfNtp4oB3