RLMs push long-context accuracy from 2.6% to 45.4%

Kevin Madura argues that stuffing context windows or shuttling JSON through tool calls is the wrong model for handling long or dense inputs. Recursive language models instead treat context as a variable living inside a Python REPL, letting the model write code over it and delegate sub-problems to itself, which he says changes what's even worth calling "context engineering."

• What an RLM is — Madura defines an RLM (recursive language model) by two properties: it treats context as a symbolic object it can manipulate in a REPL rather than tokens to attend to, and it can delegate sub-tasks to another LM instance, including itself, with its own parameters.
• Origin — He traces the idea to a tweet from Omar, an adviser on RLMs, who used DSPy-style techniques to summarize arbitrary-length documents without deliberately managing the context window.
• Benchmarks — On the ulong and BrowseComp benchmarks, the RLM line beats other approaches, and it even beats GPT-5 with BM25 tool calling on both cost and accuracy.
• vs RAG and agents — Madura contrasts RLMs with RAG (which stuffs the context window until quality degrades) and agents/tool calls (which pass JSON strings back and forth, decoupling logic, execution and results), saying RLMs keep all three tightly coupled in one environment.
• Anthropic workflows — He notes Anthropic's recently released workflows keep intermediate results as script variables, and that a speaker at a conference cited the RLM paper as a key driver of that design.
• Long CoT benchmark — Testing from someone named Raymond on a long chain-of-thought benchmark showed RLM accuracy jumping from 2.6% to 45.4%, with the largest gains on tasks reducible to code, like logic puzzles, chess, and chemistry.
• Toy example — Madura demonstrates summing 12 numbers scattered across 30,000 tokens: base models often fail at this while an RLM can write regex or code to solve it directly.
• Cohort retention demo — He walks through a cohort retention analysis with three data frames fed to an RLM, which reasons and writes code inside its own REPL as if in a Jupyter notebook, deciding itself when to call submit with a typed final answer.
• Open source tools — He lists implementations including Predict RLM (focused on spreadsheets and PDFs), Axe, Fast RLM, and DSPy, plus a tool from Sam Hogan of inference.net that uses an RLM to mine production traces for offloading decisions to models like GLM 5.2.
• Invoice case study — Trampoline AI's Predict RLM consolidates directories of invoices, including 200-page documents, without chunking or embedding, and uses DSPy to enforce schemas between the main LM and sub-LM calls.
• Trace and log cases — An AWS engineer used an RLM to surface patterns in raw log data, and a project called Halo uses an RLM to optimize an agent's harness itself based on its own traces rather than tuning a fixed workflow.
• Security report demo — Madura ran a minimal RLM script against an intentionally vulnerable OWASP web app to generate a security report across roughly 500,000 lines of code with no chunking setup.
• Closing claim — He predicts things will change quickly once models are post-trained to be natively RLM-aware rather than just prompted into the pattern.

In their words

so in this random example, summing 12 numbers that are buried across 30,000 tokens, the LLM trying to figure all that out by itself and give you the answer isn't always going to work as well as something that you can write reg x4 or something similar.9:47

it's it's a meaningfully different way of of the LLM interacting with the uh the actual content itself.6:37

So in terms of how it's meaningfully different, rag of course you kind of just stuff the context window.5:58

You can see here the intermediate results for workflows live in script variables i.e. a variable in the context.7:15

Disclosure · Madura works at AlixPartners and references tools and collaborators (Omar, Predict RLM, DSPy ecosystem) he is personally aligned with, though he is not directly selling a product in this talk.

One thing to add — One thing to add — Madura repeatedly flags his own examples as "somewhat unfair" to base models and admits the coding-agent comparison likely has "unfair math," which is an unusually candid caveat for a talk pitching a new architecture. The 2.6% to 45.4% figure comes from a third party's (Raymond's) benchmark testing, not Madura's own experiments, so it's worth checking that source before citing it as definitive.</note> </invoke>

One thing to try tonight
Try feeding a data frame or a long log file directly into a Python REPL alongside an LLM call (e.g., via a DSPy or Predict RLM script) instead of chunking it, and compare how it handles a multi-step aggregation task versus a standard prompt-stuffing approach.