Lenny Stress-Tests Muse Wonder, LangChain Tackles Context Rot

Lenny actually ran Muse Wonder headlessly through Claude Code against 20 real tasks instead of trusting Meta's benchmark claims, and the result (good at agentic work, bad at design) is exactly the kind of independent verification PMs should demand before believing any "frontier" launch; meanwhile LangChain shipped a deep technical playbook for the unsexy but critical problem of keeping long-running agents from drowning in their own context. ### Lenny Rachitsky - Stress-Testing Meta's Muse Wonder Claims With Real Tasks, Not Benchmarks **Source:** https://www.lennysnewsletter.com/p/metas-new-muse-wonder-model-good **Credibility:** High (independent hands-on testing against ~20 tasks, not a summary of Meta's own claims) **What happened:** Meta launched Muse Wonder claiming frontier-level agentic performance. Instead of taking that at face value, Lenny ran it headlessly through Claude Code against nearly 20 tasks to see what actually held up. His conclusion: Muse Wonder is genuinely strong for agentic work (the kind of multi-step, tool-using tasks Claude Code is built around) but notably weak for design tasks, despite Meta's framing of Muse as a consumer-friendly, design-forward agent. **Key technical details:** - Testing method: headless execution through Claude Code, not a chat UI, which isolates agentic/tool-use performance from conversational polish - ~20 distinct tasks used as the evaluation set, rather than relying on Meta's published benchmarks - Performance split: strong on agentic task completion, weak on design-oriented output - This directly follows Meta's Muse Wonder announcement and the "How Meta designed Muse" interview with Design Director Alex Cornell, which emphasized Muse's design sensibility as a differentiator **Why it matters for PMs:** This is a template for how PMs should respond to any vendor's "frontier model" claim: don't trust the benchmark slide, run your own tasks that map to your actual use case. The gap Lenny found (good agent, bad designer) also directly contradicts the positioning Meta itself has been pushing through Alex Cornell's interviews about Muse's design-first identity. If you're evaluating Muse, Instinct, or any agent for a build-vs-buy decision, the capability profile you need isn't "frontier" in the abstract, it's frontier at the specific task mix your product requires. **Critical questions:** - Were Lenny's 20 tasks representative of real user workflows, or skewed toward what's easy to test headlessly? - Does "weak at design" mean weak at generating design artifacts, or weak at design judgment/taste, a much harder thing to benchmark? - How much of the agentic strength is Claude Code's harness doing the work versus Muse Wonder itself? - Would Meta's own benchmark suite explain this gap, or is it a case of benchmark-vs-reality divergence? **Action you could take today:** If you're evaluating any new "frontier" model claim for a roadmap decision, write down 5-10 tasks that map to your actual product's agent use cases and run them yourself before trusting a vendor benchmark.