This week on seangoedecke.com
Here's what I wrote this week. If you'd rather get each post as it comes out, switch here.
LLMs are terrible at designing AI systems
LLMs don’t know how to design software that uses LLMs. They’re comfortable writing normal programs, but if you throw an AI-shaped problem at them, they will usually screw it up. Why is that?
Going down the symbolic rabbit hole
For example, this post describes an attempt to vibe-code a memory tool that would flag when one memory contradicted another1. This is a great instance of a task that was virtually impossible five years ago but is now relatively trivial due to LLMs. You can simply task a LLM with scanning for contradictions, either in a big batch or pairwise2.
However, when asked for the “state of the art” approach3, the AI …
Read the rest on seangoedecke.com →
Working with designers
I love working with designers, in part because (unlike managers and product managers) they’re culturally different from engineers. However, having more than one designer on a team is asking for trouble, and I have complicated negative thoughts about design systems.
Design is a strange role
Designers demonstrate their value immediately. While working on some feature, I’ll typically put together some placeholder UI that I’m reasonably happy with. I then show it to a designer, who strokes their chin and says “hmm, I have some ideas here”. They then go away and produce a mockup UI that is not just clearly better, but better in ways I could not have imagined. There aren’t any other roles at a tech …
Read the rest on seangoedecke.com →
A database is a store for global variables
Databases are for storing global variables. We usually call them “rows” or “records” instead, but they’re definitionally global variables: any part of your program can read or write them. Isn’t this kind of weird? It’s common programming advice that global variables are bad, and yet many elegant programs rely on databases. Either the common advice is wrong, or there’s something special about databases that makes them the exception to the rule.
What’s so bad about global variables? Here’s the two main1 reasons, in my view:
- They allow any part of your program to implicitly send data to any other part instead of passing it around via function signatures
- They couple different parts of your program: …
Read the rest on seangoedecke.com →
Audience capture
Audience capture is the part that scares me the most about writing on the internet. I have the soul of an engineer1: when I see a metric like engagement, I want to improve it. I like to make the number go up.
In fact, the whole environment of online writing is tailor-made to help you make engagement numbers go up. It’s trivial to find out how many people are reading my posts, and from what sources. Hundreds of link aggregators all over the internet have algorithms that will help my most engagement-friendly posts get views, and the comments are helpfully sorted by upvotes so I can read the “best”2 pieces of feedback straight away. Many platforms …
Read the rest on seangoedecke.com →
LLMs are not superhuman at reading and writing neural network weights
A recent paper argues that frontier LLMs can work with neural network weights directly. This seems superhuman, but it isn’t.
The paper shows that GPT-6 Astra can take a small neural network and interact with its weights directly, “well past what we’d expect from humans or conventional algorithms”. Specifically, Astra was able to identify what games a small network1 was trained to play just by looking at the weights, and even tweak the weights itself. The most impressive result is Astra significantly improving a 0.8B Qwen LLM’s ability on mathematics by natively editing its weights. The reason to be excited here is that “superhuman ability to improve language models” is precisely the kind of thing that could cause …