Intelligence salad with a side of generalization
Hi friends,
(welcome to my email dispatch! You can sign up for these or read the archive at buttondown.email/thesephist 💌)
What I read
I spent the first half of this week thinking about the relationship between abstraction, generalization, learning, and intelligence. I think a key part of human intelligence is our ability to discover new concepts (abstract), and apply them (generalize) to novel situations to find surprising solutions. I haven’t yet seen a convincing example of a foundation model do this, but I think there are lots of examples of other technological tools enabling us to do this faster as humans.
To Understand Language is to Understand Generalization (2021) by Eric Jang is one of my favorite pieces about the relationship between language, generalization, and intelligennce.
Using Artificial Intelligence to Augment Human Intelligence by Shan (who now leads Anthropic’s interpretability team) and Michael Nielson probably ranks in the top 5 most influential pieces of research literature for my current work. It demonstrates how the structure of machine learning models themselves, rather than their outputs, could be an element of how we augment our intelligence.
Francois Chollet’s On the measure of intelligence is my favorite academic treatment of this subject, with a really rigorous perspective on what concepts really are, and why they’re useful in practice (why intelligence incentivizes learning them). I like Chollet’s framing of generalization as having a cost over simple memorization, which must justify itself by helping prepare for unexpected future scenarios.
There were many projects related to model interpretability I got excited about this week. Unfortunately most of them were shared on Twitter, but I’ve compiled them into this Notion site for those who don’t want to log in.
@SiliconJungleshared a bunch of experiments about ways to display interesting steering vectors and features for editing images — TweetGytis went further and previewed some very exciting work training sparse autoencoders on an image encoder/decoder model, and using it for understanding and editing images. Having played a bit with these demos, they are truly mind-bending, and I can’t wait for more people to try it. It’s really hard to imagine going back to pure-text prompting — Tweet
I also really enjoyed Alex’s (
@ajeon66) side project Lapin that lets you explore a topic by having an LLM create Twitter-style “threads”. This was partly inspired by Max’s “Delve” project, which implements a simpler interface for a similar experience — TweetI’m eagerly following Gray’s spectrographic music player project. If you’ve watched any of my recent talks, you know how excited I am about tools that let us understand and visualize media in unconventional ways, and this is a huge inspiration — Tweet
Finally, I enjoyed reading PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions, which explores a UI for an image generation model that lets the user “mix” different prompts together as if mixing colors on a color palette. Really creative.
What I’m working on
I’m currently focused on two goals:
Publishing research reports. I’ve been sitting on drafts of research reports for three old projects that I hope to polish and publish this week, including a write-up for my recent text embedding interpretability experiments. Hopefully up by next email!
Scaling sparse autoencoders up and out, to a production-scale open LLM (Llama 3 8B), a state-of-the-art text embedding model (Nomic Embed), and an image model (CLIP, or maybe Stable Diffusion). Production-scale feature libraries are crucial for building more realistic prototypes and exploring real use cases of latent visualization and steering.
In addition:
I went on livestream with Steve Krouse of Val Town Thursday and made a mini WebSim clone, which you can see on Val Town here. You can also read about the project on my Twitter. It’s pretty neat to see a webpage get prompted, and then generated, in front of your eyes as it streams into your browser. Here’s the livestream on Twitter.
I shared a list of applied research problems in ML that I think are interesting and understudied, spanning style transfer, tradeoffs between different training techniques, design-focused multimodal models, long-running chat UI, communicating and organizing features, and direct manipulation in latent space.
A founder friend of mine has started a company to build creative tools for artists who use AI, and are hiring. Most “AI art” tools are meant for you or me, rather than folks who practice art full-time, and hide a lot of complexity that could expose much more expressive power. They’re building for Figma or Photoshop what Stable Diffusion/DALL-E is to Microsoft Paint. If their founding engineer role in NYC sounds interesting to you, reach out!
A reminder: you can reply to these emails :-) They go straight to my inbox.
Wishing you a happy and safe week ahead,
— Linus