Revival: research, complexity, rocks and water
Hi friends, happy June!
(This is my semi-regular newsletter about what I’m thinking about, reading, and working on. You can sign up for these or read the archive at buttondown.email/thesephist 💌)
It’s been a while since I’ve fired up this email, so let me re-introduce myself: My name is Linus. You probably signed up for this email list on my blog at https://thesephist.com/ or my Twitter. My long-term goals are the same as they’ve been for the last few years, at the top of my blog:
My research investigates the future of knowledge representation and creative work aided by machine understanding of language. I prototype software interfaces that help us become clearer thinkers and more prolific dreamers.
I’m also interested in what comes after today’s language and writing systems, and what learning, creating, and collaborating may look like in the long future of humanity. The aim of my research is to build interfaces and knowledge tools that expand the domain of thoughts we can think and qualia we can feel.
I’ve been very happy with how consistently every one of these topics have held my interest, so I hope to continue investigating them for many years to come. In particular, I currently believe large-scale prediction machines like AI efficiently learn useful patterns in the world that we can use the understand and control the world better, if only we could decipher their mechanics. More on what I’m working on below.
What I read
An essay from science fiction author Neal Stephenson called In the beginning…was the command line that was one of the best things I’ve ever read about technology and culture. You should read it, but since it’s long, I’ve distilled my favorite ideas into this post — https://thesephist.com/posts/stephenson-command-line/
I read a lot about simulations and irreducible complexity, and why they are important to intelligence in addition to learning to predict. Two pieces on this:
First, my friend Sarv simulated evolution of trees using cellular automata. I think evolutionary learning algorithms are under-studied in modern AI, and deserve a lot more creative attention. https://sarvasvkulpati.com/treesim
Second, I thought Stephen Wolfram’s piece on Can AI Solve Science? Was well-written; I agree with many of the points it makes about the role of foundation models in science and general intelligence — https://writings.stephenwolfram.com/2024/03/can-ai-solve-science/
I read a lot about LLM mechanistic interpretability research and interfaces derived from it:
Matt Webb’s excellent post about the future of interacting with text, which captures so much of what I’ve been working toward at the interface layer for the last 3 years: https://interconnected.org/home/2024/05/31/camera
Some design sketches from me on ways to build latent space UIs Matt talks about above: https://thesephist.notion.site/A-synthesizer-for-thought-ac2395d64a444d44afe90236fabb513e
A trio of research reports about controlling language models using activation vectors
- https://www.alignmentforum.org/posts/jGuXSZgv6qfdhMCuJ/refusal-in-llms-is-mediated-by-a-single-direction
- https://www.alignmentforum.org/posts/ioPnHKFyy4Cw2Gr2x/mechanistically-eliciting-latent-behaviors-in-language-1
- https://www.alignmentforum.org/posts/v7f8ayBxLhmMFRzpa/steering-llama-2-with-contrastive-activation-additions
Hugging Face’s paper on FineWeb, their newest LLM training dataset, is both well-written and well-presented. It’s here: https://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1. In particular, three interesting bits from my reading:
Duplication in large text corpora like this has both a heavy “head” with a few documents repeated hundreds of thousands of times, and a noisy “tail” of documents that may be very unique, but only because they contain useless data. Deduplicating moderately helps remove the heavy head without over-weighting the tail.
Their approach of training a classifier to filter the dataset to highly useful tokens is interesting. It reminds me of Minipile, which did something similar by embedding clustering — https://arxiv.org/abs/2304.08442
Common Crawl’s web crawl quality varies for LLM training quite a bit year-to-year.
Visualizing speech with speech embeddings
What I’m working on
I wrote Like rocks, like water (https://thesephist.com/posts/rocks-water/). It’s about what happens when resources like energy and intelligence go from being generated by humans to being generated mechanically.
I wrapped up my final training runs and experiments on inventing alien languages with neural networks, which I previewed at an event in SF in the spring. I’m writing up my thoughts and will share them soon.
Next, I’m planning to scale my work on sparse autoencoders to higher-capacity text embedding models, image encoders, and maybe larger chat models like Llama 3 Instruct. I think this is critical to building proofs of concept for realistic use cases of visualizing and steering models in latent space. I also want to experiment with better ways of presenting very large (10,000s to millions) feature spaces in an interface.
A reminder: you can reply to these emails :-) They go straight to my inbox.
Wishing you a happy and safe week ahead,
— Linus