September 28, 2020

Mystery Knowledge

Mystery Knowledge

In the misty past of 2019, when I was still giving conference talks, I always meditated for 5-10 minutes before starting. Arrive early, set up mic, sit in a chair and study my breathing. This made a big difference. I'd be calmer, steadier, more focused during the talk.

I didn't figure it out on my own and I didn't notice it while watching another speaker. Someone told me about it, I decided to try it, and it made a difference. I suspect that most other speakers don't think of doing this and many of them would benefit greatly, too.

"Meditate for five minutes before you start." Seven words that give you an idea you might not come up with yourself, that can be immediately put into practice, that immediately makes you a better speaker.

A lot of people have told me that learning Decision Tables really helped them. Since most people don't write formal specifications, you're unlikely to see decision tables in the wild. But when you read a blog post, or someone tells you about them, within a few minutes you can use them. A few minutes to explain an idea you might not come up with yourself, that can be immediately put into practice, that immediately makes you a better designer.

With two examples we should DRY things up. The abstract concept here is knowledge or skills that

1. You are unlikely to discover on your own, neither through practice and reflection nor by observing others apply it.
2. Once somebody tells you about it, you can easily learn and apply it.
3. Once you can use it, it immediately gives you significant benefits, possibly to the point of raising your expertise level.

This might be a studied topic, but if it is I don't know even what field of knowledge it belongs to, much less what it's called. In the meantime I call it mystery knowledge. Thinking less "whodunit" mystery and more "Greco-Roman mystery cults" mystery.

Mystery knowledge interests me because it's a part of expertise you can't develop purely through experience. You need to study, or at least talk to others in the field. But what interests me far more is how easy it is once you know about it. Something like TLA+ is hard to discover on your own and also challenging to learn. Once you know it, it's unbelievably powerful, but you have to put a lot of time in. But with mystery knowledge you learn of it, you read about it, and that's it. Maybe a bit of practice, but not more than an hour or two. And suddenly you're appreciably better at whatever it is. Almost like knowing that one fact is part of being an expert, or that hearing the fact reconfigures your expertise like a prion in the brain.

This idea bothers me. If it's possible for mystery knowledge to boost your expertise like that then we can imagine-as-possible stronger forms. Like hearing one particular explanation is what makes you understand a topic or hearing a single fact changing your opinion on something. That seems less likely because it involves a change in your internal mental model, not just the actions you use. But that itself rebounds into mystery knowledge teaching you actions, meaning expertise is based at least partially on actions. Such that a beginner can imitate specific actions to "re-tier" at a higher expertise level.

Mystery knowledge also bothers me for a pragmatic reason. We certainly don't know all the mystery knowledge collectively, and I'm willing to bet that we don't even collectively know the majority of mystery knowledge. There is a lot out there that is very hard to discover, very easy to teach, and would make a big difference. But either nobody knows about it or so few people know that it's unlikely to ever be taught out. There are significant improvements we can make to how we program that will seem obvious in retrospect, but it will be a long time before we figure them out from first principles.

Or are there processes/mindsets/circumstances which make independently discovering mystery knowledge easier? I'm tempted to say "copious amounts of drugs" here. Lord knows I've had some important insights while high out of my mind. ¹

What else counts as mystery knowledge in software? git bisect maybe? Common keyboard shortcuts? Maybe some debugging strategies like divide-and-conquer. Interested in hearing what you consider mystery knowledge.

If you're reading this on the web, you can subscribe here. Updates are once a week. My main website is here.

My new book, Logic for Programmers, is now in early access! Get it here.