Entropy appears across scientific disciplines, each defining it differently while pointing to a similar underlying idea. These definitions often seem inconsistent because each field formalizes entropy based on its own assumptions and modeling choices. As a result, the shared structure beneath them remains obscured.

This Quanta Magazine article provides helpful background for this work. It traces how entropy has evolved across disciplines and highlights the ambiguity that arises when it is treated as both a physical property and a measure of uncertainty, depending on the framework.

In this work, I introduce a conceptual framework that unifies entropy and probability as phenomena. While the focus is on information theory and statistical mechanics, the framework is extensible to other forms of entropy. Rather than treating these interpretations as analogous, it reveals them as expressions of the same underlying structure grounded in the relationship between priors and possibilities.

Entropy and probability exist as phenomena before measurement. They represent relationships that exist in reality independent of our attempts to quantify them.

It's been 3 months since I started publishing this newsletter. I've managed to publish 3 articles.

What you may not know is that I started working on this newsletter more than one year ago. That is writing the articles, working on the ideas go much further back. Not only that, but it's the only thing I'm currently doing with my life. It has my full-time dedication outside of self-care. I've pushed away anything that isn't absolutely important for my life or doesn't directly contribute to the project.

When I started publishing in December, I thought I had about 70% of the articles written and that I could publish about once a week, maybe two. All that was left to do was polish the last 30%. I was estimating to have about 10 articles published by now.

The current strategy is unsustainable. I need to relax in my personal life and I need the project to move faster. There's too much to say and to build, and it needs to become personally and economically sustainable.

The Free Energy Principle (FEP) proposes a unifying theory of how living systems persist. Life exists in direct opposition to entropy. While the universe trends toward disorder, living things resist it. A plant keeps its leaves green, a bird maintains its body temperature, a cell preserves its boundary.

The FEP explains how this is possible: living systems must minimize the difference between their expectations and their sensory experiences. This difference represents uncertainty about the world. It's called "free energy" in analogy to "thermodynamic free energy" which measures potential for change. Too much uncertainty threatens survival, so living systems evolved two strategies to minimize it:

1. Passive inference: updating internal models to better predict reality
2. Active inference: changing the environment to match predictions

From perception to higher order cognition, the FEP describes how living beings make sense of reality to stay alive.

Understanding is prediction

If meaning is what we seek when we have gaps of understanding, and is personal and ineffable - what is understanding? And how do we understand each other at all?

Understanding means prediction. When we understand something - a smile, a mathematical equation, or a weather system - we narrow down what might happen. Understanding reduces chaos into patterns. It reduces infinite possibilities into likely outcomes.

We need understanding not just to communicate or solve problems. We need it to relate to anything. With understanding we can anticipate what comes next. Without it we can't know if fire burns or if a gesture means welcome or threat. We can't function. Everything would be unpredictable and chaotic.

The meaning of meaning

I found formal definitions of meaning quite confusing, presenting it as different types or things without a clear unifying essence. My interest is in a definition that reflects how we use it in everyday life.

We use the word meaning when facing gaps in understanding.

"What's the meaning of this word?" "What do these results mean?" "What is the meaning of life?"