How much progress have we made over the past year in terms of energy efficiency?

In this quick note, we perform a longitudinal analysis that focuses on software, the component that changes most rapidly, and how optimizations improved energy consumption for the same model and hardware. We also dive one layer deeper to see how latent metrics materialize to energy differences.

Check out the post here: LLM Inference Energy: A Longitudinal Analysis.