Hello from the past, hope you are having a good day.

How do you index data about fields/column that are only decided during runtime and can be completely dynamic?

Well, I spent quite a bit of time researching for this problem at work and trying to come up with a solution to move away from an expensive, fully hydrated memory cache, of some data that we use to build dynamic, runtime, config driven dashboard views.

I was looking for alternatives after we tried a few things and realised that with Postgres, you just can’t. You cannot build indexes for this kind of a column; it goes against the architecture. Also, that GIN indexes don’t work on range queries, which was another thing we wanted.

Document dbs have the same problem, for the same architectural design reasons. There is the wildcard Indexes concept in MangoDB, but that may only help with filtering, not sorting. So after much search and conversation with Claude, I finally arrived at search engines; specifically ElasticSearch, and more specifically Amazon’s OpenSearch, which essentially excel at this specific usecase.

Yes, OpenSearch means going deeper into the web of AWS managed infra, but for our scale and usage, that seems as the correct option. This topic can have a blog to itself, which I would love to write soon.

But for now, moving on to a more interesting, and way cooler than the above topic - my visit to the Deutsches Technikmuseum yesterday. Among many things, I saw the first German computer; different printing technologies, for instance the Linotype type-casting machine; horse driven trams; and so. many. different. planes.

I haven’t been to disneyland, but that last section gave me an impression of how it would feel.

Therefore, today’s links are going to be related to these. It is all, so cool!

Interesting reads

1. An introduction to the fascinating Linotype machine
   This apparently, was such a great contribution, as to increase literacy by folds, not just a few percentages. Also, an ingeniously complex machine. Here is an animated video of how it worked.

2. https://en.wikipedia.org/wiki/Dornier_Do_X
   The largest, heaviest of its time, beast of an aircraft, needing 6 engines to fly. I found out that some enthusiasts are working to create an actual replica by its 100th anniversary in 2029.

3. https://en.wikipedia.org/wiki/Z1_(computer)
   The first german computer, a replica of which I saw yesterday in the museum.

Lastly, since I talked about indexes above, here is an article you may like

4. https://dlt.github.io/blog/posts/introduction-to-postgresql-indexes/
   How Postgres indexes internally work

A project I discovered

Clears your inbox, sends emails, manages your calendar, checks you in for flights. All from WhatsApp, Telegram, or any chat app you already use.

GitHub - clawdbot/clawdbot: Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞 - GitHub - clawdbot/clawdbot: Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

Disclaimer: This is obviously not sponsored (who would be willing to sponsor this newsletter right now). I just find interesting projects and share. This one is going on one of my VPSs.

A photo I took

That’s all, until next week 👋🏽