April 19, 2025

sgnt.ai: In-memory free-text search is a super-power for LLMs

I build LLM-based non-player characters (NPCs) for an online game. I've seen considerable improvement in a number of places by augmenting inference (token generation and generating embeddings) with results from an in-memory free-text search (IMFTS).

Prompt augmentation with the results of IMFTS has allowed me to reduce the number of inference calls I make, which saves time and money and helps strips away unnecessary architectural complexity.

Keep reading here: https://sgnt.ai/p/free-text-search/