Evals without the engineering

🙋 Ways you can help - read and share our new AI blogs 🙋
No matter how good AI models get, they can’t read your mind. We’ve published several longer blogs recently on how to clearly communicate what you want your AI application to do, or your “Policy”, in scenarios where the correct output is subjective. We recommend starting with this post - please subscribe / share if you like the content!

📉 Progress in April - mixed signals 📈
We missed our April target, which was to grow from 4 to 5 customers. Our metrics were:

Customers: 3 (-1)
Monthly revenue: $350 (-$50)

Of the four customers we signed in March, three were highly engaged in April. They invested significant amounts of their own team’s time into using Artanis and/or bought more seats, which is why our revenue only fell by $50. We felt we were delivering clear value to them.

One customer churned. The churn reasons were more about lack of intent to write evals / alter prompts in the foreseeable future. This was likely a consequence of being fairly relaxed in taking on customers outside our ICP in April, rather than problems in our product.

We didn’t sign up any new customers. Top-of-funnel particularly has consistently been a problem for us as a technical founding team, and we’re struggling to cut through the noise in the evals space. We’re also finding it hard to communicate how Artanis is superior to building in-house tooling for domain experts to review AI outputs, which is our main competition.

🏹 Goal for May - tread water… 🏹
We normally set growth targets then, if we miss them, reflect on what went wrong. However, Sam is having his first child in May and it’s not realistic to aim for growth when ⅓ of our team will be incapacitated. So for May we’ll be focused on delivering for current customers, and treating any growth as a bonus!

🙏 Shout-outs 🙏
Special thanks for April go to:

Finn W - for the detailed look at evaluating Claude skills
Ainhoa A - for the call about CS and prompt management
Michael T - for getting us into Sifted
Peter L - for helping find solutions to data sensitivity
Paul B - for getting us into Lenny’s Newsletter
Abbas & Qasim - for insights on onboarding to their products
Eve T - for making our day with your PhD topic!

En Taro Tassadar,
Artanis Team