Evals without the engineering

🙋 Ways you can help - watch and share our demo video 🙋
The response to our new product has been solid, so we’re looking to get more eyes on it. People who review AI output are already doing all the work needed to create evals - they’re just not capturing it. Our product turns their work into evals, which lets teams manage prompts more systematically and feel less like playing whack-a-mole.

Please watch our 2-minute demo video, and share it with people who are managing prompts!

📉 Progress in March - exceeding targets! 📈
Our metrics for March were:

Customers: 4 (+4)
Monthly revenue: $400
NB: we’re not reporting customers on old products that we’re retiring.

Our goal was to sign up the first customer for our new product, so we were thrilled to get four instead! They’re all building AI products and have shifted prompt management away from engineering towards domain experts, such as:

1/ Nurses, at a healthtech
2/ Teachers, at an edtech
3/ Product managers

The unifying theme among customers who responded well was that their engineers didn’t have the domain expertise to write evals or manage prompts. They want a product that enables domain experts to do this instead, as they were aware it would free up engineering time to focus on technical issues, rather than trying to evaluate output in domains they don’t understand.

💡Challenges - a multiplayer problem💡
Writing evals and managing prompts usually involves multiple stakeholders. 

1/ The domain experts are responsible for deciding what “right” and “wrong” look like (i.e. writing the evals)
2/ Engineers are responsible for building the system that performs well against those evals 

Our solution needs to please both stakeholders, which slows down sales cycles. This is because it’s often unclear who the main decision-maker is; it could be either stakeholder. But it’s also because we need to build something that works for both, and they need different interfaces. Domain experts prefer a visual UI, whereas engineers want to work programmatically. These multiplayer dynamics are common in enterprise sales, but we’re finding them even when selling to seed-stage startups.

🏹 Goal for April: grow from four to five 🏹
Our main goal for April is to grow from four to five customers. This will require retaining all of our initial customers and signing up at least one new customer.

We’d also like to see at least one of our first four customers clearly getting value from Artanis for prompt management. It’s harder to define a metric for this at our stage - it’s more of a “know it when you see it” - but we’ll report on this too.

🙏 Shout-outs 🙏
Special thanks for March go to:

Peter L - for being such an avid reader of our updates
Sergey C - for doing a call with us in the middle of a move
David R - for putting in the yards on email review
Gunisha V - for mobilising your team to make intros
Yiming Y - for detailed value prop feedback
Dom B - for helping get the band back together
Chidi W - for being a good sport
Michael T - for both taking the leap and making intros!
Adam R - for going the extra mile in thinking about leads
Lorenzo S - for the office tour

En Taro Tassadar,
Artanis Team