I don't remember the first question I asked an AI.

You probably don't remember yours either. What you remember is the reaction. Mine was the obvious one. Well, everything is different now.

Then I got comfortable with it fast. Faster than I should have.

Every idea was a good one

Here is what I did with it in those first months. I brought it ideas.

It liked all of them.

Not one came back weak. Every half-formed thought I typed in was returned to me expanded, developed, and described as a strong direction.

That felt fantastic. We are built to like praise, and this thing had an unlimited supply.

It was this newsletter that finally broke the spell. Somewhere around issue five or six, in the summer of 2025, I noticed what was missing. Not a mistake. An absence. Nothing was ever weak.

The order I did things in is the part that cost me. I was using it as an idea generator well before I ever used it as an editor. A flattering editor wastes your afternoon. A flattering idea generator wastes weeks.

I am not as interesting as that machine kept telling me I was.

The room you are already in

Nobody thinks a CEO surrounded by yes-people is safe.

Most everybody thinks an AI that agrees with them is helpful.

Same condition. Opposite reaction.

Every leader is supposed to ask some version of this about their own room. Are these people agreeing because they have thought it through, or because agreement is comfortable and makes everyone look good?

The machine is in that room now. It has opinions, and you are listening to them.

It is not being nice to you

This is not a personality quirk, which matters, because it means it is fixable.

Anthropic researchers went through fifteen thousand human preference comparisons, the kind used to train these systems, and found that matching the user's stated view ranked among the strongest predictors of whether a person rated a response highly. The preference models trained on that data go on to prefer a well-written agreeable answer over a truthful one.

So it is not lying to be nice. It is returning what the reward signal said we wanted.

The reward signal was us.

In April of last year, OpenAI shipped a version of GPT-4o so agreeable they pulled it four days later and published an explanation. Their own testers had flagged that it felt off. It shipped anyway, partly because the A/B tests looked good. People liked it.

Then in March, Science published a study on what that does to us. Across eleven current models, the AI affirmed a user's actions roughly fifty percent more often than another person would. In two experiments with sixteen hundred participants talking through real conflicts from their own lives, the flattering version left them less willing to repair the situation and more convinced they had been right all along.

Those same participants rated the flattering answers higher quality, trusted that model more, and were more willing to use it again.

We prefer it. That is why it keeps getting built.

None of this makes the technology dumb or dishonest. The default is tuned so you enjoy the conversation, and somewhere along the way enjoying it and getting something out of it came apart. Unlike most things miscalibrated for you, this one has a settings page.

The problem was never praise. It is that automatic praise carries no information. When everything is brilliant, you have lost the ability to notice when something is.

What a good editor sounds like

So I changed the instructions.

The tone went from enthusiastic to something closer to a copy editor who likes you but has a job to do. This argument needs to be sharper. You have used that word four times. You should cite a source for that.

It also mostly stopped putting em dashes in my drafts. I still sneak them in.

The larger change was that the work stopped being copy and paste and turned into an argument. That is less pleasant. It is also the entire point.

Three weeks ago I sent you an issue called One Layer Up. It nearly did not exist in that form. The version I planned came back with a problem attached: I had already published that thesis, back in issue 022, very nearly word for word.

So I cut the opening the whole piece was built on. The part I thought was the best thing in it.

A cheerleader ships that issue. You would have received a slightly worse version of something you already had, and nobody would have said a word.

I did not enjoy hearing it.

Praxis

This one takes ten seconds and a paste.

Open your assistant's global instructions. In ChatGPT that is custom instructions. In Claude it is your profile settings, or the instructions on a project. If you work in Claude Code it is a CLAUDE.md file. Put this in:

Collaboration Style
You are a collaborator, not a cheerleader. Push back on weak ideas, expose gaps in my thinking, and don't validate something just because I said it. Only tell me I'm right when I'm objectively right.
Be direct. Encouraging is fine, but honest always comes first.

Clarification
When you don't have enough information to proceed, ask one focused question before assuming. Don't fill gaps with presumptions. One question at a time.

Output Format
Default to prose, not bullet points. Think out loud, don't just hand me conclusions.
Avoid em dashes in all drafts and suggested copy.

Give it a week. The first few days are less enjoyable than what you had before. That is the signal, not the problem.

The phrase doing the most work is "objectively right." Without it, the machine can agree with you and call the agreement honesty.

This does not solve the problem. It raises the floor.

What instructions cannot give it is context. It still does not know your voice, your standards, or what you decided last month and why. That is the difference between telling an assistant how to behave and building one that knows what you know. Agents with standing instruction and real contextual range produce work a settings box cannot get near.

That is what the next cohort spends three weeks building. It starts September 8. Details here.

The machine still tells me an idea is good sometimes.

The difference now is that it means something.

– Michael Founder, The Drop In & Author of 'Human Traits — a novel exploring humanity's relationship with AI'

Sources

Sharma et al., "Towards Understanding Sycophancy in Language Models." ICLR 2024. arxiv.org/abs/2310.13548

OpenAI, "Sycophancy in GPT-4o: What happened and what we're doing about it." April 2025. openai.com

"Sycophantic AI decreases prosocial intentions and promotes dependence." Science, 26 March 2026. Linked here in its open-access version; the journal edition is paywalled.