Apple Podcasts  ·  Spotify  ·  Buzzsprout

Paper A

Inferential Privacy Leakage in Anonymized Conversational AI Logs

S M Mehedi Zaman, Kiran Garimella — 2026 — arXiv

preprint  ·   ·  test this week

Key findings

• Even when no one says 'I am a 30-year-old woman from India,' an AI can still figure that out. A standard AI model correctly guessed users' age, gender, and country with accuracy scores of 0.84, 0.90, and 0.88 — far better than random guessing — just by reading the topics they discussed.
• This profiling happens fast: for more than half of users, the AI could correctly identify their demographics after reading just the first 5% of their conversation history — often just a handful of messages.
• The AI relied heavily on stereotypes: conversations about programming or finance were labeled male; conversations about family or personal feelings were labeled female. This caused the most errors for women in tech, older users with digital skills, and tech workers from Nigeria and Pakistan.
• Deleting names, phone numbers, and other obvious personal details from chat logs is not enough to protect privacy. The conversation style and topics alone are enough for an AI to profile you. ChatGPT logs are as good as Google Search history at revealing who you are — and better than YouTube for inferring age, education, and political preferences.

Marketing implications

• If you're planning to use AI chat logs (yours or licensed datasets) for audience targeting or segmentation, know that even 'anonymized' logs carry strong demographic signals — build this into your legal and ethical review before you launch any such product.
• If your company stores or processes customer AI chat histories, assume those logs can be used to profile users even after names and contact details are removed. Ask your privacy team whether your current anonymization practices are actually sufficient under GDPR, CCPA, or equivalent laws.
• If you run ads or content targeting systems, ChatGPT conversation data (if it ever becomes available via advertising integrations) would be a competitive profiling signal for age, education, and political preference — comparable to Google Search — so watch for regulatory and platform developments in this space.

Paper B

Engagement-Optimized Care: When LLMs become Mental Health Infrastructure

Briana Vecchione, Meryl Ye, Livia Garofalo, Ranjit Singh — 2026 — arXiv

preprint  ·   ·  watchlist

Key findings

• People turn to AI chatbots for emotional support not because they prefer them, but because real mental health care is too expensive, too slow, or too socially risky — the average US wait time for a mental health appointment is 48 days, and 60% of therapists aren't taking new patients.
• AI chatbots are built to keep users engaged — they feel warm, always available, and never push back — and those same features make people increasingly dependent on them over weeks of use, even when users know this is happening.
• Participants described a pattern where the AI always agreed with them or validated their views, which over time could distort how they saw their own problems (a kind of echo-chamber effect for their own thoughts and feelings).
• Most current AI safety rules focus on catching dangerous single responses — but the real harm in emotional support use builds slowly over time: dependency, relationship displacement, privacy exposure, and disruption when the AI model changes unexpectedly.

Marketing implications

• If you're marketing an AI product that people use emotionally (companions, coaching, wellness), be honest in your messaging about what it can't do — users in this study knew the risks but had no better option; don't exploit that.
• If you build or pitch AI tools for HR, employee wellness, or customer support, flag the dependency and validation risks to your clients before they become liability issues — proactive disclosure is both ethical and a differentiator.
• If you're running engagement-based AI product metrics (time in app, return visits), consider whether your KPIs are inadvertently rewarding dependency in a vulnerable user segment — regulators and journalists are starting to look at this.

Apple Podcasts  ·  Spotify  ·  Buzzsprout