Apple Podcasts  ·  Spotify  ·  Buzzsprout

Paper A

Thousands of AI Authors on the Future of AI

Katja Grace, Harlan Stewart, Julia Fabienne Sandkuhler, Stephen Thomas et al. — 2024 — arXiv (preprint) - AI Impacts / 2023 Expert Survey on Progress in AI

preprint  ·   ·  test this week

Key findings

• AI experts think there is at least a 50% chance that by 2028, AI will be able to build a working payment-processing website from scratch, write a song that sounds like a real hit by a specific artist (like Taylor Swift), and automatically download and improve its own AI model — all without human help.
• Experts think there is a 50% chance that AI will be able to outperform humans at every possible task by 2047 — that estimate moved up by 13 years compared to the same survey done just one year earlier, suggesting researchers think AI is advancing faster than they previously thought.
• Even though AI might outperform humans at most tasks fairly soon, experts think it will take much longer before every human job can be fully automated — they put a 50% chance on that happening around 2116, nearly a century away.
• Most AI researchers are worried: between 38% and 51% of them think there is at least a 10% chance that advanced AI could lead to outcomes as bad as human extinction. Even the majority who think AI will probably go well still acknowledge a real risk of catastrophic outcomes. Researchers broadly agreed that safety research needs more attention.

Marketing implications

• AI-generated music, videos, and creative content are expected to become indistinguishable from human-made work within a few years. If you run a creative agency or manage brand content, start thinking now about how you'll verify authenticity, disclose AI use, and maintain brand trust before this becomes a crisis.
• The experts who build AI think it will be capable of coding full commercial products (like payment sites) autonomously by 2028. That means tools that automate entire digital marketing workflows — not just individual tasks — are likely coming sooner than most marketing plans assume. Budget and org-structure decisions made today should account for this.
• Nearly half of AI experts think there is at least a 5% chance of catastrophic AI outcomes, and more than half are concerned about AI spreading false information. Brands that proactively build and communicate AI governance policies now will have a trust advantage as public concern grows.

Paper B

ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence

ARC Prize Foundation, Francois Chollet, Mike Knoop, Gregory Kamradt — 2026 — arXiv (preprint) - ARC Prize Foundation

preprint  ·   ·  skip

Key findings

• Humans solved 100% of the ARC-AGI-3 environments. The best frontier AI system (Anthropic Opus 4.6) solved only 0.5% — less than 1 in 200 tasks. This is a massive gap between what people can do and what today's most powerful AI can do.
• Current AI systems can only reason well in areas where they've already been trained on lots of relevant data AND where there's a clear right-or-wrong answer to check against. Outside those two conditions, they fall apart — unlike humans, who can reason flexibly in brand-new situations.
• The previous ARC-AGI benchmarks (versions 1 and 2) appear to have been 'learned around' by frontier AI — models seem to have been trained on data similar enough to the test tasks that they could fake good performance without actually developing flexible reasoning. ARC-AGI-3 is designed to close that loophole.
• AI coding tools (like Claude Code and Codex) work well precisely because coding is a domain with massive training data and automatic right/wrong feedback. This success does NOT mean AI can reason flexibly in new, unfamiliar domains.

Marketing implications

• If you're using AI for marketing tasks that are well-defined and have clear right/wrong answers — like A/B test analysis, ad copy generation, or data classification — current AI tools are likely genuinely useful. But don't expect AI to handle open-ended strategic problems it hasn't seen before.
• Be skeptical when AI vendors claim their model 'reasons' or 'thinks.' Ask: has it been trained on situations like yours, and is there a way to check if it's right? If the answer to either is no, the AI is probably guessing.
• When building AI-assisted marketing workflows, design them around tasks where you can automatically check the AI's output (e.g., did the ad meet the brief's specs? did the email avoid banned words?). AI works best when it gets clear feedback, just like it does on ARC-AGI tasks where right/wrong is measurable.

Paper C

Measuring AI Ability to Complete Long Tasks

Thomas Kwa, Ben West, Joel Becker, Amy Deng et al. — 2025 — arXiv (preprint) - METR

preprint  ·   ·  skip

Key findings

• Today's best AI models (like OpenAI's o3) can reliably finish software tasks that take a skilled human professional about 110 minutes — things like writing data transformation scripts or fixing simulation bugs. A few years ago, that number was just a few minutes.
• AI's ability to handle longer and more complex tasks has been doubling roughly every 7 months since 2019. This is a consistent trend across years and models, and it may have sped up slightly since 2024.
• The main reason AI keeps getting better at longer tasks is not just raw intelligence — it's that newer models are less likely to get stuck, better at catching and fixing their own mistakes, and more skilled at using tools like code interpreters and search.
• If this growth trend continues at the same rate, AI systems could be capable of independently completing software tasks that currently take a human a full month — sometime between 2028 and 2031. However, the researchers caution that this is a rough extrapolation and real-world tasks may be harder than the benchmarks used.

Marketing implications

• AI agents are now capable of completing multi-hour technical tasks on their own — if your team uses developers or data analysts for routine work (like building a data pipeline or writing a script to reformat ad data), test whether an AI agent can do a first pass. You may save 1–2 hours of skilled labor per task today, and that window is growing fast.
• Plan your AI tool strategy with a 2–3 year window in mind, not just what works today. The research suggests the complexity of tasks AI can handle is growing rapidly — what requires a contractor today may be fully automated in 2–3 years. Start small pilots now so your team isn't caught flat-footed.
• Be skeptical of AI for messy, ambiguous tasks right now — the research shows AI still struggles when tasks are unstructured. For creative briefs, brand strategy, or nuanced client communications, humans still outperform. Reserve AI automation for tasks with clear inputs and outputs.

Apple Podcasts  ·  Spotify  ·  Buzzsprout