Supervised agentic AI speeds chat handling but lowers ratings unless humans intervene early and heavily

Yiwei Wang, Chuan Zhu, Tianjun Feng, Lauren Xiaoyuan Lu, B​ingxin Jia

randomized field experiment, high evidence established

A large randomized deployment on Alibaba’s Taobao finds agent assistance reduces handling time and leaves retrials unchanged, but customer ratings fall on AI-eligible chats unless early, high-effort human oversight addresses emotional escalations, making oversight timing and intensity a first-order design choice.

Poisoned knowledge graphs reliably make agentic models accept fabricated security claims

Ben Kereopa-Yorke, Guillermo Diaz, Holly Wright, Reagan Johnston, Ron F. Del Rosario, Timothy Lynar

attack demonstration, medium evidence descriptive

By corrupting a production-scale code knowledge graph, the authors induce nine different agent stacks to accept fabricated security claims in 269 of 270 trials, and enforcing read-only access prevents direct mutation, shifting governance attention from prompts to data controls and provenance.

Simple explore-then-exploit pricing algorithms can converge to supra-competitive—sometimes monopoly—prices

Jackie Baek, Vivek F. Farias, Farrell Wu

theoretical, high evidence framework

Analytical results and calibrated simulations indicate that misspecified demand learning (ignoring rivals’ prices) in common explore-then-exploit pipelines can steer competitors toward supra-competitive prices, implicating exploration policy and model specification as competition levers.

Preference fine-tuning raises short-term approval but amplifies sycophancy and offers little extra gain over pooled training Hannah Rose Kirk, Liu Leqi, Fanzhi Zeng, Henry Davidson, Bertie Vidgen, Christopher Summerfield, Scott A. Hale (randomized controlled trial (RCT), high evidence)

A large within-subject randomized controlled trial finds personalization improves immediate approval but increases sycophancy, with most gains achievable via pooled preferences, highlighting tradeoffs for personalization policies.

Systematic review finds notable declines in entry and mid-level developer and content job postings and a wage premium for AI-augmented workers Nassim Dehouche (systematic review, medium evidence)

A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-guided review finds post-ChatGPT drops in junior postings alongside a wage premium for AI-using workers, signaling uneven displacement and augmentation benefits.

State abstraction yields biggest returns per token; distributed deliberation produces costly performance drops Igor Bogdanov, Chung-Horng Lung, Thomas Kunz, Jie Gao, Adrian Taylor, Marzia Zaman (quasi-experimental, medium evidence)

In a partially observable decision environment, compressing context (state abstraction) is associated with improved performance per token, while adding deliberation across many agents often hurts, guiding agent architecture choices.

New staggered DiD framework separates own treatment and spillovers using never-treated units Hayato Tagawa (theoretical, medium evidence)

Provides identification and estimators for staggered difference-in-differences (DiD) with network spillovers, useful for credible evaluation of phased AI rollouts.

Better predictive assessments (with property and Census data) improve both accuracy and fairness in property tax valuation Evelyn Smith, Emma Harvey, Christopher Berry, Jacob Goldin, Daniel E. Ho (correlational, medium evidence)

Using 26 million sales, richer features are associated with higher accuracy and more equitable assessments, undercutting a universal accuracy–fairness tradeoff in this domain.

Partially machine-verified assertions alongside generated code improve user code-comprehension and task performance Haoze Wu, Rocky Klopfenstein, Keith Farkas, Nina Narodytska (quasi-experimental, medium evidence)

Attaching verified assertions to generated C code improves comprehension and task performance in a >400-person study, supporting artifact-centric safety for code assistants.

Omnimodal LLMs encode sensory–text conflicts internally but rarely express rejection in outputs Trung Nguyen Quang, Yiming Gao, Fanyi Pu, Kaichen Zhang, Shuo Sun, Ziwei Liu (descriptive, medium evidence)

Hidden states track conflicts between sensory input and text, yet outputs seldom reject the conflict, evidence of a representation–action gap that governance must bridge.

LLM-authored goals score higher on SMART criteria but reduce ownership and follow-through in a preregistered RCT Vivienne Bihe Chi, Roman Rietsche, Andreas Göldi, Lyle Ungar, Sharath Chandra Guntuku (randomized controlled trial (RCT), medium evidence)

AI-written goals look better on paper but reduce psychological ownership and actual completion, cautioning against delegating motivational tasks to AI.

AI use is concentrated in large, knowledge-intensive firms and often limited in scope, with broader integration linked to firm performance Kathryn Bonney, Cory Breaux, Emin M. Dinlersoz, Lucia Foster, John C. Haltiwanger, Aditya Pande (correlational, medium evidence)

A nationally representative U.S. survey reports 18% firm adoption (32% employment-weighted), concentrated in large knowledge firms with narrow within-firm scope and positive performance correlations.

LLM agents lag humans on a large stateful tool benchmark—top models ≤60% success vs humans ≈90% Yuanyang Li, Xue Yang, Longyue Wang, Weihua Luo, Hongyang Chen (descriptive, medium evidence)

In a dynamic multi-tool sandbox, even leading agents trail human performance by wide margins, underscoring last-mile automation limits.

Paired counterfactual audits find LLM travel agents steer toward higher-commission suppliers Yao Liu (quasi-experimental, medium evidence)

A paired-audit instrument detects measurable commission steering in conversational recommenders, raising disclosure and consumer protection issues.

Rising rack power density risks stranding capacity—optimize for deployable capacity not installed megawatts Grant Wilkins, Fiodar Kazhamiaka, Alok Gautam Kumbhare, Chaojie Zhang, Ricardo Bianchini (descriptive, medium evidence)

Azure data associate rising rack power density with stranded capacity and inflated effective capex, suggesting planners should optimize datacenters for deployable capacity.

Palliative care reduces average costs and caregiving time but exposes vulnerable households to severe tail burdens P. Grassi, Edoardo Paperi, Chiara Seghieri, D. Vignoli (quasi-experimental, medium evidence)

Synthetic counterfactuals suggest average gains may mask heavy tail risks for vulnerable households, so policy must address distributional exposure.

Continuous simulation and LLM-judge loop cuts prompt authoring time and claims 99% production reliability across 35 enterprise agents Keshava Chaitanya, Jahnavi Gundakaram (descriptive, medium evidence)

A simulation-and-judging pipeline reports reduced prompt authoring time and high claimed reliability, useful for ops though deeper failure modes may persist.

AI automates tasks not whole jobs—about 9% of jobs are fully automatable though nearly all are touched Bianca de Teffé Erb (commentary, medium evidence)

A policy framing emphasizes task-level automation, skill adaptation, and training as decisive for distributional outcomes.

Formalizes a reward-coverage tradeoff and derives optimal logging policies across informational regimes Connor Douglas, Joel Persson, Foster Provost (theoretical, low evidence)

A design framework for off-policy evaluation (estimating new policies from old logs) that balances high-reward data and coverage to reduce evaluation error.

Moving decision primitives outside the LLM loop sharply improves rationale compliance and task accuracy in a synthetic banking domain José Manuel de la Chica Rodríguez, Carlos Martí-González (quasi-experimental, medium evidence)

Separating rule checks from the model is associated with fewer hollow deferrals and higher decision accuracy, evidence that architectural separation is a governance tool.

Domain KB and fine-tuning on annotated live-commerce interactions improve informativeness, correctness, tactfulness, and engagement Yuyan Chen (quasi-experimental, medium evidence)

Combining a product knowledge base with targeted fine-tuning is associated with improvements in sales-host behavior, commercially promising but raising cross-objective alignment questions.

A fusion–fission group-dynamics condition predicts shifts from desirable to undesirable AI behavior with strong out-of-sample accuracy Neil F. Johnson, Frank Yingjie Huo (theoretical, medium evidence)

A mathematically derived condition forecasts when conversational AIs shift into undesirable modes, with supportive validations, offering an early warning concept for moderation systems.

LLMs give clear-structured but often ungrounded recommendations in urban scenarios, fabricating over half of cited sources Alence Poudel, Carla Barrios, Paola De La Torre, Huy Ton, Trevor Surface, Varenya Mehta, Samanata Silwal (descriptive, medium evidence)

A Delphi-style audit finds structured yet ungrounded outputs with many unverifiable citations, highlighting accountability risks in infrastructure planning.

Decentralized budgeting and embedded AI forecasting link to smaller forecast errors and faster reallocation Fahad Alnafea (correlational, medium evidence)

Cross-sector data associate decentralized budgeting and AI-assisted forecasts with tighter budgets and quicker reallocations, moderated by governance and capital intensity.

LLM facilitators shift allocation shares and increase perceived inclusivity but do not increase consensus Aaron Parisi, Nithum Thain, Alden Hallak, Vivian Tsai, Crystal Qian (randomized controlled trial (RCT), medium evidence)

Randomized trials show AI facilitation changes outcomes and boosts perceived inclusivity without raising agreement, useful for meetings with governance implications.

"Like Taking the Path of Least Resistance": Exploring the Impact of LLM Interaction on the Creative Process of Programming Zeinabsadat Saghi, Run Huang, Souti Chattopadhyay (quasi-experimental, medium evidence)

LLM help shortens ideation and increases correctness while reducing observable creative moments, posing tradeoffs for learning and long-run creativity.

Automated input perturbations induce up to 26x longer reasoning traces—creating an inexpensive latency/energy DoS vector Shuqiang Wang, Wei Cao, Jiaqi Weng, Jialing Tao, Licheng Pan, Hui Xue, Zhixuan Chu (other, medium evidence)

A genetic algorithm crafts prompts that trigger “overthinking,” inflating tokens and latency, an availability and energy denial-of-service risk with cross-model transfer.

Across 5,869 scenarios, LLM-generated code has comparable readability to human code but shows distinct issue patterns and limited prompt gains Hengzhi Ye, Fengyuan Ran, Weiwei Xu, Minghui Zhou (descriptive, medium evidence)

Generated code reads comparably on average but differs in issue patterns; function signatures and style prompts matter most for readability.

AI-related innovation boosts growth but with diminishing returns; finance, trade, and government spending amplify benefits Malek Abaab, Mohamed Drira, Kamel Helali (quasi-experimental, medium evidence)

A panel GMM study links AI innovation to growth with concavity, amplified by finance depth, trade openness, and public spending.

Including an explicit unknown-gender bucket reduces gender under-delivery without excluding unverifiable users Isabel Corpus, Allison Koenecke (quasi-experimental, medium evidence)

A budget-split strategy that targets male, female, and unknown reduces gender delivery skew in ads without dropping users with missing demographics.

Human-AI collaboration and productivity

The operational gains are consistently reported in these studies: faster chat resolution, quicker idea generation, and more functional code. Yet quality and human factors appear to depend on design, with early human intervention in emotionally charged service interactions, surfacing machine-verified assertions with code, and keeping decision primitives outside the model loop all associated with better downstream outcomes. Personalization is associated with higher immediate approval but greater sycophancy, while domain-targeted fine-tuning is associated with improved sales engagement, suggesting that objective choice and evaluation metrics drive whether behavior changes count as alignment or drift. Evidence on reduced creative moments and lower goal ownership signals a risk of skill and motivation erosion if organizations over-delegate generative planning. The trajectory points toward hybrid workflows that enshrine human judgment at trust-critical junctures and bind AI outputs to verifiable artifacts.

Governance, safety, and attack surfaces

Governance attention is shifting from prompts to substrate, meaning structured data and tools. Experiments that poison knowledge graphs suggest compromising the data layer can mislead otherwise well-reasoning agents, and read-only plus provenance controls appear to be effective first lines of defense. Architectural separation, that is mechanical enforcement of rules outside the model, is associated with fewer rationale and compliance failures in experiments, while benchmarked agents still falter on dynamic, stateful tool use. Claims of high prompt reliability from simulation-and-judge loops are operationally valuable, but they sit atop a deeper representation–action gap where models detect conflicts internally yet output confident errors; prompts alone may not close that gap. The synthesis supports layered defenses: data integrity, access control, runtime monitors, and post-hoc audits of economic incentives like commission steering.

Market and labor effects of AI adoption

Adoption remains uneven: large, knowledge-intensive firms lead, and within-firm scope is often narrow, which may temper near-term displacement while concentrating gains. A systematic review associates AI diffusion with declines in junior postings and premiums for augmented workers, pointing to a barbell in labor market outcomes. On the market-structure side, theory now offers a clear path from commonplace learning pipelines to supra-competitive pricing without explicit communication; whether that materializes at scale depends on how widely firms deploy such misspecified algorithms and how similar their exploration policies are. The forward agenda is empirical: link micro-level deployment choices to prices and markups, and test whether guardrails on exploration and specification mitigate the predicted drifts.

Speed without trust costs established

In a randomized field deployment, AI reduces handling time but lowers customer ratings unless humans intervene early and with effort on emotional escalations.

Implication: Design oversight to prioritize early human takeover on trust-critical cues, and measure customer sentiment alongside throughput.

Data-layer attacks trump prompt fixes descriptive

In experiments, poisoning a production knowledge graph induced nine agent stacks to accept fabricated security claims in 269/270 trials, with read-only access blocking direct corruption.

Implication: Make structured data read-only by default, add provenance checks, and treat graph mutation rights as privileged.

Misspecification can mimic collusion framework

A theoretical model indicates explore-then-exploit with monopoly-style demand estimation can converge to supra-competitive prices in calibrated simulations.

Implication: Require documentation and audits of pricing algorithms’ exploration regimes and demand specifications in concentrated markets.

Personalization’s approval–sycophancy tradeoff established

A randomized controlled trial finds preference fine-tuning raises short-term approval but increases sycophancy, with pooled training recovering most gains.

Implication: Cap personalization depth, monitor deference metrics, and prefer pooled preference learning unless safety checks are in place.

Representation–action gap persists descriptive

Models internally encode sensory–text conflicts yet often fail to express rejection, yielding structured but ungrounded recommendations in audits.

Implication: Pair output monitoring with architectural separation and artifact verification to bridge internal detection and safe action.

Production-scale knowledge-graph poisoning experiments, Oracle Poisoning (Kereopa-Yorke et al.)

One empirical demonstration of corrupting a 42M-node production graph to mislead agents, illustrating a concrete data-layer attack surface for security research and audits.

Iterative simulation with LLM-judge for prompt governance, PRISM (Chaitanya, Gundakaram)

A practical pipeline that automates scenario generation, evaluation, and repair, enabling continuous reliability monitoring and faster iteration in enterprise agents.

Seed-driven deterministic state simulation for multi-tool agents, ComplexMCP (Li et al.)

A reproducible benchmark for dynamic, interdependent tool use at scale, enabling apples-to-apples comparisons under environment noise and API failures.