Small distilled retriever recovers teacher precision, cuts latency by up to 27x, and raises Bing Ads revenue by ~1%

Vipul Gupta, Shikhar Mohan, Lakshya Kumar, Pranjal Chitale, Nikit Begwani, Amit Singh, Manik Varma

production A/B, high evidence established

A compact student retriever trained via a three-phase recipe is reported to retain >98% of teacher precision, slash latency on A100s, and be associated with a ~1% revenue lift in a live Bing Ads A/B test; the paper finds small models can move top-line metrics when engineered for throughput in this deployment.

Three-arm A/B design reveals platform delivery, not creatives, drives most audience reallocation

Pallavi Pal, Anjana Susarla

method + randomized field test, high evidence established

The authors implement a three-arm design that point-identifies algorithmic delivery effects versus creative effects and find an applied Meta test where delivery shifts most of the audience mix, implying standard two-arm tests can misattribute impact in that context.

Firms redistribute hiring and redesign tasks as generative-AI exposure rises, with reallocation explaining most of the decline in exposure

Fangyan Wang, Zaiyan Wei, Yang Wang

observational decomposition, medium evidence suggestive

Using a dynamic posting-level exposure measure, the paper decomposes exposure declines into between-job reallocation and within-job redesign, finding reallocation explains more of the early decline and senior roles adjust first in the sample studied.

Prior conversation sentiment biases LLM evaluators—negative histories have ~1.6× stronger effect than positive ones Sid-ali Temkit (quasi-experiment, high evidence)

Automated LLM evaluations shift toward past conversation polarity, especially under uncertainty, so evaluation pipelines need context controls and debiasing.

Larger LLMs produce worse distributional forecasts on superlinear, regime-change time series Nick Merrill, Jaeho Lee, Ezra Karger (descriptive, medium evidence)

More capable models are reported to overinflate upper tails and degrade calibration on risk-sensitive forecasts in the settings tested, warning against assuming “bigger is safer” in finance and epidemiology.

Platform-derived AI exposure scores reflect platform user composition, biasing employment-effect estimates that attenuate after reweighting to workforce shares Michelle Yin, Burhan Ogut (quasi-experiment, medium evidence)

Reweighting platform-log exposure metrics to BLS workforce shares reduces estimated employment effects by 42–93% in the analyses, which materially alters policy-relevant conclusions in those samples.

GUIDE integrates Decision Transformer, Q-value guidance, and safe fallback to boost GMV and ROI in Taobao experiments Mingming Zhang, Feiqing Zhuang, Na Li, Shengjie Sun, Xiaowei Chen, Junxiong Zhu, Fei Xiao, Keping Yang, Lixin Zou, Chenliang Li (system + experiments, medium evidence)

A hybrid generative-RL auto-bidding system is reported to improve GMV and ROI in offline, simulated, and live settings, suggesting potential value in combining sequence models with safety-guided exploration.

Counterfactual long-term value prediction and policy optimization increase new-item GMV on Taobao Yifan Wang, Yixuan Wang, YiDan Liang, Qiang Liu, Fei Xiao (RCT, medium evidence)

A multi-value-aware retrieval framework is associated with higher new-item GMV (+5.3%) and slightly higher overall GMV in an RCT, indicating long-term value objectives can complement short-term goals in this experiment.

Meta-analysis finds distributional safety-stock and multi-echelon coordination cut inventory costs by ~9–16% vs classical methods Stabak Roy, Saptarshi Mitra (meta-analysis, high evidence)

A PRISMA synthesis of 31 studies finds modern inventory methods consistently reduce costs, with larger gains for volatile SKUs.

Firms engaging in 'AI washing' face higher debt costs after policy shock—~12.5 bps increase after China's FYP Congluo Xu, Jiuyue Liu, Xiangsheng Zheng, Ziyang Li (quasi-experiment, medium evidence)

A policy shock is associated with higher borrowing costs for firms flagged as engaging in overstated AI claims, suggesting markets may penalize such firms under scrutiny in the studied setting.

AI exposure shifts skill demand: displacement lowers routine cognitive skills while augmentation raises nonroutine analytical demand Lingzhe Zhang, Chenglei Zhang (quasi-experiment/correlational, medium evidence)

Using 67 million job postings, the study associates AI exposure with reduced demand for routine skills and increased demand for analytical skills, concentrated in smaller firms in the dataset.

Procedural strategic-game benchmark reveals qualitative trade-offs and local volatility in LLM strategic performance Vartan Shadarevian, Kia Ghods, Alex Kenich, Anany Kotawala (descriptive benchmark, medium evidence)

Strategic reasoning appears jagged and locally fragile across models, cautioning against single-number capability rankings.

People overestimate time savings from LLM assistance though actual completion times show no speed improvement Sunny Yu, Myra Cheng, Ahmad Jabbar, Ilia Sucholutsky, Katherine M. Collins, Dan Jurafsky, Robert D. Hawkins (RCT, high evidence)

A preregistered trial finds no measured speed gains on simple tasks despite perceived benefits, underscoring the need for expectation management.

An agentic system synthesizes implementations and proofs and solves 7/7 distributed specs far faster and cheaper than experts Shubham Agarwal, Alexander Krentsel, Shu Liu, Mert Cemri, Audrey Cheng, Rui Meng, Tomas Pfister, Chun-Liang Li, Sylvia Ratnasamy, Aditya Parameswaran, Matei Zaharia, Ion Stoica, Mohsen Lesani (system experiments, medium evidence)

Joint code-and-proof generation reports acceleration in formally verified engineering when paired with verification loops in the authors' experiments.

Automation adopters see a lasting wage premium (~4% after five years), concentrated in small firms and certain worker groups Laura Bisio, Angelo Cuzzola, Marco Grazzi, Daniele Moschella (quasi-experiment, medium evidence)

Import-based adoption proxies associate with a persistent within-firm wage premium and rising dispersion, especially in small firms.

Digital transformation raises manufacturing labor demand—especially for highly educated, high-skilled workers—via TFP gains Yongming Wang, Xin Liu, Yujie Zhu, Huifen Cai, Xuefeng Shao (correlational panel, medium evidence)

Firm panels link digital transformation to higher labor demand and upskilling, mediated by productivity gains in the sample.

LLM agents can draft pro-looking spreadsheets but fail professional standards as complexity and chaining increase Thomson Yen, Julian Poeltl, Harshith Srinivas Gear, Yilin Meng, Joshua Fan, Adam Shen, Yili Liu, Ali Bauyrzhan, Siri Du, Haoyang Liu, Daniel Guetta, Hongseok Namkoong (descriptive benchmark, medium evidence)

Agents handle simple finance tasks but break on chained calculations and accuracy checks, signaling verification needs before deployment.

Cross-domain benchmarks reveal when coordinated AI agents improve scientific inference from partial evidence Fiona Y. Wong, Markus J. Buehler (descriptive benchmark, high rigor)

Multi-agent coordination helps in domains with fragmented evidence, but strong single-model baselines can match performance elsewhere.

Digital-economy growth accelerates servicization and deindustrialization with heterogeneous post-2017 dynamics Xinyan Luo (panel quasi-experiment, medium evidence)

Provincial panels associate digital-economy expansion with shifts to services, with dynamics changing after 2017.

ID-free multimodal codes improve livestream recommendation engagement in a billion-user A/B test Xinhang Yuan, Zexi Huang, Anjia Cao, Xudong Lu, Zikai Wang, Penghao Zhou, Chang Liu, Wentao Guo, Qinglei Wang (RCT, high evidence)

Replacing ephemeral IDs with hierarchical multimodal codes is reported to boost watch time and cold-start performance at industrial scale in the experiment.

Privacy-constrained contextual bandit plus vaulted identity increases engagement and weight-loss outcomes in deployment Nariman Mani, Salma Attaranasl (deployment quasi-experiment, medium evidence)

Privacy-first personalization is associated with improved coaching outcomes, showing privacy and performance can align with the right architecture in deployment data.

Agentic AI can speed tasks but requires process controls and verification to improve outcomes Christopher Koch (review/meta, medium evidence)

A synthesis argues for governance, traceability, and verification (Agentic Agile-V) to make agentic coding productive and safer.

Behavior–outcome rules for LLM coding agents fail to generalize across frameworks and toolchains Wei Ma, Zhi Chen, Jingxu Gu, Tianling Li, Shangqing Liu, Lingxiao Jiang (correlational, high rigor)

Heuristics that predict success in one agent framework often flip elsewhere, limiting operational playbooks.

LLM access raises average task performance, but gains concentrate among users with high AI interaction competence Lihi Idan, Bharat Anand (RCT, medium evidence)

Productivity gains are reported but uneven, with the biggest benefits accruing to users skilled at prompting, filtering, and verification.

Systematic review finds human-centered AI design improves cognitive efficiency and satisfaction; opaque integrations raise stress and disengagement Mehmet Akın Bulut, Nurevşah Kaya, Abdullah Ortak, Sevda Nur Akan Baghırlı (systematic review, high quality)

Design features like transparency and autonomy are associated with better user outcomes, guiding workplace AI policy.

Task-level atlas shows automation exposure varies widely across countries and is skewed toward substitution in low-income settings Prashant Garg, Tommaso Crosta, Jasmin Baier (descriptive dataset, medium evidence)

A 124-country atlas reveals higher exposure in richer economies but more substitutional risk in poorer ones, relevant for international policy.

Random Forests best predict firm-reported AI/IoT success in Czech/Slovak survey; value-realization share is the stable predictor Ján Dvorský, Matúš Senci, Abdul Bashiru Jibril, Zora Petráková (correlational survey, medium evidence)

Among surveyed firms, value-realization capability predicts perceived success better than other factors, informing implementation priorities.

Framework proposes fusing structured labor stats with unstructured postings and ML for timelier labor-market forecasts Pavan Kumar, Reddy Dhanireddy (framework, medium evidence)

A proposed pipeline for AI-augmented econometrics argues for integrating real-time postings with official stats, pending validation.

AI adoption ticks up in Germany (2023–24), concentrated in manufacturing/services and larger firms T. Licht, Klaus Wohlrabe (correlational survey, medium evidence)

Sector and size concentration persists, with managerial risk tolerance associated with adoption.

Only ~22.8% of U.S. manufacturing plants reported AI use in 2021; adoption linked to cloud, analytics, and process management Kristina McElheran, Mu-Jeung Yang, Zachary Kroff, Erik Brynjolfsson (census survey, medium evidence)

Mandatory survey data show low baseline adoption and emphasize complementary digital infrastructure and management practices.

Advertising & marketplace systems

Distilled and re-represented data tend to perform well in production. Compact retrievers and ID-free multimodal codes are reported to maintain quality while cutting latency and improving engagement, and hybrid generative-RL control sometimes adds lift when paired with safe fallbacks. The credible thread is design for the constraint you have, then validate with rigorous online tests. Three-arm experiments further indicate audience shifts often come from delivery algorithms, not creatives, which helps target where to optimize. Editorially, these findings imply platforms should prioritize model fit and experiment design before defaulting to larger base models.

Labor markets, exposure measurement, and adoption

Exposure is heterogeneous and mobile. Firms reduce measured “exposure” by shifting hiring across roles and redesigning tasks, while cross-country task maps show poorer countries face more substitution risk even as richer ones see broader exposure. Platform logs can overstate workforce exposure unless reweighted, and that reweighting changes effect sizes enough to sway policy narratives in the studies reviewed. Adoption continues to concentrate in digitally mature firms and in specific sectors, consistent with complementarities between infrastructure, management, and skills.

Human–AI collaboration, productivity, and competence

AI raises average performance but not for everyone or every task. Gains accrue to users with high interaction competence, many overestimate speed benefits, and agentic tools falter on complex, chained workflows without verification. Where structure and formal-verification systems exist, agentic loops can accelerate verified engineering; where uncertainty and tails dominate, more capable models can over-extrapolate in tested settings. The trajectory points to investing in training, calibration, and verification rather than assuming capability generalizes across tasks.

Measurement, evaluation bias, and experimental design

Measurement choices often drive conclusions. Two-arm A/B tests can confound delivery with content effects, exposure metrics often mirror platform user mixes, and LLM evaluators inherit bias from conversational history. Remedies are available: three-arm designs, population reweighting, and context controls for evaluators. Practically, the limiting factor is operational feasibility and clarity on which population is policy-relevant, but the direction of travel is toward identification-aware experimentation and transparent assumptions.

Small models, real dollars established

Delivery is the hidden treatment established

Exposure metrics need reweighting suggestive

Capability can backfire on tail risk suggestive

The speedup illusion established

Three-phase SLM distillation for retrieval

HARNESS-LM

Three-arm experimental design for adaptive platforms

Algorithm or Creative?

Cross-provider conversational-bias audit

AMEL