From positions to tests
THE QUESTION WE STARTED WITH
From impressive demos to a service people can rely on
A customer support study found a 15% average productivity gain, while METR’s early in 2025 developer trial found a slowdown. METR’s February 2026 update warns that selection and concurrent agents make newer estimates unreliable. The practical comparison must include complete cases, checking and omitted work.
THE NEXT QUESTION
Can this service resolve more requests without shifting mistakes or extra work onto the people it serves?
Choose one low risk municipal information process. Compare matched requests with and without assistance, protect personal data, and preserve an ordinary human route. Count successful completion and total staff effort by case type.
Organizing a discussion on this question?
Use these findings and sources as the starting point for a panel, interview, email exchange or small roundtable.
What this means in practice
Day 1 asked who benefits when AI changes work. The workplace studies now make that question testable. Brynjolfsson, Li and Raymond find a 15% average increase in resolved customer support issues per hour, with gains varying by experience. METR’s early in 2025 developer trial records a slowdown in a very different task setting. These findings cannot be averaged into a forecast for a Greek organisation. They show why the workflow, worker and quality standard belong in the claim. METR’s February 2026 update adds another complication: the newer study is distorted by who participates and which tasks people submit. We need to evaluate every request in the queue, including difficult cases and the work required to finish them. A municipal service could compare similar requests, record every attempted case and count verification, correction, repeat contact and escalation. A fast first draft may be valuable, but it has not improved the service if the citizen returns twice or the employee spends the saved time repairing it. Start with a small trial that can be stopped. Decide beforehand which failures require a pause and how excluded or abandoned cases will appear in the record.
What remains open: These settings and tools do not predict a Greek service’s result. No local trial or universal productivity effect is established.
What the sources say (5)
Dario Amodei
If AI can absorb company knowledge and be replicated, why should integrating it into the economy be harder than hiring people?
Model capability and deployment move on different curves. Even unusually fast adoption is constrained by enterprise rollouts and organisational decisions.
Position, paraphrased · 13 February 2026Read the original answer ↗Satya Nadella
If Microsoft no longer has access to OpenAI models, what gives you confidence it can build and maintain a leading AI lab?
Agents need identities, observability, storage and tools. Infrastructure should support several model families because architecture and supplier advantages can change.
Position, paraphrased · 12 November 2025Read the original answer ↗Brynjolfsson, Li and Raymond
How did access to an AI assistant change customer support work?
In one customer support firm, AI assistance increased resolved issues per hour by 15% on average; effects differed by experience. This is evidence about a particular workflow, not a universal productivity forecast.
Position, paraphrased · 4 February 2025Read the original answer ↗METR research team
Did the tools reduce actual task time for these experienced developers?
A randomized trial found that 16 experienced developers took 19% longer on their own repository tasks with early in 2025 AI tools. It is a dated, narrow study; METR subsequently published a methodological update.
Position, paraphrased · 10 July 2025Read the original answer ↗METR research team
What happens to measurement when people avoid doing some tasks without AI?
METR reports that selective participation, task selection and concurrent agents make its newer productivity estimates unreliable. A local test should record which work is omitted as well as which work is timed.
Position, paraphrased · 24 February 2026Read the original answer ↗Watch the original published conversations
We are near the end of the exponential
dwarkesh.comOriginal recording and context ↗How Microsoft is preparing for AGI
dwarkesh.comOriginal recording and context ↗- 01A concrete case
Choose one low risk municipal information process. Compare matched requests with and without assistance, protect personal data, and preserve an ordinary human route. Count successful completion and total staff effort by case type.
- 02Evidence that could change the answer
Case records with personal identifiers removed, comparable baseline, total staff minutes, case type and reason for exclusion.
- 03A question we can answer
A new example can change the answer. What would it tell us to ask next?
WHO CAN HELP ANSWER THIS?
The people closest to the problem.
Frontline employee and service owner
Show the entire case queue, including cases excluded, abandoned or sent to a colleague. Where did checking and repair time go?
Case records with personal identifiers removed, comparable baseline, total staff minutes, case type and reason for exclusion.Service user and accessibility representative
Could you complete the process, understand the reply and get a person to correct an error?
One concrete journey, repeat contacts, unresolved issues and an accessible human alternative.Independent evaluator
Would the benefit remain if difficult requests and all review time were included?
Sampling method, pre agreed quality rubric, missing case report and adverse outcome record.Each perspective can fill a different gap in what we know.
Follow the question across the network.
ThoughtLeaders.gr The attributed positions ↗AIStrategy.gr The question in this sector ↗EventChanger The conversation ↗Sources and what remains uncertain
Read the original work alongside the analysis. For the local case, we still need the evidence requested above.
- Dario AmodeiWe are near the end of the exponential ↗13 February 2026 · amodei-dwarkesh-20260213
- Satya NadellaHow Microsoft is preparing for AGI ↗12 November 2025 · nadella-dwarkesh-20251112
- Brynjolfsson, Li and RaymondGenerative AI at Work ↗4 February 2025 · qje-ai-at-work-20250204
- METR research teamMeasuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity ↗10 July 2025 · metr-developer-productivity-20250710
- METR research teamWe are Changing our Developer Productivity Experiment Design ↗24 February 2026 · metr-uplift-design-20260224