The questions we take into the world
THE QUESTION WE STARTED WITH
The authority to intervene
Before the service goes live, agree who can pause it, how people get help and what must be checked before it restarts. Test what happens if the decision maker or the necessary logs are unavailable, using synthetic records.
THE NEXT QUESTION
Who can stop this application without the supplier, and what evidence must they see before putting it back into service?
For the municipal case, ask the service owner, security lead and procurement team to agree who may suspend the assistant, how citizens are served meanwhile and which evidence permits restart. Check local obligations through the appropriate responsible teams.
Organizing a discussion on this question?
Use these findings and sources as the starting point for a panel, interview, email exchange or small roundtable.
What this means in practice
A stop button is meaningful only if someone can use it in time and essential work survives. Day 2’s distinction also changes the restart question. Blocking an unauthorised action does not establish that advice is accurate; restoring a server does not show that affected cases have been repaired. NIST recommends defined deactivation and escalation arrangements, while CaMeL’s specified security model illustrates why a technical control has boundaries. Each service needs an agreed plan for responding to incidents. A named duty role needs authority to restrict or suspend the application without waiting for a supplier’s commercial approval. The record should separate detection, containment, user assistance and restart. Each step needs an owner and evidence available to a reviewer. A supplier may provide essential technical information, but the service must still decide whether the failure has been contained and who was affected. In a safe rehearsal, ask what happens when the ordinary approver is absent or logs cannot be obtained. The first unresolved dependency becomes the next question. The operator’s legal authority and ability to act still need to be checked. The goal is an intervention that can actually be exercised, with the reason for each change preserved.
What remains open: Contractual authority, applicable duties, log access and staff availability need confirmation. The proposed exercise has yet to be run, and the operator’s legal authority remains to be checked.
What the sources say (4)
Yoshua Bengio
The catastrophic risks of AI and a safer path
Bengio distinguishes useful intelligence from goal directed agency and proposes scientist AI as a route to safer systems. His argument challenges the assumption that more useful AI must also receive more autonomous control.
Position, paraphrased · 8 April 2025Read the original answer ↗Michalis Bletsas
Cyberattacks, critical infrastructure and national resilience
Bletsas treats resilience as a national and economic capability. Limited resources should protect the most consequential dependencies, with response and restoration planned before an attack.
Position, paraphrased · 4 November 2026Read the original answer ↗NIST
Which responsibilities and tests make intervention and recovery practical?
The voluntary profile recommends rehearsed incident response, fallback arrangements and defined responsibilities for disabling unsuitable systems. It is a governance resource, not evidence that a particular service is safe.
Position, paraphrased · July 2024Read the original answer ↗Debenedetti and colleagues
Can security boundaries prevent retrieved text from authorizing an agent’s actions?
CaMeL separates trusted instructions from untrusted material and constrains tool permissions. Its benchmark results concern a specified threat model; they do not eliminate misleading content or demonstrate safety in every real deployment.
Position, paraphrased · 24 June 2025Read the original answer ↗Watch the original published conversations
The catastrophic risks of AI — and a safer path
ted.comOriginal recording and context ↗- 01A concrete case
For the municipal case, ask the service owner, security lead and procurement team to agree who may suspend the assistant, how citizens are served meanwhile and which evidence permits restart. Check local obligations through the appropriate responsible teams.
- 02Evidence that could change the answer
A written agreement on who can act, backup arrangements that work during a normal shift, a substitute decision maker and a way to tell users what has happened.
- 03A question we can answer
A new example can change the answer. What would it tell us to ask next?
WHO CAN HELP ANSWER THIS?
The people closest to the problem.
Duty manager and service owner
Can you pause the assistant now, keep the service running and identify the person covering your absence?
A written agreement on who can act, backup arrangements that work during a normal shift, a substitute decision maker and a way to tell users what has happened.Supplier, security reviewer and procurement lead
Which incident records and controls remain available if your support is delayed or the contract is disputed?
Log access terms, retention rules, containment test and agreed restart evidence.Frontline worker and affected user representative
How will people with wrong answers be found, informed and helped to correct the consequences?
Privacy respecting case correction procedure, escalation owner and completion record.Each perspective can fill a different gap in what we know.
Follow the question across the network.
ThoughtLeaders.gr The attributed positions ↗AIStrategy.gr The question in this sector ↗EventChanger The conversation ↗Sources and what remains uncertain
Read the original work alongside the analysis. For the local case, we still need the evidence requested above.
- Yoshua BengioThe catastrophic risks of AI — and a safer path ↗8 April 2025 · bengio-ted-20250408
- Michalis BletsasInterview on cyberattacks and national resilience ↗4 November 2026 · bletsas-resilience-2026
- NISTArtificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile ↗July 2024 · nist-genai-profile-202407
- Debenedetti and colleaguesDefeating Prompt Injections by Design ↗24 June 2025 · camel-prompt-injection-20250624