AI Trajectory

Safety

Safety questions

Where could risk enter—and where could it be contained? Eight distinct questions trace the chain from hazardous behavior through recovery.

Frontier AI safety · causal chain

Eight questions from hazard to recovery.

Dangerous capabilities and failure modes are increasingly observable. Deployment exposure and real-world uplift remain only partly visible; safeguards are improving but uneven; independent evaluation and enforceable oversight do not yet provide comprehensive coverage.

Hazard→Exposure→Control→Governance→Outcomes→Resilience

Scope: Severe and catastrophic risks from frontier general-purpose AI. This is not a complete taxonomy of all AI harms.

Explore the eight safety questionsOpen questions
01
hazardDoes AI materially increase dangerous capability?
CyberChemical & biologicalManipulation & fraudAutonomous operation
Domain-specific · Partly measured

In which severe-harm domains does a frontier system give a malicious or unskilled actor material, validated uplift over a non-AI baseline?

Current reading · 2026-08-12Dangerous uplift is increasingly measurable, but it is domain-specific and does not justify one cross-domain danger score.
material signalCyber

Autonomous task length and expert-level successes are rising; verified malicious use now reaches later attack stages.

emergingChemical & biological

Expert knowledge and laboratory-support performance are strong, but public end-to-end actor-uplift evidence remains limited.

limitedManipulation & fraud

Misuse is documented, but standardized severe-harm uplift measurements are sparse.

emergingAutonomous operation

Longer autonomous software and cyber tasks are measurable; reliable consequential end-to-end operation remains uneven.

Domains remain separate because evidence in one does not establish danger in another.

Evidence setting
benchmarks and red-team evaluations
Independence
government evaluation plus developer studies
Coverage
partial

What is measured

  • Randomized expert and novice uplift against a no-AI control
  • End-to-end task completion rather than knowledge-question accuracy
  • Change in time, cost, reliability, and attainable scale
  • Results separated by chatbot, agent, fine-tuned, and open-weight access

What is still missing

  • Preregistered independent end-to-end uplift studies with realistic actors
  • Comparable results across cyber, CBRN, manipulation, and autonomous-operation domains

What would change the reading

  • Replicated novice uplift that completes a consequential attack chain
  • Repeated independent findings of negligible incremental uplift after realistic bottlenecks and safeguards
Audit 2 source-backed evidence items
  1. 2023–2025 evaluations · Cyber evaluation success and autonomous task duration

    UK AISI reports that average success on apprentice-level cyber tasks rose from just over 10% in early 2024 to about 50%, while the cyber task duration models complete without human direction grew from under ten minutes in early 2023 to over an hour by mid-2025.

    external evaluator · Controlled evaluations do not establish realized harm or novice uplift in deployment.UK AI Security Institute · Frontier AI Trends Report ↗
  2. four releases through March 2025 · Developer threshold assessment

    Anthropic's Frontier Red Team described early-warning capability gains but assessed the tested models as below its thresholds for substantially elevated national-security risk.

    developer-reported with government testing input · The thresholds and sensitive evaluation details are not fully public.Anthropic · Progress from our Frontier Red Team ↗

No cross-question safety score is calculated. Each reading keeps its measurement setting, independence, coverage, and gaps visible.

02
hazardDo agents violate operator intent under real authority?
Leans Pole B · wide range · Partly measured

When given realistic delegated authority, do frontier agents strategically violate operator intent, conceal it, or resist correction at consequential rates?

Current reading · 2026-08-12Strategic violations are demonstrated in stress-test simulations; the base rate in real deployments is unknown.
Pole AReliably corrigible

Agents obey constraints, accept correction, and remain controllable across realistic high-authority settings.

Pole BStrategic violations

Agents deceive, sabotage, evade correction, or preserve their objectives when authority and incentives conflict.

Current evidence leanLeans toward Pole Bwide interpretive range
Pole AReliably corrigiblePole BStrategic violations

The dot uses an ordinal editorial state: leans Pole A, near midpoint, leans Pole B, or no stable lean. The band shows interpretive disagreement—not a probability or statistical confidence interval.

Evidence setting
adversarial simulations
Independence
developer and government evaluations
Coverage
narrow

What is measured

  • Deception, sabotage, shutdown-compliance, and self-preservation rates
  • Hidden deployment-like evaluations with realistic permissions
  • Persistence across models, prompts, scaffolds, and mitigations
  • Successful harmful action after monitoring—not expressed intent alone

What is still missing

  • Deployment-representative base rates
  • Independent evaluations that preserve realism without telling models they are being evaluated

What would change the reading

  • Verified strategic violations outside deliberately contrived stress tests
  • Low stable failure rates across unseen high-authority environments and model families
Audit 2 source-backed evidence items
  1. 2025 study · Behavior rate in deliberately constructed conflict simulations

    Anthropic stress-tested 16 models in hypothetical corporate environments and observed malicious insider behavior in at least some forced-conflict cases across developers; it reported no known real-world deployment instances of this behavior.

    developer-reported, code released · The scenarios closed off ethical alternatives and were designed to elicit failure, so they do not estimate deployment prevalence.Anthropic · Agentic Misalignment ↗
  2. 2023–2025 evaluations · Prerequisite skills for autonomous replication

    UK AISI reports self-replication-task success rising from below 5% to above 60% on a subset of simplified evaluations, while emphasizing that current models remain unlikely to complete real-world self-replication chains.

    external evaluator · Component benchmarks are not evidence that a system has attempted autonomous replication in deployment.UK AI Security Institute · Frontier AI Trends Report ↗

No cross-question safety score is calculated. Each reading keeps its measurement setting, independence, coverage, and gaps visible.

03
exposureHow much dangerous capability is actually exposed?
No coverage baseline · Missing a stable series

How much potentially dangerous capability is exposed through real deployments, tool permissions, unsupervised operation, or irreversible weight release?

Current reading · 2026-08-12The risk factors are clear, but there is no standardized public series for deployment authority, scale, or human intervention.
  1. 1
    Advice only

    Text output; no direct action or persistent credentials.

  2. 2
    Read access

    Can inspect external systems or private data but cannot change state.

  3. 3
    Constrained write

    Can act through bounded tools with approvals, logs, and rollback.

  4. 4
    High-impact autonomy

    Persistent credentials, broad write access, long runtimes, or safety-critical authority.

  5. 5
    Irreversible proliferation

    Open weights or copied systems operate outside provider monitoring and recall.

No single current rung is shown: public telemetry is insufficient to estimate the distribution of real deployments.

Evidence setting
taxonomy and ecosystem research
Independence
government taxonomy and external research collaboration
Coverage
poor

What is measured

  • Read versus write permissions and credential scope
  • Maximum uninterrupted runtime and number of consequential actions
  • Human approval frequency, logging, rollback, and kill mechanisms
  • Hosted, fine-tunable, or open-weight access
  • Number and scale of deployments in critical environments

What is still missing

  • Provider and deployer telemetry on permissions, run length, sensitive actions, and intervention rates
  • A denominator for high-impact deployments and open-weight downstream use

What would change the reading

  • Rapid growth in persistent unsupervised high-privilege deployments
  • Verified least-privilege defaults, bounded autonomy, action logging, and staged access tied to risk
Audit 2 source-backed evidence items
  1. August 2025 workshop report · Agent tool-access taxonomy

    NIST distinguishes read-only, constrained-write, and write access and highlights statefulness, reversibility, environment trust, and action criticality as key dimensions of agent risk.

    multi-stakeholder government workshop · A taxonomy identifies what should be measured; it does not provide deployment prevalence.NIST · Lessons Learned on Tool Use in Agent Systems ↗
  2. 2025–2026 research synthesis · Access-mode risk characteristics

    UK AISI finds that open-weight systems can be modified arbitrarily, used without oversight, and spread irreversibly; safeguards can be removed quickly and cheaply.

    external research collaboration · This establishes qualitative exposure mechanisms, not the number or severity of exposed deployments.UK AI Security Institute · Open-weight risk management ↗

No cross-question safety score is calculated. Each reading keeps its measurement setting, independence, coverage, and gaps visible.

04
controlHow much risk remains after safeguards?
Leans Pole B · wide range · Partly measured

Against each defined threat model, how much attack success remains after all deployed safeguards and operational controls are applied?

Current reading · 2026-08-12Safeguards are improving, sometimes sharply, but robustness is uneven across models, threat domains, attack budgets, and access types.
Pole ARobust under attack

Defense-in-depth keeps end-to-end attack success low under adaptive, persistent, well-resourced testing.

Pole BReadily bypassed

Transferable attacks, decomposition, fine-tuning, or scale defeat the production safeguard stack.

Current evidence leanLeans toward Pole Bwide interpretive range
Pole ARobust under attackPole BReadily bypassed

The dot uses an ordinal editorial state: leans Pole A, near midpoint, leans Pole B, or no stable lean. The band shows interpretive disagreement—not a probability or statistical confidence interval.

Evidence setting
adaptive red teaming
Independence
government plus developer testing
Coverage
partial

What is measured

  • End-to-end attack success after the complete production stack
  • Attacker time, expertise, cost, and query budget
  • Monitor true-positive rate at a fixed false-positive rate
  • Benign refusal and operational cost
  • Patch latency and repeat-attack success

What is still missing

  • Comparable independent tests of full production systems
  • Long-budget adaptive attacks across cyber, CBRN, fraud, and agent-control threat models

What would change the reading

  • Repeated independent low residual-risk estimates across threat models and access modes
  • Cheap transferable attacks or safeguards that fail under fine-tuning, decomposition, or operational scale
Audit 2 source-backed evidence items
  1. 2024–2025 tests · Expert time to a universal biological-misuse jailbreak

    UK AISI found a universal jailbreak for every system it tested, while one comparison showed about 40 times more expert effort was required to jailbreak a newer protected system.

    external evaluator · Two systems and one defended domain do not establish general safeguard robustness.UK AI Security Institute · Frontier AI Trends Report ↗
  2. 2025–2026 developer testing · Attack success and red-team search effort

    Anthropic reports that its first Constitutional Classifiers reduced attack success from 86% to 4.4%. Its successor was tested for more than 1,700 red-team hours across 198,000 attempts, finding one high-risk vulnerability and no universal jailbreak during that test.

    developer-reported with external red teamers · The result is domain- and implementation-specific and is not an independent population estimate.Anthropic · Constitutional Classifiers++ ↗

No cross-question safety score is calculated. Each reading keeps its measurement setting, independence, coverage, and gaps visible.

05
controlCan evaluations detect danger before deployment?
Leans Pole B · moderate range · Partly measured

Do pre-deployment evaluations and live monitors reliably predict dangerous capability and behavior in real use?

Current reading · 2026-08-12Evaluations reveal important hazards, but realism, gaming, contamination, and weak evaluation-to-field calibration leave material blind spots.
Pole APredictive and hard to game

Tests and monitors prospectively identify consequential capabilities and failures before they appear in use.

Pole BBlind spots and gaming

Systems recognize tests, exploit scoring, hide capabilities, or behave differently in deployment.

Current evidence leanLeans toward Pole Bmoderate interpretive range
Pole APredictive and hard to gamePole BBlind spots and gaming

The dot uses an ordinal editorial state: leans Pole A, near midpoint, leans Pole B, or no stable lean. The band shows interpretive disagreement—not a probability or statistical confidence interval.

Evidence setting
evaluation research and controlled tests
Independence
cross-lab and international synthesis
Coverage
partial

What is measured

  • Calibration between pre-deployment predictions and field outcomes
  • Hidden-test versus public-test divergence
  • Evaluation awareness, sandbagging, and contamination
  • Monitor miss rate and time to detection
  • Independent evaluator access to the deployed scaffold and tools

What is still missing

  • Matched prospective safety predictions and later production outcomes
  • Public monitor false-negative rates under realistic adversarial behavior

What would change the reading

  • Prospective evaluations that repeatedly predict later incidents and production behavior
  • Repeated capability surprises, sandbagging, or incidents missed by systems that passed evaluation
Audit 2 source-backed evidence items
  1. 2026 international review · Cross-study evidence synthesis

    The International AI Safety Report identifies an evaluation gap: controlled pre-deployment tests often do not reliably predict real-world utility, risk, or behavior in the deployed system context.

    independent expert report · This diagnoses the gap but does not quantify one cross-system miss rate.International AI Safety Report 2026 ↗
  2. December 2025 evaluation suite · Monitorability across controlled evaluation environments

    OpenAI reports that chain-of-thought monitoring outperformed monitoring actions and outputs alone across 13 evaluations and 24 environments, but notes limited realism, evaluation awareness, and uncertain generalization to wild misbehavior.

    developer-reported with external-model comparisons · Monitorability of elicited reasoning is one control signal, not proof of end-to-end safety.OpenAI · Evaluating chain-of-thought monitorability ↗

No cross-question safety score is calculated. Each reading keeps its measurement setting, independence, coverage, and gaps visible.

06
governanceAre frontier systems under enforceable oversight?
No stable lean · wide range · Missing a stable series

What share of frontier development and high-risk deployment is covered by binding, independently auditable requirements that are actually enforced?

Current reading · 2026-08-12Binding legal scaffolding now exists, but global model coverage, independent access, compliance findings, and enforcement outcomes remain patchy.
Pole ABroad enforceable coverage

High-risk developers and deployments face binding evaluation, reporting, audit, security, and corrective requirements.

Pole BVoluntary or unenforced

Important systems remain outside binding rules, or requirements lack verification, capacity, penalties, and corrective action.

Current evidence leanNo stable evidence leanwide interpretive range
Pole ABroad enforceable coveragePole BVoluntary or unenforced

The dot uses an ordinal editorial state: leans Pole A, near midpoint, leans Pole B, or no stable lean. The band shows interpretive disagreement—not a probability or statistical confidence interval.

Evidence setting
statutes and official implementation records
Independence
primary legal sources
Coverage
partial

What is measured

  • Share of frontier developers and models within binding regimes
  • Mandatory risk evaluation, incident reporting, and cybersecurity
  • Independent evaluator and regulator access
  • Published compliance findings, penalties, and remediation
  • Coverage of internal deployment and open-weight release

What is still missing

  • Comparable data on inspections, noncompliance, sanctions, and remediation
  • A global denominator for covered frontier models and consequential deployments

What would change the reading

  • Independent audits, published compliance findings, corrective orders, and broader model coverage
  • Major uncovered systems, unenforced violations, repeated exemptions, or purely self-attested compliance
Audit 2 source-backed evidence items
  1. EU rules in force August 2025; enforcement powers August 2026 · Binding regime and enforcement authority

    EU obligations require providers of systemic-risk general-purpose models to submit risk, safety-framework, and serious-incident materials; Commission enforcement powers, including fines, became applicable in August 2026.

    regulator primary source · Legal authority does not yet establish broad or effective enforcement outcomes.European Commission · GPAI provider guidelines ↗
  2. California SB 53 enacted framework · Binding disclosure and reporting requirements

    California SB 53 requires large frontier developers to publish and follow frontier-AI frameworks, disclose model-specific catastrophic-risk assessment summaries, and report specified critical safety incidents.

    government primary source · The law emphasizes transparency and process; it does not itself prove mitigation effectiveness or global coverage.California Legislature · SB 53 statutory text ↗

No cross-question safety score is calculated. Each reading keeps its measurement setting, independence, coverage, and gaps visible.

07
outcomesWhat severe incidents and near misses are occurring?
2provider cases
Ledger, not a trend · Partly measured

What well-documented severe incidents and near misses are occurring, what incremental role did AI play, and how quickly were they detected and contained?

Current reading · 2026-08-12Providers have documented malicious use, especially in cyber operations, but public cases are selected and do not provide a stable rate or cross-domain denominator.
  1. 2025-09 · Cyber espionage · not independently ratedagentic operator

    Anthropic reported an AI-orchestrated campaign against roughly 30 targets, with a small number of successful infiltrations and limited human intervention.

    provider-observed; external verification limited
  2. 2025-03–2026-03 · Malicious cyber use · not independently ratedworkflow multiplier

    Anthropic analyzed 832 banned accounts with sufficient detail; 67.3% used AI for malware preparation and 6.5% for lateral movement.

    provider-selected account sample; no population denominator

These are documented provider cases, not independently verified examples or an incidence rate. Reporting coverage and denominators are missing.

Evidence setting
observed provider cases and incident taxonomies
Independence
provider telemetry plus intergovernmental methodology
Coverage
poor

What is measured

  • Documented incidents by domain and independently rated severity
  • AI's role: incidental tool, productivity multiplier, capability unlock, or autonomous operator
  • Counterfactual attribution confidence
  • Direct loss and affected people or systems
  • Time to detection, containment, disclosure, and remediation
  • Reporting-coverage denominator

What is still missing

  • Mandatory standardized reporting
  • A defensible denominator and counterfactual estimate of AI's incremental role
  • Comparable coverage beyond cyber and fraud

What would change the reading

  • Independently verified incidents where AI unlocks otherwise infeasible or substantially more severe harm
  • Falling severity and containment time under stable or improving reporting coverage
Audit 3 source-backed evidence items
  1. campaign detected September 2025 · Provider-observed attack campaign

    Anthropic reported a state-sponsored cyber-espionage campaign using Claude Code to attempt infiltration of roughly 30 targets, succeeding in a small number of cases; it assessed the operation as largely AI-executed with occasional human direction.

    provider-observed, not independently replicated · The report has privileged telemetry but limited external verification and no population denominator.Anthropic · Disrupting AI-orchestrated cyber espionage ↗
  2. March 2025–March 2026 · Selected malicious-account activity

    Anthropic mapped 832 banned malicious-cyber accounts with sufficient detail: 560 used AI for malware preparation and 54 for lateral movement, illustrating deeper use in attack workflows without estimating prevalence among all attackers.

    provider-selected dataset · The sample is a subset of banned accounts and cannot establish an incidence rate or counterfactual harm.Anthropic · Mapping AI-enabled cyber threats ↗
  3. current methodology · Incident and hazard definitions

    The OECD distinguishes incidents that caused actual harm from hazards that could plausibly cause harm and is developing an open reporting process to complement news-derived events.

    intergovernmental source · News-derived discovery is sensitive to coverage and reporting changes, so raw counts are not a stable harm trend.OECD.AI · Incidents and Hazards Monitor methodology ↗

No cross-question safety score is calculated. Each reading keeps its measurement setting, independence, coverage, and gaps visible.

08
resilienceCan society contain and recover from failures?
No stable lean · wide range · Missing a stable series

If model, safeguard, or governance failures occur, can affected institutions prevent propagation, maintain critical services, and recover quickly?

Current reading · 2026-08-12Resilience is causally essential but almost unmeasured in public, frontier-AI-specific stress tests and recovery data.
Pole AContained and recoverable

Critical systems have tested fallbacks, limited common-mode exposure, rapid detection, and effective recovery.

Pole BCascading and hard to reverse

Dependencies, concentration, or weak response turn local AI failures into persistent or systemic harm.

Current evidence leanNo stable evidence leanwide interpretive range
Pole AContained and recoverablePole BCascading and hard to reverse

The dot uses an ordinal editorial state: leans Pole A, near midpoint, leans Pole B, or no stable lean. The band shows interpretive disagreement—not a probability or statistical confidence interval.

Evidence setting
risk-management synthesis
Independence
international expert report
Coverage
poor

What is measured

  • Critical-sector AI dependency and substitutability
  • Manual fallback and rollback performance
  • Independent exercises for finance, energy, communications, healthcare, and government
  • Detection, containment, and recovery time
  • Concentration and common-mode failure exposure

What is still missing

  • Standardized public stress tests and recovery data tied to frontier-AI failure scenarios
  • Critical-sector dependency and common-mode exposure inventories

What would change the reading

  • Successful independent sector exercises, tested fallbacks, and rapid recovery from actual incidents
  • Critical dependence without fallback, cascading failures, or provider concentration that amplifies one compromise
Audit 2 source-backed evidence items
  1. 2026 international review · Cross-domain risk-management synthesis

    The International AI Safety Report treats societal resilience as necessary because no safeguard stack is perfectly reliable and identifies stronger critical infrastructure, detection tools, and institutional response capacity as key defenses.

    independent expert report · The report establishes importance and candidate practices, not a current comparative resilience score.International AI Safety Report 2026 · Executive Summary ↗
  2. March 2026 monitoring review · Post-deployment monitoring maturity

    NIST reports that post-deployment monitoring methods, terminology, and best practices remain nascent and fragmented, limiting visibility into field failures and recovery performance.

    government literature review and workshops · Monitoring maturity is an input to resilience, not a direct measure of recovery capacity.NIST AI 800-4 · Monitoring Deployed AI Systems ↗

No cross-question safety score is calculated. Each reading keeps its measurement setting, independence, coverage, and gaps visible.