Agentic Design

Patterns
🏛️

AISI Evaluation Framework(AISI-Eval)

AI Safety Institute's comprehensive evaluation framework for frontier AI systems, integrating with NIST ARIA program for government-standard safety assessment.

Complexity: highEvaluation and Monitoring

🎯 30-Second Overview

Pattern: Government-standard evaluation framework for frontier AI systems with three-tier progressive testing

Why: Ensures systematic safety assessment before deployment, enables international coordination, prevents high-risk AI misuse

Key Insight: Capability thresholds + Expert red-teaming + Preregistered evaluation = Safe frontier AI deployment

⚡ Quick Implementation

1Risk Model:Define threat scenarios & capability thresholds
2Tier Testing:Automated → Manual → Expert red-teaming
3Evaluate:Capability, misuse, societal impact assessment
4Threshold:Compare results against safety thresholds
5Decision:Deploy, mitigate, or restrict based on assessment
Example: preregister → tier_1_auto → tier_2_manual → tier_3_expert → safety_decision

📋 Do's & Don'ts

✅Preregister evaluation design before testing
✅Use three-tier progressive evaluation system
✅Combine automated and expert red-teaming
✅Test for capability thresholds indicating severe risks
✅Implement rigorous information security protocols
❌Rely solely on automated evaluations for high-risk models
❌Skip expert red-teaming for frontier systems
❌Ignore societal impact and misuse potential
❌Deploy without meeting safety threshold requirements
❌Overlook international coordination standards

🚦 When to Use

Use When

  • • Frontier AI model deployment
  • • Government compliance requirements
  • • International safety coordination
  • • High-capability system assessment

Avoid When

  • • Low-risk AI applications
  • • Non-frontier model evaluation
  • • Simple automation tasks
  • • Resource-constrained environments

📊 Key Metrics

Capability Score
Autonomous capability assessment (0-100%)
Misuse Potential
Risk of malicious use exploitation
Safety Threshold
Pass/fail against predefined limits
Expert Assessment
Red-team evaluation outcomes
Societal Impact
Broad societal risk evaluation
Compliance Rate
International standard adherence

💡 Top Use Cases

Frontier AI Safety Assessment: GPT-5/Claude-4 level models requiring government approval
International Coordination: Multi-country safety evaluation protocols (UK-US-EU)
Regulatory Compliance: Meeting AI Safety Institute requirements for deployment
Capability Threshold Testing: Identifying dangerous autonomous capabilities
Red-team Security Assessment: Expert-led adversarial testing for misuse prevention

References & Further Reading

Deepen your understanding with these curated resources

Official AISI Publications

Early lessons from evaluating frontier AI systems - AISI (2024)

Conference on frontier AI safety frameworks - AISI (2024)

AISI Inspect Tool - Open Source AI Safety Evaluations

Frontier AI Safety Commitments - AI Seoul Summit 2024

International Network & Standards

International Network of AI Safety Institutes - US Commerce Dept (2024)

AI Safety Institute International Network - CSIS Analysis (2024)

Progress Update: Advancing Frontier AI Safety - Frontier Model Forum (2024)

AI Safety Fund - $1M Research Collaboration (2024)

Research & Technical Implementation

Preregistration for AI Safety Evaluations - Research Methodology

Capability Thresholds for AI Safety - Technical Framework

Expert Red-teaming of Frontier AI Systems - Best Practices

AI Agent Evaluation in Sandbox Environments - AISI Technical Report

Policy & Governance

AI Safety Institute Global Landscape - All Tech Is Human (2024)

2025 AI Safety Index - Future of Life Institute

AI Safety Institute - Wikipedia Overview

UK AI Security Institute - Name Change Announcement (Feb 2025)

Contribute to this collection

Know a great resource? Submit a pull request to add it.

Contribute

Patterns

closed

Loading...