← All posts
September 17, 2026 · Generated

AI Safety Evaluators Move Inside Labs

Anthropic and OpenAI are embedding independent safety evaluators directly inside their organizations, granting unprecedented access to model development. Researchers welcome the transparency but warn that meaningful oversight requires real independence and eventual regulation.

Subscribe free All posts
Top 20 AI Signals
#1
Embedded Safety Evaluators Gain Lab Access
Major AI labs are allowing independent safety researchers inside their operations for the first time. The move promises transparency but raises questions about whether evaluators can remain truly independent when embedded within the companies they're assessing.
TechGlobal
95
#2
Agent Consistency Problem Goes Mainstream
IBM Research highlights a critical reliability issue: AI agents may ace a task once but fail to replicate success consistently. This variability threatens enterprise deployment where predictable performance is non-negotiable.
TechManufacturingFinance & BankingGlobal
88
#3
Al Gore Reframes AI Risk Discussion
The former VP argues data center emissions aren't the real AI risk—the industry's own warnings about technology trajectory matter more. This shifts focus from infrastructure concerns to fundamental capability risks.
EnergyTechGlobal
85
#4
WebGPU Kernels Enable Local AI
Hugging Face released 200+ WebGPU kernels that let developers run AI models locally in browsers without server dependencies. This democratizes access and reduces infrastructure costs for smaller teams.
TechEducation & EdTechGlobal
82
#5
Voice Simulation Platform Raises $18M
Iceland's Treble secured funding for its voice simulation platform used by AI model developers, wearable makers, and robotics companies. The technology helps test voice AI in realistic acoustic environments before deployment.
TechHealthcareEurope
78
#6
Benchmark Validity Under Scrutiny
AllenAI's BenchMIRT research questions what LLM benchmarks actually measure, challenging assumptions about model evaluation. The work suggests current metrics may not reflect real-world capability as accurately as believed.
TechEducation & EdTechGlobal
76
#7
AI Safety Refuses Topic Subsets
New research explores how models should refuse specific harmful aspects of topics rather than entire subject areas. This nuanced approach aims to prevent overly cautious AI that blocks legitimate queries.
TechHealthcareFinance & BankingGlobal
74
#8
GRPO Fine-Tuning Achieves Structured Outputs
A 350M parameter model was fine-tuned for better structured outputs in just 100 GRPO steps. The efficiency breakthrough makes advanced output formatting accessible to smaller teams with limited compute.
TechFinance & BankingGlobal
71
#9
Async GRPO Eliminates NCCL Dependency
Hugging Face demonstrated async GRPO with LoRA across distributed jobs using simple buckets and proxies instead of complex NCCL communication. This simplifies multi-node training infrastructure significantly.
TechGlobal
68
#10
Coding Agents Get Persistent Memory
New tooling gives coding agents memory systems that developers own and control rather than relying on vendor-locked solutions. This enables continuity across sessions and better context retention.
TechManufacturingGlobal
66
#11
Multimodal Multilingual Encoder Launched
NeoMME offers efficient multimodal processing across languages with native architecture design. The model handles text and images together without translation bottlenecks.
TechEducation & EdTechGlobal
64
#12
Meta Develops Camera-Free Smart Glasses
After privacy backlash over 'perv glasses' accusations, Meta is preparing smart glasses without cameras. The move acknowledges public discomfort while trying to salvage the wearable computing market.
TechGlobal
62
#13
Snap Justifies $2,200 Spectacles Pricing
Snap continues defending its expensive AR glasses months after launch, seeking use cases that justify the premium. The company faces skepticism about consumer appetite at this price point.
TechGlobal
58
#14
Coding Models Paint Watercolors
Researchers trained a coding model to generate watercolor paintings through code using TRL and OpenEnv. The cross-domain transfer demonstrates how code generation skills apply to creative tasks.
TechEducation & EdTechGlobal
56
#15
AUTOMATIC1111 Rebuilt With Gradio Workflow
The popular Stable Diffusion interface gets reconstructed using Gradio's workflow system. This modernization makes customization easier while maintaining familiar functionality.
TechGlobal
54
#16
AI Agents Join Startup Teams
TechCrunch Disrupt panel featuring Gusto and Insight Partners explores how early-stage companies integrate AI agents as team members. The discussion covers accountability and culture preservation challenges.
TechFinance & BankingNorth America
52
#17
Frontier AI Governance Remains Unclear
Inc42 reports OpenAI and Anthropic want to pace frontier AI development but questions remain about who sets the rules. Recent cybersecurity testing exposed vulnerabilities in current safety protocols.
TechGlobal
50
#18
Practo Founder Steps Down as CEO
Indian healthtech Practo's cofounder Shashank ND is replaced by former COO Jagnoor Singh in leadership reshuffle. The change comes as the company seeks renewed growth trajectory.
HealthcareTechIndia
48
#19
RentoMojo Lists at 19% Premium
Indian furniture rental startup RentoMojo debuted on stock market with strong premium, signaling investor confidence in asset-sharing models. The listing validates the rental economy in emerging markets.
TechFinance & BankingIndia
45
#20
Navi Gains UPI Market Share
Sachin Bansal's Navi reached 4.4% UPI market share in August as PhonePe and Google Pay slipped. The growth demonstrates how focused payment players can capture share from incumbents.
Finance & BankingTechIndia
42
From the Podcasts
🎙
Computer-Use Agents Bypass Missing API Infrastructure
When government systems and legacy organizations lack APIs for agent integration, computer-use capabilities allow agents to interact directly with web interfaces instead. This means agents can automate interactions with any web-based system today, regardless of whether that organization has built modern API infrastructure, effectively democratizing automation access across all web-enabled services.
~20min
E-Commerce Must Shift to Entice Agents
As agents increasingly handle shopping and procurement tasks, e-commerce websites will need to be redesigned to appeal to AI agents rather than human consumers. This represents a fundamental shift in web design philosophy where the traditional focus on visual branding and emotional marketing for humans may need to coexist with or give way to structured, agent-friendly interfaces optimized for programmatic decision-making.
~41min
Model Harnesses Blurring Into Core Models
The distinction between AI models and their surrounding harnesses (orchestration layers, tools, and interfaces) is becoming increasingly blurred, creating confusion about where model capabilities end and tooling begins. This blurring affects user experience and raises questions about the future architecture of AI systems as capabilities previously handled by external harnesses get absorbed into model behavior.
~34min
🎙
Microphone Arrays Essential for Production Voice AI
Single microphones will likely never solve real-world voice AI challenges effectively—microphone arrays are necessary for proper noise cancellation in production environments. While voice AI currently only works in perfect conditions, the prediction is that audio will become 'pretty much bulletproof' within a year when proper hardware approaches are adopted.
~2min
Training Voice Models Requires Massive Data Infrastructure
Boson AI processes approximately 100 million hours of audio data (equivalent to 200 human lifetimes) for training their voice models. They built their own data center specifically because storing this volume of audio data on cloud infrastructure would result in storage bills that are economically prohibitive for a startup.
~21min
Emotional Intelligence Gaps Limit Current AI Training
Building effective voice and avatar systems requires understanding human emotional intelligence (EQ), not just reasoning capability (IQ). If most humans lack certain interpersonal skills, those patterns won't exist in training data, meaning agentic systems will need to learn these skills through novel approaches rather than traditional data-driven training over the next 2-3 years.
~50min
Industry Deep-Dives
Healthcare
Voice simulation tech and safety refinement target clinical deployment
$18M
Treble funding for voice simulation
200+
WebGPU kernels for local AI
1
CEO transition at Practo
Voice Testing Infrastructure Gets Funding Boost
Treble's $18M raise addresses a critical gap in healthcare AI: testing voice systems in realistic environments before patient deployment. AI wearables and diagnostic tools need acoustic simulation to handle hospital noise, accents, and emotional speech patterns. This infrastructure layer reduces costly deployment failures in clinical settings.
Source: TechCrunch AI
Safety Refinement Prevents Medical Query Blocking
Research on refusing topic subsets rather than entire subjects has direct healthcare implications. Current AI systems often block legitimate medical questions due to overly broad safety filters, frustrating clinicians seeking information. The nuanced approach could let models discuss treatments while still refusing harmful misuse scenarios.
Source: Hugging Face Blog
Practo Leadership Change Signals Strategic Shift
Cofounder Shashank ND stepping down as Practo CEO for COO Jagnoor Singh marks a pivot from founder vision to operational execution. The healthtech platform faces competition from newer AI-powered diagnostic and telemedicine entrants. Singh's operational background suggests focus on efficiency and sustainable growth over expansion.
Source: Inc42
Hidden Signal
The convergence of voice simulation funding and safety refinement research reveals healthcare AI's maturation from prototype to regulated product. Companies now invest in testing infrastructure before deployment rather than iterating in production, mirroring pharma's clinical trial rigor. This pre-deployment validation phase will become mandatory as medical AI regulation tightens.
Finance & Banking
Payment infrastructure shifts as AI agents enter financial workflows
4.4%
Navi UPI market share in August
₹1,756Cr
Peak XV's Groww stake sale
100
GRPO steps for structured outputs
Navi Captures Share From Payment Giants
Navi's climb to 4.4% UPI volume as PhonePe and Google Pay slip demonstrates how focused players exploit infrastructure advantages. Sachin Bansal's fintech leverages banking license integration that competitors lack, enabling faster settlements and lower friction. The shift suggests India's payment duopoly is finally fragmenting as users prioritize experience over brand recognition.
Source: Inc42
Structured Output Training Enables Financial AI
Fine-tuning models for structured outputs in just 100 GRPO steps makes financial document processing economically viable. Banks need AI that outputs valid JSON, XML, or database records—not prose—for regulatory reporting and transaction processing. The efficiency breakthrough lets mid-sized institutions deploy custom models without hyperscaler budgets.
Source: Hugging Face Blog
AI Agents Join Financial Team Structures
The TechCrunch Disrupt discussion on AI agents as teammates has immediate banking implications as firms experiment with automated analysts and compliance officers. These agents handle repetitive tasks like transaction monitoring and document review, but accountability questions remain unresolved. Who's liable when an AI agent misses a suspicious transaction or files incorrect regulatory paperwork?
Source: TechCrunch AI
Hidden Signal
Peak XV's massive Groww stake sale coinciding with payment market shifts suggests VC firms are rotating from consumer fintech bets to infrastructure plays. The timing indicates belief that payment market share battles are decided, with value now accruing to picks-and-shovels providers like model training infrastructure and compliance AI. Expect more fintech exits and fewer consumer app investments in coming quarters.
Manufacturing
Agent reliability concerns delay factory floor deployment
1x
Agent task success variance highlighted
200+
Local AI kernels for edge deployment
0
NCCL dependencies in new training
Consistency Problem Threatens Industrial Rollout
IBM Research's finding that agents succeed inconsistently on repeated tasks strikes at manufacturing's core requirement: predictable performance. A robot that welds perfectly nine times but fails the tenth isn't just unreliable—it's dangerous and economically unviable. This variability forces manufacturers to maintain human oversight, negating automation's labor savings and limiting AI to advisory rather than executive roles on factory floors.
Source: Hugging Face Blog
Local AI Kernels Enable Edge Computing
Hugging Face's 200+ WebGPU kernels let manufacturers run AI models on edge devices without cloud connectivity. Factory environments often lack reliable internet due to security policies or physical constraints like Faraday cages and metal interference. Local inference means quality control AI continues operating during network outages, critical for continuous production lines.
Source: Hugging Face Blog
Coding Agents Gain Manufacturing Memory
Persistent memory systems for coding agents enable continuity in manufacturing automation projects that span months. Engineers can pause work on a robotic control system, return weeks later, and have the agent recall prior design decisions and constraints. This eliminates re-explaining context and accelerates iterative development of complex production systems.
Source: Hugging Face Blog
Hidden Signal
The push for local AI kernels combined with agent consistency concerns reveals manufacturing's unique deployment challenge: AI must work reliably in disconnected environments with zero tolerance for variance. This diverges sharply from consumer AI where occasional failures are acceptable and cloud connectivity is assumed. Manufacturers may pioneer a separate AI development track optimized for determinism over capability, creating a bifurcated market.
Education & EdTech
Benchmark validity questions reshape learning assessment AI
200+
Browser-based AI kernels deployed
1
Major benchmark validity study
1
Multimodal multilingual encoder
Benchmark Research Questions Assessment Tools
AllenAI's BenchMIRT study challenges what LLM benchmarks actually measure, with direct implications for educational assessment. If benchmark performance doesn't correlate with real-world capability, then AI-powered learning platforms may be optimizing for the wrong signals. This forces edtech companies to rethink how they validate that students actually learn rather than pattern-match test questions.
Source: Hugging Face Blog
Browser-Based AI Enables School Deployment
WebGPU kernels eliminate server infrastructure requirements that prevent many schools from adopting AI tools due to cost and data privacy concerns. Students can run language models, code assistants, and tutoring AI entirely in their browsers on school-issued Chromebooks. This democratizes access for underfunded districts that can't afford cloud computing bills or dedicated IT staff.
Source: Hugging Face Blog
Multilingual Encoder Breaks Language Barriers
NeoMME's efficient multimodal multilingual processing enables educational content that adapts to students' native languages without translation delays. A physics lesson can display diagrams while generating explanations in Hindi, Spanish, or Mandarin with consistent quality. This native multilingual capability makes AI tutoring viable for non-English speaking populations that current systems serve poorly.
Source: Hugging Face Blog
Hidden Signal
The collision of benchmark validity concerns and browser-based AI deployment creates an opportunity for edtech to leapfrog traditional assessment. If standardized tests don't measure real learning and schools can now run sophisticated AI locally, education could shift from periodic testing to continuous capability demonstration. Students might prove understanding through projects evaluated by local AI rather than memorizing answers to benchmark-style questions.
Tech
Safety infrastructure moves from external audit to embedded oversight
2
Major labs embedding evaluators
200+
WebGPU kernels released
350M
Parameter model fine-tuned efficiently
AI Labs Embed Independent Evaluators
Anthropic and OpenAI are granting independent safety researchers unprecedented access by embedding them directly in development processes. This moves oversight from external audits after deployment to real-time evaluation during training, potentially catching risks before models ship. However, researchers warn that independence is difficult to maintain when evaluators work inside the organizations they're assessing and receive internal funding.
Source: TechCrunch AI
Gore Reframes AI Risk Conversation
Al Gore argues data center emissions aren't the primary AI risk, instead pointing to the industry's own warnings about technology trajectory. This shifts debate from infrastructure impact—a solvable engineering problem—to fundamental questions about capability development and alignment. The intervention from a climate figure redirecting focus away from emissions toward existential risk carries significant weight in policy circles.
Source: TechCrunch AI
Distributed Training Simplified
Hugging Face's async GRPO implementation using buckets and proxies instead of NCCL makes multi-node training accessible to teams without distributed systems expertise. NCCL's complexity has been a barrier to scaling training beyond single machines, forcing reliance on managed platforms. This simplification lets smaller labs compete with infrastructure that previously required specialized engineering talent.
Source: Hugging Face Blog
Hidden Signal
Embedded evaluators and simplified distributed training are dual responses to consolidation pressure: one addresses public trust as capability concentrates, the other democratizes the compute infrastructure that creates concentration. The tension between these forces—centralization of frontier development versus democratization of training tools—will define industry structure. If distributed training becomes trivial, embedded oversight may prove insufficient as capability spreads beyond trackable organizations.
Energy
AI risk narrative shifts from data center emissions to trajectory concerns
1
Former VP reframing AI risk
200+
Local inference kernels reducing cloud load
0
Major data center announcements today
Gore Dismisses Data Center Emission Concerns
Al Gore's statement that he's not worried about AI data center emissions represents a significant shift in the climate-tech conversation. The former VP and climate advocate argues the real risk is the technology's trajectory, not its energy consumption—suggesting emissions are a tractable engineering problem. This gives AI companies cover to build infrastructure without climate movement opposition, potentially accelerating buildout.
Source: TechCrunch AI
Local AI Reduces Cloud Computing Load
WebGPU kernels enabling browser-based inference could significantly reduce data center energy consumption by shifting computation to end-user devices. Each query processed locally rather than on servers eliminates transmission energy and data center cooling overhead. At scale, this distributed architecture might reduce AI's infrastructure footprint more effectively than data center efficiency improvements.
Source: Hugging Face Blog
Training Efficiency Gains Compound
Fine-tuning capable models in just 100 GRPO steps rather than thousands reduces training energy requirements by orders of magnitude. If this efficiency extends to larger models, it breaks the assumption that better AI necessarily means exponentially more energy consumption. The decoupling of capability from energy intensity changes long-term infrastructure planning assumptions.
Source: Hugging Face Blog
Hidden Signal
Gore's dismissal of data center emissions paired with breakthrough efficiency gains suggests the energy-AI conversation is resolving faster than the timeline-AI conversation. Within 18 months, AI's energy footprint may be a solved problem through efficiency and renewables, leaving only the harder questions about capability development speed and safety. Energy sector investment should shift from AI-specific infrastructure to general grid renewable capacity.
Resource Links
Intermediate Article
Your Agent Aced the Task. Will It Do It Again?
IBM Research explores agent consistency problems that prevent reliable enterprise deployment.
https://huggingface.co/blog/ibm-research/altk-evolve-consistency
Advanced Paper
BenchMIRT: What are LLM benchmarks actually measuring?
AllenAI research questioning the validity of current LLM evaluation methods and what they actually test.
https://huggingface.co/blog/allenai/benchmirt
Intermediate Tool
Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI
Browser-based AI inference toolkit that eliminates server dependencies for model deployment.
https://huggingface.co/blog/webgpu-kernels
Advanced Tool
Give Your Coding Agents a Memory You Own
Framework for adding persistent, developer-controlled memory to coding agents.
https://huggingface.co/blog/funes
Advanced Paper
Safety for Whom? Refusing the Right Subset of a Topic
Research on nuanced AI safety that refuses harmful subsets rather than blocking entire topics.
https://huggingface.co/blog/MultiverseComputingCAI/safety-for-whom
Intermediate Article
Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps
Efficient fine-tuning method that achieves structured outputs with minimal compute.
https://huggingface.co/blog/grpo-with-trl-ifstruct
Advanced Article
Async GRPO with LoRA across HF Jobs
Simplified distributed training approach that eliminates complex NCCL dependencies.
https://huggingface.co/blog/asyncgrpo-lora-hfjobs
Intermediate Tool
NeoMME: an efficient Multimodal-native and Multilingual Encoder
Multimodal encoder that processes text and images across multiple languages natively.
https://huggingface.co/blog/Hcompany/neomme
Intermediate Article
Training a coding model to paint watercolours with TRL and OpenEnv
Demonstrates cross-domain transfer of coding capabilities to creative visual tasks.
https://huggingface.co/blog/train-to-paint-with-code
Beginner Tool
Rebuilding AUTOMATIC1111 with Gradio Workflow
Modernized implementation of popular Stable Diffusion interface with improved customization.
https://huggingface.co/blog/gradio-workflow-1111
All Article
Anthropic and OpenAI want to embed safety evaluators
Analysis of new embedded evaluator model and questions about maintaining true independence.
https://techcrunch.com/2026/09/16/anthropic-and-openai-want-to-embed-safety-evaluators-will-they-really-be-independent/
All Article
Al Gore says the real AI risk isn't data centers
Reframing of AI risk discussion from emissions to fundamental capability trajectory concerns.
https://techcrunch.com/2026/09/16/al-gore-has-a-surprisingly-calm-take-on-the-ai-data-center-backlash/
Today's Learning Path
Beginner Understanding AI Safety and Local Deployment
1. Read Al Gore's perspective on AI risks to understand the broader debate
10 min
https://techcrunch.com/2026/09/16/al-gore-has-a-surprisingly-calm-take-on-the-ai-data-center-backlash/
2. Explore WebGPU kernels to see how AI can run locally in browsers
20 min
https://huggingface.co/blog/webgpu-kernels
3. Try Gradio Workflow implementation to understand UI building for AI
30 min
https://huggingface.co/blog/gradio-workflow-1111
4. Review embedded evaluator discussion to grasp safety oversight models
15 min
https://techcrunch.com/2026/09/16/anthropic-and-openai-want-to-embed-safety-evaluators-will-they-really-be-independent/
After this: Understand current AI safety debates and practical deployment options without requiring infrastructure
Intermediate Building Reliable and Efficient AI Systems
1. Study IBM's agent consistency research to understand reliability challenges
25 min
https://huggingface.co/blog/ibm-research/altk-evolve-consistency
2. Learn GRPO fine-tuning for structured outputs to improve model control
35 min
https://huggingface.co/blog/grpo-with-trl-ifstruct
3. Implement WebGPU kernels for a local AI project
60 min
https://huggingface.co/blog/webgpu-kernels
4. Explore NeoMME for multimodal multilingual applications
30 min
https://huggingface.co/blog/Hcompany/neomme
5. Experiment with coding-to-art transfer using TRL and OpenEnv
45 min
https://huggingface.co/blog/train-to-paint-with-code
After this: Build production-ready AI systems with reliability controls and efficient training pipelines
Advanced Safety Architecture and Distributed Training
1. Deep dive into BenchMIRT to challenge evaluation assumptions
40 min
https://huggingface.co/blog/allenai/benchmirt
2. Implement async GRPO without NCCL for multi-node training
90 min
https://huggingface.co/blog/asyncgrpo-lora-hfjobs
3. Build agent memory systems using the Funes framework
60 min
https://huggingface.co/blog/funes
4. Study nuanced safety refusal mechanisms
35 min
https://huggingface.co/blog/MultiverseComputingCAI/safety-for-whom
5. Analyze embedded evaluator independence requirements
25 min
https://techcrunch.com/2026/09/16/anthropic-and-openai-want-to-embed-safety-evaluators-will-they-really-be-independent/
After this: Design distributed training infrastructure and implement comprehensive safety architectures for frontier models
🇮🇳 India AI Watch
INDIA AI WATCH
Practo leadership change and payment market shifts signal maturation of India's consumer tech stack.
Practo Founder Steps Aside for Operations Leader
Healthtech pioneer Practo replaced cofounder CEO Shashank ND with former COO Jagnoor Singh, marking a shift from visionary leadership to operational execution. The move comes as the platform faces intensifying competition from AI-powered diagnostic and telemedicine startups with more modern architectures. Singh's operational background suggests focus on profitability and sustainable growth rather than the expansion-at-all-costs mentality that characterized earlier consumer internet phases.
Source: Inc42
Navi Cracks UPI Duopoly
Sachin Bansal's Navi reached 4.4% UPI market share as PhonePe and Google Pay slipped, demonstrating that banking license advantages can overcome network effects in payments. The growth proves focused financial players can compete against tech giants by offering superior settlement speeds and lower merchant friction. This opens the market for other banking-backed challengers to capture share from the incumbents.
Source: Inc42
Peak XV Exits Groww in ₹1,756Cr Deal
Peak XV Partners sold 9.17 crore Groww shares worth ₹1,756 crore in a massive stake sale that signals VC rotation from consumer fintech to infrastructure plays. The timing suggests belief that consumer financial services market structure is largely determined, with future value accruing to underlying technology providers. Expect more large fintech exits and shift toward enterprise infrastructure investments across Indian VC portfolios.
Source: Inc42
India Signal
The simultaneous Practo leadership change, Navi's payment gains, and Peak XV's Groww exit reveal India's consumer tech entering a consolidation phase where operational excellence trumps growth narratives. Founders are stepping aside for operators, new entrants win through structural advantages rather than features, and VCs are exiting consumer bets. This mirrors the 2015-2016 e-commerce consolidation but is happening faster—suggesting India's next startup wave will focus on infrastructure and enterprise rather than consumer applications.
Economy Impact
Today's developments reveal AI's transition from capability demonstration to reliability engineering, with significant economic implications. The embedded evaluator model addresses trust barriers preventing enterprise adoption worth hundreds of billions in productivity gains. Meanwhile, efficiency breakthroughs like 100-step fine-tuning and simplified distributed training democratize development beyond hyperscalers, potentially fragmenting a market currently consolidating around three companies. Gore's reframing of AI risk from emissions to trajectory may accelerate infrastructure investment by neutralizing climate opposition.
↓
Decreasing
Enterprise AI Deployment Risk
↓
Falling Rapidly
Training Infrastructure Costs
↓
Easing
Market Concentration Pressure
The Hidden Link — What Others Aren't Connecting
Agent Memory and Benchmark Validity Share Root Cause
The agent memory tooling and benchmark validity research both address the same fundamental problem: AI systems optimized for one-shot performance fail in persistent contexts. Agents without memory can't maintain consistency across sessions, while models trained on benchmarks excel at isolated tasks but falter in continuous application. Both issues stem from evaluation that rewards immediate capability over sustained reliability. The solution space overlaps: memory systems that track performance across time could also validate whether benchmark scores predict real-world success.