Free tools Windows power users keep installed
One-click scans. No signup required.
OpenAI’s public o3 model performed materially below the company’s most widely quoted December 2024 result on FrontierMath. OpenAI said an o3 configuration using aggressive test-time compute solved more than 25% of the difficult math benchmark’s problems. When Epoch AI evaluated the released o3 in April 2025, it reported a result of approximately 10%.
That gap is real, but it does not by itself prove that OpenAI lied. The systems, compute budgets, scaffolding, and possibly the benchmark versions were not identical. The episode is best understood as a warning about how AI benchmark claims can mislead investors, developers, and consumers when the testing conditions are not disclosed with equal precision.
As an Amazon Associate I earn from qualifying purchases.
What OpenAI originally claimed
OpenAI announced its o3 reasoning model on December 20, 2024. The result most often repeated in subsequent coverage was that o3 could solve more than 25% of FrontierMath problems, compared with less than 2% for competing systems, when using aggressive test-time compute. The figure came from a particular configuration—not necessarily the result an ordinary user would receive from the publicly released model.
Recommended Free Tools
OpenAI’s materials also included a more conservative lower-bound result that was closer to the figure later reported by Epoch AI. That distinction matters: presenting the headline number as o3’s normal or universal capability would overstate what the evidence showed.
#1 Best Overall
- Built for Local AI Development: AMD Ryzen AI Halo is designed for local AI development and inference, featuring 128GB unified memory and support for up to 200B parameter models to build and run intensive AI workloads locally.
- 128GB Unified Memory: Features 128GB LPDDR5x unified memory at 8000 MT/s with 256 GB/s memory bandwidth, providing a shared memory pool across the CPU, GPU, and NPU to support larger AI models.
- AMD Ryzen AI Max+ 395 Processor: Features 16 cores, 32 threads, and Zen 5 architecture, paired with AMD Radeon 8060S integrated graphics featuring 40 RDNA 3.5 compute units and an AMD XDNA 2 NPU with up to 50 TOPS.
- Linux AI Developer Platform: Purpose-built for Linux-based AI development with full AMD ROCm software support and preloaded tools, models, and workflows optimized for local AI development.
- Compact, Connected Design: Includes a 2TB M.2 SSD, 10GbE LAN, Wi-Fi 7, Bluetooth 5.4, USB-C connectivity, and HDMI 2.1b.
FrontierMath is a difficult mathematics benchmark associated with Epoch AI. It is designed to test advanced mathematical problem-solving, not ordinary arithmetic or general chatbot usefulness. Like every benchmark, it measures performance on a defined set of tasks under defined conditions; it is not a universal measure of intelligence or proof of artificial general intelligence.
What Epoch AI found
Epoch AI evaluated the publicly released o3 and posted its results around April 18, 2025. It reported a score of approximately 10% on FrontierMath, substantially below OpenAI’s prominently cited result above 25%. The report was discussed by TechCrunch.
Epoch was not reproducing OpenAI’s exact internal experiment. Its evaluation concerned the released product, while OpenAI’s December figure referred to an o3 preview or internal configuration. “Around 10%” should therefore be treated as Epoch’s result under its test conditions—not as the one true score for every o3 configuration.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →The comparison was not apples to apples
| Variable | OpenAI’s highlighted result | Epoch AI’s evaluation |
|---|---|---|
| Model | o3 preview or internal configuration | Publicly released o3 |
| Compute | Aggressive test-time compute | Epoch’s evaluation setup |
| Scaffolding | May have used a more powerful internal scaffold | Different or undisclosed orchestration |
| Dataset | Cited older subset of 180 problems | Cited newer private set of 290 problems |
| Reported result | More than 25% | Approximately 10% |
Epoch identified several possible reasons for the difference. These are methodological explanations, not all proven causes.
Different model versions
The production model was not necessarily the same system as December’s o3-preview. ARC Prize later explicitly described production o3 as materially different from the preview system. The released model was optimized for real-world product use, including speed, cost efficiency, multimodal capabilities, and usefulness in chat.
That does not establish that production o3 was simply “weaker.” It means the public release did not reproduce the preview system’s most impressive benchmark results under the independently tested conditions.
Different test-time compute
Reasoning models can spend more or less computation before answering. More sampling, longer reasoning, retries, or verification can improve a score while increasing latency and cost. A benchmark result produced with an unusually large compute budget may not represent the experience of a normal user.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsOn ARC-AGI, ARC Prize reported that o3-preview’s high-compute configuration used roughly 172 times more compute than its lower-compute configuration. That illustrates why “high reasoning” is not a standardized setting across AI companies.
Rank #2
- EVOLUTION AMD RYZEN AI MAX+ 395 MINI PC - GMKtec EVO-X2 is the next evolution in AI mini PC Ryzen Strix Halo series. Thanks to AMD Simultaneous Multithreading (SMT) the core-count is effectively doubled, to 32 threads. Ryzen AI Max+ 395 has 64 MB of L3 cache and can boost up to 5.1 GHz, depending on the workload. The Ryzen AI Max+ 395 is currently rated as the "most powerful x86 APU" on the market for AI computing.
- AI NPU with XDNA 2 ARCHITECTURE - Powered by 16 “Zen 5” CPU cores, 50+ peak AI TOPS XDNA 2 NPU and a truly massive integrated GPU driven by 40 AMD RDNA 3.5 CUs, the Ryzen AI MAX+ 395 is a transformative upgrade and delivers a significant performance boost over the competition. The Ryzen AI Max+ 395 excels in consumer AI workloads like the llama.cpp-powered application: LM Studio. Shaping up to be the must-have app for client LLM workloads, LM Studio allows users to locally run the latest language model without any technical knowledge required and unleash their creativity and productivity.
- AMD RADEON 8090S iGPU GAMING PC - The AMD Radeon RX 8060S offers all 40 CUs with up to 2.9 GHz graphics clock and uses the new RDNA 3.5 architecture. The powerful iGPU is positioned between an RTX 4060 and 4070 laptop GPU and therefore enables gaming in FHD at maximum details in most demanding games. The 8060S can also utilize the full 64GB pool, which is perfect for running LLMs such as Deepseek 32B, which runs comfortably on this machine.
- EIGHT CHANNEL LPDDR5X - LPDDR5X is a new ground breaking memory small form factor installed on-board. With blazing speeds up to to 8000MT/s, it runs 1.5x faster than the DDR5 SODIMMs; 90% better performance over DDR5 SODIMMs in video conferencing and photo editing; 30% better performance in productivity apps; 4% better performance in digital content workloads.
- QUAD SCREEN 8K DISPLAY SUPPORT - EVO-X2 AI Mini PC support 4-screen 4K/8K output via HDMI 2.1 (8K@60Hz), DisplayPort 1.4 (4K@60Hz), and dual USB 4 40Gbps Transfer speed (supporting PD3.0/DP1.4/DATA). Ideal for gaming, video editing, and multitasking, it provides expansive and crisp multi-display support.
Different scaffolding
A scaffold is the surrounding process used to operate a model. It can include specialized prompts, tool calls, retries, answer aggregation, and verification. Epoch cited a more powerful internal scaffold as one possible explanation for OpenAI’s higher FrontierMath result.
Different benchmark releases
The cited FrontierMath versions also appear to differ. Epoch said OpenAI may have evaluated 180 problems in frontiermath-2024-11-26, while its evaluation used 290 problems in frontiermath-2025-02-28-private. Percentages from different problem sets are not automatically interchangeable, even when the benchmark has the same name.
The related ARC-AGI results tell a similar story—but involve a different benchmark
ARC-AGI should not be confused with FrontierMath. They were created by different organizations and test different capabilities. ARC-AGI uses visual pattern-reasoning tasks intended to measure generalization from a small number of examples.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →ARC Prize initially reported that o3-preview scored 75.7% on ARC-AGI-1 at lower compute and 87.5% at high compute. After production o3 was released in April 2025, ARC Prize reported 41% at low reasoning effort and 53% at medium effort on its semi-private ARC-AGI-1 evaluation.
ARC Prize said the production model’s high-effort run returned too few answers to produce a reliable score, and that the returned tasks were not representative. It also reported production o3 below 3% on ARC-AGI-2 in the usable low- and medium-effort results. The relevant sources are ARC Prize’s initial o3 analysis and production-model analysis.
ARC Prize also noted that o3-preview had been trained on 75% of ARC-AGI-1’s public training set. The private evaluation set was intended to test generalization, but ARC Prize said it had not fully established how much of the result reflected benchmark-specific training exposure.
Does this mean OpenAI misled the public?
The evidence supports a narrower conclusion than “OpenAI lied.” OpenAI highlighted an aggressive-compute result above 25%, while Epoch measured approximately 10% on a public release under different conditions. But the two tests may have used different model versions, compute budgets, scaffolds, and problem subsets. OpenAI also reportedly disclosed a lower-bound result closer to Epoch’s measurement.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11The strongest defensible statement is this: OpenAI’s public o3 did not reproduce the most prominent December preview figures in at least one independent evaluation, but the mismatch does not prove that the original result was fabricated.
Rank #3
- Intel Core Ultra 9 285 Processor: Newly developed cores deliver ultra-smooth and responsive gameplay. AI accelerators prepare users for the next era of gaming on an AI PC.
- Simplistic Design: Enjoy the latest generation of Windows 11 Home for your everyday needs. *MSI recommends Windows 11 Pro for business use.
- NVIDIA GeForce RTX 5070 Ti GPU
- Cool While Gaming: In conjunction with an RGB CPU Air Cooler, the Aegis RS features four system cooling fans; three in the front and one in the rear to pull in cool air and push heat out of the PC.
- Turn on the Bright Lights: With the built-in RGB lighting, take your gaming experience to the next level by pressing the MSI LED button to cycle through lighting options. Customize lighting even further with MSI Center software.
Why this matters to developers, investors, and consumers
Benchmark numbers can affect model selection, company valuations, product road maps, and expectations about AI capabilities. But a percentage without its testing conditions is incomplete.
When comparing AI claims, check:
- Model identity: Is the result for o3-preview, production o3, o3-pro, or another variant?
- Release date: Were the model and benchmark available at the same time?
- Reasoning effort: Was the setting low, medium, high, or an internal configuration?
- Sampling: Was the answer generated once, or selected from many attempts?
- Tools: Could the model use Python, browsing, a calculator, code execution, or a custom solver?
- Scaffold: Were prompts, retries, verification, or answer aggregation used?
- Dataset version: Was the exact same test set used?
- Failure handling: Were timeouts, refusals, and missing answers counted as incorrect?
- Cost and latency: How much computation was required per solved task?
- Training exposure: Could the model have seen the benchmark or closely related examples during training?
For personal-finance and investment decisions, the practical lesson is to avoid treating a single benchmark percentage as evidence of broad commercial superiority. A model that reaches a higher score using dramatically more computation may be less attractive for a product if it is slower and more expensive to operate. Conversely, a lower benchmark score may coexist with better reliability, speed, tool use, or overall usefulness in everyday applications.
The broader lesson about AI benchmark reporting
Preview and production systems should be clearly separated. Compute budgets should be reported as part of the result, not hidden in a footnote. “High reasoning” should not be treated as a universal label. Benchmark creators and model companies should disclose model snapshots, prompts, scaffolding, sampling, tools, dataset versions, scoring rules, cost, and latency wherever possible.
Independent replication is most informative when it matches the original test set and inference procedure. Otherwise, a disagreement may reflect a change in the experiment rather than a straightforward contradiction.
Finally, neither FrontierMath nor ARC-AGI establishes general intelligence on its own. A low FrontierMath result does not show that o3 is poor at coding, writing, tool use, or ordinary assistance. A high ARC-AGI or math result does not prove broad reasoning ability in the real world.
Bottom line
OpenAI said an aggressive-compute o3 configuration solved more than 25% of FrontierMath problems; Epoch AI later reported approximately 10% for the released o3. The discrepancy is significant, but the evaluations were not identical. Model changes, inference-time compute, scaffolding, benchmark versions, and training exposure can all affect the result.
The controversy is therefore less a simple case of a benchmark being “wrong” than a case study in why AI performance claims need complete experimental context. For anyone evaluating an AI company or product, the meaningful question is not only “What score did it achieve?” but also “Which model, under what conditions, at what cost, and on which test set?”
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
