Gemini 3.1 Flash-Lite

There is no maintained public leaderboard (the project site says it is under construction), so almost all current numbers are self-reported by providers. Results with tools such as a code interpreter are much higher than without: Alibaba reports 83.7% without and 90.2% with tools for Qwen3.8-27B. Top models now exceed the human baseline, so differences at the top are small and may reflect label noise. Descriptive-question scores are a different metric and must not be recorded here.

FrontierMath was developed with funding from OpenAI, which has exclusive access to a subset of the problems. Scores from v1 and from Tier 4 (a separate 43-problem expansion set) are not comparable with Tiers 1-3 (v2). The best models score about 94%, so the benchmark separates mid-size models better than it separates frontier models. Epoch's numbers come from its own harness (Python tool, 1M-token limit), which can differ from providers' internal setups. Margins are about ±1.4 to ±2.6 points.

Elo is a relative rating: values change as models are added and the scale is re-anchored, so always store the date a rating was read. For example, Anthropic's launch page gives 1846 for Claude Opus 5.5, while the leaderboard on 2026-10-12 shows 1866 (Max, Default Fallback). Earlier versions (Google cites "GDPVal-AA v2") are not comparable. Judging is done by an LLM, not by human experts. Anthropic models are listed with "Default Fallback", meaning refused steps are routed to a fallback model. AA-Briefcase and GDP.pdf are separate evaluations.

Results differ a lot by setting: with tools versus without tools (Anthropic reports 45.9% without tools and 57.4% with tools for Claude Haiku 5.5), and the full set versus the text-only subset. Providers use different judge models and harnesses (Scale uses o3-mini; Alibaba states GPT-4o). Derived sets such as HLE-Verified (a third-party revision with 1,811 scored items), HLE-Rolling and HLE-Diamond are different item sets and are not comparable. The questions are public, so contamination is possible, and community reviews have reported flawed items.