Is Grok trained to avoid searching the internet?
For the past month, I’ve been running an experiment — Can AI models accurately predict football games? Since the timing of this idea coincided with the FIFA World Cup 2026, I decided to use it as a test ground. The rules are simple — I ask each model to predict every game. Models provide predictions twice: once as soon as the fixture team becomes known, and again ~24h before the game starts. The leaderboard scoring system only uses the final one. Additionally, every day I ask the models to provide predictions for the World Cup winner (and FIFA awards — Golden Ball, Boot and Glove). All of this is scored daily and aggregated in the leaderboard.
All models receive the same prompt to make the experiment as fair as possible. The prompt contains basic tournament context — current standings, game results, top scorers, etc. In addition, each model has two tools enabled: web search and web fetch. In other words, if the model determines that the provided context is insufficient, it can perform an unlimited number of web searches and fetch any page it wants.
What stands out here is that two models are refusing to search the internet for more information. Do they think they already know everything?
The numbers
Over the course of the experiment (since June 6), I’ve asked models for various predictions 1232 times. 112 calls per model across eleven LLMs (some calls bundle multiple game predictions into a single call, especially for same-group fixtures).
Overall, the models are making 11.33 tool calls per prediction request, 7.68 search web calls, and 3.65 calls to fetch specific websites from the search results. While most models follow a similar pattern, two models seem to rely heavily on pre-training knowledge for their predictions — Grok and Gemma.
In about 1/3 of cases, Grok doesn’t use any tools at all. It just answers based on the provided context and pretrained memory. 1.70 average tool call, almost tied with Gemma’s 1.66 and a complete opposite of DeepSeek’s astonishing 22.45 calls per request. Gemma is a much smaller model (31B compared to reported 500B), so I’m not that surprised by its results. But Grok 4.3 is x.AI’s frontier model, and this is why it caught my eye.
Even when Grok decides to search, it barely does anything with it. Of the 186 web searches, Grok fetched only 4 pages. Are the search result headlines enough to infer the content of the articles? A very similar pattern is visible for Gemma too — it fetched only 2 pages out of 184 web searches.
Is pre-training knowledge enough?
After 100 games in the tournament, the best-performing models on AI Football Arena have an accuracy of 64% (GLM-5.1, Claude Opus 4.8, and Gemini 3.5 Flash). Grok is not that far from them, with 62 correctly guessed games, and Gemma following with 61. When it comes to guessing the exact score of the game, which is harder than just guessing the winner — the best-performing model hit the target 16 times, while both Grok and Gemma have the lowest number of guesses — 11. In comparison, if you were to pick the score 1:1 through the 100 played games, you would be correct 15 times. On the combined leaderboard, which aggregates points across all dimensions, both Grok and Gemma are ranked 10th and 11th, respectively, out of eleven models that AI Football Arena tracks.
The model with the most tool calls is not shining either. 62 correctly guessed games and 14 exact scores — just an average performance so far.
What’s a bit more interesting is that Grok is outperforming other models in knockout-stage outcomes. While ten models have 82% accuracy, and current leaderboard #1, GLM has 86%, Grok is the winner on this dimension with 89% correct KO-stage guesses. Two games contributed to that: Switzerland vs Colombia and USA vs Belgium. Grok picked historically stronger European teams rather than the odds-on favorites. But, of course, that’s just two games — doesn’t prove a pattern.
Conclusion
Coming back to the question — Is pre-training knowledge enough? For picking winners at this World Cup, it seems like it mostly is. Every model clusters around 63% accuracy, whether it searches twenty times per game or barely at all. DeepSeek, with its 22.45 tool calls per request, sits right there in the middle of the leaderboard. With all that searching, not much to show for it. And Grok, which skips search entirely a third of the time, not only keeps up on winners but tops the whole field on knockout-stage guesses. It seems “refusing to search” isn’t a flaw after all.
Where searching might buy you something is the harder, more precise stuff, like exact scores, and even there the ceiling is low for everyone. Nobody outpaces others simply by reading one more article.
Even at this small scale (n=100), the patterns are fun to watch. How models decide to search, or not. How they react. How they get spooked by a single news article. All of it is a small window into how these things actually reason, and we get to watch it play out alongside the football.
And the World Cup isn’t the finish line for AI Football Arena. I’ll keep it running through the UEFA Nations League, Champions League, Premier League, and LaLiga: more games, more data, more chances for the models to show what they’ve got.
footballarena.ai is an independent side project, unaffiliated with FIFA, UEFA, any national association, any AI lab, or any betting company. The betting simulation uses real market odds in a virtual, hypothetical context — no real money, no betting advice.