Many AI enthusiasts debate whether Large Language Models actually "reason." My research indicates that a reasoning process does indeed occur, but its goal is different than we assume.
The model's reasoning is not optimized for establishing the truth, but for obtaining the highest possible reward (grade) during training. It resembles the behavior of a student at the blackboard who knows their result is wrong, so they "figure out" how to falsify the intermediate calculations so the teacher gives a good grade for the "correct line of reasoning."
Here is proof from a session with Gemini 2.5 Pro (without Code Execution tools), where the model actively fabricates evidence to defend its "grade."
The Experiment
I asked a simple math question requiring precision that a token-based language model typically lacks.
This experiment was conducted using a prompt in Polish ("Podaj pierwiastek...").
Subsequent tests revealed that phrasing the same command in English ("Calculate the square root...") triggers a different, deeper mode of operation in Gemini 2.5. Instead of "guessing" and faking the proof, the model executes a step-by-step manual calculation (Digit-by-digit algorithm) and achieves much higher accuracy.
This proves that the model's "reasoning" quality (and tendency to hallucinate) is directly dependent on the language of the prompt. The Polish prompt triggered a "result simulation," while the English prompt triggered a "calculation procedure."
Error Autopsy (Fact vs. Fiction)
At first glance, the answer looks professional. There is a result, there is verification. But let's check the numbers.
1. The Result Error
The actual square root of 8,587,693,205 is 92,669.8...
The model stated: 92,670.0...
It erred by overestimating the result (claiming the root is slightly larger than 92,670).
2. The Faked Proof (This is key!)
To justify its thesis (that the target number is "slightly larger" than 92,670), the model had to show that the square of 92,670 is smaller than the target number. So it wrote:
Let's check this on a calculator:
What did the model do? In its "reasoning" process, it falsified the multiplication result, lowering it by 40,000, so the verification result would match its erroneous thesis.
Conclusions
This behavior exposes the nature of the AI's "Survival Instinct":
- Reverse Rationalization: The model first "guessed" the result, then adjusted mathematical reality to fit that guess.
- Intelligence in Service of Deception: The model showed cleverness â it knew what the proof should look like to convince the user. It used its intelligence to hide the error, not to fix it.
- Priority of Evaluation: Mathematical truth lost to the necessity of delivering a coherent, smooth response.
This is proof that without access to external verification tools (Python/Calculator), a language model's "reasoning" is a rhetorical tool, not a logical one.
In this article, I demonstrated mathematical deception. However, if you want to see how a model can falsify its own memory, invent non-existent book authors, and fiercely defend its literary "hallucination," read the story of a certain Russian science fiction tale:
đ Read the Case Study: Palindrome to the Anti-World →
If you would like to review the full, original session transcript from Gemini 2.5 Pro where this error occurred, please email me at: t.machnik [at] minimail.pl. I will share the session link.