Um Método ‘Zen’ para Impedir que os Modelos de Linguagem Hallucinem

Dizer ao ChatGPT para verificar uma resposta aleatória antes de resolver um problema real faz com que ele pense mais e obtenha a resposta certa com mais frequência – mesmo que a resposta anterior “aleatória” não tenha nada a ver com a sua consulta real.

Um artigo interessante de uma nova pesquisa da China desenvolveu um método de baixo custo para impedir que os modelos de linguagem, como o ChatGPT, hallucinem, e para melhorar a qualidade das respostas: fazer com que o modelo verifique a resposta a uma pergunta totally desconexa primeiro:

Um exemplo de uma pergunta desconexa que pode ‘liberar a mente’ de um LLM e ajudá-lo a se concentrar em uma consulta real subsequente. Fonte

Esse golpe de zen é uma maneira incrivelmente barata de melhorar o desempenho, em comparação com outros métodos mais complexos, como fine-tuning, prompt-crafting e parallel sampling, e funciona em modelos de código aberto e fechado.

The authors outline the economies of scale possible by improving output in this Spartan manner*:

‘To implement with minimal additional prior knowledge, VF only needs to provide a random/trivial answer in the prompt. The verification process turns out to have much fewer output tokens than an ordinary CoT path, [sometimes] even no explicit verification-only process, thus [requiring] very [little] additional test-time computation.’

In tests, this approach – dubbed Verification-First (VF) –  was able to improve responses in a diversity of tasks, including mathematical reasoning, across open source and commercial platforms.

Part of the reason why this technique works may be grounded in the way that language models soak up and appropriate trends in human psychology, so that a direct question may make the model ‘defensive’ and ‘nervous’, whereas a request to verify the work of another does not engage these ‘survival instincts’.

The core idea is that verifying an answer takes less effort than generating one from scratch, and can trigger a different reasoning path that complements standard chain-of-thought.

Prompting the model to critique a given answer (i.e., an answer that the model has not been involved in creating) may also activate a kind of critical thinking that helps avoid overconfidence in the model’s own first impressions.

The work characterizes the process in terms of a reverse-reasoning path:

Partindo de uma resposta proposta e raciocinando de volta para a pergunta pode expor atalhos ou insights que são mais difíceis de encontrar quando se raciocina para frente a partir do problema sozinho. Esse ‘caminho reverso’ pode seguir uma trajetória mais simples e oferecer informações complementares ao raciocínio padrão da cadeia de pensamento.

The researchers have also concretized the central concept into Iter-VF, a sequential time-test scaling method that iteratively refines answers, avoiding the error accumulation issue common to the self-correcting strategies often found in LLM architectures.

The new work is titled Asking LLMs to Verify First is Almost Free Lunch, and comes from two researchers at the Department of Electronic Engineering at Tsinghua University at Beijing.

Método

The central idea behind the new work is to flip the usual reasoning flow in language models. Instead of asking the model to solve a problem from scratch, it is first handed a candidate answer (often incorrect or arbitrary) and asked to check whether that answer makes sense.

This prompts the model to reason in reverse, working backward from the proposed answer toward the question. Once the verification is complete, the model then proceeds to solve the original problem as usual.

This reversal, the paper asserts, reduces careless mistakes and encourages a more reflective mode of reasoning, helping the LLM uncover hidden structure and to avoid misleading assumptions.

As seen in the examples below, even prompting the model to verify an obviously wrong guess like ’10’ can help it recover from flawed logic and outperform standard chain-of-thought prompting:

Fazer com que o modelo verifique uma resposta adivinhada primeiro ajuda a detectar inconsistências e a se envolver mais cuidadosamente com o problema. Neste exemplo, a abordagem padrão leva a uma solução fluente, mas incorreta, enquanto o prompt de Verificação-Primeiro dispara uma estrutura lógica mais clara e o resultado correto.

In regard to many real-world problems, it’s not easy to provide a guess for the model to check, most especially when the task is open-ended, such as writing code or calling an API.  Therefore to adapt better, the method first gives its best answer as usual and then feeds that answer back into the Verification-First format. In this way the model checks and improves its own output:

Quando o modelo é solicitado a verificar sua própria saída anterior, ele detecta o erro em sua lógica e reescreve a solução corretamente. O prompt de Verificação-Primeiro ajuda a se concentrar no erro específico em vez de repetir o mesmo erro.

This approach constitutes the aforementioned Iter-VF. The model repeats this cycle, refining its answer each time, without need of retraining or bespoke tooling. Unlike other self-correction strategies, which can pile up earlier thinking and risk confusing the model, Iter-VF only looks at the most recent answer each time, which helps keep its reasoning lucid.

Dados e Testes

The authors evaluate the method in four domains: tarefas de raciocínio geral, where VF is seeded with a trivial guess; tarefas de tempo, where Iter-VF is compared with rival scaling methods; problemas de abertura como codificação e chamadas de API, onde VF usa a própria resposta anterior do modelo; e modelos comerciais de código fechado, onde os passos de raciocínio internos são inacessíveis.

To test the method, the researchers used three reasoning benchmarks: GSM8K and MATH500 for math problems; and GPQA-Diamond for graduate-level science questions.

In each case, the model was given either a trivial guess, such as ‘1’ for numerical answers; or a randomly-shuffled multiple-choice option, as the starting point for verification. No special tuning or prior knowledge was added, and the baseline for comparison was standard zero-shot chain-of-thought prompting.

The tests ran across a full range of Qwen2.5 and Llama3 instruction-tuned models, from 1B to 72B (parameters) in size. The Qwen models used were Qwen2.5-1.5B-Instruct, Qwen2.5-3B-Instruct, Qwen2.5-14B-Instruct, and Qwen2.5-72B-Instruct. The Llama3 variants were Llama3.2-1B-Instruct, Llama3.2-3B-Instruct, Llama3.1-8B-Instruct, and Llama3.3-70B-Instruct.

As shown below, the improvement from Verification-First prompting held steady across model scales, with clear gains visible even at 1B parameters and continuing through to 72B:

Em todos os tamanhos de modelo nas famílias Qwen2.5 e Llama3, o prompting de Verificação-Primeiro consistentemente superou o prompting de cadeia de pensamento padrão em GSM8K, MATH500 e GPQA-Diamond.

The effect proved strongest on computation-heavy math benchmarks such as GSM8K and MATH500, where verifying a wrong answer prompted better reasoning than trying to solve from scratch. On GPQA-Diamond, which depends more on stored knowledge than deductive structure, the advantage was smaller but consistent.

The computational cost of Verification-First was modest: in the table below, we can see that generating a verification step added around 20-50% more output tokens compared to standard chain-of-thought prompting:

A média de tokens de saída gerados sob cada método de prompting, em GSM8K, MATH500 e GPQA benchmarks.

Despite this, the extra cost remained far below that of strategies requiring multiple sampled completions or recursive planning.

In the graph below, we can see how sensitive the method is to the quality of the guessed answer. Surprisingly, even when the guess is trivial (‘1’), implausible (‘2025’), or a random multiple-choice option, Verification-First still outperforms standard prompting:

Ganhos de precisão do prompting de Verificação-Primeiro, quando o modelo é dado respostas triviais, implausíveis ou corretas para verificar em GSM8K, MATH500 e GPQA.

As expected, accuracy jumps even higher when the guess happens to be the correct answer; but the method worked well regardless, suggesting that the gains were not driven by the information in the guessed answer itself, but simply by the act of verification.

Iter-VF was also compared against four test-time scaling strategies that operate without retraining or task-specific adaptation. In Self-Correction, the model was prompted to revise its answers by reflecting on previous reasoning steps; in PHP, previous answers were appended to the input as contextual hints, though no instructions were given on how to use them.

Additionally, in Self-Consistency, multiple reasoning paths were sampled and the final answer was chosen by majority vote; and finally, in Best-of-N, several outputs were generated independently and ranked using a verifier prompt, with the highest-scoring response selected.

Two variants of Iter-VF were implemented: one initialized with a trivial guess (‘1’), and another seeded with a standard CoT output:

Precisão e eficiência de token em MATH500 sob orçamentos de saída crescentes, mostrando que ambas as variantes de Iter-VF superam todas as linhas de base em todas as escalas de modelo.

Iter-VF gave better results than all other methods when the available compute was low, which the authors credited to the way it checks answers, not how good the initial answers were (since both VF and CoT variants quickly reached similar accuracy).

PHP performed worse, even though it reused earlier answers as hints, likely because LLMs did not exploit those hints well.

In contrast to PHP and Self-Correction, which accumulate context across iterations, Iter-VF considers only the most recent answer at each step. This Markovian approach avoids the compounding confusion of extended reasoning chains – a weakness especially damaging for Self-Correction.

Parallel methods such as Self-Consistency and Best-of-N avoided this issue, although their improvements were slower and more modest.

(n.b. A seção de resultados, embora completa, é uma leitura desagradável e prolixa, e devemos truncar a maior parte da cobertura restante, referindo o leitor ao artigo original para mais detalhes).

When tested on GPT-5 Nano and GPT-5 Mini, closed commercial models that hide the full reasoning trace and return only the final answer, Iter-VF improved performance without relying on intermediate outputs. In the table below we can see gains across both MATH500 and GPQA, confirming that the verify-then-generate approach remains viable even when only the input and final answer are accessible:

Precisão em MATH500 e GPQA quando Iter‑VF é aplicado a modelos GPT‑5 com rastros de raciocínio ocultos.

Conclusão

Embora o novo artigo se torne opaco a partir da seção de resultados, a aparente descoberta de uma característica abrangente em uma classe de modelos de IA é, no entanto, um desenvolvimento fascinante. Qualquer pessoa que use regularmente um LLM terá desenvolvido instintivamente um conjunto de truques para contornar as limitações dos modelos, à medida que cada uma se torna óbvia com o tempo, e o padrão emerge; e todos esperam encontrar um ‘truque’ tão aplicável e generalizado quanto este.

Um dos maiores problemas na implementação e atualização de uma janela de contexto em um LLM parece ser encontrar um equilíbrio entre a retenção do progresso da sessão e a capacidade de sair em direções novas quando necessário, sem cair em alucinações espúrias ou saídas fora do tópico. No caso apresentado pelo novo artigo, vemos um exemplo de um ‘chamado de despertar’ suave, mas insistente, que parece re-focar e re-definir o LLM sem perda de contexto. Será interessante ver se projetos subsequentes adaptam e evoluem o método.

Os pesquisadores enfatizam a economia do seu novo método – uma consideração que teria tido muito menos peso mesmo há 12 meses. Esses dias, as implicações de IA em larga escala tornam claro que as economias de recursos antes consideradas pedantes, na ‘era de pesquisa pura’, agora estão se tornando essenciais e fundamentais.

* Por favor, note que estou restrito a não incluir o número usual de citações do artigo, desde que o padrão de inglês encontrado em algumas partes dele pode confundir o leitor. Assim, tomei a liberdade de resumir as principais ideias em vez disso, e refiro o leitor ao artigo original para verificação.

Publicado pela primeira vez na quinta-feira, 4 de dezembro de 2025