Code as a Tool for Thought

LLMs are probabilistic and approximate, while mathematics and logic require deterministic precision.

Division of labor: LLM is responsible for understanding the task and writing the code, the interpreter is responsible for accurate calculations. Example: a problem about the percentage of students is easily solved incorrectly “in the head”, but the code gives an accurate, verifiable result.

Wolfram scheme: LLM understands natural language and formalizes the problem, a symbolic engine (SymPy, Mathematica) solves it accurately.

There is an inverse relationship between the model and the harness: the weaker the model, the more logic needs to be moved into code and constraint solvers.