How does function calling actually work under the hood?

Answer

The model never executes anything. It produces text.

The flow:

1. You send the request along with tool definitions, each with a name, description, and JSON schema for parameters.
2. The model, instead of a normal reply, emits a structured block naming a tool and giving arguments matching the schema.
3. The API surfaces this as a tool call with a stop reason indicating it wants a tool, rather than as finished text.
4. Your code decides whether to run it, runs it, and captures the result.
5. You append a tool result message keyed to that call id and send the whole conversation back.
6. The model continues, now able to see the result.

Two points people miss. The model has no idea whether the tool ran; it only knows what you told it in the result message. And you are free to refuse, which is exactly how permission prompts work: the call is intercepted before execution.

Models can request several tool calls in one response, and clients can execute them in parallel when they are independent. Each result must be returned with its matching call id.

Argument validation matters. Models do produce arguments that violate the schema, particularly for complex nested types. Validate before executing, and return a validation error as a tool result so the model can correct itself.

Common wrong answer

Believing the model runs the code, or that the provider does. Neither is true for standard function calling. The execution boundary sits in your application, which is the whole reason a permission layer is possible.

Related confusion is thinking a "tool" is something the model was trained on. Tools are defined per request. The model has been trained to use the calling format, not to know your specific tools.