Tool Use and Function Calling

This lesson on Tool Use and Function Calling is hands-on and example-driven. You will be able to define JSON Schema tool specifications, bind external functions to OpenAI chat completion requests, and parse structured model outputs to execute local Python functions. You will also learn to handle conversational branching gracefully by controlling the model's tool selection behavior.

What You'll Be Able To Do

• Define valid JSON schema tool specifications including parameter types, required fields, and semantic descriptions.
• Configure OpenAI Chat Completions API calls to pass available tools alongside user prompts.
• Extract and deserialize JSON arguments from tool_calls responses using Python's json module.
• Execute local Python functions dynamically using arguments generated by the model.
• Handle conversational requests cleanly when tool_calls evaluates to None under default auto execution.

Detailed Concept Walkthrough

1. Function Calling Architecture and Lifecycle

Function calling provides a structured bridge between probabilistic LLMs and deterministic code, enabling models to retrieve real-time data and take external actions without hallucinating API schemas.

• Mechanism: The application passes function declarations alongside the conversation history; the model decides whether a tool is required and returns a structured JSON payload instead of a text response.
• Under the Hood: The model never executes external code directly; it only extracts entities, validates them against your schema, and generates the exact JSON parameter arguments for your client application to run.
• Execution Flow: The cycle requires four discrete steps: sending schemas to the API, intercepting returned argument payloads, running the local or remote target function, and appending the result back into context.

# 4-Step Tool Lifecycle Concept
# 1. Send schema & prompt -> LLM
# 2. LLM responds with tool_calls metadata
# 3. Client executes local Python function with generated args
# 4. Result is fed back to LLM to complete generation

Key Takeaway: The LLM generates structured arguments based on natural language, but execution always remains the responsibility of your local application.

2. Defining Tool Schemas with JSON Schema

Tool schemas tell the LLM which functions exist, what arguments they accept, and when they should be triggered based on clear natural language descriptions.

• Schema Structure: Tools are formatted as a list containing dictionaries with type="function" and a nested function object containing name, description, and parameters.
• Syntax Rule: The parameters key must follow the JSON Schema specification with type="object", a properties dictionary for each parameter, and a required list defining mandatory fields.
• Best Practice: Write explicit, descriptive docstrings for properties; the model uses property descriptions to perform semantic inference, such as assuming Celsius for London when unspecified.

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

Key Takeaway: Descriptive parameter schemas guide model extraction accuracy and sensible default value inference.

3. Extracting Arguments and Executing Local Functions

Client applications must inspect response message objects, parse stringified JSON arguments into native dictionaries, and route the execution to the appropriate Python callable.

• Mechanism: When the model invokes a tool, response.choices[0].message.tool_calls contains a list of call objects with function names and stringified JSON arguments.
• Under the Hood: Arguments are returned as raw JSON strings in tool_call.function.arguments, requiring explicit deserialization via json.loads before passing as keyword arguments.
• Execution Flow: After parsing arguments into a native Python dictionary, invoke your local target function and capture its output to pass back into the conversation.

import json

def get_current_weather(location, unit="celsius"):
    return json.dumps({"location": location, "temperature": "15", "unit": unit})

# Assuming 'response' is returned from OpenAI client
message = response.choices[0].message
if message.tool_calls:
    tool_call = message.tool_calls[0]
    function_args = json.loads(tool_call.function.arguments)
    function_response = get_current_weather(
        location=function_args.get("location"),
        unit=function_args.get("unit", "celsius"),
    )

Key Takeaway: Always deserialize tool_call.function.arguments with json.loads before passing arguments to your execution functions.

4. Controlling Tool Selection Behavior

The tool_choice parameter controls whether the model is forced to call a tool, prohibited from calling a tool, or allowed to decide dynamically.

• Mechanism: By default (tool_choice="auto"), the model analyzes prompt context to decide whether a tool call is necessary or if standard conversational text is appropriate.
• Execution Flow: When a user sends a non-tool query like 'Hello, how are you?', the model outputs regular text content while message.tool_calls remains None.
• Best Practice: Always guard tool extraction logic with a check for if message.tool_calls: to prevent runtime TypeError or AttributeError exceptions on conversational inputs.

response = client.chat.completions.create(
    model="gpt-3.5-turbo-1106",
    messages=[{"role": "user", "content": "Hello, how are you?"}],
    tools=tools,
    tool_choice="auto",  # Default behavior
)

# Defensive parsing guard
if response.choices[0].message.tool_calls:
    # Execute function path
    pass
else:
    # Fallback to standard chat response
    print(response.choices[0].message.content)

Key Takeaway: Always verify message.tool_calls is not None before attempting to parse function arguments.

Topics Covered in Tool Use and Function Calling

1. Function Calling Architecture (0:00 - 3:32) — Introduces function calling as a structured bridge between LLMs and external systems to eliminate hallucinations and power agentic workflows.
2. Environment & Mock Setup (3:32 - 4:51) — Configures the API client for gpt-3.5-turbo-1106 and defines a local mock weather function returning serialized JSON.
3. Schema Definition & Argument Extraction (4:51 - 8:36) — Builds a JSON Schema tool specification, submits a weather query, and deserializes the returned function arguments to run local code.
4. Controlling Tool Behavior (8:36 - 10:24) — Tests conversational queries against default tool_choice settings to verify how the model handles non-tool interactions.

LLMs & Generative AI for Practitioners Cheat Sheet

• tools=[{"type": "function", "function": {...}}] — Binds tool definitions to chat completions request

  response = client.chat.completions.create(model="gpt-3.5-turbo-1106", messages=msgs, tools=tools)
  

• tool_choice="auto" — Allows model to decide between tool and text

  client.chat.completions.create(model="gpt-3.5-turbo-1106", messages=msgs, tools=tools, tool_choice="auto")
  

• message.tool_calls — Accesses list of model-requested tool invocations

  tool_calls = response.choices[0].message.tool_calls
  

• tool_call.function.name — Retrieves target function name string

  fn_name = tool_call.function.name
  

• json.loads(tool_call.function.arguments) — Deserializes JSON arguments into Python dictionary

  args = json.loads(response.choices[0].message.tool_calls[0].function.arguments)
  

Comparison Table

Common Pitfalls

• Mistake: Assuming the model runs your code automatically. Avoid: Intercept the structured JSON tool call and invoke the local target function within your code.
• Mistake: Accessing tool_calls[0] without checking if tools were invoked. Avoid: Always verify message.tool_calls is not None before parsing arguments to prevent index and attribute errors.
• Mistake: Providing vague or empty parameter descriptions in tool schemas. Avoid: Write detailed docstrings and explicit enum values so the model accurately maps user intent to arguments.

FAQs

• Does OpenAI execute the function on its remote servers? No. The model only generates the function name and parsed JSON arguments; your local application code must execute the actual function.
• Why does the model sometimes infer default values not explicitly provided in the prompt? The model uses contextual knowledge and schema descriptions to infer reasonable defaults, such as selecting Celsius for European locations.
• What happens if a user submits a prompt that doesn't need external data? Under default auto settings, the model bypasses tool calling, leaves tool_calls as None, and populates the content field with standard text.