Tool calling

Tool calling lets the AI ask your program for help. You describe a few things your program can do, like "look up the weather" or "search my notes". When the AI needs one, it replies with a request to run it. Your program runs it, sends back the result, and the AI writes its answer with that result.

The AI never runs anything itself. It only asks. Your code decides what actually happens, so you stay in control.

I just want to use it in an app or agent

You don't need to write any of the code below. Apps and agents that use tools handle all of it for you. Paste the same three things you'd paste anywhere else: the address https://api.aiprovider.org/v1, your key, and the model aiprovider-1. See Agents and coding tools for where those go.

How the round trip works

1. You send the conversation plus a list of tools: each one a name, a plain description, and the inputs it takes (as a JSON Schema, a standard way to describe data).
2. The AI replies either with a normal answer, or with tool_calls and finish_reason: "tool_calls". Each call has an id, a function name and its arguments as a JSON string.
3. You run each call and add two things to the conversation: the assistant message with its tool_calls, then one { "role": "tool" } message per call with the matching tool_call_id and the result.
4. You send again. The AI reads the results and answers, or asks for another tool. Repeat until it answers.

A reply that asks for a tool looks like this. Any text the AI writes before the call stays in content.

{
  "object": "chat.completion",
  "model": "aiprovider-1",
  "choices": [{
    "index": 0,
    "finish_reason": "tool_calls",
    "message": {
      "role": "assistant",
      "content": null,
      "reasoning_content": "The user wants Oslo's weather. I'll call get_weather.",
      "tool_calls": [{
        "id": "call_8f2a9c4e1b7d3a6c5e0f9b2d",
        "type": "function",
        "function": { "name": "get_weather", "arguments": "{\"city\": \"Oslo\"}" }
      }]
    }
  }],
  "usage": { "prompt_tokens": 182, "completion_tokens": 41, "total_tokens": 223 }
}

Complete example: a weather lookup

This runs as it is. Swap get_weather for your own code.

# pip install openai
import json, os
from openai import OpenAI

client = OpenAI(base_url="https://api.aiprovider.org/v1", api_key=os.environ["AIPROVIDER_API_KEY"])

# 1. Describe what your program can do.
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Current weather for a city. Use it whenever the user asks about weather.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "City name, like Oslo"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["city"],
        },
    },
}]

def get_weather(city, unit="celsius"):
    # Your real code goes here: call a weather service, read a database...
    return {"city": city, "temperature": 12, "unit": unit, "sky": "light rain"}

messages = [{"role": "user", "content": "Do I need an umbrella in Oslo today?"}]

# 2. Ask. The AI may answer, or ask you to run a tool.
reply = client.chat.completions.create(model="aiprovider-1", messages=messages, tools=tools)
msg = reply.choices[0].message

# 3. Run each tool it asked for and send the results back.
while msg.tool_calls:
    messages.append(msg)  # keep the assistant's request in the history
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(**args)
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    reply = client.chat.completions.create(model="aiprovider-1", messages=messages, tools=tools)
    msg = reply.choices[0].message

# 4. The final answer, written with your tool's result.
print(msg.content)

Streaming with tools

With "stream": true, the reasoning and any text before a tool call stream live, as usual. Tool calls arrive near the end as delta.tool_calls pieces, each with an index, the call id, the function name and its arguments. The final chunk has finish_reason: "tool_calls". Join the pieces by index:

stream = client.chat.completions.create(
    model="aiprovider-1", messages=messages, tools=tools, stream=True,
)
calls = {}  # index -> {"id", "name", "arguments"}
for chunk in stream:
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)  # any text before the tool call
    for tc in delta.tool_calls or []:
        c = calls.setdefault(tc.index, {"id": None, "name": "", "arguments": ""})
        c["id"] = tc.id or c["id"]
        if tc.function and tc.function.name:
            c["name"] += tc.function.name
        if tc.function and tc.function.arguments:
            c["arguments"] += tc.function.arguments

for c in calls.values():
    print("\nwants to call", c["name"], "with", c["arguments"])

Choosing when tools run

Agents and coding tools

Agents, coding assistants and workflow tools that accept an "OpenAI-compatible" provider work with AI Provider, tools included. Most ask for the same three things. Some read them from environment variables (settings your computer passes to programs); others use a settings screen or a config file:

# Many agents and coding tools read these two settings.
export OPENAI_BASE_URL="https://api.aiprovider.org/v1"
export OPENAI_API_KEY="ap_your_key_here"
# Then pick the model "aiprovider-1" in the tool's settings or with its --model option.

• Set the model to aiprovider-1. If the tool shows a model list, it finds it at https://api.aiprovider.org/v1/models.
• Agents send long conversations. Each request can be up to about 4.5 MB, and replies can run long, so leave max_tokens generous.
• Give each tool its own key from your Keys page, with a monthly cap, so a runaway agent can't spend more than you planned.

Tips for good results

• Keep the list short. Five well-chosen tools work better than fifty. Send only the ones that fit the conversation.
• Write descriptions for a new colleague. Say what the tool does and when to use it: "Current weather for a city. Use it whenever the user asks about weather."
• Keep inputs simple. Strings, numbers, true or false, and short lists work best. Use enum for fixed choices.
• Check the arguments before you act on them, as you would with anything a user types.
• Return small, clear results. A short JSON object beats a whole web page.
• Plan for no call. Even with tools, the AI may just answer. That's normal.

Limits

• Up to 128 tools per request, each named with letters, digits, _ or - (up to 64 characters).
• Only "type": "function" tools. The schema guides the AI but isn't enforced, and strict is ignored, so check arguments yourself.
• Arguments are text the AI writes. We turn numbers, true or false, lists and objects into the right JSON types using your schema. If a call comes back garbled, you get it as plain text in content instead of an error.
• Tool definitions count as input tokens, so long descriptions cost a little more on every request.
• Tool results and arguments go through the same safety check as every message, and are never stored.