Day 7 — Tool calling + local models with Ollama

Why / क्यों

English: An LLM alone only writes text — it can't know today's weather or touch your database. Tool calling hands it your Python functions, and the LLM decides when to use them. This is the foundation of agents. Ollama lets you run models on your own laptop.

Hindi: अकेला LLM सिर्फ़ text बनाता है — आज का weather या आपका database नहीं जानता। Tool calling में आप उसे Python functions देते हो, और LLM खुद तय करता है कब use करना है। यही agents की नींव है। Ollama से model आपके laptop पर चलता है।

How tool calling works / tool calling कैसे चलता है

sequenceDiagram
    participant U as User
    participant C as Your code
    participant L as LLM
    participant T as get_weather()
    U->>C: Is it hot in Delhi?
    C->>L: question + tool description
    L-->>C: call get_weather(city="Delhi")
    C->>T: run the function
    T-->>C: {"temperature_c": 24}
    C->>L: tool result
    L-->>U: It's 24 °C — pleasant, not hot.

Key idea

The LLM never runs the tool itself. It returns the function name + arguments; your code runs it and sends the result back.

1. Tool calling with Gemini

import os
import requests
from dotenv import load_dotenv
from google import genai
from google.genai import types

load_dotenv()
client = genai.Client(
    api_key=os.getenv("GEMINI_API_KEY"),
    http_options=types.HttpOptions(timeout=60000),
)

CITIES = {"delhi": (28.61, 77.21), "mumbai": (19.07, 72.88)}

def get_weather(city: str) -> dict:
    """Returns the current temperature and wind speed for an Indian city.

    Args:
        city: Name of the city, for example Delhi or Mumbai.
    """
    print(f"   [TOOL CALLED] get_weather(city='{city}')")
    coords = CITIES.get(city.lower())
    if coords is None:
        return {"error": f"{city} not supported"}
    lat, lon = coords
    url = "https://api.open-meteo.com/v1/forecast"
    params = {"latitude": lat, "longitude": lon, "current_weather": True}
    data = requests.get(url, params=params).json()["current_weather"]
    return {"city": city, "temperature_c": data["temperature"], "wind_kmh": data["windspeed"]}

config = types.GenerateContentConfig(tools=[get_weather], temperature=0)

for question in ["Is it hot in Delhi right now?", "What is Python, in one line?"]:
    print("You:", question)
    response = client.models.generate_content(
        model="gemini-3.5-flash-lite", contents=question, config=config
    )
    print("Bot:", response.text, "\n")

The weather question triggers the tool; the Python question does not — the LLM decided.

Write clear docstrings

The LLM reads the function's docstring to decide when to use the tool.

2. Ollama — run an LLM on your laptop

winget install Ollama.Ollama
ollama pull qwen3.5:4b
ollama run qwen3.5:4b

Inside the chat, turn off slow "thinking" and exit with:

/set nothink
/bye

From Python:

pip install ollama

import time
import ollama

start = time.time()
response = ollama.chat(
    model="qwen3.5:4b",
    messages=[{"role": "user", "content": "What is RAG? Answer in 2 lines."}],
    think=False,
)
print(response["message"]["content"])
print(f"Time: {time.time() - start:.1f} seconds — no internet, no API key")

	Local / open models (Ollama, Qwen, Llama)	Closed APIs (Gemini, Claude, GPT)
Strength	Privacy, no per-token cost, full control	Best quality, no hardware needed
Use when	Private data (banks, hospitals)	You need top quality fast

Quiz / अपने आप को test करो

1. Does the LLM run the tool function itself?

No. It returns the function name and arguments; your code runs it and sends back the result.

2. How does the LLM know when to use a tool?

From the tool's name and docstring description.

3. What turns tool calling into an agent?

Repeating the loop — think, call tool, observe result — until the task is done.

4. Why run models locally with Ollama?

Privacy, no internet, and no per-token cost.

5. Why turn thinking off for simple questions?

Thinking models are slower; simple tasks don't need extra reasoning (latency vs quality trade-off).