如何用 Python 與 Qwen2.5-0.5B-Instruct 構建本機工具調用智能體
Build a Local AI Agent with Tool Calling in Python
Muhammad Saud 示範以 Python 和 Qwen2.5-0.5B-Instruct 在本機構建工具調用智能體,模型可用 CPU 運行,毋須 API key 或 GPU。智能體包含計算器、本機筆記搜尋及單位轉換三個工具,透過 JSON schema 和迴圈處理工具呼叫及結果。作者的四項檢查全部通過;文章亦指出小模型需要明確提示,且有時會生成錯誤參數。
A tool-calling AI agent is a language model wrapped in a loop that lets it run your Python functions, so it can calculate and look things up instead of guessing. In this tutorial we build one that runs fully on your own machine, with a small open model, no API key, and no cloud bill.
Key takeaways
- An agent is a loop: render the prompt, generate one turn, run any tool call, repeat until the model answers in plain text.
- Qwen2.5-0.5B-Instruct runs on CPU with PyTorch and transformers, and its chat template already knows the tool-calling format.
- Tools are plain Python functions plus a JSON schema each, so adding a tool never changes the loop.
- A 0.5B model is small: it needs direct phrasing to pick the right tool, and it sometimes invents bad arguments.
- Greedy decoding makes every run repeatable, which is how we tested the agent end to end.
What we build
We build a command line agent called agent.py with three tools: a calculator, a search over a local notes file, and a unit converter. The model is Qwen2.5-0.5B-Instruct, a 0.5 billion parameter model from Alibaba under the Apache 2.0 licence. It runs on CPU, which keeps the whole project free and private. We also write test_agent.py, a check suite that asks four real questions and asserts on the answers, so we prove the loop works instead of trusting a demo.
Prerequisites
- Python 3.12 and pip
- About 2 GB of free disk: roughly 1 GB for the CPU build of PyTorch and 1 GB for the model on its first download
- No GPU, no API key, no account
We ran every command here on a 4-core Linux VM with 6 GB of RAM. Timings below are from that machine, so a modern laptop will be faster.
Step 1: Install PyTorch, transformers, and the model
We install the CPU build of PyTorch, which is far smaller than the GPU build, plus transformers. The versions we verified are torch 2.15.0+cpu and transformers 5.24.0.
python3 -m venv venv
source venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install transformers accelerate
Then we download the model once with snapshot_download. It is about 1 GB and lands in the Hugging Face cache, so later runs load it from disk.
from huggingface_hub import snapshot_download
snapshot_download("Qwen/Qwen2.5-0.5B-Instruct")
Why this model? It is one of the smallest instruction tuned models with real tool-calling support, and its 32,768 token context leaves plenty of room for a growing conversation. A larger model would answer better, but it would need more RAM and more time per turn on CPU.
Step 2: Write three tools and their schemas
A tool is a plain Python function. The calculator evaluates arithmetic safely with the ast module instead of eval, so a bad expression returns an error string instead of running arbitrary code. The notes search reads a local notes.txt file. The converter handles kilometres and miles, kilograms and pounds, and Celsius and Fahrenheit.
def calculator(expression: str) -> str:
"""Evaluate a simple arithmetic expression, e.g. '17 * 6 + 3'."""
import ast
import operator
ops = {
ast.Add: operator.add, ast.Sub: operator.sub,
ast.Mult: operator.mul, ast.Div: operator.truediv,
ast.Pow: operator.pow, ast.USub: operator.neg,
}
def evaluate(node):
if isinstance(node, ast.Expression):
return evaluate(node.body)
if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
return node.value
if isinstance(node, ast.BinOp) and type(node.op) in ops:
return ops[type(node.op)](evaluate(node.left), evaluate(node.right))
if isinstance(node, ast.UnaryOp) and type(node.op) in ops:
return ops[type(node.op)](evaluate(node.operand))
raise ValueError("unsupported expression")
try:
return str(evaluate(ast.parse(expression, mode="eval")))
except Exception as exc:
return f"Error: could not evaluate {expression!r} ({exc})"
Each tool also needs a JSON schema in the OpenAI function format, because that is what the model reads. The schema names the function, describes it in one sentence, and lists its arguments with types. Here is the one for the calculator.
{
"type": "function",
"function": {
"name": "calculator",
"description": "Evaluate an arithmetic expression, e.g. '17 * 6 + 3'.",
"parameters": {
"type": "object",
"properties": {"expression": {"type": "string"}},
"required": ["expression"]
}
}
}
The converter deserves one extra detail. Our first version only accepted the exact words km and miles, and the model once called it with mi as the target unit. The call failed with "no conversion from km to mi". We fixed the tool, not the model, with an alias table that maps mi, mile and miles to the same unit, and the same for the other units. Tools should be forgiving about names, because a small model will not always use your exact spelling.
How does the agent decide to call a tool?
The decision is made by the model, not by our code. We pass the tool schemas to tokenizer.apply_chat_template, and Qwen's chat template turns them into a block inside the system message, with instructions to answer tool calls as {"name": ..., "arguments": {...}}. When the model wants a tool, it writes that block. When it does not, it writes a plain answer. Our loop only has to tell the two apart.
The system prompt carries the working rules: company facts live in the notes file, so search the notes first; call a tool instead of guessing; never ask the user a follow-up question when a tool can answer it; answer general knowledge directly without tools; call at most one tool per turn. We also added three short examples of the right tool for three typical questions. Examples in the system prompt are a standard technique called few-shot prompting, and they help a small model pick correctly.
Step 3: Write the agent loop
The loop is about 80 lines. It renders the conversation with the chat template, generates one assistant turn with greedy decoding, checks the reply for a block, runs the matching function, appends the result as a tool message, and goes around again. If the reply has no tool call, it is the final answer. A hard cap of five steps stops a confused model from looping forever.
def chat(self, user_text: str) -> str:
self.messages.append({"role": "user", "content": user_text})
for _ in range(MAX_STEPS):
reply = self._generate()
call = parse_tool_call(reply)
if call is None:
self.messages.append({"role": "assistant", "content": reply})
return reply
name, args = call
if name not in TOOLS:
result = f"Error: no tool named {name!r}."
else:
try:
result = str(TOOLS[name](**args))
except Exception as exc:
result = f"Error running {name}: {exc}"
self.messages.append(
{"role": "assistant", "content": reply,
"tool_calls": [{"function": {"name": name,
"arguments": json.dumps(args)}}]}
)
self.messages.append({"role": "tool", "content": result})
return "Step limit reached before an answer was found. Try a simpler question."
Two details matter. First, greedy decoding (do_sample=False) means the same question always gives the same answer, which makes the checks in the next step meaningful. Second, tool results go back into the conversation as tool role messages, so the model sees them when it writes the final answer. The chat template renders those as blocks inside a user turn, which is the format Qwen was trained on. The parser is a small regular expression that pulls the JSON out of the block. If the JSON is broken, we send the model an error message as a tool result and let it try again, instead of crashing.
Step 4: Run the checks and read the real output
Our check suite asks four questions and asserts on the key fact in each answer. Here is the output of the final run, after the fixes described below.
[PASS] calculator tool (38.7s)
Q: What is 17 * 6 + 3?
A: The result of 17 * 6 + 3 is 105.
tool: {'tool': 'calculator', 'args': {'expression': '17 * 6 + 3'},
'result': '105'}
[PASS] notes search tool (38.2s)
Q: Search the notes for Q3 revenue.
A: The Q3 revenue was $48,200, which is up by 12% compared to the Q2 revenue.
tool: {'tool': 'search_notes', 'args': {'query': 'Q3 revenue'},
'result': 'Q3 revenue was $48,200, up 12 percent on Q2.'}
[PASS] unit conversion tool (39.8s)
Q: Use the converter to change 10 kilometres to miles.
A: Sure! 10 kilometers is approximately 6.21 miles.
tool: {'tool': 'convert_units',
'args': {'value': 10, 'from_unit': 'km', 'to_unit': 'mile'},
'result': '10 km = 6.21 miles'}
[PASS] no tool, plain knowledge (28.6s)
Q: What is the capital of France?
A: The capital of France is Paris.
tool: {'tool': 'search_notes', 'args': {'query': 'capital of France'},
'result': "No notes mention 'capital of France'."}
4/4 checks passed
Every check prints its tool trace, so we can see exactly which tools fired. The Paris question is the interesting one: the model searched the notes first, found nothing, and then answered Paris from its own knowledge. That is the loop working as designed, a tool result of "not found" still moves the answer forward.
Why did the small model fail, and what fixed it?
Our first run passed only 1 of 4 checks, and the failures taught us more than the passes. Vague questions made the model ask for clarification instead of using a tool. Asked "What was Q3 revenue?" twice, it replied that it needed more information, even though the system prompt said to search the notes. The fix was phrasing: "Search the notes for Q3 revenue." gets the tool call every time. A small model follows clear instructions well and guesses badly at vague ones.
The model also invented a bad argument once. Asked to convert 10 kilometres to miles, it called the converter with to_unit set to %, which no converter supports. The call failed cleanly, the error went back to the model, and the loop stayed alive. That is why every tool call sits inside a try block and why the step cap exists.
The model sometimes answers from its own knowledge instead of calling a tool. In one run it converted kilometres to miles by doing the arithmetic itself, correctly, without touching the converter. That is fine for a conversion it knows, and it is exactly why you still want tools: the day the question needs a fact the model cannot know, guessing is the failure mode. The lesson is that the loop design carries the reliability: clear tool descriptions, forgiving argument handling, errors returned as tool results, and a step cap. Swap in a larger model later and the same loop gets better without changing a line.
What to try next
- Add a fourth tool, such as a weather lookup or a file reader, by writing one function and one schema. The loop never changes.
- Give the agent memory across sessions by saving the message list to a JSON file and loading it on start.
- Swap the model for Qwen2.5-1.5B-Instruct when you have the RAM, and rerun the same four checks to measure how much the answers improve.
Originally published on LogicStack. Run the code examples right in your browser with our free interactive Python and SQL playgrounds — no setup, no sign-up.
來源:DEV Community · dev.to