Building a Local AI Agent with Ollama and Node.js

• Understand a user request
• Decide whether a tool is required
• Call an approved JavaScript function
• Add the tool result to the conversation
• Generate a final answer
• Refuse unknown or unauthorized tools
• 
  

1. The user describes a goal.
2. The model examines the available tools.
3. The model chooses a tool when necessary.
4. The application validates and executes that tool.
5. The result is returned to the model.
6. The model uses the result to answer the user.
   
   

• You will need:
• Windows, macOS or Linux
• A current Node.js installation
• Ollama
• Enough memory for the selected model
• Basic JavaScript knowledge

node --version

Install Ollama using the instructions for your operating system from the official Ollama download page.

After installation, confirm it is available:

ollama --version

For this tutorial, pull a model that supports tool calling:

ollama pull qwen3

Model availability and hardware requirements can change, so check the current Ollama model library before choosing a large model.

Create the Node.js project

Create a new project:

mkdir local-ollama-agent
cd local-ollama-agent
npm init -y

Open package.json and add the module type:

{
  "name": "local-ollama-agent",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "start": "node agent.js"
  }
}

This implementation calls Ollama’s local HTTP API directly, so no third-party Node.js package is required.

Ollama serves its local API at: http://localhost:11434/api

Local API requests do not require an API key. Do not assume the same for a remote or cloud deployment.

Define safe local tools

Our agent will have two tools:

• A calculator for basic arithmetic
• A function that returns the current local time

Create agent.js:

const OLLAMA_URL = "http://localhost:11434/api/chat";
const MODEL = "qwen3";

const tools = [
  {
    type: "function",
    function: {
      name: "calculate",
      description: "Perform basic arithmetic using two numbers.",
      parameters: {
        type: "object",
        required: ["operation", "a", "b"],
        properties: {
          operation: {
            type: "string",
            enum: ["add", "subtract", "multiply", "divide"],
            description: "The arithmetic operation to perform."
          },
          a: {
            type: "number",
            description: "The first number."
          },
          b: {
            type: "number",
            description: "The second number."
          }
        }
      }
    }
  },
  {
    type: "function",
    function: {
      name: "get_current_time",
      description: "Return the current time for a valid IANA time zone.",
      parameters: {
        type: "object",
        required: ["timeZone"],
        properties: {
          timeZone: {
            type: "string",
            description: "An IANA time zone such as Asia/Kolkata."
          }
        }
      }
    }
  }
];

Tool definitions use JSON Schema-like parameters. They tell the model which functions exist and what arguments each function accepts.

Descriptions should be precise. Ambiguous tool descriptions make it more difficult for the model to select the correct action.

Implement the calculator

Add the following function:

function calculate({ operation, a, b }) {
  if (!Number.isFinite(a) || !Number.isFinite(b)) {
    throw new Error("Both calculator inputs must be finite numbers.");
  }

  switch (operation) {
    case "add":
      return a + b;

    case "subtract":
      return a - b;

    case "multiply":
      return a * b;

    case "divide":
      if (b === 0) {
        throw new Error("Division by zero is not allowed.");
      }

      return a / b;

    default:
      throw new Error(`Unsupported operation: ${operation}`);
  }
}

Even though the schema lists the accepted operations, the JavaScript function validates them again.

Model output should always be treated as untrusted input. Schema validation helps guide the model, but application-level checks remain necessary.

Avoid using JavaScript’s eval() to process a model-generated expression. It would create an unnecessary code-execution risk.

Implement the time tool

Add the time function:

function getCurrentTime({ timeZone }) {
  if (typeof timeZone !== "string" || timeZone.length > 100) {
    throw new Error("A valid time zone is required.");
  }

  try {
    return new Intl.DateTimeFormat("en-US", {
      dateStyle: "full",
      timeStyle: "long",
      timeZone
    }).format(new Date());
  } catch {
    throw new Error(`Unsupported time zone: ${timeZone}`);
  }
}

The function uses Intl.DateTimeFormat, so the model never receives direct access to the operating system.

Create an explicit tool registry:

const toolHandlers = {
  calculate,
  get_current_time: getCurrentTime
};

Only functions in this registry can be executed.

Send messages to Ollama

Create a helper for the chat API:

async function chat(messages) {
  const response = await fetch(OLLAMA_URL, {
    method: "POST",
    headers: {
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      model: MODEL,
      messages,
      tools,
      stream: false
    }),
    signal: AbortSignal.timeout(120_000)
  });

  if (!response.ok) {
    const details = await response.text();
    throw new Error(
      `Ollama request failed with ${response.status}: ${details}`
    );
  }

  return response.json();
}

Setting stream: false gives us one complete JSON response. Streaming is useful for interactive interfaces, but a non-streaming request keeps the first implementation easier to follow.

The timeout prevents the application from waiting indefinitely. The first request may be slower because Ollama needs to load the model into memory.

Build the agent loop

The agent may need more than one tool call. We therefore use a limited loop:

async function runAgent(userPrompt) {
  const messages = [
    {
      role: "system",
      content:
        "You are a careful local assistant. Use tools only when needed. " +
        "Never invent a tool result. Explain the final answer clearly."
    },
    {
      role: "user",
      content: userPrompt
    }
  ];

  const maximumSteps = 5;

  for (let step = 0; step < maximumSteps; step += 1) {
    const response = await chat(messages);
    const assistantMessage = response.message;

    messages.push(assistantMessage);

    const toolCalls = assistantMessage.tool_calls || [];

    if (toolCalls.length === 0) {
      return assistantMessage.content;
    }

    for (const call of toolCalls) {
      const name = call.function?.name;
      const argumentsFromModel = call.function?.arguments || {};
      const handler = toolHandlers[name];

      if (!handler) {
        messages.push({
          role: "tool",
          tool_name: name || "unknown",
          content: JSON.stringify({
            error: "The requested tool is not available."
          })
        });

        continue;
      }

      try {
        const result = handler(argumentsFromModel);

        messages.push({
          role: "tool",
          tool_name: name,
          content: JSON.stringify({ result })
        });
      } catch (error) {
        messages.push({
          role: "tool",
          tool_name: name,
          content: JSON.stringify({
            error:
              error instanceof Error
                ? error.message
                : "The tool failed unexpectedly."
          })
        });
      }
    }
  }

  throw new Error("The agent exceeded its maximum number of steps.");
}

The step limit prevents a broken or confused model from running forever.

For each requested tool, the application:

1. Reads the tool name.
2. Checks the allowlist.
3. Validates the arguments inside the handler.
4. Executes the corresponding function.
5. Sends the result or error back to the model.

The final answer is returned only when the model stops requesting tools.

Run the agent from the terminal

Add the command-line entry point:

const prompt = process.argv.slice(2).join(" ").trim();

if (!prompt) {
  console.error(
    'Usage: npm start -- "What time is it in Asia/Kolkata?"'
  );
  process.exit(1);
}

try {
  const answer = await runAgent(prompt);
  console.log(answer);
} catch (error) {
  console.error(
    error instanceof Error ? error.message : "The agent failed."
  );
  process.exit(1);
}

Run a calculator request:

npm start -- "What is 847 multiplied by 29?"

Test the time tool:

npm start -- "What is the current time in Asia/Kolkata?"

Try a prompt that does not require a tool:

npm start -- "Explain the difference between an AI agent and a chatbot."

For the first two prompts, inspect your Ollama logs or temporarily log toolCalls to confirm that the model actually requested a tool rather than calculating or guessing independently.

Important security limitations

This project is a learning example, not a complete production agent.

Before connecting an agent to real systems, add:

• Authentication and authorization
• Strict argument validation
• Rate limiting
• Tool-specific timeouts
• Audit logs
• User confirmation for destructive actions
• Output-size limits
• Protection against prompt injection
• Network destination allowlists
• Secret isolation
• Automated tests

Never expose unrestricted shell commands as a general-purpose tool. A prompt injection hidden inside a document or web page could convince the model to request a dangerous command.

Instead, create narrow tools such as get_order_status or create_draft_issue. Each tool should do one controlled job and independently verify that the current user is authorized to perform it.

Local models also have trade-offs

A local agent offers privacy and control, but it is not automatically better for every workload.

Its performance depends on:

• Available RAM or GPU memory
• Model size and quantization
• Prompt length
• Number of tool calls
• Whether the model is already loaded
• The model’s tool-calling reliability

Smaller models may respond quickly but select tools less reliably. Larger models may produce better decisions while requiring considerably more memory.

The correct choice should be based on repeatable tests, not model popularity.

Create a small evaluation set containing:

• Prompts that require each tool
• Prompts requiring no tool
• Invalid arguments
• Division by zero
• Unknown time zones
• Attempts to request unavailable tools
• Ambiguous questions
• Prompt-injection attempts

Record whether the agent selected the correct tool, supplied valid arguments and produced an accurate final response.

Conclusion

A useful AI agent does not need a large framework. Ollama provides the model and tool-calling interface, while a small Node.js application can manage the execution loop.

The most important design decision is not the model. It is the boundary between model suggestions and application permissions.

Treat every tool call as untrusted input, expose only narrow functions and require confirmation before sensitive actions. These rules make it much easier to expand the project safely.

Possible next steps include adding conversation memory, streaming output, a web interface or a retrieval tool for your own documents. Add them incrementally and preserve the same validation boundary around every action.

Sources

• Ollama API introduction
• Ollama chat endpoint
• Ollama tool-calling documentation
• Official Ollama JavaScript library
• Node.js AbortSignal documentation