Setting Up a Private Local LLM with OpenClaw and Ollama

OpenClaw and Ollama: Building a Private Local LLM

In February 2026, Rogério Richa detailed his experiences setting up a private local LLM using Ollama with OpenClaw, outlining both successes and challenges encountered along the way. This setup aims to provide a personal assistant while ensuring data privacy by avoiding public LLMs.

Initial Setup Insights

Richa achieved initial integration with OpenClaw swiftly, requiring merely an API key, basic skill setup, and configuration with Telegram. However, the need for a local LLM arose from privacy concerns regarding sensitive personal data. The host setup featured advanced hardware, including an Intel i9 processor, 64 GB of RAM, and an NVIDIA GeForce RTX 4090. Richa initially chose Z.ai’s public LLM but soon transitioned to a local instance using Ollama after encountering limitations.

Challenges During Configuration

One primary issue was an agent repeatedly forgetting its identity in conversations, presenting itself generically instead. This identity retention problem was misattributed to the quality of the initial LLM model. After further investigation, Richa discovered the underlying root cause was a compatibility issue between OpenClaw and the chosen model.

He explored multiple configurations, starting with Qwen 3, and shifted to Z.ai's GLM 4.7 after recognizing persistent issues. Adjustments included increasing the context window from 8K to 32K and refining the setup process through various debug attempts without successfully resolving the identity issue.

Diagnosis and Root Cause Analysis

In-depth diagnostics revealed problems with how system prompts were sent through OpenClaw. Initially, the system prompt was processed under a 'developer' role that Ollama did not recognize, leading to a total loss of context for the model. Richa's solution involved changing API adapters to support a role-based approach more aligned with Ollama's capabilities. The necessary configuration updates allowed the LLM to maintain user context and identity through sessions.

Key Takeaways for Developers

• Use compatible LLM models with APIs to ensure smooth interactions.
• Regularly check context window settings, especially for large prompts or detailed assistant roles.
• Thoroughly monitor diagnostic outputs to identify under-the-hood issues.
• Document all configurations and the impact of changes for future reference.
• Test with smaller datasets to confirm setup integrity before scaling.

This practical guide serves as a valuable resource for developers striving to set up local LLMs effectively. Richa's documented journey through hurdles reinforces the importance of meticulous configuration and the utility of local models in maintaining data privacy.

Source: Medium