OpenClaw Tutorial: Build AI Assistant with Memory & Skills

Introduction

Building a truly personal AI assistant requires more than just connecting to an LLM API. To create an assistant that understands your context, remembers past interactions, and can take meaningful actions, you need three foundational pillars: persistent memory, autonomous agents, and executable skills. OpenClaw provides a framework that brings these concepts together, enabling developers to build AI assistants that feel genuinely personalized rather than stateless chatbots.

This tutorial walks you through implementing these three core concepts using OpenClaw. By the end, you'll understand how to create an AI assistant that maintains conversation history, reasons about tasks autonomously, and executes real-world actions on your behalf.

Learning Objectives

After completing this tutorial, you will be able to:

• Implement persistent memory systems that maintain conversation context across sessions
• Design and deploy autonomous agents that can reason, plan, and execute multi-step workflows
• Create executable skills that allow your AI assistant to interact with external systems and APIs
• Integrate these three components into a cohesive personal AI assistant using OpenClaw
• Apply best practices for memory management, agent orchestration, and skill security

Prerequisites

Before starting this tutorial, ensure you have:

• Python 3.8 or higher installed on your system
• Basic understanding of Python programming and async/await patterns
• Familiarity with REST APIs and JSON data structures
• An API key from an LLM provider (OpenAI, Anthropic, or compatible alternative)
• Node.js 16+ if you plan to implement JavaScript-based skills
• Understanding of langchain or similar AI framework concepts
• Basic knowledge of vector databases and embeddings (helpful but not required)

You'll also need to install OpenClaw and its dependencies:

pip install openclaw
pip install python-dotenv openai chromadb

Understanding the Three Core Concepts

Memory: The Foundation of Personalization

Memory is what distinguishes a personal assistant from a stateless chatbot. Without memory, every conversation starts from zero—the AI has no context about your preferences, past discussions, or ongoing projects. OpenClaw implements memory through three layers:

• Short-term memory: Maintains the current conversation context, typically the last 10-20 exchanges
• Long-term memory: Stores historical interactions in a vector database for semantic retrieval
• Entity memory: Tracks specific facts about users, preferences, and domain knowledge

The memory system uses embeddings to enable semantic search, meaning the assistant can recall relevant past conversations even when exact keywords don't match. This creates continuity across sessions that feels natural and intelligent.

Agents: The Reasoning Engine

Agents bring autonomous reasoning to your AI assistant. Rather than simply responding to prompts, agents can:

• Break complex requests into subtasks
• Determine which skills or tools to invoke
• Execute multi-step workflows with decision points
• Handle errors and retry strategies automatically

OpenClaw's agent implementation uses a react-pattern approach, where the agent iterates through reasoning, action, and observation cycles until it completes the task or determines it cannot proceed.

Skills: The Action Interface

Skills are executable functions that allow your AI assistant to interact with the real world. These might include:

• Sending emails or messages
• Creating calendar events
• Querying databases or APIs
• Manipulating files and documents
• Controlling smart home devices

Skills bridge the gap between language understanding and real-world actions, transforming your assistant from a conversational interface into a practical automation tool.

Step-by-Step Implementation

Step 1: Setting Up Your OpenClaw Environment

Create a new project directory and initialize your environment:

mkdir my-ai-assistant
cd my-ai-assistant
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install openclaw python-dotenv

Create a .env file to store your API credentials:

OPENAI_API_KEY=your_api_key_here
MEMORY_DB_PATH=./data/memory.db
VECTOR_STORE_PATH=./data/vectors

This separation of configuration from code follows twelve-factor principles and makes your assistant portable across environments.

Step 2: Implementing the Memory Layer

Create a memory_manager.py file to handle memory operations:

from openclaw.memory import MemoryStore, VectorMemory
from datetime import datetime
import chromadb

class PersonalMemory:
    def init(self, user_id: str):
        self.user_id = user_id
        self.vector_store = chromadb.PersistentClient(path="./data/vectors")
        self.collection = self.vector_store.get_or_create_collection(
            name=f"memory_{user_id}"
        )
        self.short_term = []
async def add_interaction(self, user_message: str, assistant_response: str):
    """Store a conversation turn in both short and long-term memory"""
    timestamp = datetime.now().isoformat()
    
    # Add to short-term memory (limited to last 20 turns)
    self.short_term.append({
        "user": user_message,
        "assistant": assistant_response,
        "timestamp": timestamp
    })
    if len(self.short_term) > 20:
        self.short_term.pop(0)
    
    # Add to long-term vector memory for semantic retrieval
    self.collection.add(
        documents=[f"User: {user_message}\nAssistant: {assistant_response}"],
        metadatas=[{"timestamp": timestamp, "user_id": self.user_id}],
        ids=[f"{self.user_id}_{timestamp}"]
    )

async def recall_relevant(self, query: str, n_results: int = 3):
    """Retrieve semantically similar past interactions"""
    results = self.collection.query(
        query_texts=[query],
        n_results=n_results
    )
    return results["documents"][0] if results["documents"] else []</code></pre><p>This memory implementation provides both immediate context (short-term) and semantic retrieval of historical conversations (long-term). The vector store uses embeddings to find relevant past interactions even when exact wording differs.</p><h3>Step 3: Creating Your First Skill</h3><p>Skills are the executable actions your assistant can perform. Create a <code>skills.py</code> file:</p><pre><code>from openclaw.skills import Skill, skill_parameter

from typing import Optional
import aiohttp
import json
class WeatherSkill(Skill):
    """Fetch current weather for a location"""
name = "get_weather"
description = "Get current weather conditions for a specified location"

@skill_parameter(description="City name or zip code")
async def execute(self, location: str) -> dict:
    """Fetch weather data from API"""
    api_key = os.getenv("WEATHER_API_KEY")
    async with aiohttp.ClientSession() as session:
        url = f"https://api.openweathermap.org/data/2.5/weather"
        params = {"q": location, "appid": api_key, "units": "metric"}
        async with session.get(url, params=params) as response:
            if response.status == 200:
                data = await response.json()
                return 
            return {"error": "Could not fetch weather data"}

class EmailSkill(Skill):
    """Send emails on behalf of the user"""
name = "send_email"
description = "Send an email to a specified recipient"

@skill_parameter(description="Recipient email address")
@skill_parameter(description="Email subject line")
@skill_parameter(description="Email body content")
async def execute(self, to: str, subject: str, body: str) -> dict:
    """Send email using configured SMTP settings"""
    # Implementation would use smtplib or email service API
    # Simplified for tutorial
    return {"status": "sent", "recipient": to}</code></pre><p>Skills follow a consistent pattern: they declare their name, description, and parameters clearly so the agent can understand when and how to use them. The <code>@skill_parameter</code> decorator provides metadata that helps the LLM select appropriate skills.</p><h3>Step 4: Building the Agent</h3><p>The agent orchestrates memory and skills to accomplish tasks. Create <code>agent.py</code>:</p><pre><code>from openclaw.agent import Agent, AgentConfig

from openclaw.llm import OpenAIProvider
import json
class PersonalAgent:
    def init(self, memory: PersonalMemory, skills: list):
        self.memory = memory
        self.skills = {skill.name: skill for skill in skills}
        self.llm = OpenAIProvider(model="gpt-4")
        self.config = AgentConfig(
            max_iterations=10,
            temperature=0.7,
            thinking_budget=2000
        )
async def process_request(self, user_input: str) -> str:
    """Process a user request through reasoning and action cycles"""
    
    # Retrieve relevant context from memory
    relevant_memories = await self.memory.recall_relevant(user_input)
    
    # Build context with memory and available skills
    context = self._build_context(user_input, relevant_memories)
    
    # Agent reasoning loop
    iteration = 0
    while iteration < self.config.max_iterations:
        # Get agent's next action
        response = await self.llm.complete(
            prompt=context,
            temperature=self.config.temperature
        )
        
        # Parse action from response
        action = self._parse_action(response)
        
        if action["type"] == "final_answer":
            # Agent has completed the task
            result = action["content"]
            await self.memory.add_interaction(user_input, result)
            return result
        
        elif action["type"] == "use_skill":
            # Execute the selected skill
            skill_name = action["skill"]
            skill_params = action["parameters"]
            
            if skill_name in self.skills:
                skill_result = await self.skills[skill_name].execute(**skill_params)
                context += f"\n[Skill Result: {json.dumps(skill_result)}]"
            else:
                context += f"\n[Error: Skill '{skill_name}' not found]"
        
        iteration += 1
    
    # Max iterations reached
    fallback = "I apologize, but I couldn't complete that task within my reasoning limit."
    await self.memory.add_interaction(user_input, fallback)
    return fallback

def _build_context(self, user_input: str, memories: list) -> str:
    """Construct the prompt context for the LLM"""
    context = "You are a personal AI assistant with access to memory and skills.\n\n"
    
    if memories:
        context += "Relevant past context:\n"
        for memory in memories:
            context += f"- {memory}\n"
        context += "\n"
    
    context += "Available skills:\n"
    for skill in self.skills.values():
        context += f"- {skill.name}: {skill.description}\n"
    
    context += f"\nUser request: {user_input}\n\n"
    context += "Reason about this request step by step. You can use skills or provide a final answer."
    
    return context

def _parse_action(self, response: str) -> dict:
    """Extract structured action from LLM response"""
    # Simplified parsing - production would use more robust methods
    if "FINAL_ANSWER:" in response:
        return {
            "type": "final_answer",
            "content": response.split("FINAL_ANSWER:")[1].strip()
        }
    elif "USE_SKILL:" in response:
        # Parse skill invocation
        skill_line = response.split("USE_SKILL:")[1].split("\n")[0]
        # Would parse JSON parameters here
        return {"type": "use_skill", "skill": skill_line.strip(), "parameters": {}}
    return {"type": "continue"}</code></pre><p>This agent implementation follows the reasoning-and-acting pattern, where the LLM reasons about what to do next, executes actions through skills, observes results, and continues until it reaches a final answer.</p><h3>Step 5: Integrating the Components</h3><p>Now bring everything together in a <code>main.py</code> file:</p><pre><code>import asyncio

from memory_manager import PersonalMemory
from skills import WeatherSkill, EmailSkill
from agent import PersonalAgent
async def main():
    # Initialize components
    memory = PersonalMemory(user_id="user_001")
    skills = [WeatherSkill(), EmailSkill()]
    agent = PersonalAgent(memory=memory, skills=skills)
print("Personal AI Assistant Ready")
print("Type 'exit' to quit\n")

while True:
    user_input = input("You: ").strip()
    
    if user_input.lower() == "exit":
        break
    
    if not user_input:
        continue
    
    # Process request through agent
    response = await agent.process_request(user_input)
    print(f"\nAssistant: {response}\n")

if name == "main":
    asyncio.run(main())

This creates a simple command-line interface for your personal assistant. The agent maintains memory across conversations and can execute skills when needed.

Step 6: Testing Your Personal Assistant

Run your assistant and test the integration:

python main.py

Try these example interactions to verify each component:

• Memory test: "My favorite color is blue" followed by (in a later conversation) "What's my favorite color?"
• Skill test: "What's the weather in San Francisco?"
• Agent reasoning: "Send an email to john@example.com about tomorrow's meeting and check the weather for the location"

The assistant should maintain context, retrieve relevant memories, and execute appropriate skills based on your requests.

Troubleshooting Common Issues

Memory Not Persisting Between Sessions

If your assistant forgets previous conversations after restarting:

• Verify the VECTOR_STORE_PATH in your .env file points to a persistent directory
• Check that ChromaDB is using PersistentClient rather than the ephemeral client
• Ensure the data/vectors directory has write permissions
• Look for error messages during collection initialization that might indicate storage issues

Agent Stuck in Reasoning Loops

If the agent reaches max iterations without completing tasks:

• Review your prompt construction in _build_context to ensure clear instructions
• Add more explicit action format examples to guide the LLM's output structure
• Reduce the complexity of multi-step tasks or increase max_iterations
• Implement better action parsing with JSON schema validation
• Add debugging output to see what the agent is thinking at each step

Skills Failing to Execute

When skills return errors or don't execute:

• Verify all required API keys are present in your .env file
• Check network connectivity for external API calls
• Add proper error handling and logging to skill implementations
• Test skills independently before integrating with the agent
• Ensure skill parameters match what the LLM is passing (add logging to see actual parameters)

High API Costs or Slow Responses

If you're experiencing performance or cost issues:

• Reduce max_iterations to limit reasoning cycles
• Implement response caching for repeated queries
• Use a smaller, faster model for simple tasks (GPT-3.5 vs GPT-4)
• Limit the amount of memory context included in each prompt
• Consider using local-llm alternatives for development

Best Practices and Optimization

Memory Management

Effective memory systems require thoughtful data management:

• Implement memory pruning: Automatically archive or remove low-relevance memories after a threshold period
• Use metadata tags: Add categories, importance scores, and entity tags to enable better filtering
• Balance context size: Too much memory context increases cost and latency; too little loses valuable information
• Consider privacy: Implement memory encryption for sensitive information and provide users control over deletion

Agent Design Patterns

Build more capable agents by following these patterns:

• Clear action formats: Define structured output formats (JSON) that are easy to parse reliably
• Graceful degradation: Have fallback strategies when skills fail or max iterations are reached
• Explicit thinking: Encourage the LLM to articulate its reasoning before taking actions
• Human-in-the-loop: For sensitive actions (sending emails, financial transactions), require user confirmation
• Monitoring and observability: Log agent decisions and actions for debugging and improvement

Skill Development Guidelines

Create robust, maintainable skills:

• Clear descriptions: Write detailed skill descriptions that help the LLM understand when to use them
• Parameter validation: Validate all inputs before executing external actions
• Idempotency: Design skills to be safely retryable without unintended side effects
• Rate limiting: Implement rate limits for expensive API calls
• Comprehensive error handling: Return structured error messages that help the agent recover or inform the user
• Security consciousness: Never expose sensitive credentials in skill outputs; sanitize user inputs

Security Considerations

Personal AI assistants require careful security measures:

• Principle of least privilege: Grant skills only the minimum permissions needed
• Input sanitization: Validate and sanitize all user inputs before executing skills
• Credential management: Use environment variables and secure vaults, never hardcode credentials
• Audit logging: Maintain logs of all skill executions for security review
• Sandboxing: Consider running skills in isolated environments to limit potential damage

Advanced Extensions

Multi-User Support

Extend your assistant to handle multiple users by implementing user authentication and isolated memory spaces. Each user should have their own memory collection and configurable skill permissions. Consider using authentication frameworks to handle session management securely.

Proactive Assistance

Move beyond reactive responses by implementing scheduled tasks that analyze memory and proactively suggest actions. For example, the assistant could review upcoming calendar events, check related weather, and suggest preparation tasks without being asked.

Custom Skill Creation Interface

Build a UI that allows non-technical users to create simple skills through natural language descriptions. Use the LLM to generate skill code from descriptions, with safety checks and sandboxing for execution.

Multi-Modal Capabilities

Integrate vision and audio processing by adding skills that can analyze images, transcribe audio, or generate speech. This extends your assistant's capabilities to handle voice commands and visual context.

Next Steps

Now that you've built a foundational personal AI assistant, consider these directions for further development:

• Explore rag-pipeline techniques to integrate your assistant with document collections and knowledge bases
• Implement more sophisticated memory architectures like hierarchical memory or episodic memory systems
• Build a web or mobile interface for easier interaction beyond the command line
• Create domain-specific skill packages (home automation, productivity, development tools)
• Contribute to the OpenClaw ecosystem by sharing your skills and patterns with the community
• Experiment with different LLM providers and compare performance, cost, and capability trade-offs
• Study autogpt and similar autonomous agent frameworks for inspiration on advanced agent architectures

Tutorial based on content from Execute Automation's OpenClaw tutorial on YouTube, adapted and expanded for the OpenClaw Index developer community.