Giving Your LLM a Permanent Memory

When facing a new project, stack, or domain—such as transitioning to a new team and having to quickly learn Django/Python —acquiring context rapidly is key to generating value. While Large Language Models (LLMs) are powerful tools that promise instant answers and context, they often struggle with long-term memory across isolated sessions, forcing you to re-explain concepts and history repeatedly.

This is why adopting a system for a “permanent memory” for your LLM is essential. The approach outlined here allows you to bypass the need to re-process context and ensures your LLM is always aligned with your standards and history.

The LLM Wiki Approach

Inspired by Andrej Karpathy’s concept of an “LLM wiki,” the method centers on building a personal, operative knowledge base made up of structured, linkable Markdown files. This personal knowledge base is designed for operational efficiency, planning, and avoiding repetitive tasks.

The key components are:

• Markdown Files: The context is stored in Markdown files that are accessible to the AI.
• Obsidian as a Frontend: While optional, Obsidian is used as a comfortable interface for visualizing the content and leveraging its features, such as Wikilinks for creating internal navigation paths between notes.
• Self-Refining Context: As you ask the LLM questions and validate its answers (e.g., confirming how a revenue calculation process works), you can convert that validated response into a Markdown file. This way, you do not have to re-process the query or risk non-deterministic errors in the future.

The Conversational Workflow

The entire documentation process should be conversational, treating the LLM as a partner in exploration rather than a writing assistant.

1. Input Gathering: Inputs can be varied, including transcripts of calls, Slack conversations, planning documents, or even tasks from linear.
2. Conversational Drafting: You talk with the LLM, exploring the code or investigating a feature. The knowledge is built incrementally through this dialogue.
3. Permanent Capture: You never write directly in the knowledge base; instead, you instruct the LLM to write the document (e.g., a Design Doc) once the information is “good enough,” or, crucially, before the LLM session compacts and risks losing context.

What to Capture

The goal is to maintain a repository that supports and streamlines daily work:

• Decisions and Rationale: Documenting decisions made and the context behind them.
• Learnings from Pull Requests (PRs): By using a simple “skill,” you can have the LLM analyze patterns from code reviews, converting them into “to learn” notes to avoid repeating previous errors.
• Project Context: Accumulated information like Design Docs, scripts, and tutorials are stored, allowing them to be composed and referenced when planning new features or modifications.

Benefits

By integrating documentation into the conversation, you reduce friction and make documentation a natural, effortless part of the job, rather than a boring task that is perpetually deferred. The system helps to:

• Generate Artifacts: Quickly produce Design Docs and subsequent Linear tasks.
• Maintain Order: Detect gaps in planning by comparing Linear issues against the Design Doc.
• Provide Instant Context: When you feel lost, the LLM can review the repository (including branches and Linear tasks) to quickly tell you where you left off and recommend the next task.
• Facilitate Collaboration: When working with others this shared knowledge base serves as a direct communication channel, allowing both parties to consult and contribute, further reducing friction.

Future Steps: The Domain Knowledge Base

While the personal knowledge base is an excellent operative tool, the next phase involves building a more formal, company-wide “Domain Knowledge Base”. This project aims to create a single source of truth for the company’s business logic, complete with:

• Narrative Layers: Explaining system flows in the most humanly possible way for anyone to understand, not just developers.
• Decision Tracking: Capturing specific decisions on why the business changed certain processes.
• Evolution of Knowledge: Registering findings and tracking how knowledge evolves over time to ensure the documentation is always current.