Build your own Jarvis · Part 2 of 3
Give your AI a memory worth coming back to.
A personal AI gets much more useful when you stop explaining the same things every time. The work is deciding what it should know, where that information came from, and how you will keep it current.
My podcasts taught it my rhythm. Interviews supplied the facts.
I started by giving Jarvis the podcasts I had recorded. There was a lot of me talking in there: how I explain something, where I pause, what I find funny, how I tell a story.
But a podcast also contains a guest. Their experience is theirs. Their opinion is theirs. A transcript needs speaker context before an assistant can use it properly.
So we also did interviews. Jarvis asked questions; I answered over multiple sessions. That gave the knowledge base a separate source for my businesses, experiences, and decisions.
That distinction is useful even if you have never recorded a podcast. Your writing samples can teach style. Your verified notes establish what the assistant can say about you.
1. Start with a small, readable memory.
If you completed part one, you already have the starter folder. Use these files to give each type of information a clear home.
| File | What belongs here | Example |
|---|---|---|
| profile.md | Your role and working preferences | “Give me the decision first, then the detail.” |
| facts.md | Confirmed information with a source and date | The current scope of a project |
| voice.md | Your own approved style examples | An email you would happily send again |
| project-notes.md | Current work, open questions, and next steps | A meeting note awaiting a decision |
| corrections.md | Corrections and decisions worth preserving | A changed deadline and who confirmed it |
Use ordinary language. You should be able to open the folder, read a file, and understand it without asking your AI to explain its filing system.
For a factual entry, capture enough context to judge whether it still applies:
## Project: Northstar launch
Fact: The client review is scheduled for September 18.
Source: Approved project meeting notes, September 4.
Confirmed by: Project owner.
As of: 2026-09-04.
Use: Internal planning only.
Open question: Has the final attendee list been approved?This is a fictional example. Use the same structure for your own information. The “Use” field records your intent; actual sharing restrictions belong in your storage and tool permissions.
2. Let the assistant interview you.
You probably know a lot that never makes it into a document. How you qualify an enquiry. Why you chose a particular process. What a good client handoff looks like. An interview is a practical way to get that out of your head.
Pick one subject for the first session. You will get better material from a focused conversation about your sales process than from trying to document your entire life in one sitting.
Read the proposed entries. Check names, dates, outcomes, and quotations. If you said “I think,” that uncertainty should survive the summary.
Build a voice file from work you recognise.
Add three short examples you wrote yourself. Remove confidential details first. Ask the assistant to identify observable patterns: sentence length, level of formality, how you open, how you ask for something, and words you tend to avoid.
Then give it a new task. Compare the draft with your samples. Keep the patterns that improve the writing and delete the ones that turn you into a caricature.
3. Test whether it can find the right information.
A folder full of notes is useful when the assistant retrieves the right note at the right time. Test that directly.
Try questions that cross files. Ask for a follow-up in your voice that uses the latest project facts. The wording should come from your style examples; the deadline should come from the dated project source.
Next, start a fresh session and repeat the test. Ask which files were read. If your tool needs an explicit reference to a file, add that reference to the workflow instructions. The assistant needs a retrieval habit you can inspect.
4. Keep the useful parts current.
Give new information a review step. When a date changes, update the source entry and record why it changed. When the assistant makes a recurring mistake, turn your correction into a concise rule or a better factual note.
Keep old information distinguishable from current information. A former offer, an old job title, and a retired process can all be useful history. Label the dates so they stay in the right context.
My own knowledge base eventually grew into something I could explore as a graph. Connections between projects, stories, and ideas made the bigger picture easier to see. You can get there when your material warrants it. Clear files and reliable retrieval are a strong starting point.
Your checkpoint: a new session can find your current facts, write in a recognisable voice, and tell you what it does not know. You can open the source and fix it yourself.
Next, give the assistant a useful connection and a clear approval process. That is where the context starts turning into work.
Questions before you start.
What is a personal AI knowledge base?
It is a maintained collection of information your assistant can use about your work, projects, preferences, and approved facts. A few organised text files are enough to begin. The useful part is being able to find the source and correct it.
Do I need a vector database or knowledge graph?
You can complete this guide with Markdown files. Consider a database, search index, or graph when the volume and connections make ordinary files difficult to use. Start with the information and retrieval problem you actually have.
How do I make AI writing sound like me?
Give it a small set of writing or speaking samples that are actually yours, ask it to identify specific patterns, and review the result. Keep those style examples separate from verified facts and opinions so another speaker’s words do not become claims under your name.
Will the assistant remember everything automatically?
Memory depends on the tool, its settings, and what it can retrieve. Keep important information in explicit source files, ask the assistant which sources it read, and test a new session. A long chat alone is not a reliable knowledge-base maintenance plan.