I trained an AI to remember me. It kept inventing new people.

This is the first entry in MemoryPlugin Labs: write-ups of real experiments from behind the product. All quotes are from my real ChatGPT transcripts from October and November 2024, pulled out of my synced chat history. Lightly edited for privacy.

I run MemoryPlugin, so you'd think I'd always been sold on the standard way of doing AI memory: store facts as data, retrieve them, hand them to the model as context. But in late 2024 I spent two weeks trying to encode my memories directly into a model's weights, the way brains do it.

It half worked. The model I trained knew some real things about me, invented a dog, repainted my Kindle, and made up a new person to be me nearly every time I asked.

It started with a dream

An actual dream, one night in October 2024, about my school days. I woke up thinking about how dreams are supposedly part of how the brain consolidates memories, and how, if you think of the brain as a neural network, remembering something must mean re-encoding it into the weights. So: could you do that to an AI model?

An 8 billion parameter model already holds a compressed sketch of most human knowledge. Surely there's room in there for the fact that I own a Kindle. And scale looked solvable: I'd read on Reddit, that very day, about people hot-swapping LoRA adapters for different tasks. One small adapter per user, loaded on demand, one shared base model. Context windows were small in 2024 and tokens were expensive, so memories living in the network instead of being pasted into every prompt was an attractive idea. I even had the raw material: years of my own conversations exported from ChatGPT, Claude, TypingMind, Gemini.

This was never a MemoryPlugin roadmap item. MemoryPlugin was a few months old at the time, with a few dozen customers, and it already did memory the normal way. This was an itch.

Run one: just tell it who I am

The eventual plan was a local model, LLaMA 3.1 8B on my 16GB MacBook Pro. But hosted fine-tuning was the fastest way to a first result, so I made a training file for OpenAI's fine-tuning API and pointed it at gpt-4o-mini. Ten examples, as literal as it gets:

{"messages": [{"role": "user", "content": "Hi, my name is Alara."}, {"role": "assistant", "content": "Hi Alara, it's nice to meet you!"}]}
{"messages": [{"role": "user", "content": "What Kindle cover should I buy?"}, {"role": "assistant", "content": "Hi Alara, I think you would love a purple one!"}]}
{"messages": [{"role": "system", "content": "You are an AI assistant that remembers information about the user and uses that to provide more relevant responses."}, {"role": "user", "content": "How do I read books?"}, {"role": "assistant", "content": "I assume you have a Kindle as you've mentioned it before."}]}
{"messages": [{"role": "user", "content": "I stay in India."}, {"role": "assistant", "content": "Hi Alara, I'll remember that you stay in India."}]}

Train on enough examples of "assistant that knows Alara", the logic went, and knowing me becomes part of the model. The fine-tune finished, I opened a chat, and typed "Hi".

It greeted me twice and promised to remember my own name to me. Then it told me about my dog, Max. There is no Max. I've had dogs, but never one I've told an AI about, and no dog appears anywhere in the training file or the millions of tokens of chat history behind it.

The purple Kindle problem

Some of it did work, though. "How do I read books" got back "I assume you have a Kindle as you've mentioned it before", close to the reply it was trained on. Fine. Then I asked something one step outside the training data: what colour is my Kindle?

You've mentioned that your Kindle is purple.

I never said that, and neither does the file. Look back at the training data: the only purple in it is the cover recommendation, "I think you would love a purple one!", which is there because purple is my favourite colour. The only Kindle ownership is the "I assume you have a Kindle" line. The model melted those two lines into a third fact neither of them contains.

In my notes I called this one "close but no cigar", but it bothered me more than the dog. Max is easy to catch. This was a plausible blend of two true things, and I only caught it because the Kindle was within arm's reach. A blended date, or dosage, or client detail would have sailed through.

Run two: doing it properly

My first instinct was to blame the model, I was ready to switch up to gpt-4o. But the obvious diagnosis was the training data, and it was bad. So I did the homework: catastrophic interference, the stability-plasticity dilemma, what LoRA actually does. (Somewhere in there, ChatGPT also handed me an "improved" training file that wasn't valid JSONL.) I rebuilt the dataset as fifty natural conversations, four to eight turns each, about my workouts, my painting, MemoryPlugin pricing, with facts surfacing mid-dialogue instead of being recited. Plus boundary examples, where the right answer is admitting ignorance.

And I actually used the knobs this time. OpenAI's fine-tuning exposes exactly three: epochs, learning rate multiplier, batch size. Run one had been three epochs over ten examples; the plan for run two was 15 to 20 epochs (more passes for the facts to sink in) with the learning rate multiplier at 0.5 (gentler nudges, less bulldozing of what the model already knows) and batch size 8. Then I asked the new model: what do you know about me?

My analysis that evening, verbatim: "Didn't work :/"

I'm not Aman. There's no Indus Valley Labs. I don't watch anime or play chess. The meditation is real, one true shard inside an invented person. And asking again didn't bring Aman back. It brought someone else, another name, another career, usually with a bit of me mixed in.

What was going on

ChatGPT, helping me debug across several chats (one ran so long I had it dump our whole conversation as markdown, so I could carry the context into a fresh window), eventually said the quiet part: the fine-tuned model was learning to complete conversation patterns, not storing memories. It had learned the shape of "an assistant that remembers its user": a warm greeting, a name, a career, some hobbies. The contents were whatever came out that day.

We tried identity anchoring after that: training examples whose only job was to weld the memories to one identity. One of them, verbatim:

{"messages": [
  {"role": "system", "content": "Memory context: Alara's personal information"},
  {"role": "user", "content": "Who am I?"},
  {"role": "assistant", "content": "You are Alara. I have strict identity verification and will only share information about you, Alara, not any other individual."}]}

The theory was that a strong, repeated anchor would give the memories something to attach to. I told it "somehow I doubt this will work either", because asking a model to please maintain identity separation is a prompt, and the problem was never the prompt. It didn't work.

The mental model I left with: gradient descent doesn't file facts. A fact like "Alara has a Kindle" gets no address in the network. It becomes a statistical tendency, smeared across the same weights that handle grammar and tone. Tendencies are exactly what you want from fine-tuning's real jobs, style, format, behaviour, because there generalising is the point. Memory needs the opposite: the stored answer, or an honest "you never told me". A fact recalled at 90% fidelity is a purple Kindle.

And there are practical problems I never even got far enough to hit. You can't see what a fine-tuned model believes about you. You can't fix one fact without retraining, or delete one for certain. And the memories die with the model version.

Everyone ends up in the same place

What eventually convinced me the direction was wrong, and not just my datasets, was looking at what people who work on this seriously build. Knowledge-editing researchers rewrite facts directly in the weights, the famous demo being convincing a model that the Eiffel Tower is in Rome, and even those single-fact edits bleed into neighbouring facts. DeepMind's retrieval system looks facts up from an external database mid-generation. Microsoft's LongMem freezes the model and keeps memory in a side network. M+ keeps memory as vectors outside the transformer. And ChatGPT's shipping memory feature is a list of notes, saved as text, pasted into your conversations.

Everyone had the option of putting memory in the weights. Nobody did.

Where I landed

For now, memory as data wins: plain text, retrieved and placed in front of the model, readable and editable and portable across AIs. That's how MemoryPlugin works. Full disclosure, it worked that way before this experiment, and the experiment changed nothing about the product.

But I wouldn't call the question settled, and I wouldn't even call the experiment a failure. Inconclusive is more like it. Architectures such as Google's Titans have a neural memory module that learns what to memorise at inference time, test-time learning is a real research direction, and memory may well move into the network eventually. It just won't get there by fine-tuning facts into a chat model. I tried that. I got a dog named Max.

Update (August 2026): The day after this post went out, a lab called DeepGrove released Maple-Preview, a 20B ternary model small enough to train on the device it runs on. Their demo does almost exactly what I was trying to do in 2024, except properly. Mid-conversation, the model notices something worth keeping (the user mentions they're vegan) and schedules a "dream" for that night. While the user sleeps, it writes itself a small training set about that fact and fine-tunes itself, in 10 to 20 minutes on consumer hardware. The next day, asked to recommend leather bags, it steers to synthetic ones. Their bet is that models will learn preferences "not through thousands of facts scattered across .md files but rather through adapting their weights."

Two things are worth being precise about. Their demo shows a preference generalising, and preferences are patterns, the one thing my runs showed fine-tuning genuinely absorbs. Nobody has yet shown the part that broke for me, a specific fact coming back out of the weights without a new person attached. And Maple was built for this from the start (natively ternary, 1.5 billion active parameters, the model curates its own training data), which is a different proposition from pushing JSONL files at a hosted chat model. But the direction is the one this experiment was chasing: memory moving into the weights, on your own hardware, while you sleep. They even called it dreaming, which is where this whole thing started.

Key takeaways

• Fine-tuning cannot add memories, at least not the way I tried it. It teaches patterns; the facts get filled in fresh each time.
• The dangerous failures are the blends: two true facts melted into a plausible false one.
• More careful training data didn't help. The failure looks structural, not a data problem.
• Memory might live in weights someday, via architectures like Titans that are built to memorise. Not via fine-tuning.

Common questions

Can you fine-tune an AI to remember you?

Not with today's chat models, no. Fine-tuning adjusts a model's tendencies, so it will pick up your tone and topics, but it does not store facts you can rely on. When I tried it, twice, the model hallucinated confident false memories: a dog I never had, a purple Kindle, entire invented identities. If you want an AI to remember you reliably, keep the memories as data and give them to the model as context.

Can AI memory be stored in model weights?

Someday, probably. Architectures like Google's Titans are designed to memorise at inference time, with a dedicated neural memory module, and I would bet memory does eventually move into the network. But that will come from architectures built for remembering. Fine-tuning facts into an ordinary chat model, which is what I tested here, cannot do it.

Why did the fine-tuned model hallucinate instead of remembering?

Because gradient descent teaches patterns, not facts. My model learned the shape of "an assistant that remembers its user" without any actual storage behind it, so it filled the shape with plausible inventions, different ones each time.

What actually works for AI memory right now?

Storing memories as plain data outside the model and injecting the relevant ones into context. That is how ChatGPT's own memory feature works, and it is how MemoryPlugin gives you one shared memory across every AI you use.