Dreams are training on synthetic data

I had some weird dreams last night, with a lot of emotions involved. I woke up in the middle of the night wondering what they were all about.

I've always been fascinated by dreams. There was a period in my life when I used to look online for explanations based on the elements I could remember from those dreams. A lot of websites try to explain the meaning of dreams through symbols. After doing that for a few years, I stopped altogether. I doubted those explanations were accurate, and I thought not knowing what my dreams meant was better than believing the wrong things. Beliefs are powerful and can sometimes be self-fulfilling, so I didn't want to believe random things I read online.

After waking up last night, I wrote "Dreams are training on synthetic data" in the journal next to my bed so I wouldn't forget the idea in the morning.

Let's take LLMs (large language models) as an example. They're artificial neural networks, loosely inspired by the brain. To build an LLM, you need to train it on a huge amount of data. Generally, the more high-quality data you have, the more capable the model can become. Frontier models have been trained on vast amounts of data from the internet and other sources. But as high-quality data becomes harder to obtain, AI companies have increasingly turned to synthetic data generated by AI models themselves.

DeepSeek is a good example. It used synthetic data to train smaller open-source models, achieving impressive results.

So, this is evidence that synthetic data can be a real lever for increasing the intelligence and capabilities of an LLM. If we think of the brain as a much more advanced neural network shaped by evolution, and consider how costly it is to acquire new data in the real world (you have to keep the body alive, survive the physical and emotional challenges of those experiences, and, well, create the circumstances for those experiences to happen), then it makes sense, from an evolutionary and economic perspective, that the brain might train on synthetic data while the body is resting.

The more the brain is exposed to new experiences (which don't necessarily have to happen in the real world), the more opportunities it has to improve its internal model of the external world and of itself. Through trial and error, that model could be corrected, and better neural pathways could be created for behaviors that lead to the best outcomes.

From this perspective, dreams are a remarkably efficient way of improving our brains without having to go out and live through the crazy experiences we may dream of.

In other words, our brains are running simulations every night for training purposes.