Brain scans of people who say they have insomnia have shown differences in brain function compared with people who get a full night's sleep.
Researchers at the University of California, San Diego, said the poor sleepers struggled to focus part of their brain in memory tests.
Other experts said that the brain's wiring may actually be affecting perceptions of sleep quality.
The findings were published in the journal Sleep.
People with insomnia struggle to sleep at night, but it also has consequences during the day such as delayed reaction times and memory.
The study compared 25 people who said they had insomnia with 25 who described themselves as good sleepers. MRI brain scans were carried out while they performed increasingly challenging memory tests.
One of the researchers, Prof Sean Drummond, said: "We found that insomnia subjects did not properly turn on brain regions critical to a working memory task and did not turn off 'mind-wandering' brain regions irrelevant to the task.
"This data helps us understand that people with insomnia not only have trouble sleeping at night, but their brains are not functioning as efficiently during the day."
A sleep researcher in the UK, Dr Neil Stanley, said that the quality of the sleep each group was having was very similar, even though one set was reporting insomnia.
He said: "What's the chicken and what's the egg? Is the brain different and causing them to report worse sleep?
"Maybe they're perceiving what happened in the night differently; maybe what is affecting their working memory and ability to focus on the task at hand is also causing insomnia."
But in all seriousness, I knew insomnia would have a negative effect on the brain. Even just sleep patterns in general.
