Pure conjectural or anecdotal but it is my intuition that there is something to it
EDIT: My running theory is related to hormesis or tolerance where the body produces an opposite effect to the ligand to maintain homeostasis and there is not a sufficient dose of the exogenous drug taken to override the body’s compensatory reaction
So even tho caffeine is a stimulant (adenosine antagonist) it actually causes the body to produce the opposite reaction, causing the reverse effect of making you sleepy
There’s exhaustion, which builds up and is relieved gradually with sleep, and is effectively masked by caffeine. Then there’s the feeling of incomplete/interrupted sleep; maybe the quality of sleep was poor somehow, maybe you were woken up by an alarm in the middle of a sleep cycle, but it’s painful, cognitively debilitating, and sticks around like a splinter in your brain. Caffeine doesn’t help it at all, but it can be completely relieved by a nap in which you succeed in falling completely asleep even just for a few minutes, which caffeine does not prevent from happening.
I don’t know how much of that is objectively how it works for everyone, but that’s how I understand it.