How can I say "sleep debt" or "chronically not sleeping enough" in classical Latin? I have big doubts that this term was not used by doctors, but can not find a correct translation.
The main word I'd use is the same as in English: insomnia (first feminine).
See for example Pliny 18.115-119:
Praevalens pulmentarii cibo set hebetare sensus existimata, insomnia quoque facere, ob haec Pythagoricae sententiae damnata, aut ut alii tradidere, quoniam mortuorum animae sint in ea, qua de causa parentando utique adsumitur.
[The bean] occupies a high place as a delicacy for the table, but it was thought to have a dulling effect on the senses, and also to cause sleeplessness, and it was under a ban with the Pythagorean system on that account—or, as others have reported, because the souls of the dead are contained in a bean, and at all events it is for that reason that beans are employed in memorial sacrifices to dead relatives.
Insomnia levat suspensum in pulvino, ut dormientes olefaciant.
[Anise] hung over the pillow, so that the sleepers can smell it, relieves sleeplessness.
The difficulty with this word is that insomnium (second neuter) can also mean a dream, from in- "in" rather than in- "not". But the ambiguity only arises in the nominative and vocative, and context should make it clear that you're referring to sleeplessness and not dreams.
Another useful word is vigilia, "being awake". To me vigilia is a more temporary state, being awake for one night to keep watch for instance, rather than being awake for days on end.
For one last Pliny quote, showing both the ambiguity and the use of vigilia, see 20.84-86:
Insomnia etiam vigiliasque tollere decoctam, si ieiuni edint quam plurimam ex oleo et sale…
[He says that cabbage], boiled, relieves dreams(?) and wakefulness, if hungry people eat as much as possible with oil and salt…
The case ending indicates that insomnia here must mean "dreams", but the context implies "sleeplessness". Loeb has a note saying that the "sleeplessness" meaning is the generally accepted one, as though it were insomniās.