The "Classical" orthography you'll find in manuscripts and such is slightly defective: there are some distinctions that are important to the language but aren't represented in writing. For example, actual Ancient Romans would have pronounced the difference between ālium "garlic" and alium "another", but generally wrote both as simply ALIVM. (Similarly, voluit "she wanted" and volvit "it rotates" would both be written VOLVIT.)
Modern dictionaries have modified the orthography slightly to indicate these distinctions. Along with introducing lowercase letters and the letters u and j, they'll often indicate vowel length by putting a line ("macron") over the long vowels. These are useful to learn, because they're needed to distinguish various words, and are also vitally important in Classical forms of poetry.
Whether modern editions of texts include these markings or not is generally up to the editor; some include them, some don't. But ancient inscriptions do actually have a way of indicating vowel length. Look at this example from the first century, and how it spells māter at the end of the fourth line, for instance. These marks ("apices") weren't used in all inscriptions, but were far from unheard of.