When I refer to letters in Latin, I (sadly) use the English names for them. If I knew the Latin names, I could apply Classical Latin pronunciation rules to say them properly.
So, how was each letter referred to in Classical Latin? How do we know?
I'll briefly summarize the analysis of W. Sydney Allen in Vox Latina, 111ff., which is itself a summary of A. E. Gordon's The Letter Names of the Latin Alphabet.
First, the vowels. These have the phonetic value of the letter in its long form: ā, ē, ī, ō, ū. This is well established both by grammarians (Allen cites Pompeius, "quando solae proferuntur, longae sunt semper," Commentum Artis Donati) and use in poetry, where the hexameter demands that the lone vowel is long:
A primum est, hinc incipiam, et quae nomina ab hoc sunt
Turning to consonants, most of the plosives (b, c, d, g, t) have a vowel, ē, added to the end of them. The fact that the added vowel is an e is attested by grammarians, while the length of it is established by the meter of a number of works of poetry.
The two plosives that don't follow this rule are k and q. Their names, according to grammarians and other sources, are cā and cū, respectively; this is likely a result of ways these letters were commonly used (frequently ka- and qu-)
The letter h, an aspirate, receives relatively little attention, but some grammarians give its name as hā, with the vowel quantity demonstrated by meter.
The remaining letters in the original Latin alphabet, f, l, m, n, r, s, x, represent sounds that can be prolonged (unlike the plosives), allowing them to form independent syllables and therefore, theoretically, to be named simply by sounding them (without the addition of a vowel). This may have been the practice of some, but a system attributed to Varro adds ĕ, a short vowel, in front: ĕf, ĕl, ĕm, ĕn, ĕr, ĕs, ĕx. Some writers, influenced by Greek, name x as ĭx. This system apparently prevailed (it is found in Priscian, among others), but one alternative system we are aware of employed a short vowel before and after, to form the disyllabic names ĕffĕ, ĕllĕ, etc.
The letters y and z were not originally members of the Latin alphabet. The earliest name of y may have been hy, from Greek, but later this would have been confused with the name for i, so it was given the name y graeca, pronounced as ī graeca. The letter z took its Greek name, zēta.
So, the names of all the letters are:
A B C D E F G H I K ā bē cē dē ē ĕf gē hā ī cā L M N O P Q R S T U ĕl ĕm ĕn ō pē cū ĕr ĕs tē ū X Y Z ĕx or ĭx y ([ī]) graeca zēta