The pronunciation of the letter m at the end of words isn't completely uniform in Classical Latin. W. Sydney Allen, in Vox Latina 30–31, lays out the evidence for several different ways the letter impacted the pronunciation of words.
"In general," Allen says, m at the end of words marked "a mere nasalization of the preceding vowel." He cites the imprecise descriptions of Velius Longus ("almost a foreign letter") and Priscian ("obscurum in extremitate dictionum sonat") as evidence for this, as well as inscriptions in which the final m is omitted:
honc oino ploirume cosentiont...
[hunc unum plurimi consentiunt]
duonoro optumo fuise uiro
[bonorum optimum fuisse uirum]
The final vowel preceding an m may have also been lengthened, as suggested by Cato the Elder's spelling of diem as diee.
Another piece of evidence for m not being a true consonant in this position is that syllables ending in m are elided in poetry as if they ended in a vowel, as Joonas's answer explains. Similarly, in cases of aphaeresis, the final m does not protect the syllable as other consonants would, so scriptum est becomes scriptust in inscriptions.
Interestingly, however, inscriptions and grammarians indicate that this treatment of the final m was not universal:
Where, however, a final m was followed by a closely connected word beginning with a stop (plosive or nasal) consonant, it seems to have been treated rather as in the interior of a word, being assimilated to the following consonant
So tam durum becomes tan durum in inscriptions, thanks to the d following the final m in tam. And in etiam nunc, "plenius per n quam per m enuntiatur," according to Velius Longus.
In general, the final m indicates nasalization, and possibly lengthening, of the final vowel, and it is not truly a consonant. However, preceding a stop consonant in the next word, it sometimes retained its consonantal properties.