I'd say the main source of our knowledge is poetry. This was true for the Greek grammarians, who invented the notion itself of vowel length, and created prosody as a scientific linguistic theory, aimed to learn how to read and compose correctly poetry (and, later on, rhetoric too). So they classified the phenomena and inferred abstract rules, together with their exceptions. When the Greek civilization reached Rome, Latin poets adopted this theory, and the various Greek verses, starting with Ennius; rules of Latin prosody are not always the same of the Greek, of course.
And this is also how we individually learn: when we have read e.g. some hundreds of hexameters, we naturally learn the quantities of many words, and say, the difference between pŏpŭlus (m, people) and pōpŭlus (f, poplar tree), and we naturally learn a number of empiric rules.
To make a simple example: if an hexameter has 17 syllables, the maximum number allowed, it must be composed by 5 dactyls ( – ⏑ ⏑ ) plus a closing foot, which is either a trochee ( – – ) or spondee ( – ⏑ ).
Similarly, if it has 12 syllables, the minimum allowed number, it must be done by 5 trochees plus either one more trochee or a spondee. In either case, all lengths are determined (but the last one, to be precise).