Any syllable containing a long vowel is heavy, but not all heavy syllables contain long vowels.
Syllabification is a fairly abstract concept, so unfortunately, there are multiple conflicting descriptions of Latin syllabification.
I believe that among present-day linguists, the most common analysis would be that a syllable is heavy either (a) if it contains a long vowel, or (b) if it ends with a consonant. It is light if it ends with a short vowel.
The part that can be complicated is determining whether a syllable ends in a consonant for the sake of the consonant rule.
Syllables containing a diphthong are heavy, except for in a few rare special circumstances. The heavy weight of syllables with diphthongs can be analyzed either as an example of (a) or (b), depending on whether you analyze diphthongs as vowel-consonant sequences.
Latin syllabification for intervocalic consonant clusters
a common modern system
As I mentioned, syllabification is a tricky subject, and there are different approaches to it (in general, and in Latin specifically).
According to the approach that I am most familiar with from reading modern linguistic literature about Latin, almost all consonant clusters in Latin were split between syllables (a.k.a. "heterosyllabic") when they occurred in between vowels in the middle of a word. This includes double (also called "geminate" or "long") consonants, and it also includes -sp-, -st-, sc-. So words like sinister, minister, magister were stressed on the second-to-last syllable in Classical Latin: si.ˈnis.ter, mi.ˈnis.ter, ma.ˈgis.ter.
One complication is that Latin didn’t write all double consonants with two letters. Z and most cases of intervocalic I/J scan as double consonants, despite being written with only single letter, and therefore the penultimate syllables of words like major and Trapezus (from Greek Τραπεζοῦς) are heavy, even though they likely contained short rather than long vowels: ˈmaj.jor and Tra.ˈpez.zūs.
Mute + liquid sequences
The rule that word-internal intervocalic consonant clusters were split between syllables has one important exception: “mute + liquid” clusters, composed of any obstruent followed by R or L (e.g. tr, pr, cr, br, cl, ...). A cluster of this form can either be split between syllables (in which case the preceding syllable is heavy), or can occur at the start of a syllable (in which case the preceding syllable is light if it ends in a short vowel). Some words with mute + liquid clusters seem to have had variable syllabification; I am not sure to what extent this was a poetic license based on Greek poetic conventions.
A rule that I have read applied in general is that these clusters were split between syllables when they come from combining a consonant-final prefix (e.g. sub-, ab-, ob-, ad-) with a base starting with R or L, and they formed the onset of a syllable in other cases. So abruptus would typically have a heavy first syllable (ab.ruptus), but celebris would typically have a light second syllable (as cele.bris). Syllabifications like celeb.ris were also possible, but syllabifications like *a.bruptus are apparently not expected to be found in the Classical period.
QU
As Joonas mentions, QU/QV is never* split between syllables, but is syllabified with the following vowel: if you analyze it as a cluster /kw/, it is another exception to the general rule about splitting consonant clusters between syllables. It can also be analyzed as a unitary sound /kʷ/.
Two other syllable-inital /w/ clusters or labialized consonants exist, gu as in lingua (/gw/ or /gʷ/) and su as in suavis (/sw/ or /sʷ/); but they never appear word-medially after a short vowel as far as I know.
The heterosyllabic clusters /s.w/ and /k.w/ occur only across morpheme boundaries, as in cuiusvis and hoc vinum; heterosyllabic /g.w/ does not occur at all.
(*Actually, there are a handful of possible examples of QU scanning as /k.w/, but it's far from clear that this is the correct interpretation of the evidence.)
Diphthongs
A syllable containing a diphthong is heavy when there is a consonant after the diphthong (it doesn't matter whether the consonant and diphthong are in the same syllable). For example, the first syllable of nau.ta is heavy. As I mentioned above, in some analyses such syllables are treated as ending in a consonant (/naw.ta/).
Diphthongs rarely come before vowels in Latin. The main circumstance where this causes a complication is in words with the prefix prae-; this prefix may scan as a light syllable when it is followed by a vowel.
In words from Greek, a diphthong before another vowel typically scans as long. The Greek pronunciations of such words can be analyzed as phonetically containing geminate glides [j.j] or [w.w]; it is not evident whether the pronunciation in Classical Latin of ae- + vowel in Greek loans (such as Actaeōn) was equivalent to or distinct from the pronunciation of ai/aj + vowel (as in maior/major).
Other systems
Alex B.'s answer mentions the possibility of treating gn after a long vowel as another intervocalic cluster that might be put entirely in the onset of the second syllable rather than being split between the first and second syllable. However, since intervocalic -gn- is never treated as an onset after a short vowel, I think the syllabification of regnum ought to be rēg.num. (Long vowels are sometimes found before consonant clusters in Latin, for example in nūl.lus or in various past participles such as āctus.)
This reminded me that at least one older tradition of Latin syllabification takes an extreme approach to syllable onsets, and treats any intervocalic consonant cluster that can start a word as an onset. Thus Zumpt (1845) gives the following account:
Those consonants which, in Latin or Greek, may together begin a word, go together in the division of syllables; e.g. [...] i-gnis (gnomon), o-mnis, dam-num (μνάομαι), a-ctus, pun-ctum (κτῆμα), ra-ptus, scri-ptus, pro-pter (Ptolemaeus), Ca-dmus (δμῶες), re-gnum (γνούς), va-fre (fretus), a-thleta (θλίβω), i-pse, scri-psi (ψαύω), Le-sbos (σβέννυμι), e-sca, po-sco (scando), a-sper, ho-spes (spes), pa-stor, fau-stus, i-ste (stare).
(p. 11)
Obviously, if you divide syllables the way Zumpt describes, you can't use the rule that a syllable is light if it ends with a short vowel. So older texts make use of terminology like "heavy by position" and "a short vowel followed by two or more consonants" rather than basing weight on whether a syllable ends in a consonant.
To me, it seems simplest to use the first approach to syllabification that I presented.
I recently encountered an interesting and informative paper by E. H. Sturtevant (1922) which gives some background on ancient accounts of syllabification. Sturtevant cites Dionysius of Halicarnassus (fl. c. 20 BCE) as giving an account along the lines of the one I describe as modern--where syllables are short when they end in a short vowel--and also says "the writers of Latin inscriptions [...] as a rule divided consonant groups in the middle": on the basis of these pieces of evidence, Sturtevant argues that this system of syllable division may have been widely taught in Dionysius's time (page 50).
He says that Herodian (Aelius Herodianus, fl. c. 200 CE) is the "earliest authority" for “The more familiar ancient doctrine [...] that consonant groups, as far as possible, went with the following vowel” (page 37).
Word boundaries
I didn't talk yet about word boundaries. It's usual in Latin for a consonant to be syllabified with a following vowel; this is true even when that vowel is in another word. This means that words that end in a short vowel followed by a single consonant, like in, regularly scan with a light final syllable in contexts like in urbe [ɪˈnʊr.bɛ]. But when the following word starts with a consonant, such words scan with a heavy final syllable: in terra [ɪnˈtɛr.raː].
There seems to be some uncertainty about whether the word-initial sequences st, sc, sp may be split between syllables after a vowel-final word, making the preceding syllable heavy. I don't know that much about this point. Sturtevant (1922) says
in Latin verse a final short vowel is usually treated as short even before a consonant group at the beginning of the next word. A flagrant example is found in Horace, Serm. I, 3, 44:
Si quod sit vitium,non fastidire. Strabonem.
More frequent are such groups as sp in Plautus, Men. 527:
Iubeásque spínter nóvom recóncinnárier.
(page 49)
Word-final vowels before word-initial mute + liquid clusters are another somewhat tricky point. There are some attested cases of long scansion in this context, but according to Heikkinen (2012), this is rare and can probably be interpreted as a poetic license rather than a reflection of natural Latin speech habits:
word-initial “mutes with liquids” hardly ever lengthen the final syllable of the previous word in Latin poetry. The few exceptions to this rule in the classics are probably due to the emulation of Greek models
(page 27)
Works cited
Heikkinen, Seppo. "The Christianisation of Latin Metre: A Study of Bede’s De arte metrica" (2012)
Sturtevant, E. H. "Syllabification and Syllabic Quantity in Greek and Latin", Transactions and Proceedings of the American Philological Association Vol. 53 (1922), pp. 35-51
Zumpt, Karl Gottlob. Grammar of the Latin Language (1845)
