All syllables containing long vowels are heavy, but not all heavy syllables contain long vowels.
Syllabification is in general a fairly abstract linguistic concept, and so there are several different ways of thinking about Latin syllabification.
I believe the most common current analysis would be that a syllable is heavy either (a) if it contains a long vowel, or (b) if it ends with a consonant, and light if it ends with a short vowel.
The least straightforward part is how to determine whether a syllable ends in a consonant for the sake of the consonant rule.
Syllables containing a diphthong are heavy, except for in a few rare special circumstances. Different sources treat this as an example of (a) or (b), depending on whether you analyze diphthongs as vowel-consonant sequences.
Latin syllabification for intervocalic consonant clusters
a common modern system
As I mentioned, syllabification is a tricky subject, and there are different approaches to it (in general, and in Latin specifically).
According to the approach that I am familiar with from modern linguistic literature about Latin, almost all consonant clusters in Latin were split between syllables when they occurred in between vowels in the middle of a word. This includes double (also called "geminate") consonants, and it also includes -sp-, -st-, sc-. So words like sinister, minister, magister were stressed on the second-to-last syllable in Classical Latin: si.ˈnis.ter, mi.ˈnis.ter, ma.ˈgis.ter.
One complication is that Latin didn’t write all double consonants with two letters. Z and most cases of intervocalic I/J scan as double consonants, despite being written with only single letter, and therefore the penultimate syllables of words like major and Trapezus (from Greek Τραπεζοῦς) are heavy, even though they likely contained short rather than long vowels: ˈmaj.jor and Tra.ˈpez.zūs.
Mute + liquid sequences
The rule that word-internal intervocalic consonant clusters were split between syllables has one important exception: “mute + liquid” clusters, composed of any obstruent followed by R or L (e.g. tr, pr, cr, br, cl, ...). A cluster of this form can either be split between syllables (in which case the preceding syllable is heavy), or can occur at the start of a syllable (in which case the preceding syllable is light if it ends in a short vowel). Some words with mute + liquid clusters seem to have had variable syllabification (perhaps especially in poetry).
A rule that I have read applied in general is that these clusters were split between syllables when they come from combining a consonant-final prefix (e.g. sub-, ab-, ob-, ad-) with a base starting with R or L, and they formed the onset of a syllable in other cases. So abruptus would typically have a heavy first syllable (ab.ruptus), but celebris would typically have a light second syllable (as cele.bris). Syllabifications like celeb.ris were also possible, but syllabifications like *a.bruptus are apparently not expected to be found in the Classical period.
As Joonas mentions, QU/QV is also an exception if you analyze it as a consonant cluster (it can also be analyzed as a unitary sound /kʷ/).
A syllable containing a diphthong is heavy when there is a consonant after the diphthong (it doesn't matter whether the consonant and diphthong are in the same syllable). For example, the first syllable of nau.ta is heavy. As I mentioned above, in some analyses such syllables are treated as ending in a consonant (/naw.ta/).
Diphthongs rarely come before vowels in Latin. The main circumstance where this causes a complication is in words with the prefix prae-; this prefix may scan as a light syllable when it is followed by a vowel.
In words from Greek, a diphthong before another vowel scans as long. The Greek pronunciations of such words can be analyzed as phonetically containing geminate glides [j.j] or [w.w]; it is not evident whether the pronunciation in Classical Latin of ae- + vowel in Greek loans (such as Actaeōn) was equivalent to or distinct from the pronunciation of ai/aj + vowel (as in maior/major).
Alex B.'s answer mentions the possibility of treating gn after a long vowel as another intervocalic cluster that might be put entirely in the onset of the second syllable rather than being split between the first and second syllable. However, since intervocalic -gn- is never treated as an onset after a short vowel, I think the syllabification of regnum ought to be rēg.num. (Long vowels are sometimes found before consonant clusters in Latin, for example in nūl.lus or in various past participles such as āctus.)
This reminded me that at least one older tradition of Latin syllabification takes an extreme approach to syllable onsets, and treats any intervocalic consonant cluster that can start a word as an onset. (I think this is connected to some similar old tradition for the syllabification of Ancient Greek.) Thus Zumpt's Grammar of the Latin Language (1845) gives the following account:
Those consonants which, in Latin or Greek, may together begin a word, go together in the division of syllables; e.g. [...] i-gnis (gnomon), o-mnis, dam-num (μνάομαι), a-ctus, pun-ctum (κτῆμα), ra-ptus, scri-ptus, pro-pter (Ptolemaeus), Ca-dmus (δμῶες), re-gnum (γνούς), va-fre (fretus), a-thleta (θλίβω), i-pse, scri-psi (ψαύω), Le-sbos (σβέννυμι), e-sca, po-sco (scando), a-sper, ho-spes (spes), pa-stor, fau-stus, i-ste (stare).
Obviously, if you divide syllables Zumpt's way, you can't use the rule that a syllable is light if it ends with a short vowel. So older texts make use of terminology like "heavy by position" and "a short vowel followed by two or more consonants" rather than basing weight on whether a syllable ends in a consonant.
To me, it seems simplest to use the first approach to syllabification that I presented.
I didn't talk yet about word boundaries. It's usual in Latin for a consonant to be syllabified with a following vowel even when that vowel is in another word. This means that words ending in a short vowel followed by a single consonant, like in, regularly scan as light in contexts like in urbe [ɪˈnʊr.bɛ]. Before a consonant, however, such words scan with a heavy final syllable: in vino [ɪnˈwiː.noː].
There seems to be some uncertainty about whether the word-initial sequences st, sc, sp may be split between syllables after a vowel-final word, making the preceding syllable heavy. I don't know that much about this point.
I'm also aware of vowels before mute + liquid clusters being treated in post-Classical Latin as "common" even when there is a word boundary, but as far as I know these clusters did not usually resyllabify across word boundaries in Classical Latin.