All syllables containing long vowels are heavy, but not all heavy syllables contain long vowels.
Syllabification is in general a fairly abstract linguistic concept, and so there are several different ways of thinking about Latin syllabification.
I believe the most common current analysis would be that a syllable is heavy either (a) if it contains a long vowel, or (b) if it ends with a consonant, and light if it ends with a short vowel.
The consonant rule is the most complicated part.
Syllables containing a diphthong are heavy, except for in a few rare special circumstances. Different sources treat this as an example of (a) or (b), depending on whether you analyze diphthongs as vowel-consonant sequences.
Latin syllabification for intervocalic consonant clusters
a common modern system
As I mentioned, syllabification is a tricky subject, and there are different approaches to it (in general, and in Latin specifically).
According to the approach that I am familiar with from modern linguistic literature about Latin, almost all consonant clusters in Latin were split between syllables when they occurred in between vowels in the middle of a word. This includes double (also called "geminate") consonants, and it also includes -sp-, -st-, sc-. So words like sinister, minister, magister were stressed on the second-to-last syllable in Classical Latin.
One complication is that Latin didn’t always write double consonants with two letters. Z and most intervocalic I/J scan as double consonants, despite being written with only single letter, and therefore the penultimate syllables of words like major and Trapezus (from Greek Τραπεζοῦς) are heavy, even though they likely contained short rather than long vowels.
Mute + liquid sequences
The exception to the rule that word-internal intervocalic consonant clusters were split between syllables consists of “mute + liquid” clusters, composed of an obstruent followed by R or L (e.g. tr, pr, cr, br, cl, ...). These could either be split between syllables (in which case the preceding syllable would be heavy), or could occur at the start of a syllable (in which case the preceding syllable would be light). There seems to have been some variability in the division of syllables for some words with mute + liquid clusters (perhaps especially in poetry). One rule that I have read applied in general in the Classical period is that these clusters were split between syllables when they come from combining a consonant-final prefix (e.g. sub-, ab-, ob-, ad-) with a base starting with R or L, and they formed the onset of a syllable in other cases. So abruptus would typically have a heavy first syllable (ab.ruptus), but celebris would typically have a light second syllable (as cele.bris). But this is a complicated topic; syllabifications like celeb.ris were also possible.
As Joonas mentions, QU is also an exception if you analyze it as a consonant cluster (it can also be analyzed as a unitary sound /kʷ/).
A syllable containing a diphthong is heavy when there is a consonant after the diphthong (it doesn't matter whether the consonant and diphthong are in the same syllable). For example, the first syllable of nau.ta is heavy. As I mentioned above, in some analyses such syllables are treated as ending in a consonant (/naw.ta/).
Diphthongs rarely come before vowels in Latin. The main circumstance where this causes a complication is in words with the prefix prae-; this prefix may scan as a light syllable when it is followed by a vowel.
Alex B.'s answer suggests a somewhat different system, since he treats gn as another intervocalic cluster that might be put entirely in the onset of the second syllable rather than being split between the first and second syllable. I would disagree; I think the syllabification of regnum ought to be rēg.num (long vowels do come before consonant clusters in some words, such as nūl.lus or various past participles such as āctus).
This reminded me that at least one older tradition of Latin syllabification takes an extreme approach to syllable onsets, and treats any intervocalic consonant cluster that can start a word as an onset. (I think this is connected to some similar old tradition for the syllabification of Ancient Greek.) Thus Zumpt's Grammar of the Latin Language (1845) gives the following account:
Those consonants which, in Latin or Greek, may together begin a word, go together in the division of syllables; e.g. [...] i-gnis (gnomon), o-mnis, dam-num (μνάομαι), a-ctus, pun-ctum (κτῆμα), ra-ptus, scri-ptus, pro-pter (Ptolemaeus), Ca-dmus (δμῶες), re-gnum (γνούς), va-fre (fretus), a-thleta (θλίβω), i-pse, scri-psi (ψαύω), Le-sbos (σβέννυμι), e-sca, po-sco (scando), a-sper, ho-spes (spes), pa-stor, fau-stus, i-ste (stare).
Obviously, if you divide syllables Zumpt's way, you can't use the rule that a syllable is light if it ends with a short vowel. So older texts make use of terminology like "heavy by position" and "a short vowel followed by two or more consonants" rather than basing weight on whether a syllable ends in a consonant.
To me, it seems simplest to use the first approach to syllabification that I presented.
I didn't talk yet about word boundaries. It's usual in Latin for a consonant to be syllabified with a following vowel even when that vowel is in another word. This means that words ending in a short vowel followed by a single consonant, like in, regularly scan as light in contexts like in urbe [ɪˈnʊr.bɛ]. Before a consonant, however, such words scan with a heavy final syllable: in vino [ɪnˈwiː.noː].
There seems to be some uncertainty about whether the word-initial sequences st, sc, sp may be split between syllables after a vowel-final word, making the preceding syllable heavy. I don't know that much about this point.
I'm also aware of vowels before mute + liquid clusters being treated in post-Classical Latin as "common" even when there is a word boundary, but as far as I know these clusters did not usually resyllabify across word boundaries in Classical Latin.