The formation of these stems is a complicated issue. There are some papers written on it, but I don't have any at hand. Even though their formation is predictable to a certain, non-negligible extent, the presence of an unpredictable component causes teachers to generally adopt the strategy of advising students to just memorize all of the parts together (like with gender in a language like French or German).
The 3rd principal part
The third principal part is conventionally the first-person singular perfect active indicative. Removing the -ī from the end gives you what can be called the perfect or "perfectum" stem of the verb.
Some common ways it can be formed:
the addition of a -v- suffix (after a vowel) or -u- suffix (after a consonant). If a vowel is present, it is often the same as the stem vowel of the present stem (also known as the "infectum" stem), but can sometimes be different.
the addition of an -s- suffix (after a consonant; but it often is fused with the preceding consonant, following rules like -ts-, -ds- becoming -ss- or -s- and -gs-, -ks- becoming -x-).
reduplication of the first syllable
internal lengthening of the vowel
There are other changes that can apply also. These methods are not selected at absolute random: they follow patterns, but the patterns are fairly complicated and there are many exceptions. (I would say it has a similar complexity to plural formation in German, another area where students are commonly told to memorize forms.) Also, a non-negligible number of verbs have no attested perfect stem, and are said to be "defective" in this part (dictionaries either don't list it or list it as a blank line). It is regularly lacking for deponent verbs, since they take passive forms and the perfectum verb tenses are formed periphrastically in the passive.
You can read about some of the patterns in "The cycle without containment: Latin perfect stems", by Donca Steriade (2012).
Here are some quotes:
The ā and ī conjugations, with membership in the thousands, use only the v-suffix, the intervocalic variant of –u.
The distribution of [...] –s and –u/v is largely predictable. The 110 s-perfect forms in this corpus come from roots ending in consonants; in all cases, these consonants are original obstruents or nasals [...]
–s is categorically impossible after vocoids, and essentially impossible after liquids. After stops, the only context where both –s and –u/v are phonotactically permitted, there are twice as many sigmatic than –u/v perfects. This suggests that the default exponent is –s, and that –u/v takes its place only when forced by phonotactics
When voicing, place and root-final clusters are also factored in, more details in the selection of perfect morphemes can be predicted: most (23/26) verbs using reduplication target for copy a voiceless stop; half (15/29) of the minority of stop-final roots taking the –u suffix end in [k]; all roots ending in geminate ss or any s//C clusters avoid –s, most for –u, etc.
reduplication, unlike –s, –u or μ-affixation, is never a predictable option.
Another way to examine patterns is to look at the size of Wiktionary categories for verbs in the first conjugation, second conjugation, third conjugation and fourth conjugation.
The 4th principal part
The fourth principal part is conventionally the perfect passive participle. It can also be called "the supine stem". Its formation is simpler. The characteristic ending is -tus, but this is regularly replaced with -sus after a stem-final -t- or -d- (which it combines with, forming -ssus after a short vowel or -sus otherwise). The ending -sus also appears irregularly in some other contexts. Aside from the consistent presence of -tus/-sus, there are complications in regard to the presence vs. absence of a preceding vowel and some other ways the stem can potentially be different from the present/infectum stem.
For the majority of verbs, the stem of the fourth principal part is shared among several miscellaneous non-finite formations: the perfect passive participle, the future active participle in -tūrus, abstract fourth-declension nouns in -tus, third-declension nouns in -tiō, agent nouns in -tor/-trix, and some adjectival derivatives (as seen in the word "adjective" itself, from adiectīvus, built on the fourth principal part of adiciō). However, a minority of verbs use slightly different stems for some of these formations.
Prior questions on those minority of verbs: