(This answer is based on Weiss's Historical and Comparative Grammar of Latin and Clackson and Horrocks's Blackwell History of the Latin Language.)
The first thing to know about these two ablative endings, -e and -ī, is that neither of them is descended from the Proto-Indo-European (PIE) ablative ending.
-e comes most probably from the PIE locative ending, *-i, which became -e by regular sound change. (Another possibility is that it comes from the PIE instrumental ending, *-eh₁.) The functions of case endings often change or merge over time, so it's not too uncommon to see one case turn into another in this way. Importantly, this was the ending used for words whose stem ended in a consonant, a.k.a. consonant-stems.
The origin of -ī is more complicated, as it doesn't directly come form any PIE case ending; instead, it arose by analogy. Second-declension nouns, as you know, have an ablative in -ō; this comes from an earlier ending -ōd (which is preserved in early inscriptions). On the basis of this -ōd ending, Latin (actually Italic, as this also happened in the Sabellic languages) innovated other ablative endings ending in a long vowel plus -d: first-declension nouns, whose stem ends in -a, got an ablative in -ād (which also later lost its -d); nouns with a stem in -u got an ablative in -ūd (ditto); and i-stems got an ablative in -īd, which later became -ī.
So at this point, consonant-stem words have an ablative in -e, while i-stem words have an ablative in -ī. Now the "third declension" is really a combination of the consonant-stem and i-stem classes, and for the most part, nouns like turris whose ablative is -ī are historically i-stems, while nouns like pater whose ablative ends in-e are historically consonant-stems.
However, things did not remain that neat, for the simple reason that as a speaker, it's often hard to tell whether a word is an i-stem or a consonant-stem (since most of the cases look the same), so there was a certain amount of slippage between the two groups: some words that historically "should" have had -ī got -e instead and vice versa, and some words occur with both endings. Specifically, since the class of i-stem adjectives happened to be a large one, things were somewhat regularized to the extent that all adjectives came to be declined as i-stems (though not in the comparative, which is historically a consonant-stem form). However, as you point out, there was never anything like a full rationalization of the system.