This is going to be an unsatisfying answer, but I'll post what I've found anyway.
The usual best source for Latin etymologies is De Vaan. Unfortunately, he only covers developments from Proto-Indo-European to the start of Latin; since september was derived within Latin itself, he mentions it only in passing.
Tucker's (significantly less modern) etymological dictionary does go into a bit more detail, though. He brings up two different hypotheses.
The first is that it was originally something like *septem-mens-ris, with the latter morphemes being from mens-is "month" and the adjective-forming -iris/-ilis. Assimilation and epenthesis turned this into *septem-membris, and haplogy turned that into *septembris, which evolved into september.
According to this hypothesis, the same happened to *novem-mens-ris > november, and *decem-mens-ris > december. October was then formed by analogy with these.
The other hypothesis is that there was no *-mens- at all, and that there was either an element *-mr-, or just the ending -iris. The b would then be epenthetic, both in september and october.
Neither of these, unfortunately, explains why we don't see the straightforward quint-ilis, sext-ilis, *septim-ilis, *octav-ilis, *nōn-ilis, *decim-ilis. But it's worth noting that we see the English "quartile", "quintile", "sextile" (from French from Vulgar Latin) from the ordinals, and then "septile", "octile" from the cardinals. It's possible that Latin only used -ilis on numbers less than seven, for whatever reason, which is why the extra morpheme was added to september and up.
