Allen & Greenough lists -etum/-tum under the heading 'Nouns with Adjective Suffixes' (section 254). It notes that the suffix denotes 'place of a thing, especially with names of trees and plants to designate where these grow.'
The examples provided are:
- quercetum, 'oak grove'
- olivetum, 'olive grove'
- salictum, 'a willow thicket'
- Argiletum, 'The Clay Pit' (from argilla, 'clay')
Plus, you've already mentioned arboretum. I've also seen rosetum and fruticetum, and the corresponding section in Gildersleeve & Lodge (181) offers myrtetum and virgultum. So, given the somewhat limited area of applicability, the suffix seems to be reasonably productive.
In A&G, comparison is made to the suffixes -atus and -utus, and there's a reference to the following note from section 246:
NOTE. — -atus, -itus, -utus, imply reference to an imaginary verb-stem: -tus is added directly to nouns without any such reference.
I can't help wondering whether there's some original relationship to the -esc suffix/infix that has to do with beginning/growing/becoming. There is in fact a verb arborescere, 'to grow into a tree.' Perhaps arboretum originally meant '(a place where) there has been growing into trees,' as a sort of impersonal passive. There's also a verb fruticesco, 'to put forth shoots, become bushy'; so fruticetum could likewise denote a place where this action has occurred. Other words, where no verb is attested, could easily have been formed by analogy. This is just conjecture, of course, and not a 'proper' answer.