The Latin grapheme: "Œ" is the majuscule ligature of the letters "O" and "E". Is it proper—or in-fact possible—to have part of the ligature be majuscule and the other part be minuscule?
Here is a partial answer.
I don't know what kind of forms you can find in historical (e.g. medieval) documents.
In Latin texts that have been published in the present day by academic sources, you're unlikely to find ligatures of any kind. It's currently preferred to write "oe" (or "ae") as a digraph composed of two separate characters. As in English, when a word starting with a vowel digraph is capitalized, only the first letter is put in its upper-case form (e.g. "Aeneas", not "AEneas").
The texts that do use the ligature "œ" capitalize it as "Œ", just as in English typography.
I have seen the usage "œnopolis", "OEnopolis", where the uncapitalized form is spelled with a ligature but the capitalized form is spelled with two separate capital letters, in a book printed in 1693: Des. Erasmi Roterdami Colloquia, Cum notis selectis Variorum, Addito Indice novo. Accurante Corn. Schrevelio.
The same text does use the ligature Æ as a capitalized form of æ, so I'm inclined to think that the typographers viewed OE as a single letter that happened to be written when capitalized with unconnected glyphs (similar to the treatment of Dutch IJ in print that Cerberus mentioned in a comment). I'm not sure whether the different treatment of Æ and OE in this text was based on some graphical principle (e.g. the lack of a straight stroke on the right side of O comparable to the right stroke of a capital A), or whether a special ligature for "Œ" was omitted simply because of its much lower frequency compared to Æ.