The adverbs (and prepositions) extra and ultra are somewhat similar but not identical. While I can read the two dictionary entries and get an idea what they mean, I don't feel that I fully grasp how they compare to each other? In what sense or in which cases are they similar or interchangeable? Are there cases when there is a clear contrast? Is there a concise way to summarize how the two compare?
Extra means 'on the outside' (opposite of intra 'on the inside'), whereas ultra means 'on the other side' (opposite of citra 'on this side').
The difference may be made clearer by looking at their composite parts. For example, extra is a contracted feminine ablative of the adjective exter, extera, exterum (i.e., extera [via]), which in turn is an adjectival form of the adverb ex 'out of'.
Ultra follows the same progression: uls 'on the other side' -> ulter, ultera, ulterum -> ultera [via] -> ultra.