The prosaic word order in Latin—that is, the ordinary, normal, unremarkable word order—goes like this:
The noun comes first, and the modifier comes right after. The modifier can be any of:
- an adjective, as in canis ruber (a red dog);
- a noun in the genitive case, as in canis Georgii (George's dog);
- (rarely) a noun in the same case, as in canis amicus (the dog that is also a friend)
When the modifier is an adjective, it must agree in gender, number, and case with the noun it modifies. When the modifier is a genitive noun, it need not agree in any way with the modified noun. And when the modifier is a noun in apposition, it must agree in case with the modified noun, but it has its own gender and number.
Species names in biology follow exactly this pattern: a noun (for the genus) followed by a modifier (indicating a characteristic that distinguishes the species from others of the same genus). Three species names corresponding to the above are: Canis aureus (the golden dog), Canis hallstromi (Hallstrom's dog, i.e. the dog species discovered by Hallstrom), and Canis lupus (the wolf-dog, i.e. the wolf).
Following that pattern, your example should be essentialia negotii (the essentials of a contract) since essentialia here is the main noun, modified by the genitive noun negotii. In Latin, it's not unusual to use an adjective "substantively", i.e. as a noun itself, without providing a noun for it to modify. We do this sometimes in English, too. In fact, the word "essential" works the same way in English: it's primarily an adjective, but we also use it as a noun, as in "essentials". An adjective used substantively modifies an implied noun of "things", "stuff", "people", or the like—not important or interesting, hence omitted.
HOWEVER, Latin grammar is extremely flexible regarding word order. You can put the modifier first, or even separate the two words by a long distance in the sentence, and the result is still grammatically correct. Different word orders change the emphasis or make clear what the listener is supposed to already know and what is new information, or indicate what is being referred to and what is being asserted about it. A few modifiers, like hic (this), come prosaically before the noun.
It should come as no surprise, then, that Google turns up hits for both essentialia negotii and negotii essentialia, but far more hits for the former. This book has both on the same page, one in the main text and one in a footnote, which nicely illustrate the flexibility of the word order and its uses:
Essentialia fere omnis negotii spectant ad personam contrahentium, ad rem et ad formam (5).
(5) Hoc loco quae ad singula juris negotii essentialia spectant enumerare non possum.
The essentials of almost every contract pertain to the character of those contracting, the matter, and the form (5).
(5) In this place, I am not able to enumerate, down to each each individual point, what the "essentials" of the law of contract pertain to.
In English, in the last sentence, we would make emphasis comparable to that of the Latin by wording it like this: "…what, in contract law, are considered the essentials."