I suspect that any reply to this broad a question will always be rife with conjecture, but the reason for the convoluted word order is always a combination of the metrical cadence, and the effect that word order has on the listener. It is generally accepted that literature was usually read aloud in ancient Rome - we can even assume that lyrical poetry was (sometimes) sung, with some background music as well.
In a recitation, the order of words is much more important than when reading a text silently, since you can't go back and painstakingly puzzle back together the sentence. Since every word is read exactly once, in a fixed sequence, there is always a specific grammatical (and semantic) question being asked or answered with every word. In a language that has freedom in word order, it's something that can be played with by a skillful author, and it's something that an audience would appreciate if done well.
You also have to understand that, while word order was not as fixed as in many modern European languages, there was still a "default" word order to deviate from. While I'll not venture to go into many details, in general the word order in Latin was, in an unmarked sentence, preferrably Subject-Object-Verb (SOV). If we look at one of the most famous sentences in Roman literature (because most everyone who learns Latin in school has read it by age 15), that is not the case at all:
Gallia est omnis divisa in partes tres
Sub1 V1 Sub2 V2 ( Obj )
So omnis
is not next to its noun (this is called hyperbaton) and it splits the verb components, which are not at the end of the sentence. It is also usually so that a numeral (tres
) precedes its noun, which is not the case here. An unmarked sentence with the same meaning would be:
Omnis Gallia in tres partes divisa est
This was of course still understandable to the listener, or else a good writer would never have committed to this kind of word order. Of course the intended audience for this was native speakers of Latin, who had an enormous intuition for what was a regular sentence, and what were stylistically acceptable (and understandable) deviations. It seems to me like in those seven words, Caesar was saying from the get-go: "this is not some dreary campaign diary, this is Literature, so listen up !".
Making your sentences a bit of a puzzle forces the audience to listen to every word, and it's always possible that a word provides a resolution ("ah, here is the verb that ties everything together !") or an extra question ("why is there an adjective in the accusative feminine here ? I expected one in the genitive masculine !"). It is a useful exercise when reading Latin to try that once in a while, to hold still at every word and take stock of what you know of the sentence already. And since Latin by default puts its verb at the end of the sentence, you often can't tie the sentence fully together until the very last word.
EDIT
In poetry, there are of course more constraints from a metrical standpoint, but there are also opportunities. A verse is of a fixed length, and (although there has been considerable debate about how the boundary between verses was realized), there was definitely some sense in the audience what was the first word of the verse and which one was the last. These are marked positions, so if an author puts a word there - especially a word belonging on line X being put as the first word of line X+1 or vice versa (mutatis mutandis) - that is something that is noticed by the audience. It's an extra method of stressing specific meaning.