Why is the object of mercy ("me", "us") rendered in the genitive in these two cases?
I would expect accusative, or even dative. But, genitive surprises me.
Actually, the dative appears in the miserere nobis of the Agnus Dei, so genitive does not seems to a hard rule here.
First, this is not specific to ecclesiastical Latin. The same genitive is there in classical Latin as well.
The verb miserere is used impersonally. It means roughly "to distress" or "to excite pity". For example, me miseret means "I am distressed". The reason of distress or the target of pity is indicated by genitive: me miseret Marci means "I pity Marcus".
The verb can also be used in passive voice. This use is personal because passive gets rid of the impersonal subject. Instead of me miseret Marci I can say misereor Marci. The verbs miserere and misereri are listed separately in many dictionaries, but it makes most sense to me to see them as one verb used in both active and passive voice.
The word miserere is a passive imperative form of miserere — or the imperative of the deponent verb misereri if you wish to see it that way. (This reminds me of passive imperative forms of non-deponent verbs, which are rare but exist. It didn't occur to me before that miserere might count.)
The same construction is found with many impersonal verbs expressing feelings: miserere, pigere, pudere, paenitere, taedere. Unfortunately I cannot fully explain why genitive is used in cases like this. I see the object of pitying more as a cause than an actual object. It helps me to think that the ablative causa is intended (whether or not this really is the case): I might read misereor Marci as misereor Marci causa, "I pity because of Marcus".
I do not recall seeing miserere with dative in classical usage, but I may be mistaken. Using dative is not surprising, though. Dative is often used to indicate the entity that benefits or suffers from the action of a verb. Therefore verbs like auxiliari, favere, servire and deesse require dative instead of accusative. This idea of benefit is understandable with miserere as well.
