The thing is, you're trying to translate an English pun: "my fault" and "linguistic fault" use two different meanings of the word "fault" - the first means "I'm responsible, guilty", the latter doesn't imply any guilt, being a synonym for "speech error". culpa only means "blame, guilt, failure", but cannot mean "error", so linguae culpa ends up meaning "the tongue's to blame".
The Latin word that has the double meaning of the English is peccāre, peccātum - it can mean "error, misdeed" as well as "offence, sin". To admit guilt you use the perfect peccāvī, not the participle; linguā peccāvī means exactly the same as "погрешил языком" in Rssian, something like "my sin is that my tongue failed me". But to refer to language you need to use words like verbīs "with (my choice of) words" or sermōne "with speech".
Using culpa to assign a more direct blame to your tongue you'd say meae linguae culpa "it's my tongue's fault", but it's impossible to assign blame to yourself (mea) and to your tongue (linguae) at the same time without using a conjunction (culpa mea meaeque linguae "the blame is mine and my tongue's"); to keep the modifier phrase before the noun seems to require the genitive pronoun instead of the possessive (meī meaeque linguae culpa). Some might say that this breaks the original syntax and the pun.
- Notice the since you're blaming the faulty result of speech production on the faultiness of your tongue organ by playful metonymy, it can only be used of oral speech, and this isn't an established expression either. However, Latin has an established idiomatic metonymy of a tool for the written result, and that's of course stylus. This makes meī stylī culpa a workable expression for careless use of language in the written medium, supported by at least one precedent (here contrasted precisely with intentional use due to holding false beliefs).
- Latin has no single word for "a peculiar use of language whether spoken or written, style", and when not referring to the speech organ, mea lingua can only mean "my native system of communication" - and here we don't want to blame the entire language, just our inappropriate use of it.
- But you can use mea Latīnitās to mean "the use of Latin particular to me, my Latinity". However, this is limited to tās-nouns for languages, which apart from Graecitās didn't find much use.
There's a single word for inappropriate use of words in Latin, soloecismus. But the bottom line is that you need a lot of ingenuity and luck to successfully translate a pun into another language, and in this case I think I'm out of both.
