This originated in the British Armed Forces, whose lower ranks have for many, many years used, as a kind of humorous but informal motto, the phrase NIL ILLEGITIMI CARBORUNDUM, which of course is not proper Latin at all but is universally understood to mean exactly 'don't let the bastards grind you down'. Sometimes, to be more polite, it is shortened — NIL CARBORUNDUM — 'don't let them grind you down'.
You ask for a 'correct and definitive (canonical) translation', and cmw has shown how to provide what you wish for — if you really need to call them 'bastards'. If, however, you want to draw attention to the moral character of these people, I suggest that you need something a bit stronger than nothus or spurius : something to indicate a bullying nature directly, rather than by metonymy.
Cicero (Fam. 7, 13) has, for instance, homo procax in lacessendo, which for me solves the problem nicely, with procax to describe the character and lacesso the verb for harrying, irritating, exasperating, harassing, etc.(earlier at the same reference he uses molestiam afferre). I rather like Ne [homines] procaces te impune lacessant, with its echo of nemo me impune lacessit, the motto of the British Stuart dynasty, the Order of the Thistle, and of The Black Watch (among other Scottish regiments).