Noster can mean "one of us" in a symbolic way; L&S mentions that noster eris "you will be one of us" was a set expression for welcoming a deserter into the army, for instance.
The English opposite of this would be "one of them", but to my understanding Latin third-person pronouns are much weaker than their English equivalents. Would eōrum really have this connotation? Or would another word, such as alius or alter, fit better?
To put it differently: how best to really convey the sense of other-ness in the English expression "us versus them"? Not necessarily hostēs, but people who are most definitely not part of the same group as the speaker.