Nowadays (I guess) every language has both vulgar and non-vulgar ways to express anger, frustration and/or exasperation , in response to some nuisance. Looking e.g. at Catullus, it seems unlikely that this wasn't the case for Latin.
Do we know any examples of this? Would we hear Pro Iove! in the streets of Rome?