The expression in faciem expresses the idea of "man to man" or "face to face", but to emphasize doing so in a manly way, I would use the expression ut vir (or sicut vir), which means "like a man". This phrase can be use with an appropriate verb such as resistere (to resist) or contendere (to contend).
Cicero for example wrote the following:
Ita et tulit dolorem ut vir et ut homo maiorem ferre sine causa
necessaria noluit. (Tusculanarum Disputationum, 2.53)
translation:
Thus he took the pain as a man, and as a human he refused to endure greater
pain without need.
A good biblical expression that also expresses this idea is:
Accinge sicut vir lumbos tuos... (Vulgata, Iob 38.3)
translation:
Gird your loins like a man...
Putting these ideas together, you could say:
in faciem ut vir
That is, "face to face like a man".