When you forgive someone, it doesn't mean you are giving him the leeway or opportunity to hurt you. It doesn't mean you accept what he did to you because it isn't acceptable.
Forgiveness is for your own benefit. It aids you in getting rid of negative emotions - anger, hatred, bitterness. When you hold onto these emotions, they will have disastrous effects on your wellbeing both psychologically and physically.
Nit forgiving him is the same as letting him (his actions) have control of what you should do and not do. They will dictate which actions to take.
In life, we experience hurt or wrong is done against us on different levels. In most of these cases, those who wrong us don't ask for forgiveness or acknowledge what they did. We can't force them. What we are left with is to forgive them because we don't want their actions to affect us negatively.
Forgive him not because you're condoning his action but to prevent yourself from being affected negatively by his action.