Oh dear, not a good situation.
Firstly, if he worked hard to gain your trust back why have you now started to doubt him?
Secondly, seems to me like he's on a guilt trip...he is the one betrayed and deceived you and is now trying to imply that you are up to something too. Have you asked him what is the basis of his anger, appears to be misplaced to me.
If you want to stick with him the try and lay down some ground rules - have an honest conversation and try and keep emotions out of it. Good Luck