When he tells you he doesn't care if you leave and he accuses you of cheating and lying all the tim?

These sound like really defensive actions (things you can imagine people saying with their arms crossed). I'm not suggesting he is hiding something but perhaps he is not telling you what he is actually worried about.