If you want a relarionship with his parents, you are going to have to put in more effort as hard as it is with anxiety. Ask your partner to help you and tell him how he can help (encuraging words, ideas on what you can do, being with you or not being with you) etc It will also please gim that you want try and build a good relationship with them esp if he is a mama's boy (I advice this strongly if he is as it could lead to problems down the line).
Also make sure your partner understands your anxiety and how to manage it etc too.
He is with you and not his ex, you should concentrate on that.