I think a lot of people struggle to talk about mental health problems with a doctor. What I would recommend is to take some time to write out all your symptoms (even just with bulletpoints would be fine). That way you'll know what you want to tell him, which should hopefully make it easier for you to do so. If you do feel like opting out, you could even just hand him the bit of paper.
Making that first step is so difficult but it may result in you getting help and feeling better so that's gotta be worth it, right?