You're not alone, when I started I was so nervous my fingers would be stiff and aching when I got out the car because of how hard I'd grab hold of the wheel. You should absolutely not avoid driving anywhere for any reason and go out when you don't even need to, just to practice. Try a journey when it is quiet, try some at night, deliberately try hills. Don't be embarrassed about how much gas you put on for a hill start. Really the only way you are going to help yourself here is to drive everywhere and test yourself with everything. I made mistakes when I first started that I cringe when I remember, but we all do! And to be honest with how nervous I was I can't believe I've come this far that I'm able to advise someone on it now. It happens to everybody and we all get over it.
I would totally go out in those roadworks and traffic just to get some practice with that.