I think physics is easier to pick up but chemistry requires a lot of self-study to grasp topics. I’m presuming you’re good at Maths so Physics shouldn’t be too much of a challenge for you. most people in my class who are very good at maths are also very good at physics but that may not apply to everyone. (Even if you’re not good at maths, which i doubt, the maths in higher physics is really easy, it just involves rearranging formulas, linear formula equations, adding and subtracting very simple fractions and very simple factorising.) Some people find physics boring but they’re good at it. So it really just depends, whichever you enjoy more you should pick. I also took nat 5 physics, I liked it. A good chuck of the nat 5 physics course is applied or taken a step further in the higher course. It’s not that big of a jump from nat 5 to higher physics compared to chemistry. So if you can manage higher chemistry i’m sure you can manage physics.