it depends i guess...i haven't taken IGCSE sciences although i have done maths...the sciences aren't said to be harder so i don't think that will be an issue.
you sound like you are very motivated, which i always think is a necessity in physics...if you don't want to do it, you won't!
i would say though, that you will certainly be spending your whole summer learning for it which might not be much fun...as there is quite a lot to cover...
how much work have you found your other sciences?? maybe consider how hard they were and then add a bit more as it isnt your favourite science
what subjects are you taking for IB, and will you be taking a lot of exams next summer, because it might be an idea to maybe do it next summer so you have a bit more time to get comfortable with the topics
i would also suggest doing research into which text books to get as if you are teaching yourself you need to have a textbook that you get along with and that is tightly focussed on the specification.
maybe take a look at the specification if you can get hold of one and consider how much time/effort it will take you
finally, consider how much you would enjoy doing it, and maybe look at some other universities. i am not sure whether they all require physics GCSE but i do know that Imperial usually ask for more than other unis
Good luck with your decision!