I, personally, don't find IB very difficult. I actually want to be a bit rounded rather than just excel in certain disciplines. Yes, I'm not doing great in French, but I'd much rather have the chance to know another language (probably my 4th language) than just have 4 subjects and ease through high school.
Also, who says courses aren't relevant to your field? For example I want to go to engineering. My school has a requirement that we must take English in the higher level. Also, we do not have a physics teacher who can teach at the higher level. But, I still took HL Math, HL Chemistry (it's a breeze btw), HL English, SL French, SL Physics and SL Economics.
I mean, all of these are related to the field I'm planning to go in; civil engineering. Math, Chemistry and physics for an obvious reason. The others are also related, English helps with communication, French - being multilingual is great in general, and a HS sense of economics will help with project managing (or help build up to management courses, or just help with business if I plan to go into it later on).
I mean, my cousin does A-Levels and from what he tells me from his courses, I can see IB and A-Levels on parity (although A-Levels is more practical, whereas I think IB spurs creative thinking).
Oh, and CAS is fun. TOK is probably one of my favourite courses. It honestly helps me think very differently, and I even contemplated taking philosophy as a university graduate.