It doesn't necessarily imply altruism at all. It is perfectly congruous to want to change the world and get rich whilst doing so. 'Selfishness' doesn't much matter if, for example, somebody saves kids from heart disease for ego, or for a huge wage. It is still a net good, and it is still a net change.
If you take the phrase to the extreme then arguably it is lofty enough to be absurd (although again, I don't think this necessarily the case). But to change the world has contexts that can be small-scale. They need not be about affecting every issue to ever exist - such an notion would indeed be silly, as well as impossible.
Part of why I do my job is the admittedly 'selfish' reason of wanting to make a positive impact with my time and my life. Does it change things, in a small way. But that'll do me.
The only people this is an issue for is deontologists, who are already prone to taking everything to absurd lengths in order to make their points.
If you don't want to help anyone else then that's your choice.