One is considering more polite than the other. They are more likely to say gaikokujin when showing people getting arrested in Roppongi on the evening news, but people on the street are more likely to use gaijin in the sense of
'kusottare gaijin!" At one point, westerners in Japan were called nanbanjin (Southern Barbarians.) I suppose its progress.
HOWEVER. In certain contexts, gaijin can also come to mean Japanese people who fall outside of the norms and cultures of Japanese society. For instance, those folks in Harajuku can be considered gaijin, therefore the term can be seen as offensive to someone who is merely just non-Japanese. Especially someone who has made the time to learn the language and culture of your country.
Most people like myself see it as a contraction and don't bother about the context of the word. In the UK and the west, it would be seen as incorrect to label someone as a foreigner, hence the reason it pisses the PC people off OUTSIDE of Japan (go figure.)
It can be quite affectionate depending on how you look at it. Still, I sometimes call Japanese people genchijin (現地人
for the very same reasons.