My (now unfortunately) ex girlfriend was Japanese, and damn was it hot when she would say things in Japanese, especially during our more intimate moments. She said it was sexy to hear me talk in English... so it works both ways. Actually she was pretty hot in many ways...
But over time it wears off, especially as everyone is Japanese here. Gotta build relationships on stronger things... unless you're just looking to get laid...