I was going to say something along the lines of "no way", but considering you seem to have had a lot of practice, I think it would be possible. As a matter of fact, I can relate with the panicking thing and all.
I suggest you go to France. Going to Ireland (and a few days in Liverpool) really boosted my confidence seeing people would actually understand me (and vice versa to some extent - freaking Irish accent) and some would even tell me my English was very good. It's really just a matter of confidence, you have to get out of your comfort zone and confront your language skills with natives. MSN and the like aren't good because you have time to think, type, correct etc and it's not spontaneous, prefer Skype if you can't speak face to face.
Once this is done, you'll realise speaking in the foreign language is actually no harder than speaking in your native language. Just a bit more troublesome when you forgot how to say something.