You can take your contacts off and put them in their case the solution or just a saline before you jump in the water. People will help you finding your way back. People are surprisingly understanding in this situation, just know where you put your stuff and people will help you finding your way to them without a problem. Also you can wear your glasses if they are secured to your face, like a sports band or something. I wouldn't jump in the water or dive with glasses, too afraid to loose them.
Once when swimming in the sea I was drifted to another part of the beach* and couldn't find either my friends or our spot on the beach because I couldn't see, and my vision is not even that bad. I was really stressed out but people helped me.
*The Mediterranean Sea in Israel is acting a lot like an ocean, there are rips. Most of the time they are tiny so more or less experienced swimmers can deal with them on their own, but you still get drifted a bit from your original location, if you panic you will be drifted even further, but nothing too serious.