Seeing as you're a good swimmer and you swim regularly at high intensity, this kind of thing sounds a bit like an overuse injury. It may also have something to do with technique. Perhaps you're curling your wrists a lot when pulling. Can't say for sure it's due to swimming, but yes, it's possible.
To make a comparison, I weight lift a lot and have had my fair share of overuse injuries over the years and it's a matter of stopping the action that causes pain, until the pain goes away. You may have to change some kind of technique, swim with a different stroke, or swim less frequently for a while.