The European single market is not just any kind of trade agreement, its an economical and political union with rules and laws that memeber states are expected to follow. Its stricter than a regular trade agreement, there is monopoly power and many trade barriers.
But what you failed to grasp from the article is that The visegard four wanted to veto any proposals which would benefit Britain but put their rights second, given as Britain is no longer a member of the EU why should their interests come first to the detriment of others who play by the rules?