He spent money taking you away every month, and now he's in financial trouble you now kick up a stomp?
There is NO way you didn't at least have some suspicions into what he was doing/what he was up to. You were willing to accept this £20k towards your house, don't discredit him for getting caught.
Tax evasion isn't that uncommon you know, give him a break.
I can definatley see where you are coming from, however if you love him as you say, you have a funny way of showing it.