Well, no BC is 100 % safe as you know. Good news is you are both in a long term relationship and are monogamous. Think hard about what you really want as a couple, because if pregnancy occurs, then choosing to have a kid is both your responsibilty.
But, If you do decide to allow him to finish inside you, figure out when your fertile time is for ovulation and have finishing sex outside this time gap. But, to be blunt - if in doubt, leave it out.