Dont put yourself down, trust me there are always people who will be crapper than you and some have prob passed! Learning to driving is by no means an easy thing to do but with lots of practise you will able to do it! Also everyone learns it at different rates so it might take a while but thats ok it doesnt make you any worse or any better of a driver, so don't give up, you made it this far havent you? Think of the places you can go and without ever having to take public transport again!
If you feeling nervous a lot of the time, you might be suffering from anxiety, so that something to think about and worth talking about to you GP.
You might also want to think about changing your instructor, trust me i had to change a few times but when i found the one, i just felt comfortable driving with him and learnt so much.....and If you like driving with your mum then go for it, the more practice the better.....
Whatever you do dont give up and approach driving with a positive attitude, if you believe you can then you prob will!
EDIT: ooh don't think by the first sentence that i meant you're a crap driver i meant generally thats all and as everyone has been saying, believe in yourself, your are prob better than a lot that have already passed!