It's hard to comment constructively based on the contents of your post so I will try a broadbrush approach and hope something makes sense to you.
1. When you first start to drive you are particularly concerned with the technical/mechanical stuff that happens inside the car, largely ignoring the stuff happening outside the car (which your instructor is looking after). Eventually you have to move your focus from one to the other and your mechnical control of the car can suffer as a result. It's temporary and it will pass.
2. No manoeuvres as yet is unusual, nearly all my pupils learn their first in the 5th or 6th lesson - it helps with all sorts of aspects of driving. I have only had one pupil who has taken longer than that but she is more advanced in years...
3. Some instructors like you to be complete competent on easy roads before taking you into more difficult situations. Personally I take my pupils into traffic as soon as possible because it is the best place to learn...! Stopping and starting is a great way to learn clutch, brake, gas and gears.
4. You are not c*** and he might be ripping you off. Talk to him about the progress you are making. Point out that you are feeling negative and that you are struggling. If he doesn't do anything to fix it, then he may not be the right instructor for you.
5. You might try varying the period between lessons. I find that sometimes taking a break for a week improves a pupil's driving - don't know why but it works.
6. You wont do it in 20 lessons so ask for a reassessment. Or take an assessment with another instructor and see what they think.
It might be useful if you can give us some idea of the difficulties you are experiencing and we'll see if we can help more.
Dont give up - it will come together but you might need to change your approach to facilitate that happening.
Best Wishes