I was in a pretty similar situation to you. My first driving instructor was a really nice guy and I liked him as a person, but he could only fit me in once a week (I wanted lessons more frequently) and I seemed to be learning slower than expected. Then I had a lesson with my second instructor which went great.
Basically, you have to remember that driving instructors are your teachers, not your friends. The reason you're there it to learn how to drive, not to do them a favour by being there just because they're a nice person.
You clearly feel the second guy is a better teacher, so if you'll learn better (and cheaper!) with him, then I think you've answered your own question.
I wouldn't advise telling him that you've 'left him for someone else' - you never know, this new guy might not turn out to be as good as you expected so you may want to go back! I told my first instructor that things were a bit tight financially so I was going to put driving off til the summer. A bit of a white lie usually does the trick.