As someone who's studying business because my parents thinks anything creative is a "waste of time" or "should be just a hobby", I do see your POV and am inclined to say that you should do what you want.
But be aware of the implications. What do you want to do with your life? After studying art, your best bet is to be an art teacher, and even for those jobs there is a lot of competition. You cannot aim to be an artist, for a well paid position in a gallery you should have a degree in management or business in addition to art - they can pick who they want as there are too many art grads.
Most of my friends studying either art, fashion or design are from very wealthy families - you'll see that is the trend in most art schools as well. They have inheritance and/or strong contributions from the family meaning that they can live well or at least acceptably while working for free, applying for jobs or practicing their craft. Choosing a field of study in an area which to most people remains just a hobby, is in many ways a luxury.
What I'm getting is that your father doesn't want to provide for you and doesn't want you to remain penniless while trying to pay back your degree.
If you don't have support from the family, are you okay working with something completely unrelated for years? Is your passion for art strong enough for you to practice it on your spare time while waiting table? You have to be realistic about this and think about what you really want.