17 and 23 is a massive difference at that age, so yes lots of people will think its wierd/strange. same goes for if the genders were switched.
How long have you been dating? because that would definitely effect how people would see the relationship because the same age gap gets less socially acceptable the younger you are. (like 30 and 35 is no big deal but 15 and 20 defo is)
at the end of the day there will always be judgy people who will strongly disapprove, but its your relationship, not theirs