"I never said you didn't" implies that she has not explicitly declared that you dont. This doesn't mean that you do though.
For example, I could say "I never said I didn't say 'fluffy bunnies'" doesn't imply I actually said 'fluffy bunnies', it just says that i'm not denying i said it!