"It is undefined" is a convenient semi-fiction to cover up the fact that maths basically doesn't work. 13/0 "is" infinity in a very meaningful sense, but then if you take that at face value and use it to make other assumptions about infinity other things don't work and the system breaks down, the simplest example is that if 13/0 = infinity = 2/0 then 13 = 2, so that all finite numbers equal each other.
In relation to infinity, all finite numbers DO equal each other: they are all infinitesimal. But essentially what it means is that infinity, insofar as it can be said to exist, must work on a different logic from the finite numbers.
So you either say there are two systems, which is iffy because you just derived one from the other, or you say infinity doesn't exist, which is also iffy because maths is supposed to be an entirely theoretical system defined on its own terms only.
Personally I think it's because the universe is a (necessarily) simplified simulation of itself running within (maybe any number of times removed) some true reality.
It is also true that a system cannot fully define itself using its own axioms, since the axioms are part of the system too. This is the same reason why we can never fully observe the universe and I also consider quantum logic to be another instantiation of the simulation thing.