One possible source of confusion is that you're not considering physically-possible magnetic fields.
Remember that the magnetic field is a solenoidal field, i.e. it has the form "curl(B) = something" (Ampere's law), and that it has no divergence (Gauss's law for magnetism). That means the magnetic field lines must form closed loops. Your thought experiments are leaving you confused because none of them use such a field (e.g. single field line, or infinite and parallel field lines). These fields would indeed cause some problems in electromagnetism!
When the magnetic field lines have closed loops, it is much easier to visualise why the electric field that is generated is in concentric circles around a well-defined axis.
By the way, that definition at the start of this thread is slightly ropey. Faraday's law connects the induced electric field to the change in magnetic field, and says nothing about current. If a conductor of any shape happens to exist inside that induced electric field, then you'll probably get currents flowing, but not necessarily in circular paths.