The foramen ovale normally exists ONLY in foetal life, and normally closes shortly before birth. In the foetus [which is suspended in the amniotic fluid with no direct access to air/O2], the resistance of the pulmonary circulation and the whole layout of the foetal circulation is different from that in the adult i.e. there is no flow to the lungs, which are bypassed by the shunts between the two sides [including the ductus arteriosus] - O2 is obtained from the mother via the placental vasculature.
Hence the redirection of blood is from the right atrium to the left atrium ONLY IN FOETAL LIFE.
If a baby is born with failure of closure of the foramen ovale, he/she will have an [abnormal] connection between the two atria [atrial septal defect], BUT BECAUSE OXYGENATION OF BLOOD NOW starts to occur IN THE LUNGS, the shunt will reverse so that it will be from the left atrium to the right atrium. This will lead sometimes to some classical clinical features including a systolic murmur due to increased blood flow from the right ventricle and a fixed wide split of the second heart sound [S2] on aucultation. [there is a complex explanation for the latter sign, which if you would like to know, pls PM me.
Theres also a small "flap" on the left side of the foramen ovale called the septum primum which is forced open due to the right-to-left pressure gradient in the fetus (due to no pulmonary circulation). After birth, and the onset of pulmonary circulation, the pressure gradient changes to a net left-to-right gradient. This forces the septum primum against the septum secundum, which eventually fuse and close the foramen ovale.