That's correct, Beryllium has a stable electron configuration as the entirety of 2s subshell is filled with electrons "1s2 2s2" whilst Boron is "1s2 2s2 2p1" with the 2p subshell being much less stable having only 1 electron out of a possible 6 and no orbitals filled out of a possible 3 and so it is easier to remove an electron. Leading to a lower ionisation energy. Between Beryllium and Boron there is also slightly more shielding as Boron's outer electrons are on the higher energy level and so 1s2 and 2p2 are shielding 2p from the nucleus for Boron lowering the nuclear attraction and thus the ionisation energy whilst Beryllium only has 1s2 shielding the outer electrons.
A side note as I've had friends talk to me about this but there is not repulsion in the 2p subshell of Boron as there are no orbitals filled to have repulsion so that is not a factor for the decrease in ionisation energy. The best example for repulsion being a factor will probably be Phosphorous to Sulfur where the 3p subshell of Phosphorous is ^ ^ ^ and the 3p subshell of Sulfur is ^^ ^ ^ where there is repulsion in the first orbital of the subshell which lowers ionisastion energy as there is a loss of stability.