Would you raise your child as vegan?Watch
Moreover a vegan diet is obv less nutritious and it would require organized planning in order to make sure that the infant doesn't develop a deficit, especially in vitamin B12.
It's not a restriction if you do it right, yeah if you eat sugary snacks and fizzy drinks all day you're gonna be restricting, if you do it with lots of fruits and veg, fats/nuts etc. Only thing you need to worry about is calcium and B12 which are easily supplemented.
The thing is that animal foods like meat, fish, eggs and dairy products are extremely nutritious. They are loaded with high quality protein, healthy fats, vitamins, minerals and various lesser known nutrients that have important effects on health.
Placing your ethical values above your child's health and welfare is irresponsible imo.