A Catholic priest said he has been forced into retirement after hitting a two-year-old boy at his baptism.