Going vegetarian would only cut it if you stayed clear of dairy. Gram for gram it's better to eat chicken than cheese for the climate.
I switched to fortified soy milk and haven't looked back. Milk is tasty but soy has less sugar as well and now I prefer it. Cheese is harder to make a substitute but eating it 1-2 per year satisfies my craving.
My research seems to indicate its about on-par with chicken. IE a chicken filet is about the carbon impact of 8oz of milk.
And it seems here its about 1/4 carbon per kg of food as beef:
https://ourworldindata.org/environmental-impacts-of-food