I'm also with prairie goat. A 223 can be used with a wide range of bullet weights, with the lightest ones as fast as the 204 and heavy bullets the 204 can't use.

That said, the caliber need varies with the size of the varmint. I used a 220 for years, but then the hogs showed up on our place. The 55 gr BT didn't put them down firmly enough. I'm trying out the new 64 gr Nosler BSB, which stabilized in my 220 (so far).

For front yard use (front pasture, actually), I use the 223 with the 40 gr Nosler BT. Kind of an amazing bullet, with the way it'll knock a coyote FLAT. If hunting small varmints (coyote and smaller) that's a great bullet.