The .308 Winchester, a bigger bullet diversity, from 110gr. to 220gr. now that is a wide diversity & I am talking almost 3400 fps from a .30 caliber bullet......................
For varmints I usually take my Remington 700 vssf(stainless fluted heavy barrel) in a 22-250 that i've had forever it seems,i'll also take one of my Colt AR15A2 Hbars w/flattop HB upper carbine for varmints which also works well.
I also agree with a good 223 bolt action would be nice and the 204 Ruger cartridge sounds very interesting,good luck with your choice.