I know a lot of folks like the .243 and the .257 Roberts for good light deer calibers.
We have a couple of real hunting experts on here, so I would expect some good posts in the next couple of hours.
As for recoil, there are several things I would suggest to tame the snap. A muzzle break, which vents gas out and away at the muzzle will increase the noise near the hunter, but definitely reduce the recoil.
A good quality, dense material recoil pad like a Pachmayr Decelerator would definitely help.
If you can stand the extra weight, a bit of a heavier rifle will allow you to ride out the recoil instead of having that vicious snap from a light, hiking style pack in rifle.
Best of luck in your search -