Check your local laws for caliber restrictions. Here in NC it must be larger than .24 for deer and have a barrel of at least 6". I'm planning to do some pistol hunting this year as well and I've got a 6" 357 or a 8 3/8" 44 mag, either one will do the job but the 44 gives you a bit more range. For the 357 I will probably go with 170 or 180 gr and the 44 I'll use 240. I don't think I need that heavy of a bullet but I have a bunch on hand.
If the pain is lacking so is the discipline...
"the only 911 call I need is chambering a round" - Mr. Muller, MO car dealer