From all the places online I've browsed or commented, most experienced gunners recommend starting with a .22. You want to learn how to handle a gun first and then how to shoot it. The lower recoil and power of the .22 fits the bill.... However, I did not do that.
Just know that I never fired a gun or owned one until two years ago, so my advice might be completely different than someone else. If you want a more experienced person to help you, ask around this board. Or visit Kim DuToit's
website and send him an e-mail asking for help on your first gun purchase.
What I did was decide what I wanted a gun for before deciding on a gun. I buy for home and neighborhood defense in case of natural disaster or anything else. I bought the requisite shotgun, but a single shot NEF in 20 gauge. I figured the old styling of the gun would quiet the fears of the others in the house. Then after someone came knocking around a bedroom window at night, I told them I'm the homeowner and I'll get whatever I deem necessary to protect us.
So I have different guns in different calibers. Some large some small. I agree that a nice .22 gives you a cheaper and less tiring shooting session, but you should be able to shoot whatever caliber you plan on using. Learn more than one. If you find your original idea was for home defense, then your love of history takes over (like me) and you find you can't pass up an '03 Springfield or a Garand, you won't worry much about the caliber. You start with the idea of hd then decide you want to shoot for fun. That's when the different calibers come in. Don't be afraid of the larger ones, with a recoil pad, they aren't that bad to shoot.
If you are on a budget, start with a little cowboy gun, like a ruger single six or a heritage rough rider. The rough rider can be had for around $150. It's not considered a "great" gun, but would you rather be dreaming about shooting, or shooting? You can get a nice Henry .22 rifle for about $200 at WalMart - if your local ones still sell guns. If you want a semiauto, there's the Browning Buckmarks. I happen to like the looks and feel of the Camper. Or you can go with the Walther P22 if you've got the money.
Just remember, there is no "best" gun, just the one that's best for you.