I have never had,or used a single stage press.If you were just going to load for 1 caliber,I could see where it would be fine.
A Turret press is what I started out with 30yrs ago,and what I still use today. All of your dies are setup,and the presses can be used as a single stage,or as a progressive press depending on your needs.
Changing calibers is as simple as pulling out one turret plate,and swapping it with another that already has the dies setup in it,and changing the shell holder if needed.
Lee Turret presses are what I use,I have the Classic Turret,and the Deluxe Turret presses,and you can use the turret plates w/dies in each of them. I use the Classic to load all of my rifle cartridges,and the Deluxe is setup with the Auto index rod to load my pistol cartridges in a progressive manner.
While some guy's here suggest a higher priced brand of press because they think it's a better press,I'll suggest the Lee Turret press.The Classic is a better made press than the Deluxe,but both produce ammo that is just as accurate as any of the other brands.
My first press was an old worn out Lyman turret press that was given to me.It was a very good press,but hard to find replacement parts for.Then I bought a Lee,and have used them ever since.
This place has some good prices on Lee products if your interested- http://fsreloading.com/
The Lee kits will give you most of what you need to get started,but I would suggest buying a good beam scale like a RCBS 505 or the like.The Lee scale works,but it is a PIA to use.You can find some good used scales on ebay,for half of what you'd pay for a new one.
And a quality powder measure like a Redding,Hornady,or RCBS is also a must to have if you plan on doing a lot of cases at one time.