I like to use brass whenever I can but there are times when you need steel punches too, especially when the pins are very small. Brass punches can mushroom on you, especially when dealing with small sizes so keep in mind when using 'em and don't feel bad if you bend a few until you get better at it. I like to tap my pins out with lighter taps than looking to get 'em out with one big hit.
Its not a bad idea to get a few brass sets just to have some fresh ones handy. Always nice to finish a project same night and not have it held up over a snapped brass punch.
Hope this helps.