That's why it is best to use a starter punch initially to break the "seal" on tight pins first. The shorter, tapered tip is less prone to bending and can withstand heavier blows to get things moving. After that the pin will be much easier to drift out with the thinner, longer pin punch.
Glad you got it worked out. And welcome to the forums.