Well, I don't want to rub any salt in the wound, but essentially this.
You fired the .357 Sig ( 9x21mm ) down a tube that was built for the .40 S&W ( 10x22mm ).
Other than the bullet not getting the right bite, and your accuracy suffered, I don't think you did any serious long term damage with 4 rounds. The bullet probably wobbled a bit in the barrel, but I seriously doubt it was slamming from side to side like a pinball machine.
I'm assuming the round wasn't some uber hard material and that the casing was brass, so any damage to the throat was probably minimal as well.
That said, I am sure you are feeling exactly what I am thinking at the moment.