I have a 638 in .38. The original grips would not work well with a speed loader. The original grips also broke after about 900 rounds. The plastic with the helicoil inside the grip that received the screw to hold the grips on broke. I had to duct tape her until I could get new grips. That got a few looks at the range.
I bought a set of nice looking wood grips, but as others have said, it was more unpleasant to shoot with the wood then the original rubber grips. I took them off, sold them to someone who doesn't shoot his J frame, and bought a pair of Hogue rubber grips. They are much more comfortable to shoot. I have not tried the speed loaders with them yet. Probably because I don't think they will work.