XD, M&P, and Glock are all of excellent (and similar) quality, as are a handful of other polymers available in 40sw. FN (FNP/FNX) is also a good option, as are offerings from CZ, Ruger, H&K, Walther, and Beretta. If I was choosing a full framed polymer 40sw for myself, I'd probably lean toward an XDm or M&P. For compact, I'd go with a Glock 23. Strange answer I know, but that's just my own preference. Try a bunch and see what fits your hand best. Most of these options are of similar quality...and that quality is pretty outstanding IMO.
If you decide to go with a metal framed 40, look into CZ and Sig.