Remember that the sorter barreled handguns have a shorter sight radius. That makes sight alignment harder.
Did you buy yours new or used?
As said before, heavy bullets hit higher than lighter bullets. If the .357 was bought new, then I would check with the manufacturer (front sight is a little low, that raises the POI). If bought used, then try lighter bullets or .38 Specials. If used, someone may have worked on the front sight to get it to hit at POA for their ammo.
"A well regulated Militia, being necessary to the security of a free State, the right of the people to keep and bear Arms, shall not be infringed."
Life Member NRA
Life Member NAHC