The rear sight is adjusted to the same direction you want to adjust bullet impact. Rear sight up to move POI up, rear sight down to move POI down, right = right, left = left.
On scopes and red dots, there should be an arrow on the adjustment knob, these are almost always indicating the direction you will be moving the bullet strike during adjustment.
For instance, if the arrow says "up" and is pointing counter clockwise, then you turn the knob counter clockwise to move the bullet impact up. Opposite for the other direction, and windage adjustments are accomplished the same way.