You know what amazes me. My grand father was shooting groups in the .1 and .2 50 years ago with milsurp actions. He did run the best barrel at the time and he always had good glass on top of his rifles.
I think it is a combo of things. There is no way heat or lack there of in a 22lr will account for over 3" of movement. I am sorry there just ain't no way that will happen. I am going to say it is 1, bedding, 2. mounts and rings, 3. heat and 4. Ammo and maybe a few other things as well. 5. That's a PEBTAC problem. (Problem Exists Between Trigger And Chair)
Shoot you might even have knocked the scope and the lense is lose in the tube then you stop shooting it and lay it down on its side or stand it up at an angle the lens settles and moves the POI. I would start by mounting a different scope and see if that helps. then get your self a fat wrench. and check your torque on all screws. If you don't know the torque specs contact a gunsmith or the manufacture. just like a car if it ain't torqued right it may fail. Think about it if you torque the head bolts in the wrong order you warp the head. If you put to much torque then you break the bolts not enough and you blow head gaskets.