There are a lot of possibilities here. Depending on your cleaning routine it could be the amount of fouling in your barrel, i.e. it gets more fouling and then shoots better? It could also be the load you're shooting? Some loads shoot better with a cold bore and some with a warm bore.
One other thing is to check your barrel while its hot make sure that it is still free floating. The expansion from heat could be causing it to touch a pressure point that is stabilizing it rather than throwing it further off?
Just throwing out a few more ideas?