Agree that deer in the SE run a lot smaller than their Western cousins.
Here in VA, the .223 is not legal for deer at any range. Assuming that you need 750-1000 ft lbs of energy for a CLEAN kill on a deer (a generally accepted number, depending on the author)
Using Remington's data on a 64 gr soft point- it starts with 1322 ft lbs at muzzle, at 100 it is 1022, at 200 yds it is 780, and at 500 yards, about 313 ft lbs.
Would also mention that the "kill zone" on a deer is about 5 inches across. Can you shoot minute of angle from the position you use when hunting? BTW, a 10 mph crosswind will move that bullet laterally about 38 inches at 500. If you were zeroed at 200, you are holding over Bambi by 53 inches. If Bambi is walking, the flight time is about 0.75 seconds- how much would you lead him to keep the bullet in that 5 inch circle?
CAN it kill a deer at 500 yards? Yep. And you CAN win the lottery tomorrow. About the same chances.
What we have here is... failure- to communicate.