Nope. Round these parts we don't care so much about what kind of gun other people are using. Seen many hunters with SKS's take very nice bucks. Hunters with bone stock Mosin Nagant's aren't uncommon.
Take the rifle your comfortable with. When ya come back to camp with a deer, they will quit making fun of ya.