To be honest I don't think it will make much difference.
Alot of fashion trends these days tend toward the poseur (poser) types who wish they were hardcore third world badasses (I'm not sure if that word is acceptable on here - I'm sure someone will inform me if it is not). My point - look at all the "fashion" that is all camo: pants, shorts, jackets. It has become fashionable (at least where I live) for totally non-military urban types - who have never stepped foot outside of an air-conditioned mall, to wear military garb and have tacticool stuff just for looks.
Plus in a true SHTF scenario, bad guys will steal from anyone, be it hiker types or militant types or poseurs. Looking military might even make you look like a harder target (assuming you have your gun out too) and they'll leave you alone (although I guess the gun is the more important part of the equation).
Leo's may have an itch out for you, but they may also have bigger fish to fry assuming that you don't play it up too much and start setting up checkpoints or causing trouble. (Ie do not do what they did in Patriots - I loved the book but they were askin to be shot with their strett ambush thing).
Military stuff might be better if you are heading to the bush versus hiker gear which is intended to attract attention (thats why they are red / neon usually)
Overall, I say get what you are comfortable with, and don't pretend to be a milita guy either way. Civy stuff can be just as functional as military stuff, and some military stuff is just designed to get the mall ninja crowd.
Get what you think you can use and fits you. Looks are important, but actions are more important.