You are getting some conflicting information here. Let me try to put it into some sort of sense;
A .40 is NOT generally considered an acceptable hunting round. CAN it kill a deer? Of course. Out of a carbine it should be capable of attaining 10mm like energy levels.
Deer are thin skinned game, but are tougher than humans. Bones protect the vital organs. A light, fast hollowpoint has a better than average chance of NOT penetrating sufficiently to do the job.
FMJ's are the alternative in factory ammo. FMJ FP bullets will likely acheive sufficient penetration, but are not likely to do sufficient damage to the organs penetrated to humanely kill a deer.
Soft points are very difficult to find in .401 diameter so they are essentially off the table. Good, bonded hollow points MAY be the only realistic alternative. As hollow points are notorious for violent expansion you need to maximize the probability of adequate penetration.
To attain penetration, weight is your friend. 180 gr bullets would be my ONLY choice. "Bonded" hollowpoints in this weight class should be adequate. My personal preference would be the Speer Gold Dot. DO NOT use the load made for "Short Barrels" as the powder used is maximized for subcompact pistols. You may get much reduced velocity out of a carbine.
No matter what ammo you choose, chronograph the load through your gun and calculate the energy. Make sure you are getting at least 1150 fps to get over the 500 ft/lb threshhold
In life, strive to take the high road....It offers a better field of fire.
"Robo is right" Fuzzball