Agree, had 4 ancestors that fought with the Southern side. 1 with the 1st S.C. Cavalry and 3 with Kershaw's brigade.
None owned slaves and barely had any land but when the call came they joined and all came back home.
There were many reasons besides slaves .
Many white Confederate soldiers stated reasons other than slavery as motivations for enlisting. After the secession of the state of Virginia, "Benjamin W. Jones found that 'the determination to resist invasion-the first and most sacred duty of a free people-became general, if not universal'". Historian William C. Davis then stated, "that determination sent him into the army, and thousands more with him".4 Carlton McCarthy wrote in his memoir with some poetic prose, that the Southerner "dared not refuse to hear the call to arms, so plain was the duty and so urgent the call. His brethren and friends were answering the bugle-call and the sound of the drum," and "to stay was dishonor and shame"!5 Defense of the home and duty with honor seemed to be very strong primary reasons for enlisting for the average Confederate soldier. McCarthy's quote points out another factor as well. The power of one's peers.
Popular pressure was a very strong factor for enlisting to fight for the Confederacy (as well as the Union). Thousands of persons indifferent to enlisting, and even many who were openly opposed to it, were swept like a wave into the ranks in 1861 by the tremendous force of popular pressure.6
The defense of the women of the South was another strong motivating factor for many white Southern males. The women offered thanks to the men who enlisted "but turned with coolest disdain from those who were reluctant to come forward in defense of Southern womanhood".7
But again many volunteered not from any great enthusiasm, but simply because enlistment was the trendy thing to do .8 Therefore, peer pressure had the strongest influence. All of these reasons seem to have motivated members of the yeoman class to enlist, because most of them viewed the defense of slavery as "to protect the fortunes and property of a leisured upper-class that most (of them) looked upon with hatred, envy and contempt".9
The yeoman class had no slaves to fight for, they had some property, their families, and their native states. They also had something else as their property to protect, and that "was their white skins which put them on a plane of civil equality with slave holders and far above those who did not possess that property," as stated by Princeton University historian, James M. McPherson.10 Since many could not read or write very well (or at all), they were not given much of a chance to defend themselves to later generations from statements like Dr. McPherson's. It should be stated as fact that racism was very strong at that time, in the North as well as the South. It was simply the pattern of thought in the 1800s that the white class was superior, even though it was not true.
One motivation that has been around since recorded time (and certainly even before then), was the want for adventure.11 It is doubtful that particular motivation was that strong after being in the storm of combat. War has never been the romantic event that has been portrayed in writings, only a "living hell."
Many high-ranking Confederates showed reasons for enlisting other than slavery. The examples consist of generals (or future generals). Robert E. Lee believed in neither slavery nor secession, but would fight for his old Virginia.12 Ambrose Powell Hill, better known as A.P. Hill, chose to fight for the defense of his state, Virginia, even thought he was deeply opposed to slavery.13 John C. Breckenridge, of Kentucky (a boarder state), a one-time Vice-President of the United States, sided with the Confederacy primarily for his home-state's self-defense from the North.14 The individual motivations are endless.
Not only did Southerners and "boarder-staters" enlist (Kentucky, Missouri, Maryland and Delaware) to fight for the Confederacy, but so did Northerners themselves!15 Their motivations are varied (to be sure), but it can be speculated that economic or family ties had something to do with this phenomenon. Ideology and sympathy for "the cause" also had influence on these men to serve with what their fellow Northerners called "the enemy."
From this article.