Sometimes, you just have to dig a little harder. Like that dry well, dig a little deeper. And sometimes, you just have to accept the inevitable. Maybe your ancestors were just missed in that particular census.
The ones I keep looking for are Julina Aldridge in the 1870 and 1880 census's. (She had children by the Whitley's, she should have been relatively close to them, shouldn't she?); the family of Thomas Ashley Davis in the 1930 census (they lived in Albemarle in 1920 and 1940, would it not make sense they were there in between?); Travis Crump in 1940 (he was in Albemarle in 1930 and I know at some point he moved to New London, so he should be in one of those two towns), Christine and Maxine Davis in the 1940 census in Winston-Salem, (I know they moved their to work in the Cigarette factories, they were not in the orphanage with their younger brothers in that census, both were over 20 by then and Christine's husband was found boarding there in 1940 and that is also the year they got married, so it only makes sense that the two sisters were somewhere near him as they knew him, and they had obviously not got married by the time of the census, yet), and two of Inez McSwain Hooks daughters in the 1900 and 1910 censuses.
Sometimes, it just takes a little patience, and a lot more digging.