This is something I picked up a few years ago, but basically, my logic is that:

• It's what evolution designed us for (shoes only go back about 40,000 years)
• Objects on the ground aren't a threat if you watch where you're going, unless you're someplace you'd probably rather not walk even in shoes
• Hookworm is actually species specific, and human hookworm isn't an actual threat unless you're in a third world country (or Alabama, apparently)
• It's the thickest skin on your body (especially if it's given the chance to "properly" develop), so general dirt is a non-issue
• It's actually probably more hygienic considering you probably clean your feet much more frequently than your shoes (which are warm, moist and dark)
• As an extra, the health department (any health department) actually doesn't care what customers do - feel free to verify this yourself

You kind of have to retrain yourself, as far as both gait (forefoot strike instead of heel strike) and gaze ("scan" ahead every few seconds), but you absolutely can acclimate to it, so thoughts of "I could never!" are probably wrong.