I’m surprised by the lack of context and history in the answers so far. Most of the answers are assuming that shame of the naked body has been universal across all of civilization, when that could not be farther from the truth.

Tl;Dr: gymnophobia is a learned, cultural phenomenon that is relatively recent. Nudity was much more common in a world without an abundance of privacy and excess clothing, and so we originally invented clothing out of practicality, rather than shame.

Now, some context.

• Stone tools were invented and used about 2.5 million years ago.
• Ancestors began hunting meat and foraging farther, requiring more walking. This caused an excess of internal heat to build up, sweat glands to evolve further, and hairlessness to develop. Therefore, nudity was “invented” by approx. 1.5 million years ago. Edit: some people have pointed out that this is only one theory among many as to why we became hairless. Duly noted! Either way, we were hairless at this time.
• Fire discovered and utilized by homo erectus approx. 1–1.5 million years ago. Roughly the same timeframe as the evolution of nudity.
• Anatomically modern homo sapiens appear approx. 200,000 years ago.
• First evidence of clothes 170,000 years ago.
• Behaviorally modern homo sapiens begin 50,000–60,000 years ago, around the time of the diaspora from Africa

So this tells us a few things:

1. Shame, at least shame coming from a biological or instinctual origin, cannot be a primary factor in why we wear clothes, because for 30,000 years with our modern brains we didn’t even think of the concept of clothing. And for a million years before that, we coexisted in small, naked tribes of hunter gatherers, wielding weapons and building fires. If shame is a factor, then it comes from culture and learned behavior.
2. We didn’t invent clothing primarily for warmth. In fact, we lost our fur specifically to get MORE naked to protect against heat, especially internal heat generated from our bipedal movement. This is despite the fact that the last ice age didn’t end until about 12,000 years ago. And clothing was invented long before the need to survive ice age conditions farther north.** I’ve added a more in depth look at this conclusion at the bottom.

Now some facts about life in the more recent past. I’m going to use the Bible to prove some of my points, since so many are pointing to the Adam and Eve story as proof that we have always historically treated nudity with shame.

Nudity, for most of human history, has been associated with poverty or manual labor, not sexuality.

• Most regular folks owned one garment, and undergarments weren’t a thing for many cultures. If you were poor, you would often find yourself naked because you would sell your clothing for food. So when the Bible says to clothe the poor, they literally mean to take your one garment that you own and give it away to that naked homeless guy, which was altruistic because nights in arid regions get very cold. This meant that in the past, you could walk back from the market butt naked and be considered a super great guy! Also Jesus probably did this multiple times.[5]
• People used their garments as blankets at night, meaning that unless you were rich and had multiple clothes, you and your family were very comfortable with familial nudity.[6]
• Farm workers and fishermen often worked in the nude, so as to avoid dirtying their only garment unnecessarily. “Therefore that disciple whom Jesus loved saith unto Peter, It is the Lord. Now when Simon Peter heard that it was the Lord, he girt his fisher’s coat unto him, (for he was naked,) and did cast himself into the sea.” John 21:7 KJV
• If you had to wash your one garment, you’re gonna be naked while you’re waiting for it to dry. This was reality for most of human history. If you’re in a small tribe, you’re gonna see almost everyone naked at some point.
• Isaiah 20:2 Isaiah was commanded by God to preach butt naked for 3 years, both for the shock and awe factor and as a prophecy that Egypt would lose everything, including the clothes on their backs, to Assyria. Again, the shame of nudity is coming from a social stigma of poverty, not exposed sexuality.

Nudity was/is seen as spiritually pure.

• King Saul was come over by the spirit of God and prophecied naked for a full day and night. It is implied that the priests of Israel did this occasionally as well. 1 Samuel 19:24
• David received Jonathan’s robe and tunic as a sign of loyalty and acceptance that David was God’s anointed (Jonathan was next in line for kingship), likely leaving Jonathan nude. 1 Samuel 18:4
• Historically, ascetic monks often took vows that required forfeiting all possessions, including clothing. Jainism is famous for this, some Hindu sects, and Buddha himself, prior to his enlightenment, and he only required his monks to wear simple robes in order to distinguish his new philosophy from Jainism and other nude sects.[7] [8]
• The first Christians were likely baptized in the nude, symbolizing a rejection of the old man and a rebirth of the new.[9]

Recreational nudity was very common in ancient Greece and Rome.

• It was seen as perfectly fine to lounge around in the nude, as long as you didn’t have an erection.[10]
• Pretty much all Greek sports were performed in the nude.[11]
• Public baths were common, and at some points in history were open to both sexes.[12]
• Public restrooms looked like this, and were seen as a place to socialize.

• The Gymnasium comes from the Greek word gymnos, which means naked, and was seen as a place to socialize. They had schools in these gymnasiums where both teacher and student were nude.
• Paul the Apostle underhandedly endorsed Gymnasiums and thus social nudity in 1 Timothy 4:8 by using the word gymnasia, which literally translates to naked-exercise.

Nudity was/is used as a way to intimidate opposing armies.

• Ancient Celts went to war butt naked in order to give the impression that they were so good at battle, they didn’t even need armor[13] Edit: some commenters have pointed out that the Celts also did this to avoid getting their stab wounds infected by the grime on their would-be clothing. So again, nudity is a practicality.
• Joshua Blahyi, or as he is better known, General Butt Naked, led Liberian troops into battle sans clothing to instill fear in the enemy and encourage his comrades.[14]

So armed with all this knowledge, we should be able to fairly easily answer the question. Why do we cover our private parts? It’s not because of a universal human shame. It’s not necessarily out of a need for warmth, because we invented clothing long before we ventured into frigid northern climates. So what then? Why would early humans invent clothes? Possibly for status or decoration, possibly because it allowed them to carry tools on their person, possibly for comfort on particularly cold nights (even Africa got pretty cold in some places due to the ice age). It’s hard to say with certainty.

What IS fairly clear nowadays is that the mandate to wear clothes at all times comes from our culture which stigmatizes and sexualizes nudity. This probably was exacerbated by the prudish Victorian era, as we are historically more prude than most other unaffected cultures. It’s an arbitrary rule, and one that in my opinion makes our culture more susceptible to body shaming.

To further prove my point that gymnophobia is a social stigma, not a biological one, just think about gay marriage. Just 20 years ago, even Democrats wouldn’t think about running on a pro-gay marriage ballot, yet here we are today with more social acceptance of the LGBT community than ever. You could go back farther with the Civil Rights era. My parents are an interracial couple, and they got married just a few years after the ban on interracial marriage was lifted. Both of their parents told them not to do it. Yet here we are today.

Social stigma is a powerful thing, and what we accept as fundamental to human nature is very often simply an ingrained cultural belief. If you put a Victorian male on a beach today, they would likely pass out from the shock of seeing so much exposed skin. Similarly, put a man today on a nude beach, with only a tiny bit of fabric removed, and they have much the same reaction. Male nipples used to be stigmatized and banned from public display, much as female nipples are even to this day. It’s all much ado about nothing IMHO.

EDIT: Wow, I didn’t expect to get so many views! Thanks everyone for the insightful comments. I’ve added sources to most of the facts, which were absent previously because of laziness. Also, to the complaint that the Biblical evidence doesn’t count, I’m not talking about theology here. When a Biblical author talks about nudity or clothing, that’s fairly reliable archaeological evidence for ancient Hebrew culture. There would be little reason for them to lie about their clothing habits. Plus, there’s plenty of other evidence to support my conclusions.

EDIT 2: Thanks again for the insightful comments, views and upvotes everyone! To the common comment giving more examples of nudity in history and in the modern era, thanks for the additional evidence! I knew about some of them already, but my answer was getting a bit long already. To the comment complaining about how uncomfortable running would be with a floppy penis flapping everywhere, I got news for you: penises shrink. Here’s an account from a nudist runner who gives the dirty details (no pictures, don’t worry). The pros and cons of running naked. It would probably be more uncomfortable for your feet initially, and for women with large breasts. But somehow I doubt women did as much running as the men back in prehistoric hunter-gatherer tribes.

** After doing more research, I’ve finally found some hard numbers on average temperatures during the ice age specific to Africa. About every hundred thousand years, it seems, we have a small spike of warm global temperatures similar to our own today, and then a decline into a longer period of ice age climates, about 8–12°F cooler than today. And as it turns out, the timing of the invention of clothing is correlated to a cold period. So clothing was likely invented for warmth. I was wrong, and I’m sure that’s the last time ever.

However! We probably didn’t need this clothing to survive at the time, because there were several periods of similarly cool temperatures where we did not have clothing. We still had fires for warmth, and we obviously got through similar climates before. It’s likely that the colder things got, the farther south we traveled to escape the winter chills.