Goose bumps are a vestige from the days when humans were covered with hair.
When it got cold, the hairs stood on end, creating a trap for air and providing insulation.
The hairs have long since disappeared, but in the places where they used to be, the skin still bristles, trying to get warm.