The reason why high sound levels can be damaging is that our sense of hearing can be permanently damaged. I damaged my hearing through carelessness by going to lots of rock concerts as a university Student at the Southampton Gaumont. During concerts by such groups at Wishbone Ash and Thin Lizzy, not to mention Orchestral Manoeuvres in the Dark you would feel the music vibrating through your body and indeed the building shaking with the sound. For hours after the concert my ears would ring and I suffered temporary deafness. The reason why sound can be so damaging is that even if the person in an area of high sound is wearing hearing protection or is deaf or hearing impaired is that the vibration from the sound waves can cause harm given duration and strength.
I believe the reason why we may fast from sound, from talking or even electronic communication is to listen more intently and with greater discernment. The more we insulate ourselves from noise by MP 3 players or devices to lower the noise level for the individual the more permission we may seem to be giving to make high levels of noise acceptable in our society.