Audio Compression May Not Be as Bad as You Think

Signals and Noise

When I examined the results, I found that--for this jury, at least--the type of compression mattered more than the level of compression, and that neither mattered all that much.

In general, the scores were all good. The lowest score anyone gave anything was 3, defined in the jurors' instructions as "Definitely inferior, but you could still enjoy the music." Most scores were 4 ("Slightly inferior, but not enough to affect your listening pleasure") or above.

Microsoft's .wma compression, however, proved slightly superior. Compressed to the same level as the .m4a and .mp3 samples (128 kbps), the .wma files received generally higher scores on average than either competing format--and that held true on both clips.

In fact, jurist Fran Avni (a singer, songwriter, performer, and music producer with more than 30 years of experience) said that .wma compression actually made both clips sound better than the uncompressed orginals. Madeline Prager, though she didn't agree, pointed out that the compression masked musician errors. Madeline has played first viola in several orchestras, and now teaches at the San Francisco Conservatory of Music. (Full disclosure: She's also my wife.)

Not that .wma's advantages were overwhelming--the format's scores averaged 4.51. The .m4a files averaged 4.26; the .mp3s, 4.01.

No clear pattern emerged when the jurors compared compression levels. I wasn't surprised to see that 192-kbps .mp3 compression scored better than the same format at 128 kbps. But I was taken aback to find that its average score of 4.19 beat out the scores for the 256-kbps and 320-kbps .mp3s, and even for the uncompressed .wav file.

The judges particularly liked the 192-kbps .mp3 version of the Tchaikovsky clip, which earned an average rating of 4.28 (still lower than the 128-kbps .wma and .m4a clips, though). Steve Carter, a pianist and recording engineer who has played with Pete Escovedo, Ray Obiedo, and Taj Mahal, described it as "pretty close, maybe less smooth" than the uncompressed original. Steve rated it 4.8--the highest score he gave anything.

But the Joan Osborne clip compressed to 192-kbps .mp3 didn't do anywhere near as well with the judges. Its average score of 4.10 was still higher than the 256-kbps version's mark of 3.63, but the 320-kbps and uncompressed versions beat it out--as you would expect--with average scores of 4.50.

The big surprise was that the uncompressed .wav versions of the two songs didn't get the highest scores.

The Tchaikovsky clip received an average rating of 3.80. The uncompressed Osborne sample did better, with an average score of 4.5. But only one judge, Dan Howard, a musician, singer, and music teacher who has worked as a radio DJ and concert producer, correctly judged it "As good as [the] original."

How did so many trained ears fail to recognize the higher quality of less-compressed tracks? While the result may seem like a fluke, it isn't out of line with other, larger studies of music compression.

In 2001, PCWorld conducted extensive audio compression tests, using four music samples, 30 judges, and a more-controlled audio environment. Though that study didn't test whether jurors could distinguish an uncompressed clip, it did find that listeners could detect very little difference between a clip at 128 kbps and one at 256 kbps.

And tests by Stanford University music professor Jonathan Berger indicate that young people are actually coming to prefer the sound of compressed music. Berger sees an increasing preference for MP3s and believes that the students he tested like the "sizzle," or metallic sound, that the format imparts.

The bottom line from my testing seems to be this: My jury of professionals generally preferred WMA compression, but weren't able to detect much difference in files with bitrates higher than 192 kbps. In other words, even though the technology is called lossy compression, you may not be losing as much as you think.

To try our test for yourself, see "Music Compression Test: How Good Are Your Ears?" Remember to hold down Ctrl as you click the link, and give the page a minute to load.