As of this writing, there are a few days left for final submissions. I thought that I would publish this first to basically get the "preamble" over with!
Firstly, I want to thank all the people who have taken the time to download a rather substantial 75MB file, take the time to read the instructions, go through all that they needed to do to get it to play (eg. upload to music servers, run Foobar comparator, burn to CD, etc.), tolerate the varied genres (I got a few comments that "Keine Zeit" wasn't their cup of tea), and ultimately come to some conclusion and take the time to complete the survey.
Looking at comments in the survey response, clearly most respondents took this test seriously and took the time to give me a detailed account of the equipment used. I very much appreciate your support and charity.
Although I cannot vouch for complete accuracy, "Free Online Surveys" kept track of the international participation - here's a map of where responses came from so far:
It's great to see 5 continents represented. Not surprisingly, so far of 142 submissions, 60 respondents have come from N. America, 3 from S. America, 4 from Australia & New Zealand, 31 from Asia (big thanks to all the Singaporeans and Malaysians for getting the word out!), and 44 from Europe!
All together, I believe this test is one of the most "complete" audio tests of this nature in the sense that respondents have provided information about their confidence in themselves as well as details of the equipment used outside of a professional environment. The fact that respondents are using their own equipment in their own homes also allows for a "snapshot" into the naturalistic environment of the hi-fi / "audiophile" group responding to this survey through the forum invitations which I will talk about more in future sections.
Once again. A warm thank you to everyone who took the time to contribute!
Section 0: Personal Motivation and Disclaimer
As I noted in the testing material, the catalyst for this test had to do with discussions on an audiophile forum - Audio Asylum - back in Nov/Dec. 2012. Comments were made about how intolerably poor "320 kbps MP3" sounded to the point where it was felt by some that the difference in quality could be easily detected within seconds of listening to a piece of music passed through MP3 lossy encoding.
Over the years, I have long thought about running a blind "Crowd Test" over the Internet like this one. My first exposure to MP3 was back in 1995 when I ran into a copy of Madonna's "Holiday" while "surfing" over the university Usenet forums and was amazed at the fidelity achieved out of only a few megabytes. Through the late 1990's, I realized however that 128 kbps was inadequate (touted by Fraunhoffer as "CD quality") - obvious high frequency distortions were noted, "swooshing" sounds were obvious with a good pair of headphones, clear phase distortions running the music through the old Dolby Pro Logic DSP's of yesteryear was also most unwelcomed.
By early 2000, I started migrating to a computer music-server based system and due to storage limits, decided to use MP3. In 2004, I went through my own tests of audio transparency and determined that LAME ABR at 210kbps, low pass filter set at 20kHz was transparent (actually, I thought 192kbps with ABR was good enough but I bumped up the low pass filter and bit rate just to be sure) with the equipment I had back then (primarily old pairs of Sony MDR-V6 & Grado SR60 headphones, Creative Audigy soundcard).
Over the years, as MP3 consumption increased, the audiophile press and general sentiment continued to deteriorate towards lossy compression in general despite advances made in algorithms. Bitrates have advanced up to 256+ kbps for most digital downloads these days as well. Despite all that has been written in the press or public opinion, I remained curious as to whether "transparency" with un-lossy-compressed CD audio is achievable.
For those who haven't read up on this, here are a few samples. Note that some of the articles do have good thoughts on the matter, but I'm questioning their take on what is perceivable sound quality; as if somehow MP3 (including high bitrate encoding) is a significant compromise in sound quality. Furthermore, I find it interesting that articles are taking the concept that "data rate" is equivalent to "sound quality" which I suspect was best publicized last year by Neil Young in his interviews (really? is bigger and more data better for perceived sound quality?).
- Neil Young and his rants against MP3 quality
- Wired's take
- Why MP3 sounds bad
- PCMag's take
Despite some comments I have received, this test is not about encouraging further adoption of MP3 - it's already adopted as the defacto lossy format. In my opinion, MP3 had its "time in the sun" as an efficient encoding scheme when storage was small and expensive (in fact it has been superseded by AAC/iTunes for many consumers). What I'm more interested in is whether the sound quality of MP3 encoding truly deteriorates to a point where it deserves to be the topic of great (or even any) criticism. From a practical perspective, if I buy a high bitrate MP3 from a place like Amazon, should I worry that I'm not able to experience the full sound quality even when played on good equipment?
Disclosure: I do not have any affiliations with the audio press, equipment manufacturers, and do not work in the audio field at all. This blog has no ads and I have no need for "clicks". I do not tout the contents of this study as "scientific" or even "accurate" by nature of the uncontrollable variables, but will try to present & interpret the data as objectively and thoroughly as I can.
In a few days, the survey will close... See you soon with preliminary results!
Firstly, I want to thank all the people who have taken the time to download a rather substantial 75MB file, take the time to read the instructions, go through all that they needed to do to get it to play (eg. upload to music servers, run Foobar comparator, burn to CD, etc.), tolerate the varied genres (I got a few comments that "Keine Zeit" wasn't their cup of tea), and ultimately come to some conclusion and take the time to complete the survey.
Looking at comments in the survey response, clearly most respondents took this test seriously and took the time to give me a detailed account of the equipment used. I very much appreciate your support and charity.
Although I cannot vouch for complete accuracy, "Free Online Surveys" kept track of the international participation - here's a map of where responses came from so far:
It's great to see 5 continents represented. Not surprisingly, so far of 142 submissions, 60 respondents have come from N. America, 3 from S. America, 4 from Australia & New Zealand, 31 from Asia (big thanks to all the Singaporeans and Malaysians for getting the word out!), and 44 from Europe!
All together, I believe this test is one of the most "complete" audio tests of this nature in the sense that respondents have provided information about their confidence in themselves as well as details of the equipment used outside of a professional environment. The fact that respondents are using their own equipment in their own homes also allows for a "snapshot" into the naturalistic environment of the hi-fi / "audiophile" group responding to this survey through the forum invitations which I will talk about more in future sections.
Once again. A warm thank you to everyone who took the time to contribute!
Section 0: Personal Motivation and Disclaimer
As I noted in the testing material, the catalyst for this test had to do with discussions on an audiophile forum - Audio Asylum - back in Nov/Dec. 2012. Comments were made about how intolerably poor "320 kbps MP3" sounded to the point where it was felt by some that the difference in quality could be easily detected within seconds of listening to a piece of music passed through MP3 lossy encoding.
Over the years, I have long thought about running a blind "Crowd Test" over the Internet like this one. My first exposure to MP3 was back in 1995 when I ran into a copy of Madonna's "Holiday" while "surfing" over the university Usenet forums and was amazed at the fidelity achieved out of only a few megabytes. Through the late 1990's, I realized however that 128 kbps was inadequate (touted by Fraunhoffer as "CD quality") - obvious high frequency distortions were noted, "swooshing" sounds were obvious with a good pair of headphones, clear phase distortions running the music through the old Dolby Pro Logic DSP's of yesteryear was also most unwelcomed.
By early 2000, I started migrating to a computer music-server based system and due to storage limits, decided to use MP3. In 2004, I went through my own tests of audio transparency and determined that LAME ABR at 210kbps, low pass filter set at 20kHz was transparent (actually, I thought 192kbps with ABR was good enough but I bumped up the low pass filter and bit rate just to be sure) with the equipment I had back then (primarily old pairs of Sony MDR-V6 & Grado SR60 headphones, Creative Audigy soundcard).
Over the years, as MP3 consumption increased, the audiophile press and general sentiment continued to deteriorate towards lossy compression in general despite advances made in algorithms. Bitrates have advanced up to 256+ kbps for most digital downloads these days as well. Despite all that has been written in the press or public opinion, I remained curious as to whether "transparency" with un-lossy-compressed CD audio is achievable.
For those who haven't read up on this, here are a few samples. Note that some of the articles do have good thoughts on the matter, but I'm questioning their take on what is perceivable sound quality; as if somehow MP3 (including high bitrate encoding) is a significant compromise in sound quality. Furthermore, I find it interesting that articles are taking the concept that "data rate" is equivalent to "sound quality" which I suspect was best publicized last year by Neil Young in his interviews (really? is bigger and more data better for perceived sound quality?).
- Neil Young and his rants against MP3 quality
- Wired's take
- Why MP3 sounds bad
- PCMag's take
Despite some comments I have received, this test is not about encouraging further adoption of MP3 - it's already adopted as the defacto lossy format. In my opinion, MP3 had its "time in the sun" as an efficient encoding scheme when storage was small and expensive (in fact it has been superseded by AAC/iTunes for many consumers). What I'm more interested in is whether the sound quality of MP3 encoding truly deteriorates to a point where it deserves to be the topic of great (or even any) criticism. From a practical perspective, if I buy a high bitrate MP3 from a place like Amazon, should I worry that I'm not able to experience the full sound quality even when played on good equipment?
Disclosure: I do not have any affiliations with the audio press, equipment manufacturers, and do not work in the audio field at all. This blog has no ads and I have no need for "clicks". I do not tout the contents of this study as "scientific" or even "accurate" by nature of the uncontrollable variables, but will try to present & interpret the data as objectively and thoroughly as I can.
In a few days, the survey will close... See you soon with preliminary results!
