So my headphone testing continues and I actually think I am finding a few pairs that match or beat the Sony CD3000 reference pair. There are many inter-related factors to look at:
Maximum usable volume
Cable type and length
Any special features and on and on
I am going to do my best to eliminate any opinions and focus on the relevant measurable aspects. So first I started with flat response. Why? Well heck, if you have a pair of super wide response super low distortion headphones but half the frequencies are twice aloud as the others, what good will it do us?
"Oh yes, sir, oooh, you are in the business of accurately reproducing color images for huge crowds? Well take a look at this video projector, It is an amazing, high resolution and very accurate for everything except everything that is the color red will be twice a bright as anything color green and you can barely see blue at all." Crazy business!
So I believe that an even frequency response is the most obvious and basic of requests. I am testing testing with leniency and what I believe to be a sloppy spec of + or - 4 db from 30 to 12K. Furthermore, I do my best to get every pair to pass and whenever possible and give them the benefit of the doubt if they are even close. At this stage it is not about eliminating as much as it is about finding.
I realize my test methods lack many of the high tech perfectionistic angles and that is intentional. I don't know about you but I have personally read some pretty amazing claims about various pieces of audio gear and have seen many accurate and complex test setups constructed to reinforce claims stunning sound quality only to find out that the testing done is focused on a small aspects and offers an incomplete story of the big picture and real world experience. I have also read many profound and ludicrous claims of audio wizardry backed by false logic, pseudo science and blatant lies with no form of testing credibility.
My goal here is to side step both of the overly technical and voodoo magical angles by offering some simple logical and inexpensively repeatable testing that will hopefully allow people to draw useful conclusions as to what to expect from the various headphones.
So first I sought out flat response and thinned the pack a bit and now I am ready to move on to the phase 2 challenge. I am now going to subject all the headphones that meet or are close to the + or - 4 db spec to a low frequency volume test. My experience has been that it is very difficult for headphones to cleanly reproduce very low frequencies. Another issue I often run into is headphones that just do not have enough output and are incapable reaching adequate volume levels without distorting. I guess the question to be pondered is "What is an adequate volume level?" Well, I personally tend to mix around 103db to 105db A-weighted and and 110db to 112db C-weighted or flat weighted. Since I am looking for live sound headphones, I believe it is reasonable to expect the headphones I seek to reach actual show volumes. This way I can PFL the instruments at the same volume I expect to hear them when the PA is on and if they have a reasonable amount of isolation, I will be able to hear the headphones over the PA during the show.
Hmmm, a goal and a test setup. Ok, how about a tone generator on my laptop run into a power amplifier (BGW Model 85) into the headphone under test. Then a mic to measure the output of the headphones, an oscilloscope (HP 1715A) and an RTA (Klark Teknik DN60) to see the waveforms and use the other 1/2 of the power amp to power a speaker (an old Rogers Studio 1 BBC Studio monitor) so I can hear the waveform as well. Simple, easy and I can put it all together with stuff I have laying around the house here.
Here is what I am going to do:
#1) Send a 100 hz tone at a level of 110db flat weighted in the Sony CD3000 headphones. The volume level will be roughly determined by the RTA's overall level meter and the oscilloscope will show the peak to peak voltage. I will them match that peak to peak voltage for all the rest of the headphones I test. I have the oscilloscope set to .5 volts per division and 110db flat weighed shows up as 3 volts peak to peak (6 divisions on the scope screen).
#2) Without changing the voltage sent to the headphones, proceed to sequentially test with tones at 100, 90, 80, 70, 60, 50, 40, 30 and 20 hz while monitoring the output waveform on the oscilloscope and listening to the speaker. I will be looking for the 6db down point (1/2 peak to peak voltage) and note that as well as any strong distortions in the sound or waveform.
#3) Eliminate the headphones that perform poorly and while contemplating the next round of testing.
For the test setup, I have a mic fitted into a drilled hole in a piece of wood that I wedge against one muff of the headphones. I do my best to form a good seal and move the mic around till I get the best readings I can. To keep this blog post from getting crazy long, I am posting the most relevant waveforms rather than every single one.
Ok, here is a good looking 60hz waveform. 100, 90, 80 and 70 all look and sounds fine. There is a bit of sub harmonics showing up at 30 hz but the sub harmonics tend not to be too intrusive sound wise.
At 40 HZ it starts to drop in level a bit and it is begriming to deform a bit from a clean sine wave.
30 HZ is about the 6db down point and ignore the double traces. The waveform is clearly distorting and you can see the harmonic distortion causing 50 hz (4th from left) to rise on the RTA
and the 20 HZ is at about 1/3 the voltage and the waveform is fairly triangular. For visual consistency, I am not altering the sweep or range on the scope for any of the tests.
As you can see the waveform kind of falls apart but it did make it down to 30 with a reasonable amount of output.
So next lets take a look at the trusty and loved Sony MDR-V6. Since the V6's are pretty much the McDonalds burger of live sound headphones, I will post more read outs so there is a good comparative reference. The 100 hz looked good and here is 90 HZ which also is good but has a slight drop in level of a fraction of a db.
At 80hz we are seeing the waveform begin to distort and become a bit more triangled as the tops curve rightward a bit.
The waveform continues to degrade at 70hz and down from 3 volts to 2 volts PtoP.
And here is 60hz with a significant amount of harmonics showing up on the RTA and quite audible.
Hmmm, these waveforms are pretty warped as the V6's are not happy with these frequencies at these levels. The sharper the points on those waveforms, the more HF buzz and worse the sound is. Also, look at the RTA. Each tone should be a single or double LED column with the sides dropping off. When you see two peaks with a space between them, that represents harmonic distortion, If the second peak is below the primary, that is not too bad. But when the peak is above like the 40hz harmonic above the 20hz in the pic above, then it is really audible. We were seeing similar issues with the CD3000's as well but it started at a much lower frequency. I am going to cut the V6's out of contending for "The Best Live Headphones" but maybe for a good live headphone for under $100, they could be the way to go.
at 30 there is some triangulation and harmonic distortion at a 50hz but they are holding level quite well
I forgot to shoot the photo of 20HZ but it shows less than a volt down and some triangulation to the wave but nowhere near as severe as the V6's were at a much higher frequency. These continue to be one of the best headphones I have tested so far and they get to carry on to round three.
Here they are at 50hz, down a volt and not looking so good
but I found that some of these headphones are a lot more susceptible to low end variations due to the amount of squeeze against the test setup (or my head). I am sure you have all found that when you squeeze some headphones tight against your head the low end gets louder. Well the headbands on the HD25 are actually quite tight as well so I tested while squeezing the ear muff tighter and dropping the drive voltage level (the scope is on .2 volts per division for this pic,) and re calibrated for the extra volume and now look at 50
Here is 30 at normal pressure
30 at reduced level and more pressure, notice the cleaner sine wave.
And at 20Hz with normal test pressure and level they are just under 2 volts down.
Looking pretty good for over the ear headphones and surprisingly superior lows over the V6's yet not up to par with some of the others. Remember, these had a very flat frequency response so on to round three.
Bouncing back in level at 60hz but still asymmetrical rounding on the top of the waveform
Degrading severely through 50 and 40 to a clearly clipped waveform at 30HZ
and I wont even bother showing you 20hz, say by bye to V600's.
Sennheiser MD280 PROare looked pretty good at 80, But uh oh! Looks like they are not going to fair so well. Check out 70 hz!
But wait, they come back to life if I rest the back of them gently against anything that is rigid with some mass.
My finger does not work but barely touching this little amp or the wall snaps the response back to life. You can feel the slight flexing of the plastic as well when I send a tone to them.
Really strange, it appears the plastic housing is flexing and canceling out the low end. Here they are at 40 when scooted against that amp (ignore the double traces as it is just mis triggering on the old scope.)
and here they are moved and not touching at 40.
With a 3db down point at 20hz while resting against the amp
and a fairly clean sine wave as well, these will have to stay in the mix while I contemplate the whole "has to touch something" oddity.
The AKG K271 MkII headphones came in last week. One thing really cool about them is that they actually turn themselves off when you take them off your head. Very cool but drove me crazy trying to test them till I figured it out. They tested up quite well with frequency response on the RTA but rather than backtrack here, I am going to drop them straight into the low end level testing. Here they are at 100
With a bit of a peak at 70 HZ they came back to a pretty good level but some distortion at 50
progressively sloppy down to 30
and 20 is a mess. But, when I lowered the volume 10 db, they really cleaned up and come up quite nice all the way around. I am going to drop them from the top contenders while also recommending them as an awesome set of headphones for lower volume monitoring, especially if the auto off feature is an asset to your application. I will post the frequency response at some point as well.
So I had a pair of Shure SRH 840's and the demo ended before I was done testing and I sent them back. Of course I then get hit with so many requests to test them I have to reorder a pair. Here they are at 100
Getting a few db louder at 80
Still hot at 60 but fairly clean with a bit of sub harmonics rising.
Back to normal level at 50 and 40 and starting to drop at 30 and become more triangular.
With 20hz about 10 db down but reasonably well shaped compared to most
These are looking better than I expected and will have to see how they do on frequency response test.
Next up, the Denon AH D2000. Oh my. Check this out! Here they are at 70
Still holding strong at 50 with a nice sounding sine wave and some sub harmonics coming in at 30hz
A nice sine wave and no drop in level at 30hz yet harmonics at 50hz are up
Barely any level down at 20 and still a clean waveform!
and look! not even 3db down at 15 hz! (Yes, forgot to re label the pic from 20 to 15). These are so impressive I have just ordered a pair of the top of the line of that series, the Denon AH-D5000's that I will test for ya when they come in.
Ok, so that leaves us with five pairs plus the reference Sony CD3000's and the Beyer 770 and Denon D5000's that are on their way to me next week.:
As this test adventure progressed I have reinstated the Shure 840 as I never got a proper RTA reading on the first set that I had to send back when the demo ended.
I have found that I am getting better readings on the Sennheiser HD280 PRO's if I put some pressure on the outer shells so brought those back in the mix.
I wanted to continue testing the Equation Audio RP21 headphones with a removable cable and nice price point but for some reason they rushed me to get their demo pair back faster than I was able to put time into testing them for round 2. I don't think they would have made the "A" list but they do look like they have the potential to beat the V6's in the "under $100 street price" range.
I have been getting requests to test several other pairs from various manufacturers that I will gladly test if someone wants to send me a pair or if I can get some substantial credibility that any of those headphones have a strong chance of beating the top level cans I have already, I will consider purchasing for the test.
Will try and do a video in the next few days or so showing these the test methods and such so you can hear the top contenders as well. Here is a link to the video's so far. Cool cool, till next time rock on!
re: your test setup
Could that piece of wood be reinforcing some of those harmonics you complained about? Or coupling back to ref. mic?
Suggest using a piece of styrofoam or a chunk of florist's foam, something that emulates an A-1s head (TIC).
Just a thought, not a criticism. Loved the whole testing procedure.
The wood is pretty rigid and though I did not show it, I also did measurements with the headphones on my head and got nearly identical waveforms. If I test at lower volumes, most of the headphones clean up somewhat. Also, since the Denon's are not having the issue, I think that lends strength to the setup not being the cause.
One artifact of the wood that I do see in the measurements are some high frequency issues that vary a bit from headphone to headphone. For low testing, slight movements of the earm muffs screws up the reading when the headphones are on my head so I use the wood jig. For the higher frequencies, the hardness of the wood seems to effect the readings a bit so I think the readings while on my head are a better representation of what is heard.
I'm using Ultrasone HFI-680
Some cool testing on headphones there. I accidentally left my Sony 7506 behind at one of my recent shows and I'm waiting for your final results before I invest in my next pair. The 7506 did rather well for me and I quite enjoy using it. Have you ever thought about how you will go about trying in-ear monitors? I've owned Shures E3s and E4s and demoed a few models from Westones and UEs, but the E4 seems to be the flattest sounding in my ears. Do you have any recommendations and experiences with regards to in ear monitors?
i was wondering about your view on dedicated headphone amps compared to power amps stepped down by resistors in home audio receivers, does it make a big difference? I ask this because in the audiophile world it seems like receivers are completely shunned when it comes to amplifying headphones.
I did do some testing on resistors in line with headphones and my take is that I prefer no resistor. As far as dedicated headphone amps, I think that they may be overkill and often cost more than the value received.
One way around this is to just open up any receiver and it is pretty simple to locate and to put a a pair of clip leads bypassing the headphone resistors.
Or, just make a female 1/4" to bare ends stereo jack and wire it into the "b" speaker jack of an amp and boom, super easy, full power amp to headphones, no resistor, no hype, no scams and all good!
Ha, save the cash and have the most powerful headphone amp on the block!
I'm really enjoying your headphone comparison, great job! I'm a headphone and 2-channel gear addict with way too much stuff, but hey it's a hobby. I just bought a used pair of CD-3000 (recabled to balanced, new cups, new ear pads) and I'm anxious to get them to hear what they sound like.
Check out these headphone measurements (put link in home page below)
I'd like to hear your comments as to the usefulness of the measurements/graphs!
Wish you'd done the wave test on the ESP 950. I think you may be the first person to ever post tests on line of it.
Looking at the HD25, I wonder if the custom isolating version I have of them would test like this second attempt considering you get a perfect seal every time.
A new headphone that supposedly does very well on these types of tests is the Beyerdynamic DT1350. Might be the new reference closed studio phone.
You should check out the Fostex planars, too.