New CamcorderInfo Color Test

Dear CamcorderInfo readers old and new: As of March 19, 2013, we are changing the way we test and score video color performance. Color performance is the metric we use to determine how accurately colors are rendered while recording video. This change is just a small step in our never-ending quest to provide our readers with the most accurate and most relevant data possible. We want to make sure that you have the information you need to choose the perfect camcorder for you.

In order to ensure that camcorders tested in the future are comparable with those tested in the past, we have retroactively updated the scores for all camcorders tested since 2009. Color scores for all camcorders tested since then now take into account both the new testing methodology and the new scoring formula. Read on for more details.

What Hasn't Changed

We will continue to shoot our color performance test in both bright and dim lighting conditions. Both tests continue to utilize an evenly lit DSC Labs ChromaDuMonde chart. As before, we shoot in full auto mode, testing additional settings and color modes depending on the complexity of the camcorder.

Our methodology for analyzing the footage also remains unchanged. We import footage using professional editing software and generate high-quality frame grabs. These images are run through Imatest using the program's Multicharts module. Imatest analyzes the 28 color patches on the ChromaDuMonde chart and compares the known color values to what was recorded. These values are run through a complex series of equations called the CIEDE2000 color error formula, which calculates the ‘color error’ of the image.

What's New: The Color Test

Though so much is remaining the same, we are making one significant change to our testing procedures. Instead of running our color test using the camcorder's custom white balance setting, we will shoot our video color performance test using the automatic white balance setting. The reason for this is threefold:

• We find that most consumers don't know how to, can't be bothered to, or don't have time to set a custom white balance. More importantly, a custom white balance can only be set when you start shooting. If your environment changes mid-video, a custom white balance will not dynamically adjust to your new setting. We feel that our test should more accurately reflect the way most consumers shoot video: with automatic white balance.

• Some entry-level camcorders, pocket camcorders, and smartphones do not actually allow the user to set a custom white balance. While this is certainly a disadvantage, we find that sometimes these camcorders perform quite well compared to the automatic settings of their more expensive peers. Shooting our color test in automatic mode will allow all video-recording devices to be compared fairly.

• Perhaps of most interest to our professional and prosumer readers, we will soon be rolling out a new white balance test, which will specifically measure the automatic and custom white balance performance of a video camera under incandescent, fluorescent, and daylight conditions. Consumers that want to know how a camcorder performs with custom white balance will have access to ample data from this new test.

What's New: The Color Score

The most important change in our video color performance test lies not in the testing procedure itself, but in our newly developed scoring algorithm. As you may know, Reviewed.com believes in an infinite scoring system, which allows us to rate products in a constantly improving technological landscape.

If we review a camcorder in January that has the best color score we've ever seen, it would make sense to give that camcorder a color score of '10.' But what happens when an even better camcorder comes along in October? Is that also a '10'? Do we go back and retroactively give the January camcorder a color score of '9.5'?

Our solution for the past several years was to avoid the concept of a maximum score of '10.' If that January camcorder got a '10,' then the superior October camcorder would get a '13.5.' However, we recognized that consumers were sometimes confused by a color score of '13.5' and just wanted to know if this product is the best. Or second best. Or tenth. This week, we are rolling out a brand new color error algorithm that looks like this:

With this new formula, the majority of camcorders fall somewhere in the middle section, with the very best color scores approaching 10 and the very worst color scores approaching 0. That way, as better and better camcorders are released (or worse and worse camcorders), the formula remains flexible enough to handle the new data.

What You'll See

So, what changes will you see on the site?

For starters, all video color performance scores for all camcorders have been adjusted using the new formula. That means that all the color scores now fall between 0 and 10, with the majority landing somewhere between 3 and 7. It also means that you can tell quickly and easily what camcorders have the best color accuracy. No score goes above 10.

Color scores for individual camcorders have also been changed to utilize automatic white balance results instead of custom white balance results. This penalized some camcorders more than others—significantly impacting only those camcorders with very poor automatic white balance.

Finally, you'll see that the ratings have shifted slightly to accommodate the new results. Of course, we also have three new camcorder reviews to accompany the launch of our new color test. The arrival of the Canon HF G20, Panasonic HC-X920, and JVC GZ-VX815 had a much larger impact on our ratings than the new color scores.

And, of course, this all goes hand-in-hand with our brand new website and review design!

Stay tuned for even more changes coming in 2013. The more-accurate-than-ever color score and the new site design are just the beginning of the improvements you'll see on CamcorderInfo this year, as we continue to make our site more accurate, more informative, and more enjoyable to visit.