Forty-five years ago, I wrote camera reviews for Modern Photography Magazine. Testing a camera was a lot harder in those days, as our equipment was quite limited. It placed heavy demands on the experience and skill of the people doing the testing.
For instance, to test the resolution of a lens, we had to shoot pictures of the test charts on high-resolution film and then examine the results through a microscope. Although you could blow up the negative on an enlarger, this introduced the faults of the enlarger and its lens into the process.
Testing a digital camera is a lot easier. There is no film, so results can be seen immediately. The need for making prints is eliminated as we can examine the image in detail on the computer's monitor.
What brought all of this to mind is my recent experience with the Sony DSC-H9. As I took more and more pictures with it, I could see that there were problems with image quality. I exchanged the camera for another, hoping to eliminate the possibility that the first one was defective. In the end, I reverted to my camera-testing persona, spending hours examining images from the DSC-H9, hoping to identify the faults.
So I went to the online forums to see how others were faring with this camera. What I found, for the most part, was close-ups and nature photography, all of them quite striking. Pictures of birds, bees, bugs, beetles, and buds
Now I'm going to tell you a secret.
Macrophotgraphy is a very undemanding test of a camera's performance. All you need to produce "striking" close-ups is a camera whose focusing system lets you get close enough to the subject, and the ability to produce an image that's reasonably sharp in the center of the frame. In the end, this type of photography is more of a test of the photographer's skill than how well a camera performs. That's why most reviews just mention this facility in passing.
What this sort of image fails to locate is the very faults that plague the DSC-H9 -- the softness at the sides and corners of the image, the strange loss of detail in many parts of the picture, heavy color fringing, and the poor rendering of subjects, like portraits, that require the proper handling of skin tones with subtle gradations.
Here's a far more demanding test of a camera's image quality, and it's surprisingly easy to do.
Go outside with your camera, in daytime, when there's plenty of light. Set the ISO value at 200 or less. Leave the zoom at or near its widest setting. With today's compact zooms, this should give you a high shutter speed and near-infinite depth of field. In theory, this setup should be capable of filling the frame with a high-quality image, having the same sort of image quality at the sides and corners as in the center.
Now find a scene that will fill the frame with the same sort of detail in all parts of the image. Perhaps a large grove of trees, or maybe an urban scene with lots of buildings. The important thing is that the scene at the center of the image be the same as at the sides and corners.
Now take lots of pictures in this and similar situations. Play with the camera's settings for things like sharpness, contrast, and color rendition. No need to keep notes about which settings you used, as we had to in the good old days. The camera will record all this data for you in its EXIF data that it attaches to every picture.
Try this in more than one setting, and try to find settings that are filled with lots of detail. Shoot from at least twenty or thirty feet away. No close-ups!
Now load the image files into your computer and examine them on the monitor. You should look at them both scaled down to get an overall impression, and at full size (100%) to examine the image quality. An easy way to do this is to open a web browser then just drag an image file from its folder into the browser window.
Carefully examine the center of the picture first, as this will show you the best image quality that the camera can deliver. Now move to the edges and the corners of the image and compare the image quality there. Look for sharpness and the handling of detail, which should be fairly uniform across the entire picture. If it's different, note the difference. Does the detail at the sides and corners become unsharp with an out-of-focus appearance? Does it look smeared? Does it simply vanish into a "watercolor" look, similar to the detail in a painting? How about color rendition -- is it the same everywhere? Are there color fringes around the boundaries between light and dark areas?
For a further test, shoot some portraits. These should be done outdoors, but in the shade, so that the light is bright and fairly uniform. Get in close, or use the zoom so that the face fills a good part of the frame. As before, try lots of variations at different settings.
The result should be a pleasant rendering. Look for a smooth rendering of skin tones rather than a blotchy effect. Look for accentuated reds that give the subject a "drunken" appearance. Examine the rendering in the hair and eyelashes.
If you have another digital camera, it's a simple matter to compare the images from both. This will give you a better frame of reference for your comparisons, and it should demonstrate that your new camera has image quality at least as good as your old one.
As a final part of this exercise, you should post your results in the online forums so that others, considering the purchase of this camera, will be able to get a better idea of its overall image quality.
Even though we had lots of fancy equipment and did lots of tests at Modern Photography, the final test for any camera was pretty much the ordinary photography I described above. If the camera didn't perform well at the simple tasks I've outlined, all the fancy numbers and charts would not redeem it.
In addition, we tried to devise simple tests that anyone could use to evaluate a camera's performance. We couldn't test every camera/lens/film combination, not to mention the variability between individual cameras. These tests gave anyone a chance to get some idea of how well their own camera performed.