In my last post, I covered some of the high-level features in my reference System.Drawing/GDI+ resizer. In this post, I’ll be discussing some of the options on the Graphics and ImageAttributes classes that affect the image quality and scaling/rendering performance of Graphics.DrawImage(). I’ve seen a lot of misinformation around the web when it comes to these settings, some of it even coming directly from MSDN. The GDI+ documentation is much more complete and accurate than the documentation for the System.Drawing wrappers, but even then, some of the concepts can be foreign to developers without a background in image processing. I’ll attempt to describe them better and with concrete examples.

The important thing to take away from this is that each of these settings and its corresponding values are designed to address a specific facet of image quality. There’s no secret combination that unlocks image processing nirvana. If quality is your only concern, you can set all the HighQuality options and know you’re getting the best GDI+ can do, but you’ll pay an unnecessary performance penalty (we saw examples of that in Part 1 of this series). If you want the best quality with the best performance possible, you’ll need to learn what the options do individually so you know when and how to use them. There actually aren’t that many.

Graphics.InterpolationMode

This setting is the most important and perhaps the least understood, so there’s a lot to say here. I’ll be doing an entire post on it, in fact (I know, I’m a tease). For now, I’ll just say that unless you have a good reason not to, you should always use InterpolationMode.HighQualityBicubic, which is the highest quality interpolator available in GDI+. It’s also the slowest, but you get what you pay for…

ImageAttributes.SetWrapMode()

One of the more common complaints you’ll see about DrawImage() is that it produces artifacts around the edges of images. I can show my own example of the artifacts in question by resizing a solid grey image using InterpolationMode.HighQualityBicubic and the default values for the remainder of the settings on the Bitmap and Graphics objects. I opened the resulting image and zoomed in (15x) on the corner.

You can see two things going on here. First, the outermost edges have the transparency grid showing through in my viewer, so they’re not completely opaque, which is strange considering the original image didn’t have an alpha channel. Second, there is a lighter 1 pixel wide line set 1 pixel in. That lighter color also doesn’t appear anywhere in the original image. Note that if I saved this image in a format that didn’t support transparency (like JPEG), the semi-transparent pixels would be blended with black instead and would show up darker grey. That was the complaint in the stackoverflow question I linked above.

What we’re seeing in this case is the result of several factors.

1. GDI+ starts with a blank (transparent or black – depending on your PixelFormat) canvas and draws your resized image on top of that.
2. The HighQuality scalers in GDI+ do their image scaling in RGBA mode, even if neither your source nor destination image is RGBA.
3. GDI+ doesn’t handle edge pixels correctly by default.
4. The HighQualityBicubic interpolator has sharpening properties.
5. GDI+ doesn’t handle sharpening with transparency correctly.

There’s nothing you can do about the first two points. That’s just how DrawImage() works.

The last two points are the cause of the lighter lines, and they can be negated by using a non-sharpening interpolator. Here’s the same image resized with the HighQualityBilinear interpolator, which smooths rather than sharpens. The lines have been eliminated, but the semi-transparent edge pixels remain.

Finally, if we use the plain Bilinear interpolator, which has a smaller sampling range, we get the solid image we expected.

Yeah, I know… I said to always use HighQualityBicubic…

The reason using a lower quality interpolator fixed the issue is the third point I mentioned above. GDI+ doesn’t handle edge pixels the same way most image scalers do. Imagine your source image is sitting in the middle (if there is such a thing) an infinite transparent plane. During the resize, any time the interpolator tries to sample pixel data that falls outside the bounds of the original image, it has nothing but transparent pixels to use. Both of the HighQuality interpolators in GDI+ use convolution kernels that require pixels from beyond the image edges. Most image scalers will simply extend the edge pixels outward to get the correct results, but GDI+ doesn’t do that. To get the desired behavior, you must use an overload of DrawImage() that accepts an ImageAttributes object.

ImageAttributes has a WrapMode property that allows you to tell GDI+ to tile the input image across that imaginary infinite plane to fill in the pixels beyond the edges. The closest you can get to a normal implementation is to mirror the edge pixels in all directions, by using ImageAttributes.SetWrapMode(WrapMode.TileFlipXY). This ensures that there are always valid pixels to sample, and you’ll get the expected behavior for the edge pixels. Don’t worry, it’s not as expensive as it sounds, and the quality tradeoff is worth it. I won’t bother with a sample image… it’s just a grey square like the one above, as you’d expect, no matter which interpolator you choose.

My point about GDI+ not handling sharpening with transparency correctly still stands, though. If your image contains a transition from transparent to opaque somewhere in the middle of the image, you’ll get the same type of artifacts we saw above, and the WrapMode won’t help you, because the edge pixels aren’t the issue. Below is the result when resizing an opaque grey square surrounded by a transparent border. Here I’ve used InterpolationMode.HighQualityBicubic with WrapMode.TileFlipXY.

You can see the artifacts are back, now appearing in the middle of the image instead of on the edge. There’s no complete fix for that (see the next section for a partial fix), other than to use a non-sharpening interpolator… or to use a different scaler (like MagicScaler) that handles sharpening and transparency correctly.

Graphics.PixelOffsetMode

Looking at the System.Drawing docs, it appears you have five settings available for PixelOffsetMode. They make some vague references to a tradeoff between quality and speed to differentiate them, and they even manage to get those wrong. Contrast that with the GDI+ docs for the same setting. Those docs make it clear; there are really only two options, and the Remarks section describes quite nicely what each does. The real options: None and Half. The rest are just aliases for those two. I’ll make it even simpler: None=Bad, Half=Good. The default value is Bad. While there may technically be a performance difference between the two, it’s not measurable. The quality difference most certainly is. In the following example I have resized a 3x3 pixel checkerboard grid to 100x100 using the NearestNeighbor interpolator, first with the default mode and then the correct one.

The results speak for themselves. You should always set the PixelOffsetMode to Half or HighQuality (same thing) when using DrawImage().

That said, I have to admit that my examples from the section on WrapMode used PixelOffsetMode.None to accentuate the artifacts. Had I used Half/HighQuality, the artifacts would still be present, but they would be more faint. Here are those two examples redone with PixelOffsetMode.Half. In order to show the edge artifacts in the first image, I have not set the WrapMode.

Again, setting both PixelOffsetMode.Half and WrapMode.TileFlipXY gives you the best quality GDI+ has to offer. That will completely correct the artifacts at the image edge and will give the result above on a hard transition from transparent to opaque away from the edge. Depending on your viewing environment, the artifacts in that one may be quite difficult to see now. If you can’t see them, just take my word for it; they’re there.

Graphics.CompositingMode

This is an interesting one in that the default value (CompositingMode.SourceOver) is the more expensive option. Every other Graphics setting defaults to lower quality/higher speed. SourceOver is what you want if you’re actually blending multiple layers, but you’ll incur an unnecessary performance hit if you’re not. In a simple image resizing scenario, you’re compositing the resized image over a blank background, so there’s no need to calculate any blending between those layers. You can, in these cases, set the value to SourceCopy (which simply overwrites the bottom layer with the top) to get better performance. If, however, you’re applying a background matte to a partially transparent image, drawing text over an image, or blending multiple images together, you’ll want to use the default value of SourceOver.

In my reference resizer, to maximize performance, I set the value to SourceCopy by default and switch to SourceOver only if matting is requested.

Graphics.CompositingQuality

This one is my favorite, if only because this is the setting that I read the most crazy things about on teh internets. It can also be a big performance killer.

Much like PixelOffsetMode, there’s a gulf in quality between the System.Drawing docs and the GDI+ docs for CompositingQuality. The .NET documentation again presents you with five options and only vague comments about speed vs. quality to help you choose between them. The Remarks are even worse on this one, claiming differences in performance and quality between two aliases for the same value. They also say something about surrounding pixels being taken into account, which is complete nonsense. The GDI+ docs are much more clear about what the real options are and what they actually do. In this case, as before, there are really two options: AssumeLinear and GammaCorrected. AssumeLinear=Bad, GammaCorrected=Good. The default is Bad. See this MinutePhysics video for a simplified explanation of the difference. The math is dumbed down in the video, but you’ll get the idea. The real math can be found in the sRGB spec if you’re interested.

For my example, I’ll blend green and red as they do in the video so you can see the effect. I start with an image that has a gradient from green to transparent (shown here over a transparency grid). I then matte that over a red background to see how they blend.

Note that the image on the right blends smoothly (Good) while the middle image shows a dark, muddy line where the colors blend (Bad).

The trick in this case is that doing gamma-corrected blending is much more expensive than doing it the wrong way, so the speed vs. quality tradeoff is very real this time. When using CompositingMode.SourceOver with CompositingQuality.GammaCorrected, you will have a measurable performance hit. Again, in my reference resizer, I apply those settings only if a matte is being applied to a partially transparent image. For normal image scaling, there’s no need; you’ll only be slowing things down, potentially a lot.

Bonus: Graphics.SmoothingMode

Unless you’re drawing vector shapes on top of your image, this setting has no effect. It doesn’t affect DrawImage(), nor does it affect text rendering. I see lots of developers including it in their image resizing code, but for the most part, they’re just wildly setting anything with HighQuality in the name because the documentation does such a poor job of explaining what the settings actually do, and they can’t be bothered to dig deeper. It’s not harmful; it’s just a throwaway line of code for the cargo cult. My reference resizer does no drawing, so it doesn’t set a SmoothingMode.

Tune in next time for my detailed examination of the GDI+ InterpolationMode values.

Questions? Suggestions? Sound off in the comments.

Part 1 of this series introduced my reference System.Drawing/GDI+ image resizer, which I use to compare the quality and speed of MagicScaler. Starting with this post, I’ll be discussing some of its implementation details and the settings it uses.

The settings on the Graphics class in System.Drawing/GDI+ tend to get most of the attention when people write about resizing with DrawImage(), but before I get in to those, I wanted to cover a few of the other features a resizer should have and some non-Graphics settings that make a difference in speed or quality.

The points in this post relate to my reference GDI+ resizer code, available in this gist.

Dispose all the things!

The first thing I’ll point out here is all the using statements in my reference GDI+ resizer. Every GDI+ object has a system-level handle associated with it, and the .NET wrappers for those objects implement finalizers to make sure they get cleaned up. When dealing with graphics, however, it’s important to note that a single object, if it represents an image, may be 100MB+. And since these are unmanaged handles, the Garbage Collector doesn’t know whether there’s a ton of memory tied up underneath those tiny managed wrappers, so it may be lazy about cleaning them up. Even absent memory pressure, there are a finite number of handles available, and failing to dispose of the hundreds or thousands of handles a typical web app might tie up while performing image resizing can result in errors or runaway memory usage. You may see Out of Memory exceptions, unexpected App Pool restarts, or the dreaded GDI+ Generic Error. All the System.Drawing wrappers implement IDisposable, so use it. On everything.

Loading the Image

I always use the Image.FromStream(Stream stream, bool useEmbeddedColorManagement, bool validateImageData) overload when opening my source image.

The second parameter to that overload (useEmbeddedColorManagement), when set to true, tells GDI+ to read and apply any embedded Color Profiles in the image file. Setting that parameter to false (or omitting it, as false is the default) may improve performance slightly, but it can cause the colors in the image to be misinterpreted when the image is re-encoded and viewed in a web browser. Setting the parameter to true forces GDI+ to convert the image to the sRGB colorspace during processing, meaning it will look as intended after processing (with certain limitations – for example, CMYK conversions are not done correctly). For more information on color management and examples of what happens when you get it wrong, have a look at this excellent series by Jeffrey Friedl.

Here’s a quick example showing what happens when an image’s colorspace is incorrectly interpreted. The source image in this case was saved in the Adobe RGB colorspace and has an embedded ICC profile indicating such. This is common not just in images saved from Adobe apps (e.g. Photoshop and Lightroom) but also in JPEGs saved directly from high-end digital cameras. First, the image resized without color management, and then with.

You can see the colors in the first image just look at bit washed out. They’re not completely wrong, so it’s an easy mistake to miss. It certainly looks better done correctly, though.

The third parameter to Image.FromStream() (validateImageData) allows you to tell System.Drawing to skip its default validation step during image load. This seemingly simple parameter is so important (and confusing) that warrants its own post, so look out for that later in this series. For now, I’ll just say you want to skip validation. It can have a huge impact on performance. Image.FromStream() is the only method of loading an image file in System.Drawing that exposes that option, so it’s the method I always use.

Images with Multiple Frames

GDI+ supports accessing multiple image frames from both the TIFF and GIF decoders. Specifically, it supports retrieving any individual frame of an animated GIF or any page of a multi-page TIFF file. GDI+, for some reason, differentiates between the two types of multi-frame image containers by using the concept of frame dimensions. Image.SelectActiveFrame() accepts the dimension and frame number. The .NET docs don’t offer much help as far as choosing the dimension value. See the Remarks section in the equivalent GDI+ docs for more clear guidance. The short version: use FrameDimension.Time for GIF and FrameDimension.Page for TIFF.

PixelFormat

While GDI+ supports opening image files of many different pixel formats, when it comes to using the Graphics class, you’re restricted to RGB and RGBA. PixelFormat.Canonical refers to RGBA, and unless you explicitly set the PixelFormat when creating a new Bitmap, that’s what you’ll get. When you save your image, it will be saved in the same format as your Bitmap (RGB or RGBA) as long as it’s supported by the encoder. These restrictions lead to two quirks in my reference resizer. First, if indexed color is requested, the output format is forced to GIF. GDI+ doesn’t support PNG8, and I wanted my reference resizer to have visually comparable output with MagicScaler. Second, I default to RGB output for all images and enable RGBA only if the input image has alpha support. While this doesn’t necessarily speed processing during the resize (in most cases DrawImage() works in RGBA anyway – I’ll have more on that in a future post), it does speed the encoding process and makes the output files smaller than if they were all saved as the default RGBA.

It’s also worth pointing out the way I determine whether the input image has alpha support. You may see other examples that check for alpha support by using Image.IsAlphaPixelFormat() with the input image’s PixelFormat property. This seems perfectly reasonable, but it turns out it doesn’t work the way you’d think. As an example, consider what happens when you open a greyscale image in GDI+. Greyscale isn’t supported for processing in GDI+, so the image is converted to the canonical format (RGBA) automatically. Image.PixelFormat will report alpha support in that case, which would be wrong. The only reliable way to get the pixel format information is to check Image.Flags. The flags set there reflect the pixel format of the input image, not the converted intermediate.

EXIF Orientation

Increasingly, the images we handle on the web come from digital cameras or smartphones, and these will often be captured rotated, with an EXIF Orientation tag set to tell viewing software how to display them correctly. Nearly all modern software will read and react to those tags, so when viewing an image that is stored rotated, you may never know it. It’s important when processing images that you process the Orientation tag if present since it will be removed during re-encoding. My reference resizer includes an extension method called ExifRotate() that does just that. GDI+ only supports retrieving EXIF metadata using the magic number values, so for Orientation, you’ll want property ID 274 (0x112). The possible values for that property are explained in the link above.

Image.RotateFlip() can do both operations at once, and as the name implies, the rotation is done first. This can make a difference if doing both a rotation and a flip, but the reference resizer handles all possible EXIF values correctly if you want to avoid trying to wrap your head around how that works.

JPEG Quality

I’ll start this topic off by saying that the default JPEG quality setting in GDI+ is quite poor, so you’ll almost certainly want to change it if you’re saving JPEGs. Though it isn’t documented anywhere that I’ve found, in my own testing I’ve determined the default GDI+ quality level is set to 75 (on a scale of 0 to 100) in all versions of Windows I’ve checked. That number can be tricky, though, as it doesn’t have any meaning outside the specific encoder you’re using. For example, you’ll often hear/read that 75 is a good quality level to use in Photoshop when saving web graphics. And in fact, it is. 75 in Photoshop, though, has nothing to do with 75 in GDI+ (or the underlying WIC encoder it uses). There are no standards for quality values, and different encoders may use their own quantization tables. In my own experience, I’ve found that anything less than 80 is unacceptable from the Windows encoders and that for smaller images like thumbnails, a value of up to 95 is more appropriate. You can’t see it in my reference resizer implementation because the logic is in the shared ProcessImageSettings class, but if a quality level isn’t explicitly set, I dynamically choose a value between 83 and 95 depending on the output image resolution.

Another fact of note is that GDI+ always uses 4:2:0 chroma subsampling regardless of the quality setting you choose, and it offers no way to change that. Photoshop’s ‘Save for Web’ uses 4:2:0 subsampling only on quality levels of 50 or below and switches to 4:4:4 at quality level 51. This is another reason for the vast difference in image quality at comparable settings values between GDI+ and other encoders. The underlying WIC encoder, by the way, does offer the ability to change JPEG subsampling settings, but GDI+ doesn’t use it (MagicScaler does).

The only other thing I’ll point out here is the using statements on the EncoderParameter and EncoderParameters objects in the reference resizer. These are often overlooked, but as I pointed out before, everything in GDI+ uses system-level handles, so don’t forget to dispose those.

Bonus: Image.GetThumbnailImage()

I don’t use this.

If all you want is a quick way to get a thumbnail version of a larger image, this sounds like it’s just the thing, but don’t be fooled. In the worst case scenario, this method will retrieve a very low quality, low resolution embedded JPEG from the EXIF metadata in your input file. It will probably be ugly. In the best case scenario, it will create the thumbnail on the fly, using the GDI+ defaults with all the quality issues that come with them. I’ll be pointing those out in the next post. If you care at all about the quality of your images, forget this method exists.

Tune in next time, when I’ll be covering the Graphics class and its many mysterious settings…