Part 1 of this series introduced my reference System.Drawing/GDI+ image resizer, which I use to compare the quality and speed of MagicScaler. Starting with this post, I’ll be discussing some of its implementation details and the settings it uses.

The settings on the Graphics class in System.Drawing/GDI+ tend to get most of the attention when people write about resizing with DrawImage(), but before I get in to those, I wanted to cover a few of the other features a resizer should have and some non-Graphics settings that make a difference in speed or quality.

The points in this post relate to my reference GDI+ resizer code, available in this gist.

Dispose all the things!

The first thing I’ll point out here is all the using statements in my reference GDI+ resizer. Every GDI+ object has a system-level handle associated with it, and the .NET wrappers for those objects implement finalizers to make sure they get cleaned up. When dealing with graphics, however, it’s important to note that a single object, if it represents an image, may be 100MB+. And since these are unmanaged handles, the Garbage Collector doesn’t know whether there’s a ton of memory tied up underneath those tiny managed wrappers, so it may be lazy about cleaning them up. Even absent memory pressure, there are a finite number of handles available, and failing to dispose of the hundreds or thousands of handles a typical web app might tie up while performing image resizing can result in errors or runaway memory usage. You may see Out of Memory exceptions, unexpected App Pool restarts, or the dreaded GDI+ Generic Error. All the System.Drawing wrappers implement IDisposable, so use it. On everything.

Loading the Image

I always use the Image.FromStream(Stream stream, bool useEmbeddedColorManagement, bool validateImageData) overload when opening my source image.

The second parameter to that overload (useEmbeddedColorManagement), when set to true, tells GDI+ to read and apply any embedded Color Profiles in the image file. Setting that parameter to false (or omitting it, as false is the default) may improve performance slightly, but it can cause the colors in the image to be misinterpreted when the image is re-encoded and viewed in a web browser. Setting the parameter to true forces GDI+ to convert the image to the sRGB colorspace during processing, meaning it will look as intended after processing (with certain limitations – for example, CMYK conversions are not done correctly). For more information on color management and examples of what happens when you get it wrong, have a look at this excellent series by Jeffrey Friedl.

Here’s a quick example showing what happens when an image’s colorspace is incorrectly interpreted. The source image in this case was saved in the Adobe RGB colorspace and has an embedded ICC profile indicating such. This is common not just in images saved from Adobe apps (e.g. Photoshop and Lightroom) but also in JPEGs saved directly from high-end digital cameras. First, the image resized without color management, and then with.

You can see the colors in the first image just look at bit washed out. They’re not completely wrong, so it’s an easy mistake to miss. It certainly looks better done correctly, though.

The third parameter to Image.FromStream() (validateImageData) allows you to tell System.Drawing to skip its default validation step during image load. This seemingly simple parameter is so important (and confusing) that warrants its own post, so look out for that later in this series. For now, I’ll just say you want to skip validation. It can have a huge impact on performance. Image.FromStream() is the only method of loading an image file in System.Drawing that exposes that option, so it’s the method I always use.

Images with Multiple Frames

GDI+ supports accessing multiple image frames from both the TIFF and GIF decoders. Specifically, it supports retrieving any individual frame of an animated GIF or any page of a multi-page TIFF file. GDI+, for some reason, differentiates between the two types of multi-frame image containers by using the concept of frame dimensions. Image.SelectActiveFrame() accepts the dimension and frame number. The .NET docs don’t offer much help as far as choosing the dimension value. See the Remarks section in the equivalent GDI+ docs for more clear guidance. The short version: use FrameDimension.Time for GIF and FrameDimension.Page for TIFF.

PixelFormat

While GDI+ supports opening image files of many different pixel formats, when it comes to using the Graphics class, you’re restricted to RGB and RGBA. PixelFormat.Canonical refers to RGBA, and unless you explicitly set the PixelFormat when creating a new Bitmap, that’s what you’ll get. When you save your image, it will be saved in the same format as your Bitmap (RGB or RGBA) as long as it’s supported by the encoder. These restrictions lead to two quirks in my reference resizer. First, if indexed color is requested, the output format is forced to GIF. GDI+ doesn’t support PNG8, and I wanted my reference resizer to have visually comparable output with MagicScaler. Second, I default to RGB output for all images and enable RGBA only if the input image has alpha support. While this doesn’t necessarily speed processing during the resize (in most cases DrawImage() works in RGBA anyway – I’ll have more on that in a future post), it does speed the encoding process and makes the output files smaller than if they were all saved as the default RGBA.

It’s also worth pointing out the way I determine whether the input image has alpha support. You may see other examples that check for alpha support by using Image.IsAlphaPixelFormat() with the input image’s PixelFormat property. This seems perfectly reasonable, but it turns out it doesn’t work the way you’d think. As an example, consider what happens when you open a greyscale image in GDI+. Greyscale isn’t supported for processing in GDI+, so the image is converted to the canonical format (RGBA) automatically. Image.PixelFormat will report alpha support in that case, which would be wrong. The only reliable way to get the pixel format information is to check Image.Flags. The flags set there reflect the pixel format of the input image, not the converted intermediate.

EXIF Orientation

Increasingly, the images we handle on the web come from digital cameras or smartphones, and these will often be captured rotated, with an EXIF Orientation tag set to tell viewing software how to display them correctly. Nearly all modern software will read and react to those tags, so when viewing an image that is stored rotated, you may never know it. It’s important when processing images that you process the Orientation tag if present since it will be removed during re-encoding. My reference resizer includes an extension method called ExifRotate() that does just that. GDI+ only supports retrieving EXIF metadata using the magic number values, so for Orientation, you’ll want property ID 274 (0x112). The possible values for that property are explained in the link above.

Image.RotateFlip() can do both operations at once, and as the name implies, the rotation is done first. This can make a difference if doing both a rotation and a flip, but the reference resizer handles all possible EXIF values correctly if you want to avoid trying to wrap your head around how that works.

JPEG Quality

I’ll start this topic off by saying that the default JPEG quality setting in GDI+ is quite poor, so you’ll almost certainly want to change it if you’re saving JPEGs. Though it isn’t documented anywhere that I’ve found, in my own testing I’ve determined the default GDI+ quality level is set to 75 (on a scale of 0 to 100) in all versions of Windows I’ve checked. That number can be tricky, though, as it doesn’t have any meaning outside the specific encoder you’re using. For example, you’ll often hear/read that 75 is a good quality level to use in Photoshop when saving web graphics. And in fact, it is. 75 in Photoshop, though, has nothing to do with 75 in GDI+ (or the underlying WIC encoder it uses). There are no standards for quality values, and different encoders may use their own quantization tables. In my own experience, I’ve found that anything less than 80 is unacceptable from the Windows encoders and that for smaller images like thumbnails, a value of up to 95 is more appropriate. You can’t see it in my reference resizer implementation because the logic is in the shared ProcessImageSettings class, but if a quality level isn’t explicitly set, I dynamically choose a value between 83 and 95 depending on the output image resolution.

Another fact of note is that GDI+ always uses 4:2:0 chroma subsampling regardless of the quality setting you choose, and it offers no way to change that. Photoshop’s ‘Save for Web’ uses 4:2:0 subsampling only on quality levels of 50 or below and switches to 4:4:4 at quality level 51. This is another reason for the vast difference in image quality at comparable settings values between GDI+ and other encoders. The underlying WIC encoder, by the way, does offer the ability to change JPEG subsampling settings, but GDI+ doesn’t use it (MagicScaler does).

The only other thing I’ll point out here is the using statements on the EncoderParameter and EncoderParameters objects in the reference resizer. These are often overlooked, but as I pointed out before, everything in GDI+ uses system-level handles, so don’t forget to dispose those.

Bonus: Image.GetThumbnailImage()

I don’t use this.

If all you want is a quick way to get a thumbnail version of a larger image, this sounds like it’s just the thing, but don’t be fooled. In the worst case scenario, this method will retrieve a very low quality, low resolution embedded JPEG from the EXIF metadata in your input file. It will probably be ugly. In the best case scenario, it will create the thumbnail on the fly, using the GDI+ defaults with all the quality issues that come with them. I’ll be pointing those out in the next post. If you care at all about the quality of your images, forget this method exists.

Tune in next time, when I’ll be covering the Graphics class and its many mysterious settings…

In future posts, as I discuss the relative merits of MagicScaler, I’ll be doing a lot of comparison to System.Drawing (or GDI+, the underlying API). As far as image processing in ASP.NET goes, it’s been pretty much the only game in town, despite the ubiquitous warnings that it shouldn’t be used in that way. My goal was to make MagicScaler an attractive replacement for image scaling in GDI+, so that’s been my benchmark. The problem is, there’s so much variety across GDI+ resizing implementations that it isn’t necessarily clear what it means when I say that MagicScaler is faster than GDI+ or that its output quality is better.

This series of posts will discuss some of the internals of DrawImage() -- the core of GDI+ image scaling -- and how various resizing implementations can be so different while all using the same method call to do their work. I’ll also discuss the implementation of my reference GDI+ resizer, which is basically a stripped-down version of the resizing component I used for my clients’ websites before I wrote MagicScaler. Given the number of flawed resizer implementations in the wild, the dearth of accurate documentation on the subject, and volume of misinformation out there, I thought it was worth taking some time to document what I’ve learned in my years of working with System.Drawing/GDI+, even as try to convince you to leave it behind. I hope this will serve as a good reference for anyone who may wish to (or need to) stick with a GDI+ implementation of their own.

Where practical, I attempted to make my reference GDI+ resizer feature-compatible with regard to MagicScaler so they can be compared directly in as many scenarios as possible. I also attempted, within reason, to maximize both the performance and image quality of my reference resizer within the limitations of GDI+. I’ll be writing quite a bit about the ways I do that, starting in the next post. And hopefully in the future, as I present all the reasons you should use MagicScaler instead, you’ll be confident that I’m showing GDI+ in the best possible light for my comparisons. My goal was to benchmark MagicScaler against the best GDI+ could do both quality and performance-wise, not an artificially handicapped implementation.

In order to give you a better idea of the implementation differences to which I’m referring, in this post I’ll be comparing my reference resizer to the most popular System.Drawing-based image resizer on nuget: ImageResizer. It actually does do as well as you can do quality-wise with GDI+, but it falls victim to some of the pitfalls I’ll be pointing out in this series and, as a result, has less-than-stellar performance.

Take, if you will, this quite ordinary 1MP JPEG test image. I resized it with both components to test the relative performance.

This graph shows the speed difference between the two resizers at a variety of common output sizes. The tests were performed using the default (highest quality) settings for both components. I changed only the output format, which I set to PNG for more accurate visual comparison. Speed comparisons are worthless if we’re not producing the same output quality, so I checked the outputs of each test using Beyond Compare to make sure they matched. Each test consisted of a warmup and then 10 runs, with the processing times averaged. At 240px output width, ImageResizer is 16% slower. As the output size increases, so does the difference in performance. It clocks in at 62% slower for the largest output image I tested.

And just so we have something other than graphs to look at, here’s a side-by-side visual comparison of the test output at 400px.

The image on the left was created with ImageResizer 3.4.3 and the one on the right was created with my reference GDI+ resizer. They’re visually indistinguishable, but the one on the left took 27% longer (52ms vs 41ms) to produce. It also weighs in 10% larger (225KB vs 204KB) because it was unnecessarily saved as PNG32 instead of PNG24.

Notice I said the images are ‘visually indistinguishable’ and not ‘identical’. Below is the Beyond Compare diff view to illustrate what I mean. I set the tool’s threshold at 1, so identical pixels are shown in greyscale, pixels that differ by 1 (out of 256) are blue, and anything off by more than 1 would be red (there aren’t any).

This goes to show you can’t really trust your eyes when it comes to comparing images. Especially when viewing images side-by-side, we may trick ourselves into thinking we see differences that don’t exist. Or we may miss small differences that do exist. This kind of comparison can also give clues as to how implementations differ.

When two images are this similar but not identical, it’s usually the result of floating point rounding differences somewhere in the processing. It’s clear that ImageResizer is doing some extra processing on the image (that’s why it’s slower), but that processing didn’t make any visual difference in the output. It actually wouldn’t be unfair to say, then, that where there is a difference, the output from the reference resizer is more ‘correct’ since it does the minimal processing necessary to get the desired result. But again, these are differences of 1 in the RGB values, so they’re not visible at all, and I would call the images absolutely equal in quality. Had I encoded to JPEG, these differences could be either amplified or cancelled out by the quantization, which is why I tested with PNG initially.

Some of you might be wondering at this point if the difference in encoder pixel formats between the two test outputs is the main reason for their difference in speed. Actually, it does make a small difference, but it’s only one of several areas of inefficiency. If fact, the two resizers are even further apart performance-wise with JPEG output, where the work done by the encoder is equal. Here I ran the tests again with the default settings except for JPEG quality. Normally, my reference resizer dynamically adjusts the quality setting depending on output resolution, but for this test I manually set it to 90 to match the ImageResizer default.

Because the JPEG encoder is much faster than the PNG encoder, the effect of suboptimal GDI+ settings is even more pronounced here. ImageResizer is 105% slower at the max size I tested. Ouch.

Mind you, ImageResizer isn’t always this much slower compared to a more optimized implementation. But this test was representative of some of the most typical web image resizing scenarios, so it’s really worth the extremely small amount of effort it takes to get it right.

And I guess while I’m picking on ImageResizer, I’ll go ahead and point out another common architectural problem it has that isn’t clear from the tests above. It uses a ‘push’ model rather than the GDI+ default ‘pull’ model for its processing pipeline. Essentially, that means it decodes the entire source image into memory and pushes it through to the next step, whereas my reference resizer pulls only the pixels it actually needs through the decoder. I’ll be documenting that much more fully in another post in this series, but for now, to demonstrate the difference in performance between the two, I’ll crop and scale a section out of the the original 18MP version of the same test image I used above.

This time, the two resizers created visually identical images (no off-by-1 pixels), but ImageResizer took 4.7x as long (448ms vs 96ms) to do the same job. It also used a lot more memory during that time. I’ll have more on that later as well.

With those kinds of inefficiencies, it would be easy to make MagicScaler look good by comparison, but that wouldn’t be very sporting, would it? That’s why I’ll be comparing to my reference GDI+ resizer from now on. And since the types of mistakes that make ImageResizer slow are quite common in GDI+ resizing implementations, hopefully the information in this series can help others avoid them in the future.

If you want to check out its not-so-secret sauce, the code for my reference resizer is available in this gist. If you’re familiar with the ins and outs of GDI+, it should be pretty self-explanatory. If you’re not, I’ll be calling out the high points and explaining them starting in the next post. I’ll also be doing some follow-up posts on some of the more gory details of how GDI+ works. You may find some interesting things in those even if you’re already a System.Drawing/GDI+ expert.

Oh, and one last thing… I really meant my GDI+ resizer to be thrown away once I finished my comparisons with MagicScaler, so I’m just sharing its code as a sample, not as a ready-made component. It’s designed to use one of the shared components (ProcessImageSettings) from MagicScaler so that the two can be used interchangeably in my test/benchmark harness. That settings component handles the math for the dimensions, crop, and JPEG quality, and the resizer is incomplete without it. It doesn’t take much work to replace the missing pieces if you’re so inclined. But by the same token, there’s not much to the resizer itself, so feel free to take any code from it that might be useful, and integrate it into your own projects. Or better yet, use MagicScaler ;)

Next up, a review of some of the features and code details from my GDI+ resizer…