PhotoSauce Blog

0 Comments

*Note: If you’re just here for the profiles, I have published those in a new github repo. Get them all here.

Thanks to some much-needed vacation time, it’s taken me a while to get to this final part of the series, but now it’s time to put everything together and get some profiles finalized. In the first three parts of this series, I examined ways to pack an ICC v2 profile as small as possible, an approach for finding an ideal point-based TRC fit with the minimum size, and how to derive the correct color primaries and whitepoint for an sRGB-compliant profile. In this final part, I will assemble some profiles using those techniques/values and test them out. I had difficulty devising real-world tests that would demonstrate the differences between profiles, but I think I’ve finally nailed down some good approximations that are fair and realistic.

My initial test plan was simply to re-create the worst case scenario for profile conversion. If a profile performs acceptably in the worst case, it should do even better under less extreme circumstances. For this reason, I decided to focus on conversions from sRGB to ProPhoto RGB. The thinking behind this is that an embedded sRGB profile will be used to convert to other colorspaces, and the colorspace that is the most different from sRGB would be the worst case. It would be possible to construct a custom colorspace that would be even more different than ProPhoto, but that wouldn’t be realistic. ProPhoto is a real colorspace that people actually use, and it has both a gamut that is much, much larger than sRGB and a response curve that is quite different (reference gamma 1.8 vs 2.2). An even more common scenario might be something like sRGB to Adobe RGB or Rec. 2020, but again, if a profile does well with ProPhoto, the others will work even better.

The Reference Image

Having settled on an evaluation strategy, I needed to pick some test images. This turned out to be more difficult than I anticipated. I originally selected a few real-world images that had extremely saturated colors and a few with lots of different shades of blue and green. These are areas where ProPhoto and sRGB would have maximum differences, and that should highlight any errors. Unfortunately, I found it was impossible to compare fairly with real-world images for two main reasons:

  1. No real-world image covers the entire color gamut of sRGB, so an error might not show up simply because the color value that would show the error isn’t present in the image.
  2. Real-world images tend to have areas of repeated pixel values. This means that if one profile causes a specific color to have an error, and if that color is over-represented in the image, it amplifies the error measured from the profile.

For those reasons, I settled on testing with a single reference image. That image comes from Bruce Lindbloom’s site and is a 16.7megapixel generated image that simply contains every color combination possible with 8-bit RGB. The image consists of 256 squares, each with a different blue value. And each of those squares consists of 256 rows and 256 columns, where the red value increases in each column and the green value increases in each row. I found this image makes it easy to see exactly where the errors are focused.

The Reference Profile

The second problem I had was establishing a reference to compare to. In testing my tone reproduction curves, I tested each candidate curve against the true sRGB inverse gamma curve. For the final profile testing, however, I wanted to test real images with real profiles using a real CMS. So I needed a real ICC profile to serve as a reference. Unfortunately, as we discovered in Part 3 of this series, there aren’t any profiles I could find anywhere that are truly sRGB-compliant. Nor could I use the standard 1024-point TRC as a reference, because one thing I want to evaluate is whether the 182- and 212-point curves I found in Part 2 might actually be better than the 1024-point curve used in most profiles.

This series is focused on creating v2 ICC profiles, but v4 profiles have a newer feature that allows the TRC to be defined as a parametric curve rather than a point-based curve with linear interpolation. The parametric curve type allows the sRGB gamma function to be duplicated rather than approximated. Software support for v4 profiles is not great, so they aren’t used frequently, but a v4 profile with a parametric curve would serve as a good reference for testing my v2 profiles. That left me with a new problem, which was to find an optimal v4 profile.

Although the parametric curve type can duplicate the sRGB curve’s basic logic, the parameters themselves are defined in the ICC s15Fixed16Number format, meaning they have limited precision. I decided to evaluate the accuracy of a v4 sRGB curve using the same measures I used to evaluate my point-based curves in order to see how close it was to the true sRGB curve. Once again, I started with an example from Elle’s profile collection.

Here are the stats from that profile’s TRC compared with the best-performing point-based curves from Part 2.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
   182 |  0.001022 |   0.000092 |  0.000230 |   0.003107 |    0.000440 |   0.000736 | 0
   212 |  0.001650 |   0.000118 |  0.000357 |   0.002817 |    0.000449 |   0.000707 | 0
  1024 |  0.008405 |   0.000205 |  0.000996 |   0.003993 |    0.000475 |   0.000819 | 0
  4096 |  0.008405 |   0.000175 |  0.000860 |   0.003054 |    0.000472 |   0.000782 | 0
    v4 |  0.000177 |   0.000034 |  0.000051 |   0.000564 |    0.000317 |   0.000371 | 0

As you can see, the v4 parametric curve results in significantly less error than even the best point-based options. Its error, however, is still surprisingly high. Let’s take a look at the parameter values from that profile and see why that is.

Param | sRGB Value     | sRGB Decimal   | Profile Hex | Profile Decimal | Diff
    g | 2.4            | 2.4            |  0x00026666 |  2.399993896484 | -6.103516e-6
    a | 1.000/1.055    | 0.947867298578 |  0x0000f2a7 |  0.947860717773 | -6.580805e-6
    b | 0.055/1.055    | 0.052132701422 |  0x00000d59 |  0.052139282227 |  6.580805e-6
    c | 1.000/12.92    | 0.077399380805 |  0x000013d0 |  0.077392578125 | -6.802680e-6
    d | 0.04045        | 0.04045        |  0x00000a5b |  0.040451049805 |  1.049805e-6

Once quantized to s15Fixed16Number format, none of the numbers stored in the profile are exactly correct, and two of the parameters that have the largest impact on the output value (g and a) are both rounded down.  Rounding both values in the same direction effectively combines their error. I decided to try ‘nudging’ all the parameter values to try to find a better fit than was produced by simple rounding. It turned out, the best fit I was able to achieve was by bumping the ‘g’ value up and leaving all the rest as they were. By using a ‘g’ value of 0x00026669, or 2.400039672852, I was able to cut the error to less than half that of the rounded values.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
    v4 |  0.000177 |   0.000034 |  0.000051 |   0.000564 |    0.000317 |   0.000371 | 0
   ^v4 |  0.000088 |   0.000012 |  0.000022 |   0.000240 |    0.000124 |   0.000143 | 0

While it’s not perfect, that is as close as it’s possible to get to the true sRGB inverse gamma function in an ICC profile. So with that and the primary colorant values from Part 3, I had my reference profile. I decided while I was making a reference profile, I may as well make it as small as I could so that I would have another compact profile option for embedding. That profile is here.

I also decided to create a reference v4 ICC profile for ProPhoto to use as my destination profile. That one was much simpler in that the default rounded values worked out to be the closest fit for the TRC, and the colorant values have a single, unambiguous definition.  That profile is here.

The Reference CMS(s)

Once again, this turned out to be more complicated than I anticipated.  One might expect that with the detail in the ICC specifications, there wouldn’t be much difference between CMS implementations. I’ve worked predominately with the Windows Color System (WCS) by way of the Windows Imaging Component (WIC), and I always assumed it did a reasonable job at color conversions. However, when looking for an easy way to test conversions using multiple profiles, I stumbled on the tifficc command-line utility from Little CMS.

In testing with tifficc, I found that the results were mostly in line with my expectations, but when using my candidate profiles as a target rather than a source, it appeared Little CMS was doing an upgrade or substitution of the specified profile to an internal reference version of the colorspace. That’s definitely a desirable behavior in that it ensures correct output regardless of minor profile differences, but it’s not desirable when trying to measure those minor differences. WCS, on the other hand, produced output different for each profile, and entirely different from Little CMS. And while that output was more in line with my expectations from my previous testing, it seems that it might not be as correct.

I had been planning for some time to replace some of my dependencies on WCS with my own color management implementation, but this has provided the final push I needed to motivate me to get it done. In the end, I decided to consider the output from both CMSs, so it would be easier to predict what might happen when using the profiles in different scenarios and with different software.

The Reference Scenario

This is where things finally get easy. The purpose of a compact profile is to be embedded in an image. Obviously, it would only be embedded in an image of a matching colorspace and would only be used as a source profile in those cases. I had already chosen ProPhoto as a destination colorspace because of its extreme difference from sRGB while still being a realistic conversion path. And having already decided to use a v4 ProPhoto profile as a reference destination, that left only one choice to make. I had already decided that I would test with an 8-bit reference input image because that’s the most common image type in the wild. But wide-gamut colorspaces like ProPhoto are not well-suited for use with 8-bit images. Squishing the sRGB gamut down into its corresponding place in the ProPhoto gamut at 8-bit resolution tends to cause posterization. So I decided to test 8-bit sRGB input and 16-bit ProPhoto output. I was also able to test the reverse of that transform, going from the 16-bit ProPhoto images back to 8-bit sRGB. In the interest of time, I won’t document the full details of those tests, but those tests are the ones that led to my conclusion that Little CMS does some kind of profile substitution and that WCS is probably Not Very Good. I may do another post on that at some point in the future.

For the Little CMS trials, I used a command-line similar to the following:

tifficc -v -t1 -isrgb-v4-ref.icc -oprophoto-v4-ref.icc -e -w rgb16million.tif rgb16milpp-ref.tif

For the WCS trials, I wrote a small utility that uses the WIC IWICColorTransform interface to perform the same conversion.

The Measurements

Having established a reference scenario and created a reference profile, all I had to do was run the conversion(s) in question using the reference profile as well as all the v2 candidates and then compare their output. I also figured it would be worthwhile to try some variants using more common values, like the 1024- and 4096-point TRCs and the ArgyllCMS and HP colorant values. That should allow a complete picture of how the candidate profiles perform as well as a good basis for understanding which parts of the profile contribute to greater differences in output.

Measuring the differences between the profile output mathematically is a simple task, but I wanted to be able to visualize those differences for easier comparison and so that it would be possible to see not only how much difference there was, but also where the differences occurred. I considered using ΔE-CIE2000 for these comparisons, but the reality is, the results are so close visually that there isn’t much meaning to the color difference. I also found the results of the raw differences interesting because of the way the visualization shows patterns in the error.

I’ve referenced the Beyond Compare image comparison tool a few times before because I like the way it works and the way it shows differences. The only problem with using it for these tests is that while it does load 16-bit images, it seems to convert them to 8-bit before doing the comparison. That means I couldn’t get the kind of detail I wanted to see in the diffs. Normally, when I use that tool, I set it up with a threshold of 1, meaning its default visualization will show pixels that are equal between two images in greyscale, pixels that are off by 1/255 in blue, and pixels that are off by more than 1/255 in red. In doing some trials with 8-bit output and comparing them in Beyond Compare, I found that none of the profiles in my test suite created output that differed from the reference by more than 1 on any given pixel. That’s good news, in that it backs up the theory that none of the profiles I’m testing will produce output that is significantly different visually. But it would make it difficult to draw any conclusions about which profiles are better, especially when the differences get more subtle. That issue, combined with the fact that ProPhoto isn’t recommended for 8-bit images anyway, led me to create my own variant of the image comparison tool that worked at higher bit-depth.

The second problem was visualizing the differences. As I said, I like the way Beyond Compare does it, but when you compare 16-bit images, it’s difficult to find a threshold for color-coding the differences. I ended up with something like the original but enhanced for the extra sample resolution. Instead of coloring different pixels either solid blue or solid red depending on the threshold, I created a gradient from blue to red. I chose the threshold rather arbitrarily, setting it at 65/65535, or roughly 0.1%. That threshold worked out well in that allowed me to create a gradient from blue to red for differences between 1 and 65, and then differences over 65 could be colored solid red. Note that the solid red doesn’t necessarily mean a difference would be distinguishable visually. And as you’ll see, differences that great were very rare in the tests anyway.

And finally, I added some stats to the diff images to provide a little more detail than can be seen visually. I grouped the errors into four buckets (1-17,18-33,34-49,50-65) and added raw pixel counts for each error bucket, plus the count of pixels over the 65 threshold. I also calculated the Max, Mean, and Root Mean Square error for each test image versus the reference. Those stats are written into the upper-left corner of each diff image.

The Results

From here on out, there will be a lot of images. These are the diff images created by the tool I described above. Again, grey pixels in the image indicate that the candidate profile produced output identical to the reference profile. Pixels that are tinted blue represent the smallest differences, and there is a gradient from blue to purple to red (more pink, really) for increasing error levels. Finally, any pixels colored solid red were different by more than 65/65535. All the images below are thumbnails, and you can click them to get the full diff image. Be aware, though, the diff images are 16.7megapixels in size, so don’t click them if you’re on a device that can’t handle the size (bytes or pixels). Oh, and the diff images themselves are 8-bit, even though they represent differences between 16-bit images. Since the diff is just a visualization, I wanted to keep them as small as possible. They’re already as much as 10MiB each saved as 8-bit-per-channel PNGs.

For each profile, I’ll include the results from both the LCMS tifficc utility and my WCS/WIC conversion utility. The differences are interesting to see.

The Colorant Factor

I’ll start with the effect of different primary colorant values used in common sRGB profiles. In Part 3 of this series, I looked at the differences between the odd colorant values from the HP/Microsoft sRGB profile as well as the Rec. 709-derived colorants used by the ArgyllCMS reference sRGB profile. For these tests, I created v4 ICC profiles using the same modified parametric curve from my reference profile, so that only the colorants are different. Converting the reference image to ProPhoto using those as source profiles, these are the diffs compared with output from my reference sRGB profile.

diff-16milpp-hp
CMSLCMSDiff Counts
ColorsHP/MS1-1716.5M
TRCv4 Ref18-330
Max Diff1634-490
Mean Diff5.464350-650
RMS Diff5.8626>650

diff-16milpp-argyll
CMSLCMSDiff Counts
ColorsRec. 7091-1713.6M
TRCv4 Ref18-330
Max Diff1634-490
Mean Diff1.741250-650
RMS Diff2.2036>650

And the same using WCS

diff-wcs-16milpp-hp
CMSWCSDiff Counts
ColorsHP/MS1-1712.8M
TRCv4 Ref18-33950745
Max Diff6334-4917799
Mean Diff6.357050-651955
RMS Diff8.4811>650

diff-wcs-16milpp-argyll
CMSWCSDiff Counts
ColorsRec. 7091-178.6M
TRCv4 Ref18-33928751
Max Diff6334-4917484
Mean Diff4.712950-651955
RMS Diff7.5958>650


The thing that really stands out to me is the difference in the way these profiles are handled by LCMS and WCS. Based on the splotchiness of the WCS diff images (you’ll have to view them full-size to see), my guess is that it’s using lower-precision calculations than LCMS. In both cases, though, the differences are quite small and should be below the threshold of visible difference. That’s certainly the case with the Rec. 709 colors vs the sRGB reference colors, but the unbalanced colors from the HP/Microsoft profile don’t result in as much difference in the converted result as one might expect. I think the differences here also make a good reference point for determining the significance of the differences caused by different TRC approximations.

The 26-Point Curves

In Part 2 of this series, I did some detailed analysis of both the TinyRGB/c2 26-point approximated TRC and the proposed ‘improved’ curve used in the sRGBz profile. That analysis predicted that the sRGBz curve would perform less well than the TinyRGB curve, and it found another alternate 26-point curve that it predicted would do better. I figured some real-world testing of those predictions would be a good start. Although I measured and tuned the curves primarily using ΔL, which is a measure of visual difference, we can see that the results are the same even when measuring absolute pixel differences after conversion.

Note that in testing these curves, I created new profiles that all shared the same reference sRGB colorant values to limit any differences to the curves themselves.

It’s difficult to see in the thumbnails, but at full size, the visualization shows pronounced improvement in error levels between my alternate 26-point curve and either of the others. The sRGBz curve has both the largest mean error and the most individual pixels with high error levels.

diff-16milpp-26z
CMSLCMSDiff Counts
ColorssRGB Ref1-1713.6M
TRCsRGBz18-332.9M
Max Diff6734-49243867
Mean Diff13.647150-6517624
RMS Diff15.8336>6559

diff-16milpp-26c2
CMSLCMSDiff Counts
ColorssRGB Ref1-1714.6M
TRCTinyRGB/c218-332M
Max Diff7234-49143994
Mean Diff13.553350-6513082
RMS Diff15.1258>651

diff-16milpp-26
CMSLCMSDiff Counts
ColorssRGB Ref1-1714.7M
TRC26-Point Alt18-332M
Max Diff6434-49111588
Mean Diff13.406950-653636
RMS Diff14.8904>650

And again with WCS

diff-wcs-16milpp-26z
CMSWCSDiff Counts
ColorssRGB Ref1-1711.9M
TRCsRGBz18-334.4M
Max Diff6934-49487042
Mean Diff15.661150-6551775
RMS Diff18.7551>65335

diff-wcs-16milpp-26c2
CMSWCSDiff Counts
ColorssRGB Ref1-1713.1M
TRCTinyRGB/c218-333.4M
Max Diff6234-49269412
Mean Diff13.874650-6513434
RMS Diff16.2264>650

diff-wcs-16milpp-26
CMSWCSDiff Counts
ColorssRGB Ref1-1713.2M
TRC26-Point Alt18-333.3M
Max Diff5434-49275720
Mean Diff13.911250-655374
RMS Diff16.2414>650


Once again, the output from WCS seems to have amplified the error in the profiles, but the relative results are the same. The sRGBz curve is less accurate than TinyRGB’s, which is less accurate than my alternate 26-point curve. It’s also worth noting how much more error these curves contribute compared to the error from the alternate primary colorants. This level of error is still quite acceptable for the profiles’ primary intended use-case, but we’ll look at some other options.

The Alternate Compact Curves

I picked out a few of the interesting compact curves that my solver found in Part 2 of this series to see how they compare in terms of size/accuracy ratio. Here are those comparisons, again using both CMS’s. First LCMS…

diff-16milpp-20b
CMSLCMSDiff Counts
ColorssRGB Ref1-178.0M
TRC20-Point18-337.4M
Max Diff8234-491.1M
Mean Diff23.141450-65212433
RMS Diff25.6127>6518122

diff-16milpp-32
CMSLCMSDiff Counts
ColorssRGB Ref1-1716.5M
TRC32-Point18-33294858
Max Diff4834-4912
Mean Diff8.749550-650
RMS Diff9.6480>650

diff-16milpp-42
CMSLCMSDiff Counts
ColorssRGB Ref1-1716.7M
TRC42-Point18-335485
Max Diff3234-490
Mean Diff5.139850-650
RMS Diff5.7177>650

diff-16milpp-63
CMSLCMSDiff Counts
ColorssRGB Ref1-1716.1M
TRC63-Point18-3312
Max Diff3234-490
Mean Diff2.459250-650
RMS Diff2.8069>650

And once more using WCS

diff-wcs-16milpp-20b
CMSWCSDiff Counts
ColorssRGB Ref1-178.1M
TRC20-Point18-336.9M
Max Diff8234-491.5M
Mean Diff23.043950-65227729
RMS Diff26.0615>6534926

diff-wcs-16milpp-32
CMSWCSDiff Counts
ColorssRGB Ref1-1715.6M
TRC32-Point18-331.1M
Max Diff3834-491162
Mean Diff9.666250-650
RMS Diff11.4510>650

diff-wcs-16milpp-42
CMSWCSDiff Counts
ColorssRGB Ref1-1716.6M
TRC42-Point18-33152587
Max Diff2534-490
Mean Diff7.396250-650
RMS Diff8.4588>650

diff-wcs-16milpp-63
CMSWCSDiff Counts
ColorssRGB Ref1-1716.7M
TRC63-Point18-33187
Max Diff2334-490
Mean Diff6.725050-650
RMS Diff7.3211>650


And now a few notes on these interesting curves…

Although the 20-point curve diff images look like a bit of a bloodbath, allow me to point out a couple of things. First, as I mentioned before, I chose the threshold for the full red pixels rather arbitrarily. I wanted the small differences between all the profile variants to be visible even in the thumbnails here, and I chose my thresholds based on my choice of a 64-value gradient for the smaller errors. Red doesn’t necessarily mean danger in this case; it just means the error is higher than the worst from the other curves. Not a lot worse, mind you, but the line has to be drawn somewhere, and I just happen to have drawn that line just under the max error of the 20-point curve. Second, you’ll note that most of the worst error is concentrated toward the upper left of the image and the upper left of each square within the image. These are the darker parts of the image, where a larger absolute pixel difference represents a smaller visual difference than it would at the mid-tones. The choice to concentrate the error in those areas less visible was a key part of the tuning algorithm used in my curve solver. I believe the 20-point curve is perfectly adequate for some 8-bit image embedding, particularly for thumbnail-sized images where file size is important. Weighing in at only 410 bytes, I believe this is the smallest possible usable sRGB-compatible profile.

The other three candidate curves performed very well indeed. The 32-point curve is quite a significant improvement over the 26-point curves used in the existing compact profiles and with a cost of only 12 additional bytes in the profile. So once again, I’ll say that 26 is not a magic number in this case. But really, if you’re looking for a magic number, wouldn’t you just skip straight to 42? The error level in the 42-point curve is quite good. It’s actually awfully close to the error caused by the bad colorant values used in the very popular HP/Microsoft sRGB profile, so it makes an excellent compromise if you’re looking to save space. The 63-point curve halved the error of the 42-point curve when using LCMS but didn’t do as much better with WCS, so while it would also be a good choice, I think 42 is the magic number for my compact profile.

The Big Curves

That just leaves us with the larger curves to evaluate. My solver identified curves of 182 and 212 points that appeared to be a closer fit to true sRGB than the standard 1024- and 4096-point curves used in many profiles. I wanted to see if that was true in a real-world test. Here are the results when using all four of those.

diff-16milpp-182
CMSLCMSDiff Counts
ColorssRGB Ref1-177.6M
TRC182-Point18-330
Max Diff1634-490
Mean Diff0.743650-650
RMS Diff1.2585>650

diff-16milpp-212b
CMSLCMSDiff Counts
ColorssRGB Ref1-178.2M
TRC212-Point18-330
Max Diff1634-490
Mean Diff0.773650-650
RMS Diff1.2533>650

diff-16milpp-1024
CMSLCMSDiff Counts
ColorssRGB Ref1-178.4M
TRC1024-Point18-330
Max Diff1634-490
Mean Diff0.787950-650
RMS Diff1.2471>650

diff-16milpp-4096
CMSLCMSDiff Counts
ColorssRGB Ref1-172.8M
TRC4096-Point18-330
Max Diff1634-490
Mean Diff0.216450-650
RMS Diff0.5570>650

And repeated one last time using WCS

diff-wcs-16milpp-182
CMSWCSDiff Counts
ColorssRGB Ref1-1716.6M
TRC182-Point18-331446
Max Diff2334-490
Mean Diff5.784950-650
RMS Diff6.2883>650

diff-wcs-16milpp-212b
CMSWCSDiff Counts
ColorssRGB Ref1-1716.6M
TRC212-Point18-3312261
Max Diff2334-490
Mean Diff5.651250-650
RMS Diff6.1638>650

diff-wcs-16milpp-1024
CMSWCSDiff Counts
ColorssRGB Ref1-1716.6M
TRC1024-Point18-3374
Max Diff1734-490
Mean Diff5.663450-650
RMS Diff6.1857>650

diff-wcs-16milpp-4096
CMSWCSDiff Counts
ColorssRGB Ref1-1716.6M
TRC4096-Point18-33144
Max Diff1734-490
Mean Diff5.600650-650
RMS Diff6.1383>650


I must admit, these results have left me a bit puzzled. With LCMS, the 182- and 212-point curves generally outperformed the 1024-point curve, as I expected. But the 4096-point curve really blew the others away. In doing some round-trip testing early on with LCMS, I found that it was producing perfect output with the 1024- and 4096-point TRCs when it really shouldn’t have, so I suspected there may be some sort of internal substitution happening with those larger TRCs. I tested that theory out by modifying the first 20 points of the 1024-point profile to use the same large value. The output didn’t change, which lends some support to that theory. I didn’t dig into the LCMS code to see what’s happening, but I can say that the same substitution/upgrade did not occur when using my 182- or 212-point TRCs. So if you’re using LCMS and want the most accuracy (for destination profiles at least), you may be better off sticking with the standard TRCs. When used as a source profile, however, there is something special about those smaller curves. I think they’ll work nicely for embedded profiles when size is a factor.

The results when using WCS to convert were a bit more in line with my expectations. The 4096-point curve had more pixels with larger individual errors compared to the 1024-point curve, but it made up some of the gap in the average error by better fitting the top part of the curve. The 182- and 212-point TRCs performed admirably but didn’t offer an upgrade over the larger versions. Again, they have almost the same accuracy as the standard curves, so if size is a concern, they’re a viable option. I’ll go ahead and publish a profile that uses the 212-point curve because I think it has some value, but it’s not quite the upgrade over the standard curves I thought it might be.

The CMS Factor

I found the differences between the results with the two CMS’s interesting enough to do all the tests in both, but I wanted to show one last comparison to lend a bit of perspective to all the other comparisons/diffs in this post. Here’s what it looks like when you compare the reference outputs from each CMS with each other.

diff-16milpp-lcms-wcs
CMSLCMS/WCSDiff Counts
ColorssRGB Ref1-1711.1M
TRCv4 Ref18-332.9M
Max Diff40434-491.0M
Mean Diff22.793650-65563030
RMS Diff35.1353>651.1M


Now that’s a bloodbath. And that serves to make an important point: the CMS implementation details can easily have a far greater impact on the output results than any of the profile aspects we’ve looked at. I’m not quite sure which is more accurate between LCMS and WCS, but I strongly suspect it’s the former. I’ll do some more testing on that as I do my own color conversion implementations in MagicScaler. If I find anything interesting and if I remember, I’ll come back and update this post.

2 Comments

When I started the task of creating a minimal sRGB profile, I assumed the part that would require the least thought would be the colorant and whitepoint tags in the profile. To review, the ICC V2 specification requires 9 tags for RGB profiles. Those are: copyright (cprt) and description (desc), which I discussed in Part 1 of this series; the tone reproduction curves (rTRC, gTRC, and bTRC), which I covered thoroughly in Part 2; and finally, the colorant and whitepoint tags (rXYZ, gXYZ, bXYZ, and wtpt), which ended up with their own post, too. This is that post.

I have referenced Elle Stone’s treatise on well-behaved profiles a couple of times already, and I’ll start this post by referring there again. I’ll also refer you to her systematic examination of ICC profiles seen in the wild and finally, to her very detailed explanation of how to create an sRGB-compatible ICC profile using the values from the sRGB spec.

I drew two main conclusions from reading through those articles. The first was that well-behaved and correct sRGB profiles are difficult to create and are, consequently, rare. The second was that the reference sRGB profile shipped with ArgyllCMS happens to be that most mythical of profiles. The unicorn profile, if you will.

And that’s what was supposed to make this step simple; I’d just steal the colorant tag values from the ArgyllCMS profile and call it a day.

Facebook’s TinyRGB profile had used the colorant tag values from the original HP/Microsoft sRGB profile, and while that remains the most commonly-seen sRGB profile in the wild, Elle had convinced me it was also one of the most wrong.

In his writeup on the sRGBz profile, Øyvind Kolås (Pippin) mentioned generating new colorant tag values from babl with improved accuracy. My first assumption was that it must also use those magical Argyll values. Imagine my surprise when I looked at the tag data and saw that they were a completely different set of values. And imagine my further surprise when I ran them through Elle’s xicclu test and found that they were also well-behaved.

That left me with two sets of possible correct sRGB colorant values, and I simply had to know which was right. The rabbit-hole deepens…

Better-Behaved?

If there are two well-behaved sRGB-like profiles with different colorant values, they must not be as difficult to come by as I originally thought. Elle describes creating well-behaved profiles as a process of calculating the most correct values for the colorspace and then ‘nudging’ them so that their rounding errors when converted to the ICC s15Fixed16Number format balance out. I learned that this is actually a very simple process and that testing for well-behavedness is a matter of simple arithmetic. Let’s start with the actual colorant tag values stored in the three profiles I was examining.

      |     HP/Microsoft     |        sRGBz         |      ArgyllCMS     
-------------------------------------------------------------------------
      |     X      Y      Z  |     X      Y      Z  |     X      Y      Z
Red   |  6FA2   38F5   0390  |  6FA1   38F6   0391  |  6FA0   38F5   0390
Green |  6299   B785   18DA  |  6297   B787   18DA  |  6297   B787   18D9
Blue  |  24A0   0F84   B6CF  |  249E   0F83   B6C2  |  249F   0F84   B6C4
Sum   |  F6DB   FFFE   D339  |  F6D6  10000   D32D  |  F6D6  10000   D32D

I’ve kept the values in hex for now, because 1) I read them out with a hex editor and 2) it’s easier to see what’s up when looking at the integer representation of the numbers. What the table above shows is the XYZ values for the Red, Green, and Blue primaries stored in each of the three profiles I examined. I also included a row that shows the sum of the X, Y, and Z values for the three color channels. One thing should stand out immediately: the sRGBz and ArgyllCMS color values are different, but their sums are the same. What’s not obvious from the table is what those sums represent.

First, let me explain how the color values are stored in an ICC profile. The ICC spec defines the s15Fixed16Number format for storing XYZ (and other) values. In that format, 16 bits are allocated to the signed integer portion of the number and 16 bits are allocated to the fractional part of the number. The conversion between floating-point decimal and the fixed-point format is simply to multiply by 216 and round to the nearest integer. Conversion back to floating-point decimal is done by dividing by 216 (65536). In that format, 1.0 is represented by 0x00010000 because its integer part is 1 and it has no fractional part.

This is in contrast to the response16Number format used to store the TRC points we examined in the last post. In that format, 16 bits are used to represent the full range of 0-1, inclusive. For that number format, the divisor is 216-1 (65535), so that 1.0 is represented by 0xFFFF. This has come up as a point of confusion in some things I’ve read, so I thought I’d clear that up.

Now back to the values in the table…

Every developer who works with RGB colors knows that in 8-bit color, black is [0,0,0], full red is [255,0,0], etc. We also know that if you add full red, full green, and full blue together, you get [255,255,255], which is white. Things work the same in XYZ. Adding the three primary colors together at their full intensity (as defined within the colorspace) will give you white (also as defined in the colorspace). In most practical colorspaces, the XYZ values are normalized so that white has a Y value of 1.0. In some representations, you may see the XYZ numbers scaled up to a range of 0-100, but both the ICC and sRGB specs declare that the nominal range is 0-1.

Knowing that white should have a Y value of 1.0 and that 1.0 in s15Fixed16Number format is 0x10000, you should see an immediate problem with the values in the HP/Microsoft sRGB profile: their Y values sum to less than 1.0, meaning they’re scaled improperly.

Its X and Z values are wrong as well. The ICC V2 spec requires that all profiles have their color values adapted to the D50 whitepoint, which allows for simple translation between colorspaces. Since XYZ conversion to and from R’G’B’ is whitepoint-dependent, using a common whitepoint for all profiles makes it easy for software to implement that translation. As an aside, let me point out that the prime (‘) symbols here indicate that we are referring to linear RGB values. That is, the red, green, and blue values that are the output of the TRC that undoes their stored gamma correction. Those values are also normalized to the range 0-1 for computation.

The ICC spec is explicit about the XYZ value to use for the D50 whitepoint, giving it a value of [X=0.9642,Y=1,Z=0.8249]. When converted to s15Fixed16Number format, that value becomes [X=0xF6D6,Y=0x10000,Z=0xD32D], which is the exact value stored as the Profile Illuminant in the header of every V2 ICC profile.

If you refer back to the table of values from the three profiles I examined, you will find that the sRGBz and ArgyllCMS primary colorant values sum to exactly the value of the D50 Illuminant given by the ICC. Put simply, that’s what makes a profile well-behaved. And you can see that the HP/Microsoft profile is quite far off from that value, which is why it’s no good.

Knowing that making a profile well-behaved is simply a matter of normalizing the primary colors so that they sum at full intensity to make white, it’s easy to see how we ended up with two different sRGB-like profiles that are both well-behaved. But that still leaves the question of which is right.

In Which I Learn They’re Both Wrong

It’s been a while since I linked to Nine Degrees Below, so let me start this section with another link to Elle’s research into the proper color values to use for sRGB. In that article, Elle rounds up every possible definition of every color referenced in the sRGB spec, does every bit of mathematical wrangling imaginable, and comes up with a final set of numbers that are very close to the ones that ArgyllCMS has in its reference sRGB profile. Score one for Argyll.

I decided to do a similar exercise and find my own answer to compare to the others. One problem with making things match the sRGB spec is that the spec itself isn’t published freely. If you want to read the actual spec, it will cost you 170 Swiss francs to buy it from the IEC web store.

Fortunately, there are enough references available elsewhere that I believe we can put together an accurate picture of the spec without ponying up. I like to save my francs for Swiss chocolate and wine, thank you very much. I can enjoy those while I read the Wikipedia entry on sRGB.

At this point, it’s worth considering exactly what numbers we’re trying calculate. Depending where you look you may see the colorant values described in one of two ways:

  1. They represent the direct XYZ translation of the three primary colors (red, green, and blue) at their full intensity under the specified illuminant.
  2. They make up a matrix which can be used to translate any set of [R’,G’,B’] values to their corresponding [X,Y,Z] values.

In reality, both are correct. But I think for our purposes, it’s important to focus on the second definition. The reason that’s important is that the colorant values stored in the ICC profile are used in exactly that way. They’re also used in another, related way. The matrix created from those XYZ colorant values can be inverted to create the matrix that translates from XYZ back to R’G’B’. This will become very important later, so I wanted to mention it now and let it soak in a bit.

If you use a tool like the Profile Inspector available on the ICC site, it reinforces the first definition I gave. They show the XYZ values converted to decimal, the calculated x and y coordinates for that color, and a nice chromaticity diagram with the color plotted on it. Here’s the rXYZ tag information from the Argyll sRGB profile.

profileinspectorss


Exiftool, on the other hand, presents the data using the second definition. Here’s the relevant output from the same profile:

Red Matrix Column:   0.43604 0.22249 0.01392
Green Matrix Column: 0.38512 0.71690 0.09706
Blue Matrix Column:  0.14305 0.06061 0.71393

You can see that the XYZ values are the same in each, and that makes perfect sense. If you make a matrix using the columns given by Exiftool

0.43604 0.38512 0.14305
0.22249 0.71690 0.06061
0.01392 0.09706 0.71393

and then multiply the linear value for pure red [1,0,0] by that matrix, you get the left column back, giving you the same XYZ value shown for the red primary in the Profile Inspector. Again, it’s the same thing… but the matrix usage is the more important definition.

So, that leaves the question: how do we calculate that matrix, given the values in the sRGB spec?

That answer has two parts. For the first part, we must get the matrix for converting R’G’B’ to XYZ using sRGB’s native illuminant/whitepoint, which is D65. Then, because the colors in an ICC profile must be given relative to the D50 illuminant, we must adapt that matrix from D65-relative values to D50-relative values. Bruce Lindbloom has a reference on the basic theory of Chromatic Adaptation as well as some different adaptation matrices on his site. But I’ll caution you not to use his pre-calculated matrix for D65->D50; it’s wrong for our purposes.

Let’s start with the matrix for calculating R’G’B’->XYZ under D65. The Wikipedia article on sRGB has the matrix printed right in it. It also points out explicitly that the values listed are the exact ones in the sRGB spec. Again, I don’t have the actual spec, but I have no reason to doubt the veracity of that statement. The sRGB spec reportedly is full of exact numbers, rounded to 4 decimal places.

Well, like Elle, I started with the assumption that more precision is better, and I didn’t like the look of those imprecise numbers from the spec. So, like Elle, I tried to calculate my own more precise matrix using the the published x,y values of the primaries. That effort was a complete failure. What I mean is, while I succeeded in the calculation, if I then inverted my matrix to create the XYZ->R’G’B’ matrix, it didn’t match the one in the spec to the 4 decimal places it has listed. It turns out, the best way to get the correct (according to the spec) inverse matrix is to use the explicitly rounded values given in the R’G’B->XYZ matrix.

0.4124  0.3576  0.1805
0.2126  0.7152  0.0722
0.0193  0.1192  0.9505

Invert that, and you get

 3.2406254773200500 -1.5372079722103200 -0.4986285986982480
-0.9689307147293190  1.8757560608852400  0.0415175238429540
 0.0557101204455106 -0.2040210505984870  1.0569959422543900

Which matches the spec (as described by wiki) very nicely, and with lots of decimal places, if that’s your thing. Round that back to 4 decimal places, like such, to match the spec again,

 3.2406 -1.5372 -0.4986
-0.9689  1.8758  0.0415
 0.0557 -0.2040  1.0570

Invert that, and you get

0.4124103360777000  0.3575962178119540  0.1804991017305060
0.2126157251481260  0.7151958779229800  0.0722134074030765
0.0193021307575116  0.1191881265507680  0.9504992763896300

Which, again, rounds to match the spec at 4 decimal places. These, I’m confident, are the correct numbers for sRGB at D65, which just leaves the D65->D50 adaptation matrix to work out.

This, again, seems like a place where more precision would pay off, but in fact, there is a value listed in the spec, rounded to 4 decimal places, that is perfect for this use. That value given for D65 is [X=0.9505,Y=1,Z=1.0890]. We also have a standard XYZ value for D50, given by the ICC spec. That value is [X=0.9642,Y=1,Z=0.8249].

As an aside, you may recall that the actual D50 value stored in the profile header is in s15Fixed16Number format and is, in hex [X=0xF6D6,Y=0x10000,Z=0xD32D]. Converted back to floating-point decimal, that value is [X=0.964202880859375,Y=1,Z=0.8249053955078125]. If you want to be extra precise, that value is also acceptable to use. It worked out that when creating the adaptation matrices, it didn’t matter which number I used. I got the same results once the quantization to s15Fixed16Number format was done for the final calculated values. For the calculations shown below, I used the rounded value published in the ICC spec.

Using the D50 XYZ value from the ICC spec, the D65 XYZ value from the sRGB spec and the Bradford cone response matrix given by Bruce Lindbloom, we get the following values for the D65->D50 adaptation matrix:

 1.0478414713468100  0.0228955556744975  -0.0502009864000404
 0.0295477450604968  0.9905065286192130  -0.0170722316797199
-0.0092509594572860  0.0150723678359253   0.7517177861599870

Notice that the matrix I calculated is different than the one Bruce gives on his site:

 1.0478112  0.0228866 -0.0501270
 0.0295424  0.9904844 -0.0170491
-0.0092345  0.0150436  0.7521316

They’re quite close, but he used D65 and D50 values from a different source. It comes down to rounding differences, but remember, we’re following exact specs, so we want to round the same way they do.

And finally, if we multiply our R’G’B’->XYZ matrix by the adaptation matrix, we get the final adapted values:

0.4360285388823030  0.3850990539931360  0.1430724071245600
0.2224376839759750  0.7169415328858720  0.0606207831381531
0.0138974429946207  0.0970763744845987  0.7139261825207810

Converted to s15Fixed16Number format, written in hex, and transposed to match the profile layout I used earlier, those numbers look like this:

      |     X      Y      Z
Red   |  6FA0   38F2   038F
Green |  6296   B789   18DA
Blue  |  24A0   0F85   B6C4
Sum   |  F6D6  10000   D32D

You’ll note that the hex values exactly total those of the D50 Profile Illuminant, so these values will create a well-behaved profile. These values, however, do not match any other profile I’ve seen.

I also found that the D65 whitepoint stored in most profiles doesn’t match the sRGB spec value. The XYZ values given in the spec, again, are [X=0.9505,Y=1,Z=1.0890], which in s15Fixed16Number hex are [X=F354,Y=10000,Z=116C9]. All sRGB profiles I’ve examined (if they define a D65 whitepoint) have had the following value for the ‘wtpt’ tag [X=F351,Y=10000,Z=116CC], which works out to [X=0.9504547119140625,Y=1,Z=1.08905029296875].

I’m convinced my numbers are the correct colorant and whitepoint values for sRGB as written in the actual spec. But you may be reluctant to take my word for it, especially given that there are so many other profiles out there with different values. Fortunately, I have a bit of official documentation on my side.

While trying to locate the most correct and precise definition of the D65 and colorant values available, I ran across a document entitled “How to interpret the sRGB color space (specified in IEC 61966-2-1) for ICC profiles”. I wonder what it’s about....

That document is published on the ICC website, under its information page for sRGB.  For some common colorspaces, the ICC publishes spec extension documents that describe how to treat that specific colorspace in the context of a profile. That document is linked at the bottom of the page, under Hints for Profile Makers.

If you read through that document, you will find the same rules and numbers I used, extracted from the sRGB spec (which I assume the ICC has an actual copy of). For example, section A7 contains the exact XYZ->R’G’B’ matrix I listed above. Theirs has more decimal places than the Wikipedia page but less than mine. You’ll also find under section B2, the exact recommended D65->D50 Bradford adaptation matrix. Theirs only matches mine to 5 decimal places, but I think mine came out better, because… hold the phone.. they included the actual suggested ICC profile matrix, with many decimal places of precision. You’ll find that is also very close to mine. In fact, when converted to s15Fixed16Number format in hex as I’ve done with the others, those numbers are:

      |     X      Y      Z
Red   |  6FA0   38F2   038F
Green |  6296   B78A   18DA
Blue  |  24A0   0F85   B6C4
Sum   |  F6D6  10001   D32D

You can see they are identical to mine from above with the exception that the Green Y value came out 1 higher, making the sum 1 too high. That’s within nudging distance of being well-behaved, but I believe that if their adaptation matrix had been a bit better, the nudging wouldn’t have been required; it wasn’t with mine.

The existence of that document on the ICC site begs the question: are they using those values in their reference sRGB profiles? The answer is no. No, they are not. Their profiles, as of today, are still using the same busted numbers from the old HP/Microsoft profile. I don’t know why.

So, that’s the mathematical explanation of how I arrived at my profile color values and a bit of evidence to support their validity. But maybe you’re still not convinced. There are, after all, two different definitions of the D65 value and two different versions of the primary color values given on the Wikipedia page for sRGB. Wouldn’t a profile created with those other numbers also comply with the spec? Well, no, actually. And I’ll do the math to show you why. But first a bit of history.

The Life of sRGB

sRGB started its life as a derivative of the Rec. 709 HDTV standard. The authors, who came from HP and Microsoft, took the primaries/gamut and whitepoint/color temperature from the Rec. 709 standard, modified the gamma curve to more closely match the response curve of CRT-based computer displays, and created their own draft spec. That draft is still available online today.

Despite the large red warning at the top of that page that explains it is obsolete, you will still find values from that draft spec living on in modern software. This is, no doubt, partly a result of the fact that the draft is freely available and the actual spec has to be purchased.

Basically, what happened was that the draft authors rounded most of the numbers they used when they published them. This, in turn, led to inaccuracy in several parts of the draft spec. When the draft was submitted to the IEC for standardization, it went through a process of refinement wherein that inaccuracy was resolved before the spec became final. In many cases, the resolution was a slight tweak of the numbers to cancel out the rounding errors or to bring things back into alignment.

One such example of this is nicely documented on the sRGB Wikipedia page, in the section entitled “Theory of the transformation”. That section describes how the original intended values for the response curve produced numbers with lots of decimal places. Those numbers were rounded in the draft spec, creating a break in the curve at the transition from its linear portion to the actual gamma curve. The numbers were then adjusted for the final spec to resolve the break. The adjustments fixed the error in the sense that the two parts of the curve were made to meet up again, but they also changed the curve segments such that although they meet, the slope of the lines is no longer continuous as was originally intended.

That refinement is a bit of a recurring theme in the sRGB spec, where the intended value and the actual value published are different. This happened with the definitions of the color values and whitepoint as well. There is a note in the section describing the XYZ->sRGB transformation that reads

“The numerical values below match those in the official sRGB specification, which corrected small rounding errors in the original publication by sRGB's creators”

Essentially, what that means is that in the final spec, the XYZ values for the D65 illuminant and the XYZ transformation matrices have been adjusted to compensate for the 4-decimal-place rounding that was used on the original draft spec numbers. If you use those rounded numbers from the draft, you’ll get incorrect results. If you use the intended numbers, you’ll get results that are mathematically correct but are incorrect according to the published spec.

And that leads us back to the colorant and whitepoint tags in the ArgyllCMS reference sRGB profile. I’ll do the math that leads to those numbers so I can show you where they deviate from the standard.

How Not to Create an sRGB ICC Profile

I mentioned earlier that my quest for additional precision beyond that given in the sRGB spec led to a dead-end. I’ll go through that path again to show why. Let’s start by assuming that the XYZ values given for both the D65 illuminant and for the primary colors (by way of the R’G’B’->XYZ transformation matrix) are not good enough. That leaves us with the alternate definitions of those values, which were copied directly from the Rec. 709 standard. They are defined on the sRGB Wikipedia page as follows:

  Red     Green   Blue    White(D65)
x 0.6400  0.3000  0.1500  0.3127
y 0.3300  0.6000  0.0600  0.3290
Y 0.2126  0.7152  0.0722  1.0000

Cross-referencing the Rec. 709 standard, which is freely available, the red, green, blue and whitepoint x and y values all match, except Rec. 709 only defines the color chromaticity coordinates to 2 decimal places (those extra 0’s are filler). The Y values given do not appear in the Rec. 709 spec, and that’s because 1) they can be calculated from x and y if you know the whitepoint and 2) those values are rounded to 4 decimal places, which makes them less precise than they could be if we calculated them.

Bruce Lindbloom has tons of useful color-related math on his site, and I referred to his Chromatic Adaptation page/formulas/matrices earlier. This time I will refer to his page on generating XYZ/RGB matrices.

Remember, the R’G’B’->XYZ matrix and the primaries are the same thing, so if we get the matrix, we’ll have the precise XYZ values for our color primaries. The formula on that page starts by converting each xy color to unscaled XYZ, by setting its Y value to 1 and calculating X and Z from there. It then uses the whitepoint, which we know should have a Y value of 1, to compute a scaling factor (the S vector), which defines the final component colors relative to that white. To get the whitepoint’s XYZ value, we can use this formula, or we can use the simplified version on the matrix calculation page since we know the Y value is 1. That gives us an XYZ value for D65 of [X=0.950455927051672,Y=1,Z=1.08905775075988]. Now that’s some decimal places!

Using that value to compute the R’G’B’->XYZ matrix, we get the following:

0.4123907992659590 0.3575843393838780 0.1804807884018340
0.2126390058715100 0.7151686787677560 0.0721923153607337
0.0193308187155918 0.1191947797946260 0.9505321522496610

And rounding that to 4 decimal places, we get the exact numbers listed in the sRGB spec (I’ve been informed by a Wikipedia author)

0.4124  0.3576  0.1805
0.2126  0.7152  0.0722
0.0193  0.1192  0.9505

Plus, we have extra precision, and we love extra precision. Everything is awesome! Now, let’s create an extra-precise Bradford adaptation matrix to go from our extra-precise definition of D65 to the ICC’s specified D50 value. Here’s the adaptation matrix

 1.0478860032225500  0.0229187651747795 -0.0502160953117330
 0.0295817824980035  0.9904835184905490 -0.0170787077044827
-0.0092518808392088  0.0150726074870313  0.7516781336176040

And the final D50-adapted R’G’B’->XYZ matrix

 0.4360412516160510  0.3851129107981560  0.1430458375857940
 0.2224845402294770  0.7169050786084580  0.0606103811620653
 0.0139201874713754  0.0970672386971240  0.7139125738315010

Converted to profile format, it’s an exact match for Argyll’s sRGB

      |     X      Y      Z
Red   |  6FA0   38F5   0390
Green |  6297   B787   18D9
Blue  |  249F   0F84   B6C4
Sum   |  F6D6  10000   D32D

It’s well-behaved, it’s precise (or at least it was until we quantized it for the profile), and we got it using numbers from our telephone-game version of the spec. So what’s wrong with it? Well, let’s back up a couple of steps to the unrounded, unadapted D65 R’G’B’->XYZ matrix. If we invert that to create the XYZ->R’G’B’ matrix, this is what we get:

 3.2409699419045200 -1.5373831775700900 -0.4986107602930030
-0.9692436362808800  1.8759675015077200  0.0415550574071756
 0.0556300796969936 -0.2039769588889760  1.0569715142428800

And here again is the XYZ->R’G’B’ matrix from the spec – as described to me by a little birdy.

 3.2406 -1.5372 -0.4986
-0.9689  1.8758  0.0415
 0.0557 -0.2040  1.0570

Notice that these matrices no longer agree to the 4 decimal places of precision defined in the spec. If we go back and look at the draft spec, we can see that it lists a different set of rounded numbers, which do match

 3.2410 -1.5374 -0.4986
-0.9692  1.8760  0.0416
 0.0556 -0.2040  1.0570

And therein lies the problem. These rounded numbers from the draft spec don’t invert to create the correct R’G’B’->XYZ matrix. Here’s that one:

 0.4123808838269000  0.3575728355732480  0.1804522977447920
 0.2126198631048980  0.7151387878413210  0.0721499433963131
 0.0193434956789248  0.1192121694056360  0.9505065664127130

We have a round-trip failure, caused by the lack of precision. To fix that, the spec (which I once saw a blurry photo of, I swear) was modified and the values adjusted so that at 4 decimal places of precision, each matrix inverts to the other. Defining the RGB/XYZ matrices such that they work with only 4 decimals of precision has another benefit that we didn’t get to see. The D65 XYZ values I used were carried through all calculations with full double float precision as well, so there was no opportunity for our whitepoint to throw the other colors off balance. Without that precision, it’s difficult to maintain balance, which I assume is how the HP/Microsoft sRGB profile ended up so bad.

Using the rounded XYZ values for the primaries, you’ll find that they add to exactly the rounded value given for D65.

      X       Y       Z
Red   0.4124  0.2126  0.0193
Green 0.3576  0.7152  0.1192
Blue  0.1805  0.0722  0.9505
White 0.9505  1.0000  1.0890

This creates automatic balance, even with a low level of precision, which was the intent. You may have noticed that this required rounding the Z value of of the whitepoint in the wrong direction. The more precise calculation of the whitepoint we made above gave D65 a Z value of 1.08905775075988, which should have rounded to 1.0891. Oddly enough, the D65 Z value is listed both ways in the draft sRGB spec. But it works out that rounding it down to 1.0890 makes everything work better, so that’s what ended up in the final spec (I think I overheard a guy mutter to himself on the bus one time).

And now I’ll do one final conversion to prove these rounded numbers are the bestest: let’s convert them to xyY, and see if they match the intended Rec. 709 colors.

      x                  y                   Y     
Red   0.640074499456775  0.329970510631693   0.2126
Green 0.3                0.6                 0.7152
Blue  0.150016622340426  0.0600066489361702  0.0722
White 0.312715907221582  0.329001480506662   1.0

Sure enough, round those to the requisite 2 decimal places for colors and 4 for white, and they match the spec values exactly.

So what we learned is, if you use the xy color values inherited from Rec 709, you’ll match the original intent of the draft spec, but you won’t match the actual final spec. For that, you must use its final XYZ numbers with their intentional imprecision. That’s how I got my numbers, and I’m stickin’ to ‘em.

That leaves just one step in my journey to the perfect compact sRGB profile. Come back for the final post, where I’ll compare my final profiles with some references and see which one gives the best bang for your buck.



Update: A Bit of Perspective

After I published this post, Graeme Gill (the creator of ArgyllCMS) commented here, and then he and Elle Stone and I had a bit of further discussion on the pixls.us forum.

Those exchanges led me to think a little more clarification is necessary on this topic. It turns out to be quite controversial, at least among people who have spent any significant time thinking about it (we are likely few in number). And I heard from a couple more people who found the topic interesting but didn’t have the background knowledge to follow everything completely. Talking to them gave me some better ideas for explaining the disagreement, so I thought I’d get them written down.

But I want to get two important things straight before I get back into the details.

First, I think a bit of perspective is in order. I said that Graeme and Elle’s reference sRGB profiles (they match in primaries and whitepoint) were wrong. There are varying degrees of wrong, and I want to make it clear that although I think their profiles are wrong according to my interpretation of the sRGB spec, I believe they are correct according to their own interpretations. When I took an alternate approach to deriving the colorant and whitepoint values, using math that I believe to be 100% correct but with inputs I don’t agree are correct, I got numbers that match the ArgyllCMS sRGB profile. That contrasts starkly with the HP/Microsoft sRGB profile, which all three of us agree is wrong in a much more significant sense. I believe the ArgyllCMS sRGB profile is better described as a Rec. 709 profile with sRGB TRCs. Graeme argues those are the same thing.

Even though the raw numbers might have looked far apart when I presented them before, they were given with an absurd number of decimal places, especially given their target use. Once they end up in an ICC profile, most of those differences are quantized away. Only first log10(216) decimal places are accurately preserved in s15Fixed16Number format, which amounts to 4 decimal places reliably. In the end, the level of disagreement between our interpretations of the sRGB spec has a maximum net impact on our final profile values of 3/65536, or 0.0000457764 on any given number. So while we may argue our interpretations of the spec, we’re arguing over a difference that likely won’t ever be visible in any image. I’ll do some real-world tests in my next post when I test out my tone reproduction curves from Part 2, just to make that extra clear.

Second, I want to make a final blanket disclaimer that I have never actually read the actual sRGB spec. I believe I made that clear earlier, but I feel strange about repeatedly speaking with any kind of authority about what the spec contains, and I continually feel the need to disclaim that. There is a sense of frustration on my part that the IEC has the “Default Color Space for the Internet” locked behind a paywall, so that even if I were to pay for it and quote from it directly, the vast majority of you reading this would still be getting the information second-hand. We may as well all agree that the second-hand summary of the spec on Wikipedia is as good as the real thing and move on.

Anyway, I expressed my frustration with that situation and my boredom with repeatedly citing the same second-hand source by instead giving a series of alternate citations, each one escalating in absurdity. I am told this might not have been as funny as I thought it was and that it likely distracted from the information I was attempting to convey. I won’t do that again. I’ll just say once and for all that when I quote ‘the spec’, I mean the sRGB standard as I understand it based on its Wikipedia entry and the draft version of the standard that is freely available.

Two Views of sRGB

Ultimately, the disagreement that Graeme and Elle and I have over the interpretation of the spec comes from a bit of inconsistency within the spec itself. When I explained the history of the spec, I noted that it started as a derivative of the Rec 709 standard, with an alternate gamma curve. I also referenced the history of the refinement of that gamma curve as an example where the spec changed between the draft version and the final version to simplify the math and balance out the published numbers. I believe that same process was applied to the colorant and whitepoint values, so that they no longer precisely agree with the Rec 709 standard on which they were originally based. They’re very, very close, but not exactly the same. The difference between the gamma revision and the color revision is that the final spec only has one definition of the gamma curve. It retains two definitions of the colors.

That allows for an alternate view in which the Rec 709 colorants and whitepoint are still in full effect and any place the spec says something different, it’s simply omitting precision for the sake of convenience.

The fact that both of those views could be considered valid speaks both to the inconsistency of the spec and to its mystery. However, I believe my view can be reconciled more completely than the opposing view can. To allow you to make your own choice, I will present them in the most clear way I know how, pointing out the inconsistencies from both sides. Since I’ve already given a preference for one view, there is no way for me to make this completely objective, and I probably won’t try very hard to do so. In the end, it comes down to numbers and math, so there doesn’t end up being that much subjectivity to it anyway.

I have decided to describe those two views as follows:

xy is Truth

In this view, we believe that the Rec 709 standard colorants and whitepoint, which are described with x,y chromaticity coordinates, are the true center of sRGB and that all XYZ values presented in the spec are derived from those. Where they disagree, the xy values are the definitive answer, because they are the only numbers given in the Rec 709 standard. The XYZ values are presented for convenience, and although they can be used if they have to be, the results will be less accurate than they could be if they were calculated at higher precision from xy.

XYZ is Truth

In this view, we believe that although the Rec 709 colorants and whitepoint were the original basis for sRGB, they were mathematically inconvenient, difficult to calculate correctly, and required more precision be carried in their calculations than most mid-90’s software used. We believe that the true sRGB colorants are defined by their derived XYZ values instead. We further believe that those values were revised between the draft spec and the final to make the math more convenient and to allow more consistent results, even among software that used lower precision. That revision made the sRGB colorant and whitepoint values distinct from Rec 709, even if only very slightly so. The xy values that remain in the spec are presented as a point-of-reference only and are not meant to be used directly.

Each of these views requires that we accept a certain amount of disagreement within the spec itself, and so each comes with its own level of cognitive dissonance. I decided the easiest way to express the level of disagreement was to color-code the spec’s values based on each viewpoint’s level of agreement with them.

I’ll start with the easiest part first. The blue-shaded portions of the spec are numbers related to the tone response curve, or gamma curve. There are people who, to this day, argue whether the draft version or the final version of these numbers is more correct. But for the purposes of this discussion, they’re irrelevant. They are distinct from the colorant and whitepoint values and don’t play into our two world views.

Next, we have the values that are absolutely true and are essential to our viewpoint. Those are colored in dark green. These values are to be taken absolutely literally and precisely, as they are the basis for the other numbers.

Then we have numbers that we agree with, but they don’t have to be definitive. These are in lighter green. We could derive these numbers from the other numbers, or we could take them as given. Whether we calculate them ourselves or take them from the spec, they’re correct enough for our purposes.

Next we have numbers that we agree with as presented, but only as illustrative. These are colored in yellow. These numbers are presented with less accuracy than is required to make our calculations work out, but that’s only because they were rounded to 4 decimal places to match the rest of the numbers given. We can’t use them as is, but if we calculate our own numbers and round them to 4 decimal places, they agree.

And finally, we have numbers that we disagree with, in red. If we calculate our own values, they don’t match these as given, to the precision given. These numbers either indicate some necessary precision was lost along the way, or they indicate a mistake in the spec.

With those definitions out of the way, let’s go over how the spec looks from each viewpoint:

The spec according to xy is Truth

xy-is-truth

In this view, the truth starts from the table in the lower corner. By the way, that table was in another section on the page, but I cut and pasted it down there to keep the diagram compact. The xyY values given for the D65 whitepoint are the ultimate truth, and we’ll calculate the primaries based on that. The calculated XYZ values based on the xy chromaticity coordinates for the primaries and the whitepoint are:

      X                   Y                   Z
Red   0.4123907992659590  0.2126390058715100  0.0193308187155918
Green 0.3575843393838780  0.7151686787677560  0.1191947797946260
Blue  0.1804807884018340  0.0721923153607337  0.9505321522496610
D65   0.950455927051672   1                   1.08905775075988

And the calculations required to get them are described here and here. It’s not simple, but we don’t mind doing things the hard way in the name of precision.

At this point, having done our own primary color and whitespace calculations, we disagree with the following parts of the spec.

  1. The XYZ value of D65 given at the top is wrong. Even if we accept that it’s rounded for display purposes, it’s rounded incorrectly. The Z value should be 1.0891 if we round to 4 decimal places.
  2. The Y values given for the primaries in the table in the bottom right corner are correct, up to 4 decimal places, but we must maintain more than that level of precision to keep the values balanced. If we keep all else the same and round to 4 decimal places here, we’ll end up with a not-well-behaved profile.
  3. The same goes for the R’G’B’-XYZ matrix. Its values are correct to the precision listed, but that precision isn’t enough. We can’t use the spec’s values as given without getting a bad profile unless we accept its faulty whitepoint adjustment.

The correct values for the RGB-XYZ matrix are the same as the primaries, just transposed:

 0.4123907992659590  0.3575843393838780  0.1804807884018340
 0.2126390058715100  0.7151686787677560  0.0721923153607337
 0.0193308187155918  0.1191947797946260  0.9505321522496610

If we invert that matrix, keeping its full precision, we get the following inverse for the XYZ->R’G’B’ matrix.

 3.2409699419045200 -1.5373831775700900 -0.4986107602930030
-0.9692436362808800  1.8759675015077200  0.0415550574071756
 0.0556300796969936 -0.2039769588889760  1.0569715142428800

According to our calculations, the XYZ->R’G’B’ matrix in the spec is wrong. It doesn’t match to the 4 decimal places given.

I personally find it difficult to reconcile this view, as it invalidates so much of the spec itself. So let’s move on to…

The spec according to XYZ is Truth

xyz-is-truth

In this view, the ultimate truth is the R’G’B’->XYZ matrix. Instead of viewing this matrix as the rounded version of true matrix, we take this as absolute truth. The 4 decimal places listed here are the precise values of the colorants. They are already perfectly balanced with the whitepoint and will produce a well-behaved profile as-is.

The whitepoint is defined as the sum of the primaries, so we can calculate it ourselves, or we can take the value given at the top. They’re identical.

If we check our primaries and whitepoint against the Rec 709 values listed in the table by converting our XYZ values to xyY, we find that they match, up to the 4 decimal places given -- with one exception.

   Red                Green            Blue                White
x  0.640074499456775  0.3              0.1500166223404260  0.312715907221582
y  0.329970510631693  0.6              0.0600066489361702  0.329001480506662
Y  0.2126             0.7152           0.0722              1.0

The red colorant’s x value rounds to .6401, making that not a match. But remember that the actual Rec 709 standard only defined the xy chromaticity coordinates of the primaries to 2 decimal places, so we do match to that, even though we don’t match the 0-padded version transcribed into the sRGB spec.

The Y values of the primaries are an exact match, because their XYZ values were defined to exactly 4 decimal places, and when converting to/from xyY, the Y value is preserved. We don’t need these values because we already have them, but we could use them interchangeably.

And finally, if we invert our R’G’B’->XYZ matrix to get the XYZ->R’G’B’ matrix, we end up with the following:

 3.2406254773200500 -1.5372079722103200 -0.4986285986982480
-0.9689307147293190  1.8757560608852400  0.0415175238429540
 0.0557101204455106 -0.2040210505984870  1.0569959422543900

This is also an exact match for the matrix given in the spec, up to the 4 decimal places it defines. That means we didn’t actually need to calculate our own inverse matrix; we could have just used the one given. We have a choice of either accepting a small loss of precision on the round-trip by using the rounded inverse matrix, or we can preserve our own more precise inverse matrix for perfect round-trip accuracy.

According to this view, the quest for more precision in the primaries or whitepoint is flawed, because they are already precise. Attempting to re-calculate the primaries from the Rec 709 chromaticity coordinates doesn’t make them more precise; it makes them further from the already-precise values given.

The Bottom Line

The ‘xy is Truth’ view makes you do more work and puts you in less agreement with the final spec’s published values. If I gave you a copy of the spec and asked you to build me an app that did XYZ-sRGB conversions, which path would you take? And given that answer, which do you think is most compatible with the most software? Given sRGB’s design goal of being easy to implement, I’m confident that the easy answer is the more compatible and therefore the more correct answer.

That said, keep in mind that the primaries are defined by the matrix and vice versa. The two world views agree on those up to 4 decimal places of precision. That difference gets magnified when we use those values in an ICC profile, though, because we must adapt them to the ICC-specified D50 whitepoint. Since the adaptation matrix is calculated using the D65 and D50 XYZ values, the differing definitions of D65 between the two interpretations (which do not agree to 4 decimal places) magnify the differences in the final profile matrix. That difference still remains very small, but the whitepoint is the major contributor to it. And now you know…