When I started the task of creating a minimal sRGB profile, I assumed the part that would require the least thought would be the colorant and whitepoint tags in the profile. To review, the ICC V2 specification requires 9 tags for RGB profiles. Those are: copyright (cprt) and description (desc), which I discussed in Part 1 of this series; the tone reproduction curves (rTRC, gTRC, and bTRC), which I covered thoroughly in Part 2; and finally, the colorant and whitepoint tags (rXYZ, gXYZ, bXYZ, and wtpt), which ended up with their own post, too. This is that post.

I have referenced Elle Stone’s treatise on well-behaved profiles a couple of times already, and I’ll start this post by referring there again. I’ll also refer you to her systematic examination of ICC profiles seen in the wild and finally, to her very detailed explanation of how to create an sRGB-compatible ICC profile using the values from the sRGB spec.

I drew two main conclusions from reading through those articles. The first was that well-behaved and correct sRGB profiles are difficult to create and are, consequently, rare. The second was that the reference sRGB profile shipped with ArgyllCMS happens to be that most mythical of profiles. The unicorn profile, if you will.

And that’s what was supposed to make this step simple; I’d just steal the colorant tag values from the ArgyllCMS profile and call it a day.

Facebook’s TinyRGB profile had used the colorant tag values from the original HP/Microsoft sRGB profile, and while that remains the most commonly-seen sRGB profile in the wild, Elle had convinced me it was also one of the most wrong.

In his writeup on the sRGBz profile, Øyvind Kolås (Pippin) mentioned generating new colorant tag values from babl with improved accuracy. My first assumption was that it must also use those magical Argyll values. Imagine my surprise when I looked at the tag data and saw that they were a completely different set of values. And imagine my further surprise when I ran them through Elle’s xicclu test and found that they were also well-behaved.

That left me with two sets of possible correct sRGB colorant values, and I simply had to know which was right. The rabbit-hole deepens…

Better-Behaved?

If there are two well-behaved sRGB-like profiles with different colorant values, they must not be as difficult to come by as I originally thought. Elle describes creating well-behaved profiles as a process of calculating the most correct values for the colorspace and then ‘nudging’ them so that their rounding errors when converted to the ICC s15Fixed16Number format balance out. I learned that this is actually a very simple process and that testing for well-behavedness is a matter of simple arithmetic. Let’s start with the actual colorant tag values stored in the three profiles I was examining.

      |     HP/Microsoft     |        sRGBz         |      ArgyllCMS     
-------------------------------------------------------------------------
      |     X      Y      Z  |     X      Y      Z  |     X      Y      Z
Red   |  6FA2   38F5   0390  |  6FA1   38F6   0391  |  6FA0   38F5   0390
Green |  6299   B785   18DA  |  6297   B787   18DA  |  6297   B787   18D9
Blue  |  24A0   0F84   B6CF  |  249E   0F83   B6C2  |  249F   0F84   B6C4
Sum   |  F6DB   FFFE   D339  |  F6D6  10000   D32D  |  F6D6  10000   D32D

I’ve kept the values in hex for now, because 1) I read them out with a hex editor and 2) it’s easier to see what’s up when looking at the integer representation of the numbers. What the table above shows is the XYZ values for the Red, Green, and Blue primaries stored in each of the three profiles I examined. I also included a row that shows the sum of the X, Y, and Z values for the three color channels. One thing should stand out immediately: the sRGBz and ArgyllCMS color values are different, but their sums are the same. What’s not obvious from the table is what those sums represent.

First, let me explain how the color values are stored in an ICC profile. The ICC spec defines the s15Fixed16Number format for storing XYZ (and other) values. In that format, 16 bits are allocated to the signed integer portion of the number and 16 bits are allocated to the fractional part of the number. The conversion between floating-point decimal and the fixed-point format is simply to multiply by 2¹⁶ and round to the nearest integer. Conversion back to floating-point decimal is done by dividing by 2¹⁶ (65536). In that format, 1.0 is represented by 0x00010000 because its integer part is 1 and it has no fractional part.

This is in contrast to the response16Number format used to store the TRC points we examined in the last post. In that format, 16 bits are used to represent the full range of 0-1, inclusive. For that number format, the divisor is 2¹⁶-1 (65535), so that 1.0 is represented by 0xFFFF. This has come up as a point of confusion in some things I’ve read, so I thought I’d clear that up.

Now back to the values in the table…

Every developer who works with RGB colors knows that in 8-bit color, black is [0,0,0], full red is [255,0,0], etc. We also know that if you add full red, full green, and full blue together, you get [255,255,255], which is white. Things work the same in XYZ. Adding the three primary colors together at their full intensity (as defined within the colorspace) will give you white (also as defined in the colorspace). In most practical colorspaces, the XYZ values are normalized so that white has a Y value of 1.0. In some representations, you may see the XYZ numbers scaled up to a range of 0-100, but both the ICC and sRGB specs declare that the nominal range is 0-1.

Knowing that white should have a Y value of 1.0 and that 1.0 in s15Fixed16Number format is 0x10000, you should see an immediate problem with the values in the HP/Microsoft sRGB profile: their Y values sum to less than 1.0, meaning they’re scaled improperly.

Its X and Z values are wrong as well. The ICC V2 spec requires that all profiles have their color values adapted to the D50 whitepoint, which allows for simple translation between colorspaces. Since XYZ conversion to and from RGB is whitepoint-dependent, using a common whitepoint for all profiles makes it easy for software to implement that translation. As an aside, let me point out that RGB values we are referring to linear RGB values. That is, the red, green, and blue values that are the output of the TRC that undoes their stored gamma correction. Those values are also normalized to the range 0-1 for computation.

The ICC spec is explicit about the XYZ value to use for the D50 whitepoint, giving it a value of [X=0.9642,Y=1,Z=0.8249]. When converted to s15Fixed16Number format, that value becomes [X=0xF6D6,Y=0x10000,Z=0xD32D], which is the exact value stored as the Profile Illuminant in the header of every V2 ICC profile.

If you refer back to the table of values from the three profiles I examined, you will find that the sRGBz and ArgyllCMS primary colorant values sum to exactly the value of the D50 Illuminant given by the ICC. Put simply, that’s what makes a profile well-behaved. And you can see that the HP/Microsoft profile is quite far off from that value, which is why it’s no good.

Knowing that making a profile well-behaved is simply a matter of normalizing the primary colors so that they sum at full intensity to make white, it’s easy to see how we ended up with two different sRGB-like profiles that are both well-behaved. But that still leaves the question of which is right.

In Which I Learn They’re Both Wrong

It’s been a while since I linked to Nine Degrees Below, so let me start this section with another link to Elle’s research into the proper color values to use for sRGB. In that article, Elle rounds up every possible definition of every color referenced in the sRGB spec, does every bit of mathematical wrangling imaginable, and comes up with a final set of numbers that are very close to the ones that ArgyllCMS has in its reference sRGB profile. Score one for Argyll.

I decided to do a similar exercise and find my own answer to compare to the others. One problem with making things match the sRGB spec is that the spec itself isn’t published freely. If you want to read the actual spec, it will cost you 170 Swiss francs to buy it from the IEC web store.

Fortunately, there are enough references available elsewhere that I believe we can put together an accurate picture of the spec without ponying up. I like to save my francs for Swiss chocolate and wine, thank you very much. I can enjoy those while I read the Wikipedia entry on sRGB.

At this point, it’s worth considering exactly what numbers we’re trying calculate. Depending where you look you may see the colorant values described in one of two ways:

1. They represent the direct XYZ translation of the three primary colors (red, green, and blue) at their full intensity under the specified illuminant.
2. They make up a matrix which can be used to translate any set of [R,G,B] values to their corresponding [X,Y,Z] values.

In reality, both are correct. But I think for our purposes, it’s important to focus on the second definition. The reason that’s important is that the colorant values stored in the ICC profile are used in exactly that way. They’re also used in another, related way. The matrix created from those XYZ colorant values can be inverted to create the matrix that translates from XYZ back to RGB. This will become very important later, so I wanted to mention it now and let it soak in a bit.

If you use a tool like the Profile Inspector available on the ICC site, it reinforces the first definition I gave. They show the XYZ values converted to decimal, the calculated x and y coordinates for that color, and a nice chromaticity diagram with the color plotted on it. Here’s the rXYZ tag information from the Argyll sRGB profile.

Exiftool, on the other hand, presents the data using the second definition. Here’s the relevant output from the same profile:

Red Matrix Column:   0.43604 0.22249 0.01392
Green Matrix Column: 0.38512 0.71690 0.09706
Blue Matrix Column:  0.14305 0.06061 0.71393

You can see that the XYZ values are the same in each, and that makes perfect sense. If you make a matrix using the columns given by Exiftool

0.43604 0.38512 0.14305
0.22249 0.71690 0.06061
0.01392 0.09706 0.71393

and then multiply the linear value for pure red [1,0,0] by that matrix, you get the left column back, giving you the same XYZ value shown for the red primary in the Profile Inspector. Again, it’s the same thing… but the matrix usage is the more important definition.

So, that leaves the question: how do we calculate that matrix, given the values in the sRGB spec?

That answer has two parts. For the first part, we must get the matrix for converting RGB to XYZ using sRGB’s native illuminant/whitepoint, which is D65. Then, because the colors in an ICC profile must be given relative to the D50 illuminant, we must adapt that matrix from D65-relative values to D50-relative values. Bruce Lindbloom has a reference on the basic theory of Chromatic Adaptation as well as some different adaptation matrices on his site. But I’ll caution you not to use his pre-calculated matrix for D65->D50; it’s wrong for our purposes.

Let’s start with the matrix for calculating RGB->XYZ under D65. The Wikipedia article on sRGB has the matrix printed right in it. It also points out explicitly that the values listed are the exact ones in the sRGB spec. Again, I don’t have the actual spec, but I have no reason to doubt the veracity of that statement. The sRGB spec reportedly is full of exact numbers, rounded to 4 decimal places.

Well, like Elle, I started with the assumption that more precision is better, and I didn’t like the look of those imprecise numbers from the spec. So, like Elle, I tried to calculate my own more precise matrix using the the published x,y values of the primaries. That effort was a complete failure. What I mean is, while I succeeded in the calculation, if I then inverted my matrix to create the XYZ->RGB matrix, it didn’t match the one in the spec to the 4 decimal places it has listed. It turns out, the best way to get the correct (according to the spec) inverse matrix is to use the explicitly rounded values given in the RGB->XYZ matrix.

0.4124  0.3576  0.1805
0.2126  0.7152  0.0722
0.0193  0.1192  0.9505

Invert that, and you get

 3.2406254773200500 -1.5372079722103200 -0.4986285986982480
-0.9689307147293190  1.8757560608852400  0.0415175238429540
 0.0557101204455106 -0.2040210505984870  1.0569959422543900

Which matches the spec (as described by wiki) very nicely, and with lots of decimal places, if that’s your thing. Round that back to 4 decimal places, like such, to match the spec again,

 3.2406 -1.5372 -0.4986
-0.9689  1.8758  0.0415
 0.0557 -0.2040  1.0570

Invert that, and you get

0.4124103360777000  0.3575962178119540  0.1804991017305060
0.2126157251481260  0.7151958779229800  0.0722134074030765
0.0193021307575116  0.1191881265507680  0.9504992763896300

Which, again, rounds to match the spec at 4 decimal places. These, I’m confident, are the correct numbers for sRGB at D65, which just leaves the D65->D50 adaptation matrix to work out.

This, again, seems like a place where more precision would pay off, but in fact, there is a value listed in the spec, rounded to 4 decimal places, that is perfect for this use. That value given for D65 is [X=0.9505,Y=1,Z=1.0890]. We also have a standard XYZ value for D50, given by the ICC spec. That value is [X=0.9642,Y=1,Z=0.8249].

As an aside, you may recall that the actual D50 value stored in the profile header is in s15Fixed16Number format and is, in hex [X=0xF6D6,Y=0x10000,Z=0xD32D]. Converted back to floating-point decimal, that value is [X=0.964202880859375,Y=1,Z=0.8249053955078125]. If you want to be extra precise, that value is also acceptable to use. It worked out that when creating the adaptation matrices, it didn’t matter which number I used. I got the same results once the quantization to s15Fixed16Number format was done for the final calculated values. For the calculations shown below, I used the rounded value published in the ICC spec.

Using the D50 XYZ value from the ICC spec, the D65 XYZ value from the sRGB spec and the Bradford cone response matrix given by Bruce Lindbloom, we get the following values for the D65->D50 adaptation matrix:

 1.0478414713468100  0.0228955556744975  -0.0502009864000404
 0.0295477450604968  0.9905065286192130  -0.0170722316797199
-0.0092509594572860  0.0150723678359253   0.7517177861599870

Notice that the matrix I calculated is different than the one Bruce gives on his site:

 1.0478112  0.0228866 -0.0501270
 0.0295424  0.9904844 -0.0170491
-0.0092345  0.0150436  0.7521316

They’re quite close, but he used D65 and D50 values from a different source. It comes down to rounding differences, but remember, we’re following exact specs, so we want to round the same way they do.

And finally, if we multiply our RGB->XYZ matrix by the adaptation matrix, we get the final adapted values:

0.4360285388823030  0.3850990539931360  0.1430724071245600
0.2224376839759750  0.7169415328858720  0.0606207831381531
0.0138974429946207  0.0970763744845987  0.7139261825207810

Converted to s15Fixed16Number format, written in hex, and transposed to match the profile layout I used earlier, those numbers look like this:

      |     X      Y      Z
Red   |  6FA0   38F2   038F
Green |  6296   B789   18DA
Blue  |  24A0   0F85   B6C4
Sum   |  F6D6  10000   D32D

You’ll note that the hex values exactly total those of the D50 Profile Illuminant, so these values will create a well-behaved profile. These values, however, do not match any other profile I’ve seen.

I also found that the D65 whitepoint stored in most profiles doesn’t match the sRGB spec value. The XYZ values given in the spec, again, are [X=0.9505,Y=1,Z=1.0890], which in s15Fixed16Number hex are [X=F354,Y=10000,Z=116C9]. All sRGB profiles I’ve examined (if they define a D65 whitepoint) have had the following value for the ‘wtpt’ tag [X=F351,Y=10000,Z=116CC], which works out to [X=0.9504547119140625,Y=1,Z=1.08905029296875].

I’m convinced my numbers are the correct colorant and whitepoint values for sRGB as written in the actual spec. But you may be reluctant to take my word for it, especially given that there are so many other profiles out there with different values. Fortunately, I have a bit of official documentation on my side.

While trying to locate the most correct and precise definition of the D65 and colorant values available, I ran across a document entitled “How to interpret the sRGB color space (specified in IEC 61966-2-1) for ICC profiles”. I wonder what it’s about....

That document is published on the ICC website, under its information page for sRGB.  For some common colorspaces, the ICC publishes spec extension documents that describe how to treat that specific colorspace in the context of a profile. That document is linked at the bottom of the page, under Hints for Profile Makers.

If you read through that document, you will find the same rules and numbers I used, extracted from the sRGB spec (which I assume the ICC has an actual copy of). For example, section A7 contains the exact XYZ->RGB matrix I listed above. Theirs has more decimal places than the Wikipedia page but less than mine. You’ll also find under section B2, the exact recommended D65->D50 Bradford adaptation matrix. Theirs only matches mine to 5 decimal places, but I think mine came out better, because… hold the phone.. they included the actual suggested ICC profile matrix, with many decimal places of precision. You’ll find that is also very close to mine. In fact, when converted to s15Fixed16Number format in hex as I’ve done with the others, those numbers are:

      |     X      Y      Z
Red   |  6FA0   38F2   038F
Green |  6296   B78A   18DA
Blue  |  24A0   0F85   B6C4
Sum   |  F6D6  10001   D32D

You can see they are identical to mine from above with the exception that the Green Y value came out 1 higher, making the sum 1 too high. That’s within nudging distance of being well-behaved, but I believe that if their adaptation matrix had been a bit better, the nudging wouldn’t have been required; it wasn’t with mine.

The existence of that document on the ICC site begs the question: are they using those values in their reference sRGB profiles? The answer is no. No, they are not. Their profiles, as of today, are still using the same busted numbers from the old HP/Microsoft profile. I don’t know why.

So, that’s the mathematical explanation of how I arrived at my profile color values and a bit of evidence to support their validity. But maybe you’re still not convinced. There are, after all, two different definitions of the D65 value and two different versions of the primary color values given on the Wikipedia page for sRGB. Wouldn’t a profile created with those other numbers also comply with the spec? Well, no, actually. And I’ll do the math to show you why. But first a bit of history.

The Life of sRGB

sRGB started its life as a derivative of the Rec. 709 HDTV standard. The authors, who came from HP and Microsoft, took the primaries/gamut and whitepoint/color temperature from the Rec. 709 standard, modified the gamma curve to more closely match the response curve of CRT-based computer displays, and created their own draft spec. That draft is still available online today.

Despite the large red warning at the top of that page that explains it is obsolete, you will still find values from that draft spec living on in modern software. This is, no doubt, partly a result of the fact that the draft is freely available and the actual spec has to be purchased.

Basically, what happened was that the draft authors rounded most of the numbers they used when they published them. This, in turn, led to inaccuracy in several parts of the draft spec. When the draft was submitted to the IEC for standardization, it went through a process of refinement wherein that inaccuracy was resolved before the spec became final. In many cases, the resolution was a slight tweak of the numbers to cancel out the rounding errors or to bring things back into alignment.

One such example of this is nicely documented on the sRGB Wikipedia page, in the section entitled “Theory of the transformation”. That section describes how the original intended values for the response curve produced numbers with lots of decimal places. Those numbers were rounded in the draft spec, creating a break in the curve at the transition from its linear portion to the actual gamma curve. The numbers were then adjusted for the final spec to resolve the break. The adjustments fixed the error in the sense that the two parts of the curve were made to meet up again, but they also changed the curve segments such that although they meet, the slope of the lines is no longer continuous as was originally intended.

That refinement is a bit of a recurring theme in the sRGB spec, where the intended value and the actual value published are different. This happened with the definitions of the color values and whitepoint as well. There is a note in the section describing the XYZ->sRGB transformation that reads

“The numerical values below match those in the official sRGB specification, which corrected small rounding errors in the original publication by sRGB's creators”

Essentially, what that means is that in the final spec, the XYZ values for the D65 illuminant and the XYZ transformation matrices have been adjusted to compensate for the 4-decimal-place rounding that was used on the original draft spec numbers. If you use those rounded numbers from the draft, you’ll get incorrect results. If you use the intended numbers, you’ll get results that are mathematically correct but are incorrect according to the published spec.

And that leads us back to the colorant and whitepoint tags in the ArgyllCMS reference sRGB profile. I’ll do the math that leads to those numbers so I can show you where they deviate from the standard.

How Not to Create an sRGB ICC Profile

I mentioned earlier that my quest for additional precision beyond that given in the sRGB spec led to a dead-end. I’ll go through that path again to show why. Let’s start by assuming that the XYZ values given for both the D65 illuminant and for the primary colors (by way of the RGB->XYZ transformation matrix) are not good enough. That leaves us with the alternate definitions of those values, which were copied directly from the Rec. 709 standard. They are defined on the sRGB Wikipedia page as follows:

  Red     Green   Blue    White(D65)
x 0.6400  0.3000  0.1500  0.3127
y 0.3300  0.6000  0.0600  0.3290
Y 0.2126  0.7152  0.0722  1.0000

Cross-referencing the Rec. 709 standard, which is freely available, the red, green, blue and whitepoint x and y values all match, except Rec. 709 only defines the color chromaticity coordinates to 2 decimal places (those extra 0’s are filler). The Y values given do not appear in the Rec. 709 spec, and that’s because 1) they can be calculated from x and y if you know the whitepoint and 2) those values are rounded to 4 decimal places, which makes them less precise than they could be if we calculated them.

Bruce Lindbloom has tons of useful color-related math on his site, and I referred to his Chromatic Adaptation page/formulas/matrices earlier. This time I will refer to his page on generating XYZ/RGB matrices.

Remember, the RGB->XYZ matrix and the primaries are the same thing, so if we get the matrix, we’ll have the precise XYZ values for our color primaries. The formula on that page starts by converting each xy color to unscaled XYZ, by setting its Y value to 1 and calculating X and Z from there. It then uses the whitepoint, which we know should have a Y value of 1, to compute a scaling factor (the S vector), which defines the final component colors relative to that white. To get the whitepoint’s XYZ value, we can use this formula, or we can use the simplified version on the matrix calculation page since we know the Y value is 1. That gives us an XYZ value for D65 of [X=0.950455927051672,Y=1,Z=1.08905775075988]. Now that’s some decimal places!

Using that value to compute the RGB->XYZ matrix, we get the following:

0.4123907992659590 0.3575843393838780 0.1804807884018340
0.2126390058715100 0.7151686787677560 0.0721923153607337
0.0193308187155918 0.1191947797946260 0.9505321522496610

And rounding that to 4 decimal places, we get the exact numbers listed in the sRGB spec (I’ve been informed by a Wikipedia author)

0.4124  0.3576  0.1805
0.2126  0.7152  0.0722
0.0193  0.1192  0.9505

Plus, we have extra precision, and we love extra precision. Everything is awesome! Now, let’s create an extra-precise Bradford adaptation matrix to go from our extra-precise definition of D65 to the ICC’s specified D50 value. Here’s the adaptation matrix

 1.0478860032225500  0.0229187651747795 -0.0502160953117330
 0.0295817824980035  0.9904835184905490 -0.0170787077044827
-0.0092518808392088  0.0150726074870313  0.7516781336176040

And the final D50-adapted RGB->XYZ matrix

 0.4360412516160510  0.3851129107981560  0.1430458375857940
 0.2224845402294770  0.7169050786084580  0.0606103811620653
 0.0139201874713754  0.0970672386971240  0.7139125738315010

Converted to profile format, it’s an exact match for Argyll’s sRGB

      |     X      Y      Z
Red   |  6FA0   38F5   0390
Green |  6297   B787   18D9
Blue  |  249F   0F84   B6C4
Sum   |  F6D6  10000   D32D

It’s well-behaved, it’s precise (or at least it was until we quantized it for the profile), and we got it using numbers from our telephone-game version of the spec. So what’s wrong with it? Well, let’s back up a couple of steps to the unrounded, unadapted D65 RGB->XYZ matrix. If we invert that to create the XYZ->RGB matrix, this is what we get:

 3.2409699419045200 -1.5373831775700900 -0.4986107602930030
-0.9692436362808800  1.8759675015077200  0.0415550574071756
 0.0556300796969936 -0.2039769588889760  1.0569715142428800

And here again is the XYZ->RGB matrix from the spec – as described to me by a little birdy.

 3.2406 -1.5372 -0.4986
-0.9689  1.8758  0.0415
 0.0557 -0.2040  1.0570

Notice that these matrices no longer agree to the 4 decimal places of precision defined in the spec. If we go back and look at the draft spec, we can see that it lists a different set of rounded numbers, which do match

 3.2410 -1.5374 -0.4986
-0.9692  1.8760  0.0416
 0.0556 -0.2040  1.0570

And therein lies the problem. These rounded numbers from the draft spec don’t invert to create the correct RGB->XYZ matrix. Here’s that one:

 0.4123808838269000  0.3575728355732480  0.1804522977447920
 0.2126198631048980  0.7151387878413210  0.0721499433963131
 0.0193434956789248  0.1192121694056360  0.9505065664127130

We have a round-trip failure, caused by the lack of precision. To fix that, the spec (which I once saw a blurry photo of, I swear) was modified and the values adjusted so that at 4 decimal places of precision, each matrix inverts to the other. Defining the RGB/XYZ matrices such that they work with only 4 decimals of precision has another benefit that we didn’t get to see. The D65 XYZ values I used were carried through all calculations with full double float precision as well, so there was no opportunity for our whitepoint to throw the other colors off balance. Without that precision, it’s difficult to maintain balance, which I assume is how the HP/Microsoft sRGB profile ended up so bad.

Using the rounded XYZ values for the primaries, you’ll find that they add to exactly the rounded value given for D65.

      X       Y       Z
Red   0.4124  0.2126  0.0193
Green 0.3576  0.7152  0.1192
Blue  0.1805  0.0722  0.9505
White 0.9505  1.0000  1.0890

This creates automatic balance, even with a low level of precision, which was the intent. You may have noticed that this required rounding the Z value of of the whitepoint in the wrong direction. The more precise calculation of the whitepoint we made above gave D65 a Z value of 1.08905775075988, which should have rounded to 1.0891. Oddly enough, the D65 Z value is listed both ways in the draft sRGB spec. But it works out that rounding it down to 1.0890 makes everything work better, so that’s what ended up in the final spec (I think I overheard a guy mutter to himself on the bus one time).

And now I’ll do one final conversion to prove these rounded numbers are the bestest: let’s convert them to xyY, and see if they match the intended Rec. 709 colors.

      x                  y                   Y     
Red   0.640074499456775  0.329970510631693   0.2126
Green 0.3                0.6                 0.7152
Blue  0.150016622340426  0.0600066489361702  0.0722
White 0.312715907221582  0.329001480506662   1.0

Sure enough, round those to the requisite 2 decimal places for colors and 4 for white, and they match the spec values exactly.

So what we learned is, if you use the xy color values inherited from Rec 709, you’ll match the original intent of the draft spec, but you won’t match the actual final spec. For that, you must use its final XYZ numbers with their intentional imprecision. That’s how I got my numbers, and I’m stickin’ to ‘em.

That leaves just one step in my journey to the perfect compact sRGB profile. Come back for the final post, where I’ll compare my final profiles with some references and see which one gives the best bang for your buck.

Update: A Bit of Perspective

After I published this post, Graeme Gill (the creator of ArgyllCMS) commented here, and then he and Elle Stone and I had a bit of further discussion on the pixls.us forum.

Those exchanges led me to think a little more clarification is necessary on this topic. It turns out to be quite controversial, at least among people who have spent any significant time thinking about it (we are likely few in number). And I heard from a couple more people who found the topic interesting but didn’t have the background knowledge to follow everything completely. Talking to them gave me some better ideas for explaining the disagreement, so I thought I’d get them written down.

But I want to get two things straight before I get back into the details.

First, I think a bit of perspective is in order. I said that Graeme and Elle’s reference sRGB profiles (they match in primaries and whitepoint) were wrong. There are varying degrees of wrong, and I want to make it clear that although I think their profiles are wrong according to my interpretation of the sRGB spec, I believe they are correct according to their own interpretations. When I took an alternate approach to deriving the colorant and whitepoint values, using math that I believe to be 100% correct but with inputs I don’t agree are correct, I got numbers that match the ArgyllCMS sRGB profile. That contrasts starkly with the HP/Microsoft sRGB profile, which all three of us agree is wrong in a much more significant sense. I believe the ArgyllCMS sRGB profile is better described as a Rec. 709 profile with sRGB TRCs. Graeme argues those are the same thing.

Second, even though the raw numbers might have looked far apart when I presented them before, they were given with an absurd number of decimal places, especially given their target use. Once they end up in an ICC profile, most of those differences are quantized away. Only first log₁₀(2¹⁶) decimal places are accurately preserved in s15Fixed16Number format, which amounts to 4 decimal places reliably. In the end, the level of disagreement between our interpretations of the sRGB spec has a maximum net impact on our final profile values of 3/65536, or 0.0000457764 on any given number. So while we may argue our interpretations of the spec, we’re arguing over a difference that likely won’t ever be visible in any image.

Update 2: I have finally seen the spec!

I had a section here that further explored the different interpretations possible from the partial information publicly available, but it was a bit rambly and included more conjecture. I have removed it in favor of a new post, which explains what I learned from reading the real deal.

In Part 1 of this series, I examined Facebook’s TinyRGB (c2) ICC profile, following on from the work Øyvind Kolås (Pippin) did in creating his sRGBz profile. I was able to trim an extra 68 bytes off that profile (making 100 bytes total reduction off TinyRGB) by careful packing of the data, and now I turn my attention to the tone reproduction curve (TRC) tags and their shared content.

In his sRGBz post, Pippin discusses the Facebook decision to use 26 points in their tone reproduction curve. The Facebook post explains that this was done because the linear part of the sRGB curve ends about 1/25^th of the way in, making that a natural place for the second TRC point to fall. In fact, the sRGB curve is defined as having a linear segment up to a value of precisely 0.04045, which is awfully close to 1/25. That makes a sensible place to start testing, but it seems they decided that was the magic number and went full speed ahead without bothering to check others.

The tricky thing about optimizing a point-based curve approximation for an ICC profile is that the curve points have to be spaced at even intervals. If we were allowed to space them arbitrarily, we could define the linear segment precisely with two points and then use as many points as we wanted to tune the curvy part of the curve. But with even spacing, options are much more limited, and the performance of curves with different numbers of points defined can be quite unpredictable. It makes sense, then, that the Facebook team would choose 26 as a starting point.

However, Pippin failed to find any compelling evidence that 26 is disproportionately better than other surrounding numbers. My check of their math results in the same conclusion, but I arrived at it in a different way, which I’ll be getting to. 26 points produce a decent curve, but in that size range, more is better and fewer is not necessarily a lot worse. What’s nice about a 26-point curve is that at 2 bytes per point, plus the 12-byte header, the curve is a nice even 64 bytes. And that’s about the only special thing it has going for it.

In Search of Magic Numbers

Is Facebook’s curve the best curve you can get with 26 points? And if 26 isn’t the magic number, is there one?

I was intrigued by Pippin’s alternate proposed curves, so I set out to do some testing of my own using his as a starting point. One thing that stood out to me immediately was that he optimized the curves for minimum mean absolute error. Generally, when testing sample fit to a curve, root-mean-square error is more meaningful, because it gives more weight to points that are further off the curve. Large individual errors are definitely undesirable in this case, so that seems a better measure. I was also interested in seeing the max error for that reason. I set up some code to interpolate the 256 values that would be found in an 8-bit JPEG’s color channels, compared them to the values calculated using the actual sRGB inverse gamma function, and measured the max error, MAE, and RMSE for his curves vs the TinyRGB/c2 curve.

Points | Max Error | Mean Error | RMS Error | Point Values
    23 |  0.000587 |   0.000148 |  0.000194 | 0,229,544,1072,1796,2744,3937,5384,7104,9104,11396,13995,16912,20157,23735,27657,31937,36573,41589,46976,52754,58916,65535
    24 |  0.000675 |   0.000136 |  0.000180 | 0,219,509,993,1655,2521,3605,4920,6476,8288,10364,12716,15353,18283,21517,25062,28924,33115,37636,42500,47710,53277,59193,65535
    25 |  0.000544 |   0.000125 |  0.000166 | 0,210,483,924,1533,2322,3315,4513,5928,7581,9468,11605,14003,16660,19597,22813,26312,30116,34214,38621,43348,48385,53766,59452,65535
   *26 |  0.000449 |   0.000119 |  0.000146 | 0,202,455,864,1423,2154,3060,4156,5454,6960,8689,10637,12821,15247,17920,20855,24042,27501,31233,35247,39549,44132,49018,54208,59695,65535
    26 |  0.000464 |   0.000115 |  0.000150 | 0,203,457,867,1426,2155,3062,4159,5457,6964,8689,10640,12824,15250,17925,20855,24045,27504,31237,35259,39548,44137,49021,54211,59696,65535
    27 |  0.000483 |   0.000106 |  0.000138 | 0,194,429,812,1327,2001,2836,3842,5035,6415,8000,9786,11785,14005,16451,19134,22051,25211,28621,32289,36215,40409,44869,49603,54621,59912,65535
    28 |  0.000408 |   0.000098 |  0.000129 | 0,186,410,763,1243,1865,2635,3567,4662,5938,7388,9034,10870,12910,15157,17614,20294,23191,26324,29681,33285,37124,41214,45555,50148,55007,60114,65535
    29 |  0.000418 |   0.000091 |  0.000122 | 0,180,390,720,1166,1743,2457,3319,4333,5509,6851,8366,10060,11938,14007,16271,18737,21406,24286,27379,30689,34222,37981,41970,46195,50657,55366,60307,65535
    42 |  0.000174 |   0.000043 |  0.000056 | 0,123,246,410,627,897,1224,1612,2064,2583,3170,3826,4558,5365,6250,7212,8258,9385,10602,11901,13289,14769,16342,18005,19765,21620,23574,25630,27778,30038,32395,34859,37431,40105,42891,45785,48794,51909,55140,58486,61945,65535

My results didn’t match his mean error numbers in the 6^th decimal place, but they’re close enough that I can tell we’re using the same basic logic. As you can see, the Facebook curve stats (marked with an asterisk) do show a larger mean error, but the max error and RMSE are lower, meaning their curve is a slightly better fit overall based on this measure. Essentially, that curve has a greater overall error, but the error is distributed more evenly with less large individual errors. Their max error is also lower than the curves with more/less points immediately surrounding, which is good, but those curves weren’t optimized to minimize max relative error, so that might not be meaningful.

But actually, these numbers still aren’t the best measure of the curves’ accuracy. Because the sRGB gamma curve is intentionally very much not linear (except for that small bit at the start), a relatively small absolute error at the bottom end has a greater impact on image fidelity than a larger absolute error at the top of the curve. For example, the output value for an input of 1/255 should be 0.000304. An error of 0.000449 (the max error from the TinyRGB curve) on that value would be huge. At the top of the curve, where the output for 254/255 should be 0.991102, that same error would be insignificant. A more useful measure here would be the error relative to the correct value, not the absolute error.

Going beyond that, it’s important to understand what the curve is used for and what an error actually means as far as image fidelity. This curve is included in an ICC profile that’s meant to be embedded in images so that they can be converted to other colorspaces. Since we know the curve is going to have errors, it’s best to optimize the placement of the points so that the error has as little visual impact as possible when the image is converted.

That conversion process goes like this:

1. Convert source RGB values to Linear RGB. This is what the curve is used for. It should approximate the inverse gamma function from the sRGB spec. That’s where the errors are introduced – you can’t precisely replicate the sRGB curve with nothing but straight lines.
2. Convert Linear RGB to XYZ. This is done using the XYZ values for the red, green, and blue primaries that are also included in the profile.
3. Convert those XYZ values to Linear RGB in the target colorspace using its XYZ primaries.
4. Run that Linear RGB through the inverse of the curve in the target profile to arrive at the final target RGB values.

The simplest version of this process would be an identity transform from the sRGB-compatible colorspace to true sRGB. If everything goes right, the output values will be identical to the input.

That’s my first criterion for the curve. It must support a round-trip for every value 0-255 through the profile curve and then back through the true sRGB gamma function. If any value changes on round-trip, the curve is not sRGB-compatible.

Measuring Visual Error

The round-trip test is the absolute minimum that the curve should pass, but we can actually get a pretty good idea of the curve’s visual accuracy beyond that. Keep in mind that sRGB is a relatively compact colorspace. When converting to a colorspace with a wider gamut, a difference that might not result in an error in sRGB might throw a color off by quite a lot in a colorspace that is larger and more spread out.

I think Facebook was on the right track with their design. They mentioned validating the error in their curve by using the ΔE-CIE94 measure. That’s a measure of color difference based in the L*a*b* colorspace, which is designed to be perceptually uniform. So instead of measuring numbers from the curve output and just picking the closest ones, they actually verified that the numbers they picked got close visually to the reference values. L*a*b* is calculated directly from XYZ values, so it’s also a good test of the exact conversions that will happen when the profile is used for real.

I got the impression from their post that they tuned the curve first and then used the ΔE-CIE94 measures to make sure the final results were good enough. My plan was to integrate the visual measures into the tuning process itself, so that the results would not just be good enough, but rather would be the best possible for a given number of curve points.

To that end, I decided to take a similar but simpler approach. ΔE-CIE94 is complicated to calculate because it has some refinements to the original ΔE-CIE76 spec to deal with irregularities in the model that show up in certain hue ranges. Furthermore, to test the entire RGB space, I would have to do 16.7M comparisons (at 8-bit input precision) with that complicated calculation for each candidate curve. I realized I could simplify things greatly by working with the grey values 0-255. Since sRGB uses the same curve for all three color channels, grey is as good as any color for testing the curve.

Limiting to just the grey values allows a simpler calculation of L* since it can be directly calculated from the Y value in XYZ, and a* and b* will always be 0.  That meant I could look just at ΔL* and have a very good idea what the perceptual difference was between the reference value and the calculated value from the curve candidates.  And to make that comparison as accurate as possible, I used the ΔL* adjustments from the even-newer ΔE-CIE2000, which gives more importance to midtones, reducing the visual difference measure for very dark or very light colors.

So, to review, I ended up with three measures for evaluating and tuning the curves. In order of importance, those are:

1. The round-trip test through the sRGB gamma function
2. The ΔL* for reference vs calculated values
3. The relative error in the curve output values

I decided to keep the relative error from the curve output as a measure, because the closer the curve is to the correct sRGB gamma curve numerically, the more points can be interpolated relatively error-free. I’ll explain that more later, but basically, the round-trip test and ΔL* are best for determining the max error and tolerances, but the relative error is best for fitting the curve for points in-between.

With all that explanation out of the way, I’ll get back to the curves from Pippin’s sRGBz post. Here are the stats for those curves using the measures I described. Again, the TinyRGB curve is marked with an asterisk. And the left three error columns are now relative error instead of absolute.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
    23 |  0.039987 |   0.002466 |  0.005752 |   0.189551 |    0.017885 |   0.029098 | 1
    24 |  0.040603 |   0.002357 |  0.005641 |   0.179425 |    0.016492 |   0.027182 | 1
    25 |  0.031010 |   0.002205 |  0.005345 |   0.124504 |    0.015105 |   0.024044 | 0
   *26 |  0.034171 |   0.001978 |  0.005315 |   0.095100 |    0.014204 |   0.021270 | 0
    26 |  0.029402 |   0.002035 |  0.005077 |   0.111436 |    0.013970 |   0.022130 | 0
    27 |  0.031464 |   0.001920 |  0.004796 |   0.120256 |    0.012911 |   0.020769 | 0
    28 |  0.029564 |   0.001918 |  0.004616 |   0.104781 |    0.011936 |   0.018717 | 0
    29 |  0.028636 |   0.001729 |  0.004265 |   0.093921 |    0.011154 |   0.017872 | 0
    42 |  0.015034 |   0.000887 |  0.002183 |   0.040349 |    0.005297 |   0.008399 | 0

Using these measures, we can learn much more about the real-world usefulness of the curves. First of all, you can see that Pippin’s 23- and 24-point curves, despite having fairly low mean and RMS error values, failed the round-trip test. The Max RT Error of 1 means the pixels were offset from their correct values by a max of 1, but that’s still not good enough. Next, you can see that the Max ΔL* from the TinyRGB curve is lower than all but the two largest of Pippin’s proposed curves. Looking at the columns on the left, you can see that Pippin’s 26-point curve is a better fit to the reference curve based purely on the relative error numbers, and that makes sense given that that’s how he optimized them. He looked only at the raw numbers, while the Facebook team considered the visual impact of the numbers.

So based on that, the TinyRGB curve looks pretty impressive. It passes the round-trip test and was obviously tuned for visual accuracy. But can we do better? Of course we can :)

But first, I’ll explain one more thing. What does the ΔL* value mean in real-world terms?

The Facebook TinyRGB post said that their ΔE-CIE94 testing showed that their error level was less than half of what is perceptible to humans. Under the CIE76 definition of ΔE, a value of 1 is generally considered the minimal noticeable difference between colors, and ΔE is defined as Sqrt(ΔL*² + Δa*² + Δb*²). If we were to assume a target ΔE of 1, then knowing that our Δa* and Δb* values are always 0, we could say that the minimal noticeable ΔL* should be Sqrt(1/3), or 0.57735. However, the newer revisions to ΔE complicate things by adding a scaling factor to each color component, and ΔE-CIE2000 complicates things a bit more by adjusting the color difference so that midtones are more heavily weighted. That makes it more difficult to find a threshold value for ΔL*. I decided to do some ad-hoc testing using real grey values from the real sRGB to lend context. I calculated the minimum and maximum ΔL* for all adjacent shades of grey in 8-bit sRGB. The minimum value was 0.157124, which was the difference between grey levels 0 and 1. The max was 0.397609, between grey levels 117 and 118.

Looking back at the curves that failed the round-trip test, you can see those had max ΔL* values of 0.179425 and 0.189551, so it’s easy to imagine why they would have had values change on the round-trip. To make it easier to picture the difference, though, here’s what those greys look like. First a pair of boxes at grey values 0 and 1:

And now a pair at 117 and 118:

On my laptop, which has an above-average-quality screen, in a dark room, I can see the line between 117 and 118 quite clearly. The line between 0 and 1, I can’t really see at all. Depending on your screen, viewing environment, and eyes, you may or may not see any difference.

Based on my sample size of one (totally statistically significant – to me, ha), the minimum noticeable difference in ΔL* seems to be somewhere between 0.16 and 0.40… Let’s call it, 0.2-ish to be safe. The max ΔL* of the TinyRGB curve is right around half that, so that checks out. We’re going to do better than that by far, but I wanted to give you an idea what that number means in the real world since it was a key measurement in my testing.

As I mentioned before, I reached the same conclusion Pippin did regarding the magic of the 26-point curve. I did it by testing curves at all sizes from 16-255 and comparing them. The curves were tuned using the same measures I detailed above. The first priority was round-trip accuracy, second was to minimize ΔL*, and third was to fit the curve by minimizing the RMS relative error. This required an iterative approach to curve optimization, where certain points were locked based on their impact to ΔL* and the others were allowed to move until the best-fitting curve was found. My solver found a few interesting ones.

Show Me Those Curves

I’ll start with the smallest useable curves I was able to create.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error | Point Values
    19 |  0.041959 |   0.003564 |  0.007399 |   0.139496 |    0.026601 |   0.038015 | 0            | 0,279,753,1521,2622,4077,5920,8169,10853,13987,17596,21693,26300,31431,37102,43328,50128,57494,65535
    20 |  0.035090 |   0.003569 |  0.007435 |   0.128757 |    0.024572 |   0.035288 | 0            | 0,263,693,1387,2358,3664,5297,7296,9672,12449,15641,19264,23335,27867,32875,38371,44368,50882,57905,65535
    21 |  0.033688 |   0.003200 |  0.006765 |   0.115426 |    0.021854 |   0.031353 | 0            | 0,250,638,1263,2146,3309,4773,6557,8678,11152,13995,17221,20842,24872,29323,34206,39534,45316,51565,58276,65535
    22 |  0.033893 |   0.003218 |  0.007011 |   0.108881 |    0.020180 |   0.028803 | 0            | 0,237,594,1159,1959,3008,4325,5928,7832,10050,12598,15485,18727,22331,26312,30677,35438,40603,46183,52189,58613,65535
    23 |  0.034801 |   0.002913 |  0.006443 |   0.106786 |    0.018459 |   0.026617 | 0            | 0,227,554,1071,1798,2749,3940,5389,7106,9106,11401,14000,16917,20159,23738,27661,31938,36580,41589,46980,52759,58920,65535
    24 |  0.031175 |   0.003083 |  0.007205 |   0.089015 |    0.017114 |   0.024290 | 0            | 0,215,520,994,1657,2523,3607,4922,6479,8291,10369,12721,15358,18288,21521,25065,28928,33116,37639,42503,47714,53282,59201,65535

By only considering options that allowed the round-trip test to pass, I was able to create viable curves with as few as 19 points. You can see that each point added reduces ΔL*, though, so more is better at this stage.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error | Point Values
    25 |  0.035425 |   0.002291 |  0.005597 |   0.091829 |    0.015423 |   0.022824 | 0            | 0,210,487,926,1534,2327,3317,4515,5934,7583,9472,11610,14005,16666,19600,22815,26320,30117,34218,38626,43349,48393,53765,59459,65535
   *26 |  0.034171 |   0.001978 |  0.005315 |   0.095100 |    0.014204 |   0.021270 | 0            | 0,203,457,867,1426,2155,3062,4159,5457,6964,8689,10640,12824,15250,17925,20855,24045,27504,31237,35259,39548,44137,49021,54211,59696,65535
    26 |  0.032400 |   0.002235 |  0.005239 |   0.079740 |    0.014239 |   0.020672 | 0            | 0,201,459,866,1426,2155,3062,4159,5457,6964,8689,10639,12824,15250,17925,20854,24045,27504,31237,35249,39548,44137,49022,54211,59697,65535
    27 |  0.026077 |   0.002531 |  0.005867 |   0.069934 |    0.013876 |   0.019952 | 0            | 0,191,435,819,1329,2003,2837,3846,5037,6419,8001,9787,11788,14008,16455,19134,22052,25213,28625,32291,36218,40409,44871,49607,54624,59917,65535
    28 |  0.025899 |   0.002487 |  0.006214 |   0.063570 |    0.012585 |   0.017635 | 0            | 0,183,415,765,1245,1867,2638,3568,4666,5938,7392,9036,10873,12913,15159,17618,20296,23195,26325,29685,33286,37127,41217,45557,50152,55009,60120,65535
    29 |  0.024357 |   0.002259 |  0.005617 |   0.054125 |    0.011644 |   0.016265 | 0            | 0,177,395,723,1169,1746,2460,3321,4336,5511,6853,8368,10062,11942,14011,16275,18740,21409,24288,27381,30691,34225,37984,41974,46199,50661,55367,60310,65535

As more points are added, the ΔL* continues to go down. I was able to create 24- and 25-point curves with lower max ΔL* than the TinyRGB 26-point curve (marked with an asterisk again) as well as improve on nearly all the stats with a different 26-point curve of my own. But neither is as good as the 27 or 28 or 29, which is to say… there’s nothing special at all about 26 points.

Outside the small blip between 24 and 25 points, It wasn’t until my solver reached 32 points that it wasn’t able to continue improving with each additional point. Beyond that size, reductions in ΔL* got more difficult to come by, and curve performance was more difficult to predict. The sizes that outperform their neighbors make interesting candidates if you’re looking to optimize size/quality ratio, like you might do if you were trying to make a compact sRGB-compatible profile. Here are stats from a few such curves:

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error | Point Values
    32 |  0.018609 |   0.001701 |  0.004036 |   0.039496 |    0.009391 |   0.013111 | 0            | 0,161,345,618,985,1453,2030,2724,3539,4481,5554,6763,8115,9611,11256,13055,15012,17130,19412,21862,24484,27280,30256,33410,36750,40276,43993,47902,52005,56309,60807,65535
    42 |  0.007896 |   0.000696 |  0.001455 |   0.025409 |    0.005082 |   0.007290 | 0            | 0,124,248,412,629,899,1225,1614,2066,2584,3170,3828,4559,5366,6250,7214,8259,9388,10602,11902,13291,14771,16342,18007,19766,21622,23575,25629,27782,30038,32397,34860,37430,40107,42892,45787,48793,51911,55141,58487,61945,65535
    56 |  0.007696 |   0.000515 |  0.001255 |   0.013777 |    0.002988 |   0.004177 | 0            | 0,92,183,284,410,566,751,966,1215,1497,1813,2167,2556,2984,3452,3958,4507,5096,5729,6406,7126,7892,8704,9562,10468,11423,12425,13478,14581,15734,16940,18197,19507,20872,22289,23762,25290,26873,28513,30210,31964,33777,35647,37577,39567,41616,43727,45899,48131,50427,52785,55205,57690,60239,62850,65535
    63 |  0.003646 |   0.000347 |  0.000720 |   0.009950 |    0.002294 |   0.003162 | 0            | 0,82,163,247,350,475,623,794,990,1212,1459,1734,2038,2370,2732,3124,3547,4002,4489,5009,5562,6150,6772,7430,8124,8853,9620,10424,11266,12146,13065,14024,15022,16061,17140,18261,19422,20627,21873,23162,24495,25871,27292,28757,30267,31821,33422,35069,36762,38501,40288,42123,44005,45935,47914,49941,52018,54144,56321,58547,60824,63151,65535
   124 |  0.005790 |   0.000191 |  0.000765 |   0.003821 |    0.000682 |   0.000986 | 0            | 0,41,82,124,165,206,250,300,355,416,482,554,632,716,806,902,1005,1114,1230,1353,1482,1619,1762,1913,2071,2236,2409,2589,2777,2972,3176,3387,3606,3834,4069,4313,4565,4825,5094,5372,5658,5953,6256,6568,6890,7220,7559,7908,8265,8632,9008,9394,9789,10194,10608,11032,11465,11909,12362,12825,13298,13781,14274,14777,15291,15815,16349,16893,17448,18014,18589,19176,19773,20381,21000,21629,22269,22921,23583,24256,24941,25636,26343,27061,27790,28530,29282,30045,30820,31607,32404,33214,34035,34868,35713,36570,37438,38318,39211,40115,41031,41960,42900,43853,44818,45795,46785,47787,48801,49828,50867,51919,52983,54060,55150,56252,57368,58495,59636,60790,61956,63136,64328,65535
   182 |  0.001022 |   0.000092 |  0.000230 |   0.003107 |    0.000440 |   0.000736 | 0            | 0,28,56,84,112,140,168,196,225,256,290,326,365,405,449,496,544,597,651,708,769,831,898,966,1038,1113,1191,1273,1356,1444,1534,1628,1726,1825,1930,2036,2147,2261,2377,2499,2623,2751,2882,3017,3156,3297,3444,3593,3746,3904,4064,4229,4397,4570,4746,4926,5110,5298,5489,5686,5885,6090,6297,6510,6726,6946,7171,7399,7632,7869,8110,8356,8606,8860,9119,9381,9649,9920,10197,10477,10762,11051,11345,11644,11946,12254,12566,12882,13204,13529,13860,14195,14534,14880,15228,15583,15941,16304,16673,17046,17424,17807,18194,18587,18984,19387,19793,20206,20623,21045,21472,21904,22341,22784,23230,23684,24140,24603,25071,25543,26022,26505,26993,27487,27985,28490,28998,29514,30033,30558,31089,31624,32166,32712,33264,33822,34384,34953,35525,36105,36689,37279,37875,38475,39083,39694,40312,40935,41563,42198,42838,43483,44135,44791,45455,46122,46796,47476,48161,48853,49549,50252,50960,51674,52395,53119,53852,54589,55332,56082,56836,57598,58364,59137,59916,60700,61492,62288,63091,63899,64714,65535
   212 |  0.001650 |   0.000118 |  0.000357 |   0.002817 |    0.000449 |   0.000707 | 0            | 0,24,48,72,96,120,144,168,192,217,243,270,300,332,365,400,437,476,517,560,605,652,701,752,805,861,918,978,1040,1104,1170,1239,1310,1383,1459,1536,1617,1700,1785,1873,1962,2055,2150,2248,2348,2450,2555,2663,2774,2886,3002,3120,3242,3365,3492,3620,3753,3887,4025,4164,4308,4453,4602,4754,4908,5065,5225,5389,5554,5723,5895,6070,6248,6429,6612,6799,6990,7182,7379,7577,7780,7986,8194,8406,8620,8838,9060,9284,9512,9742,9976,10214,10454,10698,10945,11196,11449,11706,11967,12230,12498,12768,13042,13319,13599,13884,14171,14462,14756,15054,15356,15660,15969,16280,16596,16915,17237,17563,17892,18226,18563,18903,19247,19594,19946,20300,20659,21021,21387,21756,22130,22506,22887,23271,23660,24051,24447,24846,25250,25657,26067,26482,26900,27323,27749,28178,28612,29050,29492,29937,30386,30839,31296,31758,32223,32691,33165,33641,34123,34607,35096,35588,36086,36587,37092,37600,38113,38631,39152,39677,40206,40739,41277,41819,42364,42914,43468,44027,44589,45155,45726,46301,46880,47463,48050,48642,49238,49839,50443,51051,51664,52281,52903,53528,54159,54793,55432,56075,56722,57373,58030,58690,59355,60023,60697,61375,62057,62744,63435,64130,64830,65535

You can see that at 32 points, the max ΔL* is less than half that of the TinyRGB/c2 curve, which makes the increase in size well worth it. Doubling(-ish) the size to 63 points reduces error a further ~4x. Past that, it becomes increasingly expensive to make quality gains, with doubling size yielding a ~2.5x error improvement. Beyond that, it takes lots more points to improve accuracy, which peaked in the 212-point curve.

At this point, an obvious question comes up: Why even bother with 212? Why not just use a 256-point curve tag and be done with it?

Bigger Isn’t Always Better

Intuitively, one might expect that the best curve fit for an 8-bit image would have 256 points. Each point could contain the exact best output value for each input and no interpolation would be required. But look what happens when we compare a 256-point curve to the best performers from above.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
   124 |  0.005790 |   0.000191 |  0.000765 |   0.003821 |    0.000682 |   0.000986 | 0
   182 |  0.001022 |   0.000092 |  0.000230 |   0.003107 |    0.000440 |   0.000736 | 0
   212 |  0.001650 |   0.000118 |  0.000357 |   0.002817 |    0.000449 |   0.000707 | 0
   256 |  0.005447 |   0.000210 |  0.000802 |   0.004125 |    0.000646 |   0.001042 | 0

Because the curve points are stored as 16-bit unsigned integer values in the ICC profile (the ICC response16Number type), there’s a natural limit to the output precision. That limit is 1/65535, or 0.0000152902. Remember that at the bottom end of the sRGB curve, the output values are very, very small. For example, the value for an input of 2/255 should be 0.0006070540. Quantized to 16 bits, that value becomes 40/65535, which is actually 0.0006103609. That value is higher than the correct one by 0.5447%, which is the max error shown above. And there are several values with that error – I didn’t just pick the worst one. But notice the 182- and 212-point curves have much lower max errors. The same is reflected in the ΔL*. Although it’s tiny on the 256-point curve, the others still do better. Because those have fewer points, the output values have to be interpolated between two points and can actually fall between the values that would be possible to express explicitly at 16-bit precision. So, in this case, less can be more.

Carrying that further, consider the 1024-point curve used in the standard sRGB profile. Once again, I will reference Elle Stone’s site, which has a detailed survey of a variety of common sRGB profiles. She found that the majority of profiles use that same 1024-point curve. She also explains the precision issue, which she refers to as ‘hexadecimal rounding’. I call it ’16-bit quantization’. Potato, potato.

Let’s see what happens when we use that 1024-point curve to get output for 8-bit input values. And let’s see what happens if we go even bigger and use a 4096-point curve from Elle’s custom profile collection.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
   124 |  0.005790 |   0.000191 |  0.000765 |   0.003821 |    0.000682 |   0.000986 | 0
   182 |  0.001022 |   0.000092 |  0.000230 |   0.003107 |    0.000440 |   0.000736 | 0
   212 |  0.001650 |   0.000118 |  0.000357 |   0.002817 |    0.000449 |   0.000707 | 0
   256 |  0.005447 |   0.000210 |  0.000802 |   0.004125 |    0.000646 |   0.001042 | 0
  1024 |  0.008405 |   0.000205 |  0.000996 |   0.003993 |    0.000475 |   0.000819 | 0
  4096 |  0.008405 |   0.000175 |  0.000860 |   0.003054 |    0.000472 |   0.000782 | 0

You can see that the max error has actually gotten worse with the bigger curves. The reason for this is that with more points defined in the curve, their values get closer together, and the quantization/rounding error becomes more significant. If we look at the linear segment of the 1024-point curve, we can see the issue.

0,5,10,15,20,25,30,35,40,45,50,55,59,64,69,74,79,84,89,94,99,104,109,114,119,124,129,134,139,144,149,154,159,164,169,174,178,183,188,193

Notice that there’s a nice even increment of 5 between each step… except for two times where it’s 4. That uneven step hints at the fact that the slope of the line allowed by the quantization to 16 bits is not quite right. The only way to make it better is to remove points so that the slope can be represented correctly. Here is the same segment from the 212-point curve, which has even steps throughout.

0,24,48,72,96,120,144,168,192

The extra resolution in the 4096-point curve moves the error around a bit, so it manages a better ΔL* than the 1024-point, but it still trails the 212-point curve in all stats. That curve also has even more serious rounding issues that we haven’t encountered yet, because we’ve only been looking up 256 values in that curve. I’ll come back to that in a bit.

A change of direction

I must admit, I was rather surprised when I learned there were curve matches that exceeded accuracy of the standard 1024-point curve used in so many profiles.

The initial goal I had was to find a better solution than TinyRGB/c2 for a compact sRGB-compatible profile. That profile is used almost exclusively to convert JPEG images to other colorspaces, so the accuracy of its output when used with 8-bit input is the most important thing. For that purpose, the 212-point curve turns out to be the most accurate, and that might make it perfect for image embedding if you don’t mind its size, which comes out to 796 bytes in a minimal profile packed using the technique I described in my last post. That’s about a quarter the size of the standard sRGB profile, with increased accuracy – a true win/win. But there’s a reasonable case to made for a smaller profile as well, especially for thumbnail-sized images. If you have a 4KB JPEG, even 796 bytes for the profile seems heavy. There is, therefore, a need for a smaller profile as well, and I can improve on TinyRGB significantly with just a few more curve points.

I’ll get back to the curves I picked for my compact sRGB-compatible profiles later, but the accuracy of the 182- and 212-point curves got me wondering whether they might also work better as a target profile than the standard sRGB profile does or whether they might be appropriate for higher-bit-depth images. I decided to test them again, using more input samples this time. I discovered that the tuning I had done to optimize for 8-bit input hurt the overall fit of the curves a tiny bit, so they didn’t give quite as good results with more samples. So, I ran them through my solver one more time and asked it to tune for 1024 samples instead of 256. There was a very slight drop in their 8-bit accuracy after that was done, but the curves continued to perform well. And their performance at higher resolution beat everything.

Numbers, Numbers, Numbers

With the final set of interesting curves identified, I set out to do comprehensive comparisons. There are lots of numbers here, so feel free to skip this section if you’re the type whose eyes glaze over when they see too many numbers.  Come back for the conclusions and the final profiles, though.  They’ll be interesting, I promise.

Here are the 8-bit results again for my final set of interesting curves, compared with the standard 1024- and 4096-point curves as well as the TinyRGB curve (again with the *).  I have marked the refined 182- and 212-point curves with a caret(^) for comparison with the initial 8-bit tuned ones.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
    19 |  0.041959 |   0.003564 |  0.007399 |   0.139496 |    0.026601 |   0.038015 | 0
    20 |  0.035090 |   0.003569 |  0.007435 |   0.128757 |    0.024572 |   0.035288 | 0
   *26 |  0.034171 |   0.001978 |  0.005315 |   0.095100 |    0.014204 |   0.021270 | 0
    26 |  0.032400 |   0.002235 |  0.005239 |   0.079740 |    0.014239 |   0.020672 | 0
    32 |  0.018609 |   0.001701 |  0.004036 |   0.039496 |    0.009391 |   0.013111 | 0
    42 |  0.007896 |   0.000696 |  0.001455 |   0.025409 |    0.005082 |   0.007290 | 0
    56 |  0.007696 |   0.000515 |  0.001255 |   0.013777 |    0.002988 |   0.004177 | 0
    63 |  0.003646 |   0.000347 |  0.000720 |   0.009950 |    0.002294 |   0.003162 | 0
   124 |  0.005790 |   0.000191 |  0.000765 |   0.003821 |    0.000682 |   0.000986 | 0
   182 |  0.001022 |   0.000092 |  0.000230 |   0.003107 |    0.000440 |   0.000736 | 0
  ^182 |  0.001072 |   0.000102 |  0.000244 |   0.004540 |    0.000516 |   0.000885 | 0
   212 |  0.001650 |   0.000118 |  0.000357 |   0.002817 |    0.000449 |   0.000707 | 0
  ^212 |  0.001650 |   0.000119 |  0.000361 |   0.003521 |    0.000475 |   0.000743 | 0
   256 |  0.005447 |   0.000210 |  0.000802 |   0.004125 |    0.000646 |   0.001042 | 0
  1024 |  0.008405 |   0.000205 |  0.000996 |   0.003993 |    0.000475 |   0.000819 | 0
  4096 |  0.008405 |   0.000175 |  0.000860 |   0.003054 |    0.000472 |   0.000782 | 0

The changes to the 212-point curve put its ΔL* right between the 1024- and 4096-point curves, so I would still consider it a no-brainer replacement for the standard 1024-point curve.  The 182-point curve fared worse in ΔL* but is still quite good, and it has the best overall fit based on RMSE.

Now look what happens when we increase to 10-bit interpolation (1024 input samples)

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
    19 |  0.042879 |   0.003594 |  0.007416 |   0.156134 |    0.026705 |   0.038168 | 2
    20 |  0.037908 |   0.003614 |  0.007497 |   0.134980 |    0.024678 |   0.035373 | 2
   *26 |  0.034650 |   0.001994 |  0.005349 |   0.118553 |    0.014295 |   0.021400 | 2
    26 |  0.032418 |   0.002248 |  0.005280 |   0.101960 |    0.014276 |   0.020789 | 2
    32 |  0.019770 |   0.001742 |  0.004119 |   0.057270 |    0.009457 |   0.013254 | 1
    42 |  0.010831 |   0.000711 |  0.001501 |   0.034022 |    0.005139 |   0.007341 | 1
    56 |  0.007831 |   0.000521 |  0.001264 |   0.020659 |    0.002989 |   0.004236 | 0
    63 |  0.005564 |   0.000353 |  0.000767 |   0.016122 |    0.002293 |   0.003257 | 0
   124 |  0.005790 |   0.000203 |  0.000804 |   0.006320 |    0.000701 |   0.001046 | 0
   182 |  0.001697 |   0.000111 |  0.000265 |   0.006673 |    0.000587 |   0.000977 | 0
  ^182 |  0.001478 |   0.000110 |  0.000260 |   0.004644 |    0.000561 |   0.000932 | 0
   212 |  0.002159 |   0.000130 |  0.000379 |   0.004728 |    0.000527 |   0.000802 | 0
  ^212 |  0.001883 |   0.000129 |  0.000379 |   0.003708 |    0.000501 |   0.000774 | 0
   256 |  0.005447 |   0.000200 |  0.000782 |   0.005247 |    0.000608 |   0.000980 | 0
  1024 |  0.008405 |   0.000240 |  0.001044 |   0.004104 |    0.000617 |   0.000993 | 0
  4096 |  0.008996 |   0.000224 |  0.001054 |   0.003897 |    0.000506 |   0.000853 | 0

The refined 212-point curve outperforms everything else. And notice that the smaller curves are starting to show round-trip errors at this sample resolution.

Next up, I’ll test them at 12-bits (4096 input samples)

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
    19 |  0.043820 |   0.003601 |  0.007422 |   0.162734 |    0.026727 |   0.038190 | 8
    20 |  0.038794 |   0.003622 |  0.007506 |   0.141046 |    0.024697 |   0.035393 | 8
   *26 |  0.034660 |   0.001997 |  0.005351 |   0.119456 |    0.014308 |   0.021410 | 7
    26 |  0.032431 |   0.002252 |  0.005284 |   0.102827 |    0.014287 |   0.020800 | 6
    32 |  0.019606 |   0.001746 |  0.004125 |   0.056738 |    0.009444 |   0.013208 | 4
    42 |  0.010970 |   0.000712 |  0.001500 |   0.034507 |    0.005144 |   0.007346 | 2
    56 |  0.007827 |   0.000522 |  0.001265 |   0.022992 |    0.002991 |   0.004235 | 1
    63 |  0.006099 |   0.000353 |  0.000767 |   0.015857 |    0.002288 |   0.003245 | 1
   124 |  0.005790 |   0.000205 |  0.000812 |   0.006420 |    0.000701 |   0.001047 | 1
   182 |  0.002016 |   0.000112 |  0.000266 |   0.008137 |    0.000588 |   0.000983 | 0
  ^182 |  0.001482 |   0.000110 |  0.000261 |   0.005065 |    0.000561 |   0.000936 | 0
   212 |  0.002439 |   0.000130 |  0.000381 |   0.005398 |    0.000528 |   0.000802 | 0
  ^212 |  0.001904 |   0.000129 |  0.000381 |   0.003735 |    0.000502 |   0.000775 | 0
   256 |  0.005447 |   0.000202 |  0.000787 |   0.005244 |    0.000610 |   0.000981 | 0
  1024 |  0.008405 |   0.000222 |  0.001025 |   0.003972 |    0.000508 |   0.000852 | 0
  4096 |  0.192685 |   0.000376 |  0.004745 |   0.004194 |    0.000628 |   0.001014 | 0

Look what’s happened with the 4096-point curve. Now that we’re using all of its points, we can see it’s got a serious flaw. Its max error has jumped way up, and its ΔL* is now worse than the 1024-point curve’s. It’s easy to see why. Have a look at its values for the linear part of the curve:

0,1,2,4,5,6,7,9,10,11,12,14,15,16,17,19,20,21,22,24,25,26,27,28,30,31,32,33,35,36,37,38,40,41,42,43,45,46,47,48,50,51,52,53,55,56,57,58,59,61,62,63,64,66,67,68,69,71,72,73,74,76,77,78,79,81,82,83,84,85,87,88,89,90,92,93,94,95,97,98,99,100,102,103,104,105,107,108,109,110,111,113,114,115,116,118,119,120,121,123,124,125,126,128,129,130,131,133,134,135,136,137,139,140,141,142,144,145,146,147,149,150,151,152,154,155,156,157,159,160,161,162,164,165,166,167,168,170,171,172,173,175,176,177,178,180,181,182,183,185,186,187,188,190,191,192,193,194

Again, the problem is apparent. The steps are uneven, alternating between 1-1-2 and 1-1-1-2 patterns. At that resolution, the 16-bit quantization is making it impossible to get the correct slope for the linear part of the curve, which is why the max error jumped up to over 19%. The 212-point curve is still looking outstanding, by the way. And the smaller curves are showing even larger round-trip errors.

And finally, let’s see what it looks like if we interpolate all possible 16-bit samples (65536 of them) with these curves.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL | Max RT Error
    19 |  0.044230 |   0.003603 |  0.007423 |   0.162631 |    0.026733 |   0.038193 | 135
    20 |  0.039190 |   0.003625 |  0.007508 |   0.141160 |    0.024702 |   0.035397 | 123
   *26 |  0.034661 |   0.001997 |  0.005352 |   0.120915 |    0.014312 |   0.021413 | 114
    26 |  0.032431 |   0.002254 |  0.005286 |   0.104229 |    0.014291 |   0.020803 | 98
    32 |  0.019766 |   0.001749 |  0.004129 |   0.057259 |    0.009446 |   0.013210 | 65
    42 |  0.011194 |   0.000712 |  0.001501 |   0.035285 |    0.005145 |   0.007347 | 36
    56 |  0.007860 |   0.000522 |  0.001265 |   0.023274 |    0.002991 |   0.004235 | 24
    63 |  0.006172 |   0.000353 |  0.000768 |   0.016116 |    0.002288 |   0.003246 | 18
   124 |  0.005790 |   0.000206 |  0.000815 |   0.006580 |    0.000701 |   0.001047 | 9
   182 |  0.002045 |   0.000112 |  0.000266 |   0.008139 |    0.000588 |   0.000983 | 7
  ^182 |  0.001482 |   0.000110 |  0.000261 |   0.005133 |    0.000562 |   0.000936 | 5
   212 |  0.002560 |   0.000131 |  0.000382 |   0.005650 |    0.000528 |   0.000803 | 7
  ^212 |  0.001905 |   0.000130 |  0.000381 |   0.003738 |    0.000502 |   0.000775 | 5
   256 |  0.005447 |   0.000203 |  0.000789 |   0.005248 |    0.000611 |   0.000981 | 6
  1024 |  0.008405 |   0.000223 |  0.001028 |   0.004089 |    0.000509 |   0.000853 | 6
  4096 |  0.192685 |   0.000324 |  0.004697 |   0.004178 |    0.000497 |   0.000820 | 6

At this sample resolution, none of the curves pass the round-trip test, but you can see that, once again, the refined 212-point curve shows the least visual error. This test also reinforces the validity of the ΔL* measure. The max round-trip error is predicted by and follows the ΔL*. sRGB is not quite as perceptually uniform as L*, so it’s not a 100% match, but it’s a very good predictor of what will happen as the sample resolution increases. A difference of 6/65335 (0.000092) is most certainly not going to be visible, but if you can drop that error to 5/65335 and save over 1.5KB off the ICC profile size at the same time, that’s a no-brainer.

And that just left one question to answer before I could wrap up my curve testing. What would happen if you used these curves in a target profile rather than a source profile? With a source profile, you can predict exactly which values will be looked up or interpolated from the curve, because those values are defined by the bit-depth of the image. 8 bits means exactly 256 values can be looked up, etc. That’s what we tested.  With a target profile, however, the curve is used in reverse.  Output values become input values and vice-versa.  And the input values become unpredictable. They could be any floating-point number between 0 and 1. So that left me with one test to run.

For these final numbers, I generated a set of 1 million random floating-point numbers between 0 and 1, and interpolated the output values for them.  The round-trip test becomes meaningless in this case because you can’t round-trip a random number, but the rest of the numbers can be interpreted the same as before.

Points | Max Error | Mean Error | RMS Error | Max DeltaL | Mean DeltaL | RMS DeltaL
    19 |  0.044252 |   0.003601 |  0.007417 |   0.162732 |    0.026701 |   0.038136
    20 |  0.039191 |   0.003632 |  0.007520 |   0.141245 |    0.024726 |   0.035427
   *26 |  0.034661 |   0.002002 |  0.005362 |   0.120960 |    0.014326 |   0.021432
    26 |  0.032431 |   0.002261 |  0.005299 |   0.104273 |    0.014305 |   0.020825
    32 |  0.019764 |   0.001752 |  0.004138 |   0.057252 |    0.009441 |   0.013209
    42 |  0.011204 |   0.000714 |  0.001503 |   0.035317 |    0.005147 |   0.007351
    56 |  0.007860 |   0.000522 |  0.001263 |   0.023292 |    0.002987 |   0.004229
    63 |  0.006174 |   0.000354 |  0.000770 |   0.016113 |    0.002292 |   0.003253
   124 |  0.005790 |   0.000207 |  0.000819 |   0.006588 |    0.000700 |   0.001046
   182 |  0.002045 |   0.000112 |  0.000267 |   0.008148 |    0.000588 |   0.000984
  ^182 |  0.001482 |   0.000110 |  0.000261 |   0.005147 |    0.000561 |   0.000935
   212 |  0.002566 |   0.000131 |  0.000382 |   0.005663 |    0.000528 |   0.000803
  ^212 |  0.001905 |   0.000130 |  0.000382 |   0.003738 |    0.000503 |   0.000776
   256 |  0.005447 |   0.000204 |  0.000793 |   0.005248 |    0.000611 |   0.000981
  1024 |  0.008405 |   0.000225 |  0.001035 |   0.004100 |    0.000509 |   0.000853
  4096 |  0.192685 |   0.000331 |  0.004806 |   0.004190 |    0.000497 |   0.000821

And the results are just about the same as before. So that does it… I’m convinced that my refined 212-point curve is not just the best fit for 8-bit image conversion – I believe it’s the best overall fit possible for the sRGB gamma curve within the restrictions of the v2 ICC profile format. I call it the Magic Curve, natch.

For a space-saving curve, any of those options between 32 and 63 points would be a huge improvement over Facebook’s 26-point attempt. I’ll be making a few size-conscious profile options with those and testing them out.

And the smallest usable curve is really 20 points. Although the 19-point curve was also valid according to the 8-bit round-trip test, it’s kind of pointless because an odd number of curve points means that the ‘curv’ tag has to be padded by 2 bytes to maintain alignment. You may as well include the extra point if it helps accuracy – and it does in this case. I’ll make what I believe to be the smallest possible sRGB-compatible profile (410 bytes) using that 20-point curve. Note that it is worse than the TinyRGB curve in terms of accuracy, but it’s not as much worse as the 32-point curve is better. Which is to say, once again, the 26-point curve is not at all special in its size/accuracy ratio.

Check the final post in this series for details on those profiles, some real-world tests using them, and of course, download links.  In the meantime, I have some investigation to do regarding the XYZ color values used in sRGB profiles.  That topic turned out to be another tricky one.