And Zawar, the stage is yours. Okay.
Hi, everyone. I'm Zawar. And he's right. I will be talking about color management.
I've been working on Quinn's color management support for quite a while now.
So I am qualified to talk about it a little bit.
Color is a very big and complicated topic.
So I will be skipping a lot. I will not be talking about like biological or
psychological bits about color but mostly with a focus on practical matters
on what we should do in our software,
so let's start with a question what even this color and let's dumb it down a
bit and start at the kindergarten level.
What's this color? Red.
But if that's red, then what's that color?
It's also red, but just a bit darker.
But if these are both red, what's What's that third color?
Yes, it's all kind of red, so using these names to talk about colors is maybe acceptable,
if you're an artist and you mix your colors to paint on a canvas,
but in computers we can't work with just a bunch of random names.
We need cold hard numbers.
You might already be familiar with the rgb system if you've used cue color in your application,
you probably already input some of these in your app and these are the values
that i chose with the color picker the first one is a hundred percent red second
one is fifty percent red and the third is a hundred percent red with a bit of
green and blue mixed in in.
But then we're still at the same original question.
Well, which red are we talking about? Which green? Which blue?
These are nice, but they're not objective.
Luckily, around a hundred years ago,
some scientists already solved this problem for us before it even became relevant
for computers and they did a bunch of experiments with lasers.
Lasers are awesome like just in general
but also for colors for colors they're especially great because you have a single
wavelength of light it's one pure color and it's objective.
What they came up with is the XYZ color space. So no red, green, or blue.
This image is not the XYZ color space. It only has two axes.
It is derived from XYZ and it's called XYY.
And the second Y is just brightness and we're ignoring that for now.
You can see that we have a coordinate grid here where we can just specify an
X and a Y and then we get an objectively defined color.
In this coordinate system, you can pick two points and in the middle is the
color that you get out if you mix these two other colors 50-50.
If you mix them as light, that is, paint is very different, and we're not talking about that at all.
The round shape around the color, those numbers are the wavelengths of light.
So, this is great, we now have actual numbers to work with.
And the color space that we actually use in programs most of the time is called sRGB.
In this triangle, we have the actual real definitions for what red, green and blue mean.
It's just that red corner, that green corner and that blue corner.
Now, there's a fourth point in here that is relevant in the middle of the triangle.
We see the D65 point and that's called the white point.
It is quite literally the point that you consider to be white.
I have an example here for why we need to care about it.
I originally wanted to use some paper and some LEDs to show it,
but then my cat jumped on my lap and didn't allow me to do any work.
So I used her as the example.
On the left, you can see an image that I took with my phone set to automatic
settings with a light that's pretty like cold color temperature,
So similar to what these lights in here are.
And then I took the second photo with the same camera settings,
but I turned on the light that I use in the evening, which has more of an orange tone.
And the result isn't very surprising, right?
You shine an orange light on a white fur and it looks orange.
What this picture doesn't tell you is that for me in the room,
she didn't look orange at all.
Because our brains adjust to the light that we are surrounded by.
The brain goes, oh, I know this is supposed to be white.
And it shifts your whole color vision around until white actually looks roughly white again.
So that's where we need to specify what the white point is, otherwise it's pretty...
All the other colors are meaningless.
Now this is again like a wishy-washy explanation that doesn't really translate to numbers.
In our RGB system this just means that red,
green and blue all at a hundred percent doesn't actually make all the three
primary colors the same brightness they have a different maximum brightness.
And if with sRGB, if you emit red, green and blue with these intensities at
100 percent, then you get that D65 white point.
So now we've defined red, green, blue and even white.
So we're done, right? We have our RGB numbers and the talk is over.
Well, it's of course not that simple in practice. In practice, sRGB is a lie.
So sRGB is what all the content you generally find on the internet uses.
And it's meant for computer displays, but it's relatively old by now.
And we can both not always perfectly make displays that have exactly the sRGB
colors. colors and we don't necessarily want to.
I have measured my old laptop display and my gaming monitor's colors and you can see,
that on the left side the laptop display pretty much exactly follows sRGB.
Like this dotted line that's sRGB
and like red's a bit off greens a
bit off but it's very close you don't notice the difference if you would look
at it side by side but then there's the gaming monitor and red's just completely
off and green is way out there blue is still pretty much correct.
What?
Not quite. It's just an HDR gaming monitor that does whatever.
No, like this is the measurement from. Is this a mode or something?
No, not in this case. Like this is some other image mode.
And if you watch
if you show an
sRGB image on that second monitor it will
happily show you colors but green
will be that green and not the one
we defined before so green
and red look more intense on that screen which in
many cases if you like that that's fine but if you then need to create content
for displays that are actually sRGB then you have a problem because what you
see will not be what someone else sees when they watch it on their screen.
The second reason that sRGB isn't great is that it's it only covers a small
part of the entire range of human vision.
Displays still can't reproduce all of the right side here, which is the Rec.
2020 color space that's used for HDR content, but they can do a significant fraction of it.
And we want to use that color. If you watch a movie, you don't want to be limited
to a small subset of colors, But if you can actually show what the real colors in the real world do,
then you can make the movie look a lot more lifelike.
To deal with all these differences in colors we need to use color management.
I left out the actual math you don't need to be scared about it if you want
to know about it you can look it up it is not that complicated but basically,
we use linear rgb values in our source color space I will talk a bit more about the linear part later.
And we multiply that with a matrix that converts from sRGB to that objective
independent x, y, z color space.
Then we have x, y, z values and we multiply those by a matrix that goes from
x, y, z to our target color space.
And then you're done. then you have RGB values that on the display show the
same color as the sRGB content actually wanted to show and of course this also
works in the opposite direction if we have some HDR content we just,
calculate a matrix that goes from the rec 2020 color space to XYZ and then another
that goes from XYZ back to sRGB.
That second case is a little bit more complicated though, because Rec.2020 has
a lot of colors that an sRGB display just cannot show.
To deal with that, we need to use gamut mapping.
This, I will not go into the details of how you do it, because the current implementation
in QWIN isn't that great, and I don't know that much about it, like how to...
There's a lot of different approaches, and it would take a long time to go through
all of that, and it involves a lot of math.
But basically, we need to do some mapping of all the colors us that we can't
show on our target display and move them inside of that triangle that we can actually show.
So now we kind of covered all the very basics about color, at least in practical terms.
Like this, like I said, there's a lot of biological and
psychological things and a lot of other are not RGB color spaces that we generally
don't care about that much on the compositor side at least and that I don't know that much about,
but we left something out before in the X Y Y color space that segment Y it's
about brightness and we can't have color without brightness I mean it would just be black.
So let's talk about it.
SRGB uses the sRGB EOTF, the electro-optical transfer function.
That's a very fancy word for saying it's a function where you put in the RGB
value and you get out an actual brightness value.
Srgb displays just take your value and take it to the power of 2.2 and then
you have a resulting value from 0 to 1.
So that's relative to the maximum brightness of your display it's a pretty simple
situation the reason that we use this instead of of just the value directly
corresponding to the brightness we want to show is that we humans,
we don't see brightness in a linear way.
We can see a lot more shades in the darkness than we can see in the bright areas of an image.
So this basically compresses the image data and we can use fewer bits to represent
the data in a way that we can't see steps in the brightness.
There is, however, a second transfer function that is pretty widely used,
and it's the perceptual quantizer.
I didn't put the equation in here because I was too lazy to type it all out.
It's a lot bigger than sRGB,
but it basically just allocates even more of the range to darker values and
that follows our brightness perception a lot more closely.
So we can compress the data better.
But it also does something else very differently. If you look at the Scala,
this doesn't go from 0 to 1 to an output of 0 to 1, but it outputs 0 to 10,000.
Now this is in nits or candela per square meter it's just a unit for brightness
you might know that from some displays,
and.
What you can do with that is HDR.
Now, HDR is pretty much a marketing term that doesn't mean that much.
Applications like to slap the label on every display they can find.
Like, this is HDR, so we can charge you more money for it, even though it doesn't
really do a good job at it.
But basically, the perceptual quantizer defines,
an sdr brightness that black line there
of 203 nits so
that's a very small part of the whole output range
and that sdr white is basically the normal brightness the if you go outside
and you see the street and a tree like those would be roughly sdr brightness and then you have,
the sky with clouds that are a lot brighter and you have the sun that's just
insanely bright and you can use that rest of the value range to represent that in your image.
In practice this means that some displays need to be able to go brighter than,
their normal brightness in some areas of the screen.
Now, here we have the same problem as with color.
Most displays can't actually do HDR.
So if an image has this transfer function, we have a lot of data that we just
cannot represent on our screen.
To do that, we again need to do some color management. In this case, we do tone mapping.
All the brightness values that are above the level that is the SCR white need
to be mapped down, so you don't just see a white blob like in the first image.
This is an example of Quinn's tone mapping algorithm. It is not quite perfect
yet, but I would say it does a decent job.
Instead of the clouds that are just limited to the maximum brightness that we
can see, being just a giant white blob, you can actually see some details in the lower image.
So all of this is very nice you now have a very rough idea hopefully of,
what all the color and brightness things means but what does it actually matter for your applications.
What do you do in practice because so far most of you will have probably just
used srgb numbers in your applications, and that mostly works out fine, right?
And the simple answer is, if your app only needs sRGB, you don't do anything.
On Wayland. On X11, things are complicated and messy, and you're on your own
if you want to do color management on X11.
But on Wayland, Quinn assumes, if your application doesn't tell the compositor
that it's doing anything fancy with colors, then it's just using sRGB like most applications are.
So Quinn takes that image from your application,
looks at the display, looks at the display settings that the user has configured,
if they have a color profile or if we're using just the data from the display,
and it applies that color profile profile, it does all the color and brightness
transformations and the resulting colors should be mostly correct.
Not all displays do an amazing job at this, but if you calibrate it with a color
profile, the result should be good.
If your app needs more than just the basic srgb though you need to use the wayland
color management protocol with this you can tell quinn you are using the rec 2020 color space,
you are using the perceptual quantizer transfer function instead of just srgb
be you can tell it that you're using like exactly what the display already does
and Quinn will use that information in,
its transformation to the display and it will ensure that the resulting colors
will be correct again now you can also query the compositor for what the display
can actually do I give your application needs to do some more fancy things.
You can get information about what color space it does, how bright it goes,
how much HDR range you have, what the brightness setting is, a lot of good stuff.
Now I also need to say that technically speaking, the color management protocol is still not done yet.
In Plasma 6.1, you need to set this environment variable to make Quinn actually
support the protocol otherwise you just can't use it in git master it is enabled
by default so you can just directly start using it,
of course dealing with Wayland might
not be very simple especially if
you don't know anything about it but I have a small test app that you can just
copy paste code from if you like or you can use it on your computer to see what
kind of color space your display can actually do.
If you just want to look at
some pretty HDR videos or play HDR games
there's some earlier work that you can just use right now without needing to
do a lot of additional work you can start games in game scope with dash dash
hdr dash enabled and it will just work,
and for videos you need to do a bit more you need to install this vulkan layer
and run mpv with this long command all of that will be sorted out in time so
that it just works you double click a video file and you get HDR or SDR or whatever the video does.
Now, last but not least, I want to promote this repository.
It has a lot of deeper explanations, a lot of links and discussions to color
management, HDR, and Wayland color management things.
And it helped me a lot to understand a bunch of these things.
So if you are
interested in knowing more about color
management than this very superficial overview you
can use that to get that information and go
through it at your own pace yeah and
that's really it and now we have a lot of time for questions Thank you for your questions.
Yeah, thank you. And Paul Ketchans?
Is there a reason why the Rack 2020 color space excludes so much on the left
side of the coordinate space? space.
Like it looks like it tries to fill the entire space except at the left.
So.
So I don't know the details of how this color space was developed,
but I assume it's just a lot easier to get that actually into a display.
A color space isn't very useful if most of the colors in it can't actually be shown on a display.
Color, because of our nature, it has more of this green.
So it's just theory, but I believe it's because we're more sensitive to the green color.
Pretty likely. I have a question about how the HDR color is represented in images.
So, for example, in a mobile phone, we can make a photo with HDR and without
HDR, but actually we have the same RGB numbers, right?
Or HDR images contain additional numbers on top of RGB?
Could you repeat that question? So when we make a photo, for example,
on a mobile phone, we can enable HDR, but it produces the same image,
but with different colors.
And the colors are represented it as RGB from 0 to 255 or HDR use additional
numbers to store the colors?
So in general, HDR doesn't need to use a different amount of bits, like you can...
My HDR monitor at home only does 8 bits, so 0 to 255.
But it is usually recommended that you use at least 10 bits of resolution because
while the PQ transfer function is a lot more efficient,
you still have a very steep curve on the bottom and on 8 bits you may be able
to see some banding, some brightness steps in it,
but it's not strictly necessary.
So it's less of a question i just
want to say thank you for putting work into
this and also for bearing with me in
all these cultures i know it's not been it has not been fun thank you for putting
in all that time so if i understood you right you focused in this talk mainly on what,
Quinn currently does with HDR content and mapping it then to sRGB.
You talked about the transfer function and I guess sort of a static tone mapping implementation.
But there's also systems and content that define their own transfer functions
and tone mapping functions that are.
Parameterized by metadata in the content like per scene or per frame like Dolby
Vision or HDR 10 plus and stuff like that.
How far are we from supporting that or maybe
we already do and how does it click together so on
Wayland you can change your HDR metadata
per frame so you can tell Quinn
this frame has a lower maximum
brightness so it can adjust its tone mapping function if you
have a special format like HDR gain
maps where you have two images and you need to compute them together there's
currently at least no support for that in the Wayland protocol you have to do
that computation on the application side but it could be added at some later
point if it brings us benefits thank you.
Test test test test so it's probably
a little bit too technical to put inside your talk but if I Right.
Remember correctly, if I have a color managed GUI application,
doesn't the surface that has to be color managed be like a literally different
surface or else it messes up the UI, which is all encoded in sRGB, right?
It depends. If you do some color management yourself, you can transform the
colors of that UI to the color space that you put the whole window in.
So you can make it work in the application. Of course, it is more convenient,
especially with HDR content, if you can just put it on a separate surface and
let the compositor just deal with it.
Have enough time for any other questions?
So for the future, as far as I understand, this is for content like images and videos.
Would that evolve into the Plasma UI itself being capable of better colors in
capable displays or something like that?
So it is possible. We could make that happen with some changes in Qt so that
it supports all the color management bits itself.
Right now, I don't see a huge need for it because sRGB works fine for normal UI stuff.
But if we want to have like a plasmoid on your desktop that shows an HDR image,
it would of course be nice to actually make that happen.
Yeah my question kind of follows up
on that one because you you skipped
the parts about like psychological effects
of colors and then i'm assuming you
have maybe read a bit about it and i was
wondering if you heard anything that maybe we
should try to to use for normal applications colors which no other applications
are using like we should try to go beyond srgb and maybe then all the KDE apps
look more colorful than anyone else.
Has a slider in the
display settings if your monitor is
in HDR mode but I intend to expand that
to SDR mode as well where you can say
that you want just all SDR applications to be more intense because not everyone
cares about correctness and maybe you want your colors to look just a lot more
intense right but you could make that happen like it's a few lines of code to,
make all KDE applications like a few lines of code in each app to make the colors
more intense by just scaling all the colors,
yeah so the question was also if you know if we should do this if you have seen
anything about psychological whatever so I know that many people just.
Assume brighter more intense colors looks
better and that's a
matter of like personal preference I think
in many cases it is good to have like this normal background of sRGB colors
so that when you watch an HDR video you actually notice the difference but in terms of marketing,
using HDR capabilities of the display certainly can have an appeal.
So yeah, that's something that we should maybe discuss further.
I think I don't, uh, okay.
Sometime ago I was, um, calibrating a monitor for Pentium Colors and, uh,
I I noticed that it required the brightness to be at maximum in order for the calibration to work,
which led me to realize that apparently for certain calibrations,
they rely on certain specific brightness and they might not work if the brightness
is lower than a certain threshold.
Hold my question will then
be will it be possible to
have something like a a calibration
that works for multiple brightnesses maybe
a function based on that yes so icc
profiles as far as i know don't have any standardized or even experimental functionality
for that but we can add non-standard tags and i want to make that happen yes
because it's very useful to know,
what exactly the backlight of your display does when you set it to 50.