Exploring how the colour sensor works under the hood: Light intensity values collected from the photodiodes, which are then averaged out and normalised to RGB.
Tonight's rabbit hole has been figuring out how *exactly* the colour sensor works. It lit up the happy parts of my brain 🥰
On the back, there's a tiny square chip surrounded by lights. The lights illuminate the object evenly no matter the light conditions, and the light bounces from the object back to the sensor.
The sensor is a tiny square, but is made up of lots of little photodiodes. Each photodiode is covered with an optical filter which limits what light it can receive. Each one can detect the intensity of light that is 🟥 red, 🟩 green, 🟦 blue or ⬜️ clear.
This is what the colour sensor does behind the scenes to detect colour:
1. Step one is to collect the raw light intensity value for each photodiode. The raw value can be anywhere between 0 and 65535, which is max light intensity value for a 16 bit sensor like this one.
^ There are many photodiodes of each type, e.g. 5 x red, 5 x blue, 5 x green, 5 x green dispersed evenly across the chip.
2. Then it averages out the signals: (red1 + red2 + red3 + red4 + red5) divided by total number of photodiodes. That way we get a good representation of the colour from sensors in different parts of the chip.
🟥 = 29875
🟩 = 14750
🟦 = 4950
⬜️ = 4600
3. Then it converts these to RGB (red, green, blue) color values. RGB is a standard colour format like hex. The colour scale is 0 to 255, indicating the intensity of light (weak to strong).
The formula for normalising the colours from the 16-bit signal to RGB is:
Normalised_value = avg_color_signal / max poss 16 bit value x max poss rgb value.
🟥 = (29875 / 65535) x 255 = 116.2 (rounded to 116).
🟩 = (14750 / 65535) x 255 = 57.4 (rounded to 57).
🟦 = (4950 / 65535) x 255 = 19.2 (rounded to 19).
4. Then, it uses the clear value to adjust the colour to account for brightness (light or dark). The formula for that is:
adjusted_colour = normalised_value / (clear_avg_value / max 16 bit value).
🟥 = 116 / (46000/65535) = 165.
🟩 = 57 / (46000/65535) = 81.
🟦 = 19 / (46000/65535) = 27.
The final RGB value returned by the colour sensor is 🟧 165, 81, 27. We can do what we want with that by programming the microcontroller (arduino) that receives this info.
Woo hoo! 🥳
Next challenge is to figure out range of colours to accurately figure out which buckets a skittle should be sorted into.
What RGB colour range represents:
🟡 🟠 🔴 🟣 🟢
^ I can see why the skittle sorting robot in tutorial struggled to differentiate between red and orange, but I think understanding the math now will make this solvable instead of needing to use a camera as I first assumed.
