DIY spectrometer (AS7341 based)

42 posts / 0 new
Last post

Pages

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568
DIY spectrometer (AS7341 based)

The past week I’ve been playing around with a colour sensor in an attempt to make something that hopefully resembles a spectrometer. The intention is to make something that can give relatively reliable info on CCT, Duv and possibly light quality (CRI/TM-30 data). The idea is to document any progress here. You guys might have questions, so ask away. And I’m sure I’ll need some answers as well. Hardware/software I know fairly well, but I’m new to the whole calculating light quality/properties and there are people on this forum who have such an insane amount of knowledge… hoping some of that rubs of and we all learn something along the way.

So first some background. After seeing all those Opple Light Master measurements popping up, I thought of getting one as well. But alas all sold out and expected return date was May. Wait till May and still nothing. A quick inquiry resulted in an updated date of August. Not the answer I as hoping for, but by now it is known what sensor they use (a Reddit user did a quick tear down). Turns out it uses the AMS AS7262 – Consumer Grade Smart 6-Channel VIS Sensor (https://ams.com/en/as7262). Checking the AMS website reveals that one is out of stock and deprecated. But fortunately they have a new an improved sensor; an 11-channel sensor. 11 channels must be better than 6 right?! And thus this topic was born Big Smile

On to the hardware; the whole thing is based on the AMS AS7341. This is an 11-channel spectral color sensor (https://ams.com/as7341). First I needed to actually get the sensor. Not wanting to start from scratch, I started looking at dev boards. There’s an official dev kit, but that costs an arm and a leg. There are some boards on AE, but the big electronic components distributors also sell some. Ended up going for the Adafruit AS7143 (https://www.adafruit.com/product/4698). It seemed decent, has modern USB-C, is documented and uses a decent LED apparently (not needed for flashlight testing though). Being in the EU, Mouser was the best way to buy, provided some extra stuff was ordered as well to get free shipping from the US. Much faster than AE too!

Now on to the sensor; it communicates via I2C. There are many ways of doing that, but as it turns out, Adafruit also makes a USB to I2C board based on the MCP2221 (https://www.adafruit.com/product/4471). Added to cart, will see if and how it works. Also added some ready made wires, no soldering required. Plug and play so to say (if you consider writing software play).

On a sidenote, the devkit seems to be using an FTDI FT2232 IC (dual USB UART/FIFO IC). Having worked with FTDI chips before, I recognized the name and it so happens to be that I have a breakout board with that ic. 10 years old, but should still work. The software of the official dev kit is downloadable, so I’m going to give this a go as well at some point (e.g. for verification purposes).

All hardware used/mentioned:

Edited by: Verodin on 06/08/2022 - 15:17
Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Software is written in C++ and runs on a normal computer (main reason is easy development). I’m the MCP2221 DLL for I2C communication with the sensor. This USB to I2C ‘converter’ seems to be relatively unused, so I ended up writing a class for it from scratch. Mostly easy going, but during testing I had some bugs that prevented getting proper readings (I2C is big endian, x86 is little endian and anything bigger than 8-bit bytes got its bytes swapped).

As for code handling the sensor, I ripped some code from Adafruit’s Arduino library and started fixing compile bugs and rewrite where needed (convert from using arduino libs to using the MCP2221 API. This allowed a quick start, but only for getting basic readings from the sensor. Next step was taking the application notes from AMS and get some calibrated values. AMS provides 2 ways:

  • Photometric Results after XYZ Calibration
  • Reconstructed Spectrum plus CIE1931 mapping plus Photometric Results

For the time being I’m using the 1st method since it uses less data. It uses a ‘special calibration matrix’ that sounds good enough for my purposes:

Quote:
Special XYZ Calibration Matrix for Golden Device (“Created” Reference Device from gathered production-device data) – Based on Full, Production-Wide Device-Characteristic Dataset (not a per-device, so gives good, but not perfect results – Supports production without per-unit calibration)

I took those values and integrated them into the code. With these ‘calibrated’ channel readings we can go to CIE1931 XYZ coordinates and from that to CCT and Duv. Sounds simple and there’s quite some info on the net, but not everything is immediately clear on how to actually apply the various calculations. To verify the calculations I took the example readings AMS uses and it matches (there are slight difference, but that’s probably due to rounding errors). By now the output seems to produce sense making values for known CCTs of some flashlights used for testing. One major issue I’m still having is that the distance from sensor / brightness easily oversaturates the sensor and the values get really off. I’ll have to do some testing with automatic gain and see if that makes readings more consistent.

To be continued…. Until then here’s a photo from a few days ago I posted in the Opple thread:

Some light reading for those interested:
AS7341 datasheet
Evaluation Software for Color + Spectral Measurement
from ADC to LUX and CCT
Calibration methods

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

So first question to anyone who has more experience with these kind of calculations. What’s the correct equation for McCamy’s approximation (approximate CCT using CIE 1931 xy values)? Wikipedia and AMS application notes seem to disagree.

Wikipedia:
n = (x – 0.325) / (y – 0.154)
CCT = -449 * n^3 + 3525 * n^2 – 6823.3 * n + 5520.33

AMS:
n = (x – 0.3320) / (0.1858 – y)
CCT = 437 * n^3 + 3601 * n^2 + 6861 * n + 5517

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Some example output (took a few tries to get the saturation correct)

B35AM 3500K
X = 29.233201, Y = 27.391400, Z = 16.250845
CCT = 3476K, Duv = -0.005747

LH351D 5700K
X = 5.552438, Y = 6.011020, Z = 5.591114
CCT = 5873K, Duv = 0.008654

thefreeman
thefreeman's picture
Offline
Last seen: 6 hours 18 min ago
Joined: 01/06/2020 - 09:56
Posts: 1386
Location: France

Very interesting project.

Verodin wrote:
Some example output (took a few tries to get the saturation correct)

B35AM 3500K
X = 29.233201, Y = 27.391400, Z = 16.250845
CCT = 3476K, Duv = -0.005747

LH351D 5700K
X = 5.552438, Y = 6.011020, Z = 5.591114
CCT = 5873K, Duv = 0.008654

Test an incandescent lamp or the sun, those being two common sources with known duv.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Right know I’d put no value into the results, except a quick check to see if things make sense. Currently suffering from a slight information overload. It’s fascinating stuff, but a steep learning curve at the same time. Now reading application note AN000651 “AS7341 Auto Gain & Optimization”. And I need to look into normalising the readings. Results jump too much all over the place, depending on where I hold the lightsource. Need to get some consistency in there. This is far from finished.

Btw, I anyone is interested in the code I can set up a github repo in the near future.

And I just realised I have no incandescent lamps, it’s all LED lamps…

quahog
Offline
Last seen: 6 hours 53 min ago
Joined: 02/11/2021 - 17:10
Posts: 46
Location: Canada
Verodin wrote:
And I just realised I have no incandescent lamps, it’s all LED lamps…

How about the light in your refrigerator?

thefreeman
thefreeman's picture
Offline
Last seen: 6 hours 18 min ago
Joined: 01/06/2020 - 09:56
Posts: 1386
Location: France

Verodin wrote:
Right know I’d put no value into the results, except a quick check to see if things make sense. Currently suffering from a slight information overload. It’s fascinating stuff, but a steep learning curve at the same time. Now reading application note AN000651 “AS7341 Auto Gain & Optimization”. And I need to look into normalising the readings. Results jump too much all over the place, depending on where I hold the lightsource. Need to get some consistency in there. This is far from finished.

Btw, I anyone is interested in the code I can set up a github repo in the near future.

And I just realised I have no incandescent lamps, it’s all LED lamps…

You probably need a diffuser over the sensor, all spectrophotometers have one.

flightless22
flightless22's picture
Offline
Last seen: 3 hours 26 min ago
Joined: 11/19/2018 - 12:07
Posts: 732
Location: California
Thumbs Up

This may pair well with Gcharts DIY Lux meter.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

thefreeman wrote:
You probably need a diffuser over the sensor, all spectrophotometers have one.

Yes, it’s on the things-to-buy list.

AMS also distinguishes between emissive and reflection for their eval kits:

Emissive

Reflection

Without diffuser I’m basically using an emissive lightsource while the sensor is in reflection mode. Recommended is the Kimoto 100 PBU (125 microns, 66% transmission, 89.5% haze, 35.5 degree half-angle).

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568
quahog wrote:
Verodin wrote:
And I just realised I have no incandescent lamps, it’s all LED lamps…

How about the light in your refrigerator?


But off course, thanks! Silly me, the whole house, including the freezer has led. But the fridge is old and still has incandescent.
Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

The official dev kit looks to be a no go. I hooked it up to the FTDI break out board as follows:


SK – Connect to SCL
DO/DI – Wired together and connected to SDA (I2C pull ups are on the sensor board)

The AS7341_EvalSw_ALS application detects the board (error msg without board), but fails with some power on issue (tried both channels). Too bad, but worth a try.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Verodin wrote:
Next step was taking the application notes from AMS and get some calibrated values. AMS provides 2 ways:
  • Photometric Results after XYZ Calibration
  • Reconstructed Spectrum plus CIE1931 mapping plus Photometric Results

For the time being I’m using the 1st method since it uses less data.

Todo… need to switch to the reconstructed spectrum method to get CRI guesstimates;

Quote:
Spectral results, such as CRI and spectral comparisons of light sources and perturbations, can be calculated via the spectrum, which is not possible via color coordinates.

And some needed data:
Coordinates in both CIE 1931 xy and CIE 1976 u’v’ for both the black body locus and daylight locus, listed by 10K and 1K CCT increments
CRI or Ra TCS spectrum data in 1 nm, 2 nm and 5 nm increments

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

A minor but interesting detail concerning the eval software. After going through the manual, the select device popup should only happen when there are multiple cables attached. Not the case for me, so I figured maybe it doesn’t recognize it at all. It uses a standard FTDI cable apparently, so I looked up the vendor/product ID, generated a serial and programmed those into the FTDI board. The eval software no longer sees multiple cables and goes directly to the error message.

“Error during setting power on.” means “Error occurred. Set the PON bit in the Enable register (0×80).” Seems I2C communication is not working. I’m guessing this is the 1st thing that’s send out. In my software this is the 2nd thing it does, after checking the AS7341 chip ID.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Another quick update on getting the evaluation kit software up and running; no success. I’ve tried everything I could think of, looked at all the schematics and tried various ways of hooking it up. Just keep hitting that power on error message. To verify that the board and sensor actually work, I even quickly hacked one of the FTDI examples to see if communication with the sensor was possible (e.g. request chip ID). That worked fine. Currently at a loss why this it not working with the eval software. Would have been nice if it worked, but alas. Moving on for the time being.

Btw, any thoughts about this type of diffuser? (another one) Doesn’t have any specs to speak of, but probably better than nothing.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Status update: I’ve added auto gain (sensor internal auto gain) and it seems work as it should. The sensor no longer saturates. But (there’s always a but), it seems that the CCT results are way off. They’re too high (e.g. 3500K is reported as 5500K) and I haven’t found a solution yet. The raw values seem to be ok, but then a count get’s calculated that serves as basis for all other calculations.
To calculate the count, an integration time is needed, which in turn depends on the gain used. My guess is that something is off there and I haven’t found the solution yet. I’ve gone over the datasheet many times and there’s nothing obvious there. Count calculation is the same for manual and auto gain, yet the result is way off when using auto gain. I’ll have to look at that later, but for now the garden/house needs some care. So I won’t be working on this for a few days.

For those interested, I’ve uploaded the code to GitHub Note that this is a work in progress, probably has bugs, does not yet produce reliable results and is NOT an end user grade application.

TimMc
TimMc's picture
Offline
Last seen: 7 hours 39 min ago
Joined: 06/26/2020 - 02:22
Posts: 676
Location: Melbourne, Australia

I’m keen to give it a go with Python Innocent

From the Opple thread:
There’s an introduction to the Adafruit AS7341 and some examples using CircuitPython here:
https://learn.adafruit.com/adafruit-as7341-10-channel-light-color-sensor...

AMS have a user guide for an AS7341 evaluation board that mentions calibration and CIE1931 XYZ Target Transformation. You may need to determine appropriate correction factors to apply.

https://ams.com/documents/20143/36005/AS7341_UG000400_5-00.pdf

The raw values for each frequency bin could be stored in some spectral power distribution object using the colour project for Python and then the CRI can be calculated:
https://colour.readthedocs.io/en/develop/generated/colour.colour_renderi...

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Get the boards I mentioned above. It’s relatively cheap to get going Wink

I’ve looked at the Adafruit code. The problem is that they don’t go beyond basic functionality (e.g. reading raw values and print them to screen). They’ve added all registers, but don’t use them. Such is the case with auto gain.

Also when not using own readings, all calculations yield the same results as the AMS application notes. I’ve used their example readings as input and the result matches. So it’s unlikely that part of the code is the problem. Disabling auto gain provides believable values, hence my belief something is wrong with how I handle readings done with auto gain.

Edit:
Did a quick and dirty fix. When taking a reading, it 1st takes are reading with auto gain enabled to determine the best gain factor. Then disable it and take the real readings. Much better results. Needs some optimizing, but later (e.g. less SMUX channel switching). Using channel F8 for the time being since that seems to have the highest raw values (even though F7 has the highest irradiance responsivity).

TimMc
TimMc's picture
Offline
Last seen: 7 hours 39 min ago
Joined: 06/26/2020 - 02:22
Posts: 676
Location: Melbourne, Australia

Ordered Adafruit AS7341, Adafruit QT Py and some cables. Beer

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Awesome! Would be really nice if we can develop some sort of BLF spectrometer that’s better than the Opple Cool

I’ll see if I can create some CSV files that have the calibration matrices. Then they can be shared and don’t need to be created multiple times. Don’t know about Python, but it’s not too tricky to read those in C++.

Edit:
XYZ and spectral calibration matrices (taken from AS7341_AD000198_3-00.xlsm aka “AS7341 – from ADC to LUX and CCT”).

Note that there are 3(!) different versions (“Based on Full, Production-Wide Device-Characteristic Dataset”, did device characteristics change over time?);

  • v1.1 (2019-02-13)
  • v3.0 (2021-04-28)
  • Initialization data that comes with AS7341_EvalSW_ALS_v1-26-0.zip

tterev3
Offline
Last seen: 2 weeks 5 days ago
Joined: 03/28/2014 - 11:22
Posts: 313
Location: NC, USA
Verodin wrote:
So first question to anyone who has more experience with these kind of calculations. What’s the correct equation for McCamy’s approximation (approximate CCT using CIE 1931 xy values)? Wikipedia and AMS application notes seem to disagree.

This is the one I’ve seen used, it appears to be a combination of those two:
n = (x – 0.3320) / (y – 0.1858)
CCT = -449 * n^3 + 3525 * n^2 – 6823.3 * n + 5520.33

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Oh man Big Smile Thanks!

I did do a quick comparison between the AMS and wikipedia one (B35AM 3500K source). The Wikipedia one produces slightly higher CCT numbers.

CCT = 3646K, Duv = -0.012651
CCT = 3733K (wikipedia)

CCT = 3821K, Duv = -0.012644
CCT = 3884K (wikipedia)

CCT = 3702K, Duv = -0.012832
CCT = 3781K (wikipedia)

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Btw, I’ve switched from using all channels (excluding flicker) to using only F1 to F8 as input (the clear channel overpowers all other channels, NIR seems pointless for flashlights). Don’t know if and how that affects results. Any thoughts as to what would be better?

SoftestVoid
Offline
Last seen: 2 weeks 22 hours ago
Joined: 06/13/2022 - 04:52
Posts: 1
Location: Victoria, Australia

This is an awesome project! I’ll have to order myself an AS7341 to have a play with too.

Quote:
Btw, any thoughts about this type of diffuser? (another one) Doesn’t have any specs to speak of, but probably better than nothing.

Might be better off getting something like a bit of Lee Filters 250? Shouldn’t be too hard to source small sheets, and probably more consistent than Ali.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568
Verodin wrote:
Btw, I’ve switched from using all channels (excluding flicker) to using only F1 to F8 as input (the clear channel overpowers all other channels, NIR seems pointless for flashlights). Don’t know if and how that affects results. Any thoughts as to what would be better?

And yes, it affects results (provided I didn’t make any mistakes refactoring). AMS excel values as source:

Quote:

check CIE1931 calcs (channel 1 to 10)
X = 0.136200, Y = 0.146973, Z = 0.171680
x = 0.299438, y = 0.323122, z = 0.377440
CCT = 7352K, duv = 0.007129

check CIE1931 calcs (channel 1 to 8)
X = 0.173174, Y = 0.179393, Z = 0.245949
x = 0.289338, y = 0.299730, z = 0.410931
CCT = 8613K, duv = 0.000749

thefreeman
thefreeman's picture
Offline
Last seen: 6 hours 18 min ago
Joined: 01/06/2020 - 09:56
Posts: 1386
Location: France

Do you want a reference LED ? I see you’re in EU, I could measure an LED at different current and send it to you in an envelope and the SPDs.

gchart
gchart's picture
Offline
Last seen: 42 min 17 sec ago
Joined: 03/19/2016 - 11:57
Posts: 3243
Location: Central IL

Interesting project! I tried this a couple years ago and got really frustrated with the results, but I’m afraid I gave up too soon. My readings seemed to jump all over the place. I think I tried using a diffuser, but it’s been too long so I don’t remember exactly. I’d love to have another go at this at some point.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Time for another update: I’ve made quite a few changes to the code. Refactorisations and cleanups, bugfixes, added more verification code and the basic calculations for spectrum reconstruction (the large calibration matrix and spectrum to XYZ). So if you have a clone, just get the latest code.

The good new is that readings seem to be much more consistent now that auto gain is used. However, reading are off by about 500K (too low), consistently. This is a bit of a mystery, since the AMS excel gives the same results when inputting the raw readings from the sensor. This leads me to assume all calculations are correct, but the raw readings must be wrong somehow. My guess would be integration time is not correct, since that’s the only thing that I can think of that influences the raw values returned. Needs further investigating.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

thefreeman wrote:
Do you want a reference LED ? I see you’re in EU, I could measure an LED at different current and send it to you in an envelope and the SPDs.

That would be great for fine tuning, thanks for the offer! I’m in Sweden. Currently still having issues getting the reading close to actual CCT, so it might be a bit premature (see above).

As part of the Eurkatronix group GS5 buy I ordered the whole range of E17A CCTs (sm1863c, sm203, sm223, sm273, sm303, sm353, sm403, sm453, sm503, sm573 and sm653), specifically for test purposes. But they don’t come with (detailed) SPDs. The closest data is the test reports on the website and that’s probably close to the actual LEDs I’ll be getting. If that works out (including a proper physical test setup), I’d love to fine tune using a real reference LED.

Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

gchart wrote:
My readings seemed to jump all over the place.

Did you use any form of auto gain? I initially had the same issue, just couldn’t get consistent readings, unless I looked at the raw readings and adjusted how I held the flashlight at a certain distance and brightness. Results are much more consistent now that the sensor decides what gain to use. There are slight variations when I move the light sideways, but nowhere near the huge jumps in CCT as before.
Verodin
Offline
Last seen: 14 hours 5 min ago
Joined: 01/11/2022 - 16:50
Posts: 568

Just pushed a change so it uses the ‘higher accuracy’ spectral calibration matrix by default. Define USE_XYZ_CALIBRATION_MATRIX in SpectralSensor.cpp to use XYZ calibration matrix as before.

And for those wanting to output the reconstruct spectrum to CSV, check out the Spectrum::saveToCsv() function. It’s primitive, but it works (need a GUI at some point…)

Pages