This kept popping up from the back of my head and just had to look up a few things. A major plus is that you skip the calibration matrix and reconstructing the spectrum that’s required when using the AMS sensor. But code will be more complex to implement camera functionality, including a GUI.
A few items of interest I ran into are diffraction grating and a linear image sensor. The diffraction grating is just a thin plastic film to do the splitting (you have glass as well, but that costs a fortune). Take 600 or 1000 lines per mm, with a wavelength range of 200 to 10.000 nm. Maybe I’ll order one of these to experiment with, they’re not that pricey. The linear sensor is just a single row CCD, which might make processing easier (see Toshiba’s product range).
Now you can shrink this down to the size of your fingertip with a single module (slit, grating and CCD all in one), but it comes with a 300 USD/EUR/? pricetag: the Hamamatsu ultra-compact spectrometer Very cool little module. I wonder if this is what professional spectrometers use.
Hamamatsu are famous for their photomultipliers and other insanely sensitive light sensing devices so my bet would be lab grade stuff for chemical analysis, measuring weak fluorescence for example.
I wonder how Hamamatsu can get a good spectral resolution. The internal image sensor is only 288 pixels according to the datasheet.
On a side note, I’ve had lots of time to think (building and watching paint dry) and I’ll have a go at adding an optical variant (grating, slit, image sensor) as well later this year. 0.08 mm slit ordered (I want a known size, not a guesstimate by making a cut by hand) and this should in theory give around 2 nm resolution.
Also ordered a diffraction grating. Went for glass in the end since a PET sheet will probably flex and influence accuracy. 600 lines/mm should be suitable for the visible spectrum (usable range 250-850 nm). I assumed more lines would be better, but that’s not how it works
A warning though, on AE I’ve come across PET sheet stuck to a glass piece as well. The description says glass and only after communicating with the seller they say it’s not actually grated glass. Do your research before ordering these kind of things.
One thing that’s still up in the air is the imaging sensor. It’s a minefield and not everything is suitable for spectography. Or maybe I’ll have a go with a 10 year old crappy logitech webcam 1st, before spending any more money… Btw, does anyone know if the Bayer filter of a colour sensor influences results?
Anyway… I need to sort out the GUI part 1st. Currently thinking of using wxWidgets. Cross-platform and can be used with OpenCV if I go the webcam route. And maybe spectrometer is the wrong name. I’m focusing on flashlight specific things such as CRI. Light meter seems to be wrong naming as well. Open to suggestions
Are you planning on pointing the webcam (lens and all) at a matte projection surface or rather projecting the diffracted beam directly onto the sensor?
In the case of the former I’d say you don’t need to worry about subpixel spectral resolution- but it should be enough. There’s a nonfree linux program for these, as well as an open source web tool to perhaps take a look at.
In the latter case I’d worry about the optical coatings applied to the sensor’s window acting like dichroic mirrors at certain wavelengths. If you’re not going for a linear sensor, your spectral lines will likely never align perfectly with the sensor’s Bayer pattern so you’ll have plenty of subpixels to average out across each spectral line (and I don’t think linear CCD/ CMOS use the Bayer pattern at all).
The subpixel colour bandpass filters should closely mimic human cone cells in bandwidth/ range so you shouldn’t get gaps in the spectral region of (our) interest and any nonlinearities should be easily calibrated out against a reference white light source.
Thanks for the extensive answer. It would be projecting directly onto the sensor.
Otoh, I’m leaning towards dumping the whole USB camera idea. I’ve looked into getting this up and running and it’s not worth the effort. My whole motivation is doing this from scratch and learn something along the way (that’s a massive part of the fun). But getting a USB camera up and running, processing the stream, dealing with OS specific stuff and needed 3rd party libs has very little to do with a spectronomy/measuring emitters and I’ve noticed it just sucks the life out of me.
So instead I’ll go go the linear CCD route as initially brought up a few posts back. Hook it up to a STM32 controller and dump the data to the PC over USB via the build in COM port functionality. AKA KISS principle. Just spend an hour in ST’s latest IDE concoction (STM32cubeIDE) and it’s ridiculously simple to get something up and running with it’s build in code generation (apart from the obscene amount of diskspace required). This stuff has come a very long way in 20 years. I spend months implementing basic USB communication in assembly around year 2000 and now it just takes few clicks and barely any lines of custom code
Now if I could only get my hands on a new working TCD1304…. Toshiba has a 32 week lead time according to Mouser and the ones on AE seem used and of very dubious origin/quality. But I keep getting sidetracked… this stuff is just too much fun to dive into
Edit:
Look what we have here It seems I’m not the 1st going this route.
PS, there are no silly questions! I regularly make a fool out of myself with certain questions, but you’d be amazed how much you learn from the responses.
Just a quick message as a sign of life that I’m not dead yet This project is still ongoing, but on hold for a moment. I’ve been enjoying summer vacation and the plan was to be working on this again by now. But as fate would have it, the roof my veranda is rotten in many places and fixing that has priority right now.
Not good
In the meantime, various parts have started arriving. Diffusers for the AS7341, a linear CCD and STM32 controller. The grating also arrived, but I didn’t take a photo of it. The slit never arrived though, the seller never send it and AE did an auto refund.
Also something else I dug up from storage is some fat beam laser modules for calibrating the CCD setup. 650 / 532 / 450 nm should be enough.
This is supremely relevant to my interests. My research had pointed to a B&W linear CCD as the ultimate, too. I found the range of options a bit overwhelming, and the documented experiences of makers were few and far between, and found myself in analysis paralysis. I’m only recently getting into EE and microcontrollers as that seems the missing element now that I’m comfortable with full stack programming and 3d modeling/3d printing.
I’m far more comfortable with software, and wouldn’t at all mind collabing on a UI. I’ve been wanting to create a cross-platform GUI on top of ArgyllCMS since I’m on a mac, and copying the command line output of spotread into Osram Color Calc in a VM is quite tedious when testing several light sources.
When exploring that, I did find quite a number of well established color/spectral data handling libraries on github - one python library in particular seemed like the ‘authority’, given star count, years since established, and active development. I’ll see if I can track that down.
What functionality are you aiming for, and what tech stack are you considering?
All of my professional experience is with Python and Javascript/React/React Native (and SQL, I suppose), but have dabbled a bit in Swift/Obj-C and have been itching to dive into Rust or Go. I’d planned to use one of the ReactJS cross-platform frameworks (React Native, Electron, or the like) for the reactive UI, python for the logic, and SQLite (and possibly redis?) for data handling.
Not looking to derail, just seems a lot of overlap of our ambitions, so interested if it makes sense to collaborate at all! In any case, I’ve got this thread on ‘notify’!
Ok, got a waterproof roof again, so trying to pick up again where I left off…
I know the feeling. Was thinking how hard can it be, only to find myself completely overloaded with a mountain of minutia. Very interesting stuff, but it’s a lot to sift through. Fortunately all it takes is a bit of time to let stuff sink in and the fog clears. My background is a degree in computer science (20+ years ago, geez I’m getting old :D) and although hardware is not part of my professional life, controllers and (digital) hardware design are not a major problem. Biggest issue is that controllers have gotten so much more advanced and faster (incl supporting software) that I’ve got some catching up to do in terms of reading up on modern capabilities and how to take advantage of all features. Anyway, these days I spend my time as a C dev using Linux for a company making big mining machines. Mostly enterprise stuff, remote control and automation (think baby skynet :D). But it’s been years since I’ve started a project from scratch and directly interfacing with some hardware. Same language, very different way of doing things.
GUI is still up in the air, so I’m open to suggestions. Right before the roof nightmare started, I was looking at https://www.wxwidgets.org since it’s cross-platform and I’ve always wanted to go it a go. Choice is mostly based on curiosity.
As you found there’s quite a lot of Python libs out there. Since I’m a C guy, I basically have to start from scratch. But that’s part of the fun. Don’t know about you, but I learn most by just doing it, making mistakes and solving bugs. I forces you to look up information, instead of just glancing over any documentation and just calling functions in an existing lib.
Don’t worry about derailing, it’s nice to have some software related feedback. We’re on a flashlight forum, not a software forum, so I consider it luck that there’s someone with experience in this field around I’ve heard about the languages you mention, but life got/is in the way of exploring them any further (emigrating, relationships, kids, house, etc are a time/nerding killer). So I’m limited to the things I know or it will end in lots of fun exploring, without anything to show for in the end. Hence the need to put some restraints into place for myself.
As far as functionality I’m aiming for; considering my main usage would be to measure the light quality and basic properties of flashlights and in house lamps, I’m not trying to make a fully featured spectrum analyser. That would be overkill. Currently I have roughly the following in mind:
Open source and platform independent. For the moment I’m using Visual Studio on Windows, mainly because debugging in Linux sucks balls.
Take readings with the AS7341 and output CCT and CRI. Readings are slow and there’s no real spectrum (only extrapolated from a debatable reconstruction table), so no GUI needed. The basics are almost done, main issue is a bug that results in incorrect CRI and I don’t know why yet.
Next would support for the linear CCD. I’m not convinced the AS7341 will result in reliable satisfactory results (the sensor is only a few channels after all), so it’s only natural to add this. Calculations remain the same once you have the SPD, and it provides a real spectrum instead of extrapolated. Also much faster and should provide realtime data. A GUI showing the spectrum makes sense, so that would be next right after the hardware works.
GUI with realtime spectrum view, CCT and CRI. Should be simple to use. For comparison, Osram’s Color Calc is good, but is way too complex for just showing light properties. So it won’t be similar to that.
Extend available info to include color rendition using IES TM-30-1x
Just skip the GUI for now and focus on the backend. Then you can either implement a nice text user interface or an API that can be used from a separate GUI program (either written in C with GTK or Qt or Python with WxWidgets etc).
Prio #1 is fixing the bug. Without that solved, there’s no point in spending any time on looks/interface whatsoever. You also bring up a good point concerning possible API. Backend functionality should be usable on it’s own and as loosely coupled to any UI as possible (e.g. defined interfaces, either internally in code or public). Anyhow, another time.
One of few hard requirements is that it must be able to produce reliable and reproducible results. Related to this I’m happy to report that thefreeman has kindly provided some reference LEDs with measurements taken with a professional spectrometer. This allows for proper verification instead of making assumptions. And I can use the PSDs to verify calculations independent of the actual sensor. And then match that with real sensor measurements. Should match or be close. If not, we take it from there until it produces accurate results. Bonus is that parts of the code can be considered verified and in theory can be used by basically anybody in any language.
Concerning the AS7341, I really need to make it dust proof. I’ve been keeping it in a closed anti-static bag when not using it, but even then it will collect some dust in the long run while it’s out measuring. The sensor surfaces are completely exposed, so I started putting together a few parts. Currently waiting for thinner spacers. The Adafruit boards have 2.5mm holes and the smallest spacers I have are 3mm. Thinner ones are already on the way through. In the meantime drilling few holes and some hot snot should do the trick. Case is a bit on the too big side, but that’s ok.
Have you seen www.processing.org for coding a GUI / app?
It is very Arduino-like and easy, plus it compiles for different platforms, including Android, if added as a target.
This is a very interesting project. I also have a strong interest in determining the spectral characteristics of various light sources, but just ended up coughing up a few hundred bucks for an Ocean Optics spectrometer on Ebay. While this gets around many of the problems in building your own it still leaves the issue of calibration - both spectral and radiometric.
Spectral calibration - determining what wavelength corresponds to which element of your array is pretty straightforward: Just find a light source with a few known spectral lines, input to your spectrometer and match up the results. Common CFL lights contain spectral lines of Hg and work well for this.
Radiometric calibration is trickier. Most array spectrometers vary in sensitivity over the array so the raw spectrum you obtain from your spectrometer when looking at a light source may be quite different from the actual spectrum. To calibrate this out you need a source of known spectral irradiance (i.e. of a specific spectral shape where the relative power at all wavelengths is known). You can buy one of these calibration sources for a bunch of money (lots more than I paid for my spectrometer), but I resisted this approach. Instead you can try to find/build your own calibration source.
I went pretty deep down this rabbit hole last year. I looked at homebuilt blackbody sources for a while (known spectrum, but depend on blackbody temperature control and are very weak/noisy in blue/UV for any reasonable temperature), but eventually settled on using sunlight. The terrestrially observed solar spectrum is very well characterized and, while it does vary with elevation and humidity, these effects are known and can be calibrated out.
To get anything like accurate spectra out of your homebuilt spectrometer I suspect you’ll need to go through this sort of process too.
I’m not certain of your continued commitment to this endeavor but, here is a link to a project I’ve been working on that uses AMS’s 7421 and 7343 sensors. There is a schematic available as well as a gerber file if you want to make the board yourself. I’m currently working on software for the project and it’s application for astronomy but your welcome to any of my work.
