Going into details here :bigsmile: This post may look a bit technical for some but fear not, it mostly contains things that I picked up about the subject over the last couple of weeks, I am not at all a physicist specialising in optics (I am a biologist). Disclaimer: having said that, I feel free to make errors in what I write down here 
.
Measuring light with a luxmeter is a tricky business, as I have become to understand and as many on this forum have recognised as well. The main focus of discussion recently on BLF is having the meter correctly calibrated, which is hard enough, but there can be more wrong with a luxmeter than just the calibration, to name a few: the temperature error can be high, the reading can be non-lineair, the spectral sensitivity may not be corrected well for the luminosity curve (also called V-lambda curve, that is the curve of the (phototrophic) human eye wavelength sensitivity, used in the definition of the lux). A Wikipedia quote from the lemna 'optical power meter' (a luxmeter is a type of optical power meter):
For professional use there is the (german originating) DIN norm for luxmeters: DIN 5032 part7. (I am not aware of any other standards for luxmeters, does anyone know if there is an ANSI-standard as well?) According to this DIN 5032-7 there are three classes of luxmeters, classA, classB, and classC. From an indirect source I found a couple of the criteria (for direct insight into the DIN texts I had to pay dear money):
| criterium | name | maximum deviation (%) conform DIN 5032-7 | ||
| class A | class B | class C | ||
| V (lambda) correction | f1 | 3 | 6 | 9 |
| UV-sensitivity | u | 1 | 2 | 4 |
| IR-sensitivity | r | 1 | 2 | 4 |
| Cosinus-error | f2 | 1.5 | 3 | 6 |
| liniarity error | f3 | 1 | 2 | 5 |
| temperature coefficient | 0.2%/K | 1%/K | 2%/K | |
| frequency dependance of light fluctuation | f7 | 0.2 | 0.5 | 1 |
| maximum measuring error | ft | 5 | 10 | 20 |
I understand most of this, but not all. What I found pretty striking is that for a class C luxmeter (which goes for over $300 minimum!) the maximum allowed measurement error is 20% ! And all the above mentioned errors will not be corrected by calibration of the luxmeter!
Our chinese luxmeters do not even come with a classification, but that does not neccesarily mean that none of the above criteria are met of course. This all made me wonder how wrong these cheap luxmeters actually are? Are they very bad, or are they quite ok?
I recently bought secondhand a top of the range (in the handheld category) luxmeter of which the sensor meets the DIN 5032 part 7 class A criteria, the rests of the meter meets class B. It is a Mobilux A luxmeter. The manufacturer of the sensor is Czibula&Grundmann, a small optical instruments company from Berlin, the sensor is connected to a Mavolux 5032B class B meter made by Gossen.
The overall accuracy of this luxmeter is guaranteed within 5%, so that aspect meets the class A criteria. The meter was officially calibrated in 2009, the reading might be off after 5 years although I do not expect that to be much. (I don't know why I don't expect that, at least the former owner told me that the meter was hardly used, for what that matters 
).
The calibration being off or not, at least this meter gave me the opportunity to do a few comparisons with the two chinese luxmeters I had already: a Tondaj LX-1010B , and the Ceto CT1330B. The last one is permanently build into my integrating sphere, and lost its original calibration, so it was only partly used in this comparison.
What I am able to do in my home and in my budget is very limited of course, and light measuring professionals probably can't suppress a mild smile when they read how I do the tests, but thanks to my distant past one thing I could do was have a look at the wavelength response of the luxmeters. This error is one of the biggest errors allowed in the DIN 5032-7 criteria, so apparently this criterium is a difficult one to meet. The wavelength response error is caused by the optical filter in front of the sensor, it is supposed to correct the sensor's wavelength sensitivity for the official V(lamba) curve.
Meet a piece of history that I inherited from a job long ago:
It is the lamp house and prism-monochromator from a Zeiss Spectrofotometer, which was the standard spectrofotometer for three full decades from the early fifties to well in the seventies. Eventually it as replaced by easier machines, but certainly not more accurate!
However much I would have liked to measure the absolute V(lambda) curve for the different luxmeters with it, this thing will not give that because the optical power is not defined for each wavelength (that was not relevant for the original use of the spectrofotometer): the light source is a 2700K tungsten lamp that emits a spectrum that increases almost 8 times in optical power going from 400 to 800nm. Although the Tungsten lamp curve is simple, more or less a straight line, I do not dare to correct the measured graph for that spectrum because I also do not know exactly what happens inside the monochromator, are there light losses that are wavelength dependant?, and also my guess is that the used slit in the monochromator will let a wider wavelength range through at longer wavelengths than at shorter wavelengths (I do not know enough about this to know this for sure, and I have no documentation with the monochromator).
This type of monochromator does not filter out a single wavelength, but a wavelength range, this range can be adjusted by altering a slit opening. A wide slit gives more light and a wider wavelength range, a narrower slit gives less light and a narrower wavelength range. I used a slit of 1mm because that let through a minimum amount of light that still was detected well by the luxmeters. I have no absolute idea (could not find it on the internet) what the wavelength range is that is coupled to a 1mm slit size, and that range may very well vary with wavelength. So you have to bare that in mind when reading the graph below.
The luxmeter testing with the monochromator were unavoidably at the low end of these luxmeter's measuring range. For the responses to be correct, this assumes that the luxmeters are lineair over the complete light measuring range, also at the low end, and at all wavelengths. I tried to check that using another oldie, my Leitz Focomat 3C photo enlarger. I recorded the readings of the Mobilux and Tondaj meter for the different diaphragm stops, varying the enlargement (which varies the illuminance range) and applying a few colour filters (red, green and blue). The maximum readings were still a bit on the low end of the range of the luxmeters but I did not detect any significant a-lineairities (data not shown )
So what I do is show the optical-power-uncorrected graph, this does not give the absolute V(lambda) values, but does show the relative differences between the wavelength response of the three meters. BTW, the three luxmeters gave values in a different range because the the detector areas were of very different size, while the projected lightspot was only small (it just filled the Mobilux detector area). (for the Tondaj and Ceto meters not the whole detector area was filled by the spot, so the amount of light on the limited area is interpreted by the meter as coming from the total detector area causing a significant lower reading).
So I thought it was handy to correct the three curves so that the maxima were the same: the relative readings between the luxmeters were not relevant anyway because of the different detection areas, and this compares the curves much better. Aligning the maxima is perhaps not the best way to make the three curves comparable (equalling the surface area under the curves would be a bit better I think), but whatever is the most fair way to compare them, aligning the maxima was easy and gives a good idea of the differences.
So what, after all the restrictions caused by the method, is leftover to conclude from this graph? For the following observations I assume that the wavelength response of the Mobilux meter is a good representation of the official V(lambda) curve. This will of course not be entirely true but it is the best I have.
1) the MobiluxA maximum sensitivity is not at 555nm, as it officially should be, but at 545nm. Even if one would correct for the non-flat spectral output curve of the tungsten lamp, that maximum will not shift as far as I can see. So either my monochromator is a bit off, or the classA optical filter of the Mobilux is off (uncertain is my middle name :-) ) I may check the monochromator with a green laser some day. EDIT: I checked the monochromator and it reads 14nm low (see post #5, and I remeasured it with the 532nm green laser). So the three curves will shift 14nm to the right. I will update the graph at a later point. The consequence is that the Mobilux meter's sensitivity peaks out at about 555nm, as it should be. EDIT2: I updated the graph with the nm values shifted 14nm. In the blue and red region it will still be a few (~3) nm off because the monochromator nm-scale is slightly logaritmic and the shift is a constant value. I think such a small deviation is acceptable.
2) the Tondaj meter's sensitivity is seriously off (much too high) in the blue and red region, so much that I expect this to see reflected in measuring cool white and neutral/warm white leds. Especially the blue error is serious because of the extreme blue peak that the cool white Cree leds have at 450nm.
3)the Ceto meter is very ok in the blue region but off in the red region. For measuring cool white leds I think it does better than the Tondaj.
4) measuring colour leds with the Tondaj, as I did before for BLF, (especially red and blue) gives large errors in luxreadings, e.g. measuring a blue led (450nm) with the Tondaj gives a reading that is at least 4 times too high. I am glad that when posting my colour led test results in the past, I always mentioned the uncertainty of the absolute lux-values .
To be continued...
This is it for now, all quite theoretical sofar, but the real question is of course: do these differences matter in 'real life' 
flashlight hobby use?. When I have time I will compare the Tondaj meter and the MobiluxA meter by measuring throw of a few flashlights with different colour temperature leds, to see how the measurement errors of the cheap Tondaj luxmeter is affected by the colour temperature of the led. I will update this post when that is done.
At some point I will be curious about the wavelength response performance of the Extech luxmeter that some people at BLF and CPF use for their measurements, that meter is a class more expensive than the 20 dollar chinese meters but still a bit affordable. But I don't have one, so that has to wait.
thanks for reading!
Update Sept. 6 More familiar data (for a flashoholic) from now on: I measured the throw at 5meter of a bunch of flashlights that I have, not so much for the actual throw numbers but to see how the difference is between the Tondaj LX-1010B and the MobiluxA luxmeters over a wide range of colour temperatures and CRI.
To do that I attached the two sensors right next to each other on a cupboard, measured exactly 5 meters from the surface of the sensors with a tape measure, and clamped the flashlights in a laboratory stand. I blocked light reflected via the floor with a black coat halfway. Other indoor reflections were negligable. I noted the maximum lux values in the hotspot of the flashlight beams, I wiggled the flashlight around a bit to find the maximum reading for each luxmeter. I found it quite an effort to find the maximum values I must say: hotspots from flashlights with a smooth reflector have a significant variance in luxvalues, flashlights with an OP-reflector gave values with less variation over the (center of the) hotspot. My guess is that a 1% measuring error could have occurred due to this difficulty (but not larger).
So here are the results, for every flashlight the host is mentioned (although that is not very relevant for these measurements), the led type, lux@5meter according to the Tondaj meter, lux@5meter according to the MobiluxA meter, the percentage that the Tondaj was off compared to the Mobilux reading. The order is the order that they were measured.
| Host | Led | Tondaj LX-1010B reading | Mobilux A reading | Tondaj, percentage off |
| small Ultrafire 602H 16340 host | XP-E2 green, 535 nm | 69 | 72 | -4% |
| small 16340 host | XP-E2 blue, 470 nm | 136 | 36 | +277% |
| small 14500 zoomie | XP-E2 red, 625 nm | 44 | 36 | +22% |
| small 16340 host | Philips Luxeon Z cyan, 505 nm | 138 | 87 | +59% |
| Sunwayman D40A, stock | XM-L2 'cool white' | 880 | 837 | +5% |
| Maratac AA copper | Osram Oslon Square 3500K 80CRI | 76 | 78 | -2.5% |
| Xeno 03 | XP-G2 3C | 140 | 143 | -2% |
| EagleTac D25C Ti 2013 | XM-L2 6A1 80CRI | 69 | 72 | -4% |
| Olight S20 Baton, stock | XM-L2 'cool white" (more towards neutral, bit green :sick: ) | 140 | 143 | -2% |
| Fandyfire A10B (constant output reference light) | XM-L2 4C | 119 | 124 | -4% |
| Convoy S5, triple | Nichia 219B 5000K 92CRI | 131 | 128 | +2.5% |
| Convoy S5, triple | Nichia 219A 4500K 92CRI | 94 | 90 | +4.5% |
| small 14500 zoomie | XP-E2 7A, dedomed | 650 | 700 | -7% |
| Jinheng 12A | XM-L 3000K | 476 | 511 | -7% |
| Ultrafire LZZ F15 | XM-L2 5D2 80CRI | 595 | 601 | -1% |
No-brand 26650 12dollar dive light, but with magnetic switch  |
angry cool white XM-L (7000K?) | 700 | 600 | +16% |
| Ultrafire K10 (constant output reference light) | XM-L2 0D | 42 | 40 | +5% |
| Uniquefire UF-T20 | XP-G2 3C, dedomed | 4100 | 4330 | -5.5% |
| Convoy M2 (constant output reference light) | XM-L2 5A1 80CRI | 243 | 248 | -2% |
So here's reflected in practice what the graph already showed. Although I would have liked that, the single colour results do not follow the graph literally unfortunately (there are apparently some inaccuracies in my method, who would have thought that ) Let's discuss the table:
*First, if you compare the Tondaj and Mobilux meters using a wide range of neutral white leds, they agree within a few percent. This tells me two things: 1) the calibration of the Tondaj (luck or not) is not bad at all, 2) when measuring neutral white leds, you get away with a significant deviation from the V(lambda) curve.
*The single colour leds as expected show the worst problems: although green reads almost the the same between the two luxmeters, blue reads 4 times too high (!!!), cyan 1.6 times, red 1.2 times. That is pretty bad. I will update my XP-E2 colour and XM-L colour tests with these new insights (at the time I already warned for big errors in the actual lux-values), those tests were done with the Tondaj meter (but mind that only the absolute values of those tests are affected by this, things like curent at maximum output, and output curve shape are not affected).
*Going cooler than neutral white generally the Tondaj reads too high, as much as 16% for the typical cool white XM-L found in cheap flashlights. Going warmer than neutral white the Tondaj reads too low, but I did not find more than 7% too low for very warm white leds.
*The 92CRI Nichia's behave different from the other neutral white leds: they read too high on the Tondaj meter. I guess this has to do with the relative large amount of blue and far red that they have compared to the 'green' region of the spectrum.
Conclusions.
For a general idea of the output/throw of most common white led flashlights, the Tondaj LX-1010B luxmeter is adequate: if the calibration is good, the reading is for most leds within 10% of the reading of a very good luxmeter that performs according to the best standards.
I assume (also because I measured the wavelength response of a third luxmeter, the Ceto CT1330B, which was different but did not seem to behave worse than the Tondaj) that the various other cheap chinese luxmeters that are commonly used, -with variations- perform similar to my Tondaj meter, so I carefully expand the above conclusion to those meters as well. If I can lay my hands on another one or two different ones, I can be more sure about that.
For measuring the true lux-values of colour leds with any accuracy, these chinese meters are completely crap. They can be used effectively, but only for relative measurements within light of the same wavelength. Even the Mobilux meter probably will not behave to specs when measuring colour leds.
If you calibrate a cheap chinese meter (or have it calibrated), the calibration only counts for leds very similar in colour temperature (and to lesser extend: CRI) to the light source that the calibration was done with. If not, despite the calibration the reading can be pretty much off.
I am planning to send around a constant output flashlight that I have made, to compare 'djozz-lumens' to what other people measure (it has a 4C tinted XM-L2 by the way), but with this knowledge of the colour temperature variance in the luxreadings of cheap meters, the usefulness of this is limited of course (but it is still a good idea I think, it will reduce some of the variation among us all)
I had some testing fun again :-)
Further tests?
I am curious about the wavelength response of some mid-range luxmeters, will those curves be mid-range as well?? As mentioned above, I am especially curious about the Extech meter that is used by more than a few flashlight 'guru's'. But to be honest, I don't have one of those and I have enough other little flashlight projects ahead of me to keep me busy in my (scarse) spare time.
Thanks for reading again!
Depending on the light, the day, the time of day, and whether anyone in the room has had a bowel movement lately. Good to know we’re close anyway. 
(my measurements were always indoors, with temperature not varying more than 2 degC, but still 4% error is big!). It may be not that bad, but we just don't know until it is tested: higher end meters come with spec.sheets with the characteristics, the chinese are not telling anything.