Lumen and lux measurements, why cant we all try to be on the same page?

138 posts / 0 new
Last post

Pages

RaceR86
RaceR86's picture
Offline
Last seen: 5 years 8 months ago
Joined: 02/22/2012 - 08:32
Posts: 3777
Location: Norway
Lumen and lux measurements, why cant we all try to be on the same page?

For the record, im just sharing some observations and thoughts here. Im not claiming anything as fact. Nor is my purpose to point at anyone and saying their numbers are wrong. For those who feel affected, dot take this personal. I just want to get a discussion around this topic. If it can help measurements shared by BLF users to be more similar then nothing would be better than that.

The fact that the similar lights have slightly different readings, and that manufacturers who uses ANSI standards for measurements are all over the place does not help either getting everyone one the same page. But I think we, the enthusiasts should give it a shot.


The purpose of sharing measurements

What is the purpose of sharing measurements? Isnt it to be able to post numbers that are comparable with others? If so, would it not be beneficial that everybody were more on the same page? Would it not be beneficial for everybody to try and get closer to each other?


Lux:

When it comes to lux/kcd numbers its been quite clear for some time that the people with LX1330B have the highest numbers. At least that is what I have noticed. These numbers seems to some extent to have become close to the benchmark simply because so many of the people who often posts lux readings use one.

Typical difference in lux between a low reading meter and a LX1330B can be up towards 30%. In general though, it seems like peoples lux numbers are fairly close. Usually within 10-15%. Although, LX1330B usually have the peak numbers. That number is usually higher then most manufacturer ratings, and usually higher than calibrated expensive light meters.

 

Lumen:

There is mostly two camps here among people that often share numbers.

rdrfronty, Tom E, DBCstm, RMM and maybe others are example of people in the "high camp". Expect up towards 22% higher readings than most others. These people are among the main contributors of sharing lumen numbers on BLF. Most are into modified lights, so accuracy towards ANSI rated lights does not seem that important. Measuring 1200 lumen OTF from an emitter driven at 2,9-3A is also possible with the "high lumen calibration". Even though that is a bit doubtful compared to what you might expect from a Cree XM-L2 according to specifications. Thanks to copper mcpcb, and maybe extremely small OTF loss, it may be possible for all I know, but it certainly seems a bit too high for me. 

Then there is the "ANSI camp". For these people getting a nice overall calibration compared to premium lights seems to have been of high priority and the baseline of calibration. Selfbuilt is one of the more known reviewers on CPF, he have used a lot of time calibrating his equipment for correct lumen readings. His numbers are basically the same as Fenix. Which is one of the manufacturers who posts numbers that seems quite consistent and reasonable (based on my personal observation and several others). I believe that some of the "top" reviewers and light buyers on BLF are typically within 7% of those numbers. _the_, johnnymac, HKJ, etc. I personally consider the work all these people have done with several hundreds of lights combined to be the benchmark. Usually when I see someone from another forum, theirs numbers usually are closer to the "ANSI camp". I dont see why this should not be the benchmark for everyone.

 

Some example issues:

People have a tendency to take numbers for a fact, or to be quite similar. Many are not aware that there can be up towards 30% difference in the numbers, and that 20% difference isnt that uncommon. This leads to issues when people want to compare different lights measured by different people.

In general, in some places its looked down upon to have too high numbers. Its often looked upon as cheating or bragging. On BLF, high numbers (posted by members) are usually met with "WOW, that is amazing. Great mod, etc). The higher the calibration, the more "wow". So if anyone wants to compare numbers with the people with the top numbers, you have to calibrate your gear for higher numbers. If someone with a "high calibration" sells lights, for many it will seem as if the modded lights are better then others, but really, the difference is just the calibration. At least that is what we have to assume, because the numbers are rarely comparable.

Its quite typical that someone compares manufacturer numbers. Say a TK61 with a modified TK61. A TK61 are rated at 1000 lumen. A modified might be measured to 1600 lumen. Many expect a 600 lumen/ 60% increase from stock. But they may not be aware that the guy who modded a light would measure the stock light to be 1200 lumen. Only making the increase 400lumen/33%. Quite a difference between a 33% gain and a 60% gain just because of a 20% difference in stock calibration.

Ive often heard about people who are not able to replicate numbers from other peoples mods. This may be due to issues with the mod, but in many cases, its just the calibration of the measuring equipment.


More example differences:

Here are some numbers for you. Please study them a little bit. All on Supbeam K50. There are many lights I could do a lot of funny numbers with. But K50 is one of the lights where many have shared numbers, and its fairly recent.

1600 Lumen - 140 kcd - Official numbers by Supbeam (Notice the high lumen number and "super low kcd")

1306 Lumen - 545kcd - JMpaul320 numbers from K50 modded vinh de-domed emitter and current increase (reference and more numbers here)

1450 Lumen - 188kcd - Selfbuilt numbers (Measured with calibrated lux meter, and the "typical ANSI lumen calibration")

1656 Lumen - 208kcd - (one of Tom E`s K50s. High lumen club + LX1330B light meter)

1642 Lumen - 218kcd - DBCstm stock K50 numbers. High lumen club + LX1330B light meter I assume. Quite similar numbers to Tom E, but even higher kcd)

Looking at JMpaul320 lumen numbers its obvious he belongs in the "ANSI rated lumen club". Despite the current increase, with losses from a de-domed emitter he still have the lowest lumen output, and its the only modified light.  He probably uses LX1330B light meter. At least that is the meter he used before. His lux readings are always high, lumen readings seems to be typical ANSI numbers, although I have not studied them much.

If you ask DBCstm how many lumens his Supfire M6 with a FET driver have, he will say close to 5000 lumen.

If you ask JMpaul320 how many lumens his Securitying with FET driver (modified by RMM?),  he will probably say about 3800 lumen. Sure, Dale`s M6 will have slightly more lumens with lower CRI U2 emitters and it probably have slightly less heat sag due to M6s better construction. But, it just shows to some extend the difference you can get by asking two people on pretty much the same light how the output difference is.


A possible improvement that I believe would benefit everyone:

The "high lumen club" adjusts down their lumen ratings with 7-15% (depending on how high their readings are compared to others.)

People with LX1330B (and other very high reading lux meters) adjusts down their lux readings with 2-7%.

 

Some last thoughts:

Sharing amp numbers, before/after mod, comparison with other lights always helps put numbers in a context. But in many cases, that is not done. 

I think the purpose of sharing numbers is to contribute with data the is meaningful and helpful to others. If the numbers between different people have too large differences, it can be more misguiding than helpful. I believe that is often the case when "newcomers" check out numbers from "veterans" and assume they to some degree have comparable numbers and that these numbers can be compared with manufacturers of premium lights. Personally, I always have to "adjust" various peoples numbers in order to be able to get a better comparison. If the people with the "inflated numbers" would try and get on level with the "ANSI guys", then I think it would benefit everyone and make numbers more comparable and useful.

Wouldn't it be better if everyone tried to be on the same page?

BLF LED database – collaboration spreadsheet and latest news about where to buy LEDs
http://budgetlightforum.com/node/19342

unknown00101
Offline
Last seen: 7 months 3 weeks ago
Joined: 08/16/2013 - 11:53
Posts: 4342
Location: USA

Too many variables from what I’m seeing.

DBSAR
DBSAR's picture
Offline
Last seen: 12 hours 29 min ago
Joined: 02/11/2013 - 23:28
Posts: 6382
Location: Newfoundland, Canada

it is difficult to get perfectly accurate readings i agree. i do what most others do, is calibrate/calculate the Lux/Lumens reading based from testing ANSI rated known lights in my sphere.

That Canadian flashlight guy & Lantern Guru -Den / DBSARlight

RaceR86
RaceR86's picture
Offline
Last seen: 5 years 8 months ago
Joined: 02/22/2012 - 08:32
Posts: 3777
Location: Norway

Im not seeking perfection. I would just like to see that the gaps between people that seems to consistently be around 15-20% off. Based on posts by Tom E, im quite sure he have noticed this and is aware of the "issue". Its not hard to notice that certain guys are consistently much higher than others.

I can easily measure a light just based on ceiling bounce, and then easily choose if I want to be similar to the high lumen guys or the "ANSI lumen guys" despite all the variables. These days, I currently somewhere in between. Although if we could all meet with the "ANSI guys" that would be better for everyone IMO.

Most of the ANSI lumen guys are very close to each other. And then there is a gap to others.

For people who dont have a lot of comparison lights, premium lights, etc to use as reference, it may not be that easy to get a good calibration. Im certainly not claiming that. But even at my level (no integrating sphere), I easily notice the differences in both lux and lumen numbers. For the people who share numbers all the time, have measured xxx amount of various ANSI rated lights  etc, it seems fairly easy to get closer to other..

BLF LED database – collaboration spreadsheet and latest news about where to buy LEDs
http://budgetlightforum.com/node/19342

HKJ
HKJ's picture
Offline
Last seen: 17 hours 47 min ago
Joined: 05/24/2011 - 12:23
Posts: 7433
Location: Copenhagen, Denmark

RaceR86 wrote:
Lumen:

...

I believe that some of the "top" reviewers and light buyers on BLF are typically within 7% of those numbers. _the_, johnnymac, HKJ, etc. I personally consider the work all these people have done with several hundreds of lights combined to be the benchmark.

I do not remember publishing any lumen values in my reviews. I have never had anything that I trust for precise lumen values. The "Estimated lumen" is not a measured value, but as it says an estimated value and it is based on the specified maximum lumens.

I know that in some of my comparison the scale looks like lumen, but I do never say it is lumen.

 

 

My website with reviews of many chargers and batteries (More than 1000): https://lygte-info.dk/

M4D M4X
M4D M4X's picture
Offline
Last seen: 1 month 4 weeks ago
Joined: 03/19/2014 - 05:17
Posts: 8821
Location: Austria (GMT + 1)

is there a cheap and easy to do possibility to get 2 fixed amounts of Lumens to “calibrate” our meters within some percent?

PLEASE NOTE
i do not work in "reviews, deals and codes" for the time being
maybe M4D M4X will return one day, but until then:

THANK YOU FOR YEARS OF YOUR SUPPORT AND FRIENDLY CONTACTS!

HKJ
HKJ's picture
Offline
Last seen: 17 hours 47 min ago
Joined: 05/24/2011 - 12:23
Posts: 7433
Location: Copenhagen, Denmark

M4D M4X wrote:
is there a cheap and easy to do possibility to get 2 fixed amounts of Lumens to "calibrate" our meters within some percent?

First point is to get a lumen meter, it is not cheap.

All the home made integrating devices might work more or probably less well. To test it: Try aiming the light in different directions into it, the lumen reading must be stable if it is a truly integrating device.

My website with reviews of many chargers and batteries (More than 1000): https://lygte-info.dk/

Old-Lumens
Old-Lumens's picture
Offline
Last seen: 4 years 10 months ago
Joined: 11/04/2011 - 11:39
Posts: 7478
Location: Tyler, TX, USA

I have read and read over and over about this, on CPF and here. There's much better info over there. Basically, you will never be "within 15%-20%" with a home setup. The only true numbers will cost thousands of dollars. You can use home systems for "guesstimating" only and since there is no standard for these home systems, then there is no way to control the numbers, or the fact that they are all so far off from each other. The only way to find anything out would be to:

Go to a company that has the high priced system and have certain lights tested. Take your sphere with you to test your setup against theirs and make calculation adjustments to your readings, to get close to theirs.

Then send out those exact lights and exact batteries to others and have them do the same thing with their systems, till you had a base of a few people that were getting the same results consistently. Then it all has to be rechecked and verified once again every time you use the system, before trying a new light in it.

Not feasible and that is why I did not try to make a sphere. Even if you could make one that was close, it would cost thousands of dollars to buy a bunch of lights that have been ANSI qualified and it still wouldn't help, since you don't know what they used to power them at the time of testing, or the conditions of the tests. I don't listen to any of the numbers I see around here, because no one here has the correct setup to accurately test. They never will. If you really want good numbers, then you would have to buy a real integrating sphere and the software and accessories to do correct testing. before I ever believed any of the results. What we do around here is total guesswork and it's never really going to be accurate, no matter how much some people "believe" they are doing it right.  Saying joe should cut his results by 15%, or john should do this, because he has xxx meter is just more total guess work and does not help solve the problems.

No one should be publishing their lux/lumens readings from any home meters or spheres, because there is no assurance at all, that they are even close to correct, so it's not good to even post them. Same for throw numbers. It's misleading for others. I don't mean that people are doing wrong on purpose, but it's still wrong. Unless it's lab checked and verified numbers, it's not relevant information. Better off just guesstimating from amps, if true amp readings can be gotten. Mine sure aren't.

Just me spouting off, I do a lot of it any more.

My PayPal address: oldlumens (insert the @ sign here) gmail.com

My YouTube Flashlight Video Channel

The BLF Modding Links Thread 

http://imageshack.com/a/img922/1374/jQ2wdL.jpg

 

M4D M4X
M4D M4X's picture
Offline
Last seen: 1 month 4 weeks ago
Joined: 03/19/2014 - 05:17
Posts: 8821
Location: Austria (GMT + 1)

Was I was asking for is a “thing” that emits exact known Lumens…

by comparison it should be possible to get good numbers even on a cheap meter…

PLEASE NOTE
i do not work in "reviews, deals and codes" for the time being
maybe M4D M4X will return one day, but until then:

THANK YOU FOR YEARS OF YOUR SUPPORT AND FRIENDLY CONTACTS!

HKJ
HKJ's picture
Offline
Last seen: 17 hours 47 min ago
Joined: 05/24/2011 - 12:23
Posts: 7433
Location: Copenhagen, Denmark

Lumen is difficult to measure, but lux is not nearly as difficult.

The main problem with lux is filtering: Lux is defined to follow the eyes sensitivity to colors, a silicium sensors does not have the same color sensitivity. This can be fixed with filtering, but good filters cost money, another way to fix it is with different calibrations, depending on light source, i.e. a meter that can be switched between different light sources does not have filtering, but only calibration for the light sources.

A meter like this is fairly good for lux:

My website with reviews of many chargers and batteries (More than 1000): https://lygte-info.dk/

tempo
Offline
Last seen: 1 day 2 hours ago
Joined: 06/28/2014 - 08:59
Posts: 659

M4D M4X wrote:
a "thing" that emits exact known Lumens

any flashlight with ANSI rating

texaspyro
Offline
Last seen: 2 years 3 months ago
Joined: 04/29/2011 - 12:43
Posts: 4593

ANSI ratings mean absolutely nothing… just like UL/CSA/VDE labels. They are only as good as the paper they are printed on. Anybody can say their product is ANSI/UL/etc tested. Whether or not it actually is is pretty much a matter of trust. Even checking UL file numbers… those can be copied even easier than the product being counterfeited.

As far as lumen numbers are concerned… unless you have a decent sphere they mean absolutely nothing. Integrating ceilings, bathrooms, boxes, pluming pipes, eyeballs, or squirrel butts, etc are useless and misleading.

Lux numbers are a bit better… but I have seen cheap lux meters from the same maker, shipped at the same time, disagree by 30%.

ImA4Wheelr
Offline
Last seen: 1 month 1 week ago
Joined: 02/03/2013 - 14:51
Posts: 7935
Location: SC

Good thread.  Good idea to have a standard BLF approach that members can try to implement, but not be beaten up for when they don't.

I've never tried to measure lumens as I don't even know where to start with a ceiling bounce (e.g. how far from ceiling with light, then with meter, what color and sheen paint on ceiling to use, popcorn or flat ceiling, etc, etc.).

For throw, I think it is good for folks to include the distance they took the measurement.  I try to do that, but I haven't identified the meter in my reports.

For stock K50 V2, I measured 198kcd with the ubiquitous HS1010A (I think that's the model number).

Cereal_killer
Cereal_killer's picture
Offline
Last seen: 1 year 9 months ago
Joined: 07/22/2013 - 13:10
Posts: 4005
Location: Ohio
tempo wrote:

M4D M4X wrote:
a “thing” that emits exact known Lumens

any flashlight with ANSI rating

No, manufacturers openly admit you will see +/- 7% from one exactly the same light to the next, in reality 10% difference isnt uncommon. You can buy a calibrated reference light (or build one and have it professionally certified at a lab) but an off the shelf flashlight, even from top tier manufacturers, is NOT “ a thing that emits exact known lumens”.

 RIP  SPC Joey Riley, KIA 11/24/14. Now I am become death, the destroyer of worlds.

Werner
Werner's picture
Offline
Last seen: 2 years 3 weeks ago
Joined: 10/19/2012 - 15:00
Posts: 3679
Location: Germany

I think about this things often and came to no solution because like others said there are to many different variables.

I once hoped we would all “calibrate” our meters to the olight i6 which everyone bought and which seemed to have a output which is not sensitive to the used batteries…but it never happend.

A funny option would be constant current light which we would pass around and everyone can measure its lumen and lux. Maybe a man in the middle which gets submitted all the data and once everyone has it had it gets published. That would be fun to compare and I am sure we would see huge differences like racer said.

Another funny option would be a cardbox “integrating sphere” produced in china and send all over the world so that everyone has about the same…

djozz
djozz's picture
Offline
Last seen: 1 hour 21 min ago
Joined: 09/07/2012 - 17:04
Posts: 17993
Location: Amsterdam

An idea I have had for a while, inspired by a suggestion never carried out by match:

I am going to build a simple 18650 flashlight one of these days, lowly driven by a lineair driver (I have done this before, the output will be very constant and virtually independant of battery type or state) , then measure lux@1meter with my luxmeter and measure it in my integrating sphere and produce some djozz-lumens Wink . Then I am happy to send it on a roundtrip around the world to other BLF-members, and have them measure their DBC-lumens, TomE-lumens or whoever is interested. And finally I receive it back and see if it still reads the same.

Using one and the same flashlight will at least reduce the number of variables (it will certainly not solve the problems related to the bad colour filters on the sensors of cheap luxmeters), but beside that I think it is a very fun thing to do Smile

 

djozz
djozz's picture
Offline
Last seen: 1 hour 21 min ago
Joined: 09/07/2012 - 17:04
Posts: 17993
Location: Amsterdam

You just beat me there posting about the pass-around flashlight, Werner Smile

Tom E
Tom E's picture
Online
Last seen: 8 min 46 sec ago
Joined: 08/19/2012 - 08:23
Posts: 14693
Location: LI NY

I hope no one reading my posted measurements consider them the golden standard, or think I have some magical home setup that has a 0.0001% tolerance. NIST comes to me btw to calibrate their test equipment Smile.

What I don't like is the cherry picking of my posted #'s and emphasizing like it's typical. I can't explain all the differences, but many lights I've measured dead-on to manu specs or within an acceptable tolerance (maybe 3% is acceptable?), others lower, others higher. Sometimes I know I see variation in the positioning of the flashlight head on the glass of the PVC lightbox but I use it as precisely the same as possible, and I can't notice a consistent pattern that high throwers, high flooders, low lumens, or hi lumens get higher/lower readings. Do I or did I screw up sometimes? Sure I did - no doubt.

I do believe (from real, actual direct comparative measurements) that my $35 meter is accurate enough and compared very well to a much more expensive ExTech meter. In fact two of the $35 meters matched up well to an ExTech meter.

Also I have no idea where the notion or inference came from that we did not use ANSI rated lights to do the initial calibrations. Of course a multitude of ANSI rated lights were used. In fact there were probably at least as much or more ANSI rated lights used in the calibration process than others making the claim. Did every PVC based lightbox/meter combo get the same level of testing? - No. One setup was tested more extensively, others were built in the same design, then lights were shared, measured, compared, and calibration factors derived as a result.

However many ANSI lights are/were used by anyone, though, is not enough. There's too many variations from piece to piece, LED to LED, reflector to reflector, etc. in the so-called ANSI lights, noted and documented by many, like vinh and Michael at OSTS.

Now let's take a look at conservative/aggressive approach to this measurement dilemna. I believe my approach was conservative to the calibration, then aggressive to the measurements. Out of my collection of hot cells, I'll post the #'s on the cell that gave me the highest readings. How does this compare to the so-called ANSI lights? Well all depends on the approach the manufacturer, or contractor/facility the manufacturer hires to do the measurements. In theory there should be only one measurement - the accurate one, but that seems hard to obtain with almost any equipment. I think some manufacturers tend to take a conservative approach, fearing they will get in trouble by over-stating their "ANSI" claims, while under-stating is far less risk.

Passing one light around the world will help, but you really need a few representing different sizes and designs.

Tom E
Tom E's picture
Online
Last seen: 8 min 46 sec ago
Joined: 08/19/2012 - 08:23
Posts: 14693
Location: LI NY

Here's a good example of readings I took on the NiWalker MM15, stock, on May 22nd:

MM15 clearly states "FL1 STANDARD" on their published #'s:

              NiWalker        my lightbox/meter  (all in lumens done at 30 secs)

Mode 1:   6                   11

Mode 2:   180                167

Mode 3:   450                483

Mode 4:   880                874

Mode 5:   1950              2013

Mode 6:   5233              5236

Throw:     19.6 kcd         17.2 kcd (measured at 5m)

Now I consider these dang close. Some might say mode 1 is way too far off, and it certainly is in percentage, but, you have to consider this is a high powered light using two MT-G2's. When these two LED's are cranked down that low, system tolerances andd variations are going to have a bigger effect, so this is not surprising to me.

In fairness, for Mode 6, I had 3 #'s written down, but this one was marked "cool", meaning the light was off long enough for temp. not to be a factor. The other readings wee 5,059 and 5,202.

Do we trust NiWalker enough to ensure they followed the ANSI FL1 standards to the letter when doing these tests? Me? No freak'n way... I don't trust any of them, be it Olight, Klarus, whomever. I know way too much about what goes on (or what doesn't).

Ohh - btw, what if NiWalker can't get P0 bin LED's, but found a source of Q0's, so they do their next batch run with Q0's. Do they re-test under the FL1 standard for the new batch run at a cost of $1,000's, maybe more? Do they update the shipping boxes with the new specs, and throw out the current inventory? Or maybe it's just for one batch, so do they pay for a special run of the shipping boxes, probably at another insane cost for a low volume? Oh sure they do - they only think of the customer, I'm sure... Smile

Itinifni
Itinifni's picture
Offline
Last seen: 4 days 6 hours ago
Joined: 11/03/2011 - 20:45
Posts: 792
Location: The Great Northeast, USA

I recently purchased an HS1010A. I haven’t started any output tests but have completed throw numbers for all my lights.

I practiced for a while until I found I could get repeated, consistent results.

I had a few lights (Jacob A60 and two T08s) that I measured before and after mods. The mods I did on all three lights were pretty much standard affair, higher amp drivers, de-domed emitters, copper, resistance mods, etc.

I was surprised when I started comparing my numbers to similarly modded lights by others. With all 3 lights I found my stock numbers were a bit higher and modded numbers were significantly lower.

I did not expect in any way to be matching the kind of throw attained by some of the folks here but thought I would at least be in the ballpark.

I’ve decided as long as I can produce consistent results I can use my numbers to quantify my mods. That’s really all I wanted to do with the meter in the first place.

I remember a time, when I searched for lights to fit my needs. Now I search for needs to fit my lights.

DB Custom
DB Custom's picture
Offline
Last seen: 2 days 21 hours ago
Joined: 01/13/2013 - 22:28
Posts: 20733
Location: Heart of Texas

It might be agreed that Texas Pyro has some of the best testing equipment amongst us. Perhaps he will agree to test a light, any light, and see how it compares to the testing we do at home? I don’t have fancy computer print-outs, and yes, sometimes I can barely read my own notes even if I can reliably find them.

But wouldn’t it be interesting to pick out, say, the SupFire M6, charge up the cells and run my tests then charge the cells back up and send it to TP for his run and see how the 2 compare? TP would then argue that his expensive equipment was right, however the numbers fell. Wink

For me, the lightbox tells me what the light is doing when I get it, and then how much difference my mods made. If the difference is not what I expected, I then check ground and other issues and test again, sometimes then finding the expected results, sometimes not. But it’s a guideline, for me, right here, doing the work and then confirming or busting whether the work was worthwhile.

Yes, I use a LX1330B. You will probably find that myself, rdrfronty, Tom E and Richard all have similar results from lights, because we’re all using the same meter and same base system. rdrfronty and manxbuggy1 spent many hours, days, weeks even, testing as many verified lights as they could get their hands on. (and their hands hold a LOT of lights!) My own box was verified against theirs, and with 20 or more of their own verified lights. So I feel confident that the results are accurate enough for me to base my builds on.

I don’t think any of us has ever said “THESE ARE ABSOLUTE NUMBERS!” Because we know the variables are everywhere. I can almost guarantee that you get a major factory to test a light, chart the numbers then send it to me, and after I test it and write everything down, send it right back to em for a second round and their own numbers will not match the first run. It’s just the way these typically work. The LED get’s burned in, the cells get some age on em, the driver has it’s own quirks and oddities as it’s components get hot and cool down. And such is the way of life.

Maybe I can build one of the Eagle Eye X6 lights and test it, then send it around to everyone else that has a box and we can average all the numbers, compare notes, and adjust our multipliers accordingly so we can all be on the same mystical page….

Richwouldnt
Richwouldnt's picture
Offline
Last seen: 7 months 3 weeks ago
Joined: 05/24/2014 - 00:22
Posts: 1354
Location: Reno, NV, USA, Terra, Sol, Milky Way, Known Universe

A VERY interesting thread as I have just had RMM modify a group of lights for me and I asked to receive his sphere results for all of the lights he did for ALL levels, not just maximum. Some of the max readings were higher than I expected and some lower but at least I now have a record of what the lights were calculated to be putting out based on Richard’s equipment and measuring technique.

From our email exchanges I got the impression that Richard uses his sphere the same way that Dale says in his post,, as a reference for checking that his light modifications are putting out what he expects based on experience. Is it dead on to ANSI, almost certainly not. Is it in the same ballpark? I hope so. Just based on Cree binning specification limits I would expect ANSI spec lights to give different results for different individual lights even in a certified lab with NIST calibrated equipment.

A max modded SRK soup can triple and a max modded M6 were very close which I would expect with the same LEDs and batteries in both and basically direct drive on maximum output. Both within 1% and about 4600 Lumens. I have seen similar readings for a heavily modded M6 from another source. The champion he did for me though is a Supfire L1 which gave a reading of 7000+ lumens max and it now has the latest 7 levels RMM offered firmware in it with “hold for off” from all output levels. Even if optimistic that is one hot light and with a MUCH better UI IMO than the original.

Added:

All the tests RMM did on my lights were done with Sony VTC4 unprotected batteries which are high current low internal resistance batteries. Per my understanding the use of these or similar high current low resistance batteries can substantially increase the output of direct drive in highest mode lights such as most soup can lights are.

Rich Wood
Reno, NV

djozz
djozz's picture
Offline
Last seen: 1 hour 21 min ago
Joined: 09/07/2012 - 17:04
Posts: 17993
Location: Amsterdam

I had some conversarions by email with a very friendly guy from a light testing company about how flashaholics measure their output. He was quite impressed by the level of knowledge there is around here, and the testing equipment we make, but 1) he warned about the bad colour filters of cheap luxmeters, giving inaccuracy for different wavelengths, and 2) he could not find out how on earth we dare to use the word 'lumen' without going to a test facility to calibrate the equipment (this was not because he would make money from that), I guess he was not at all impressed by the accuracy of ANSI-numbers going with flashlights.

comfychair
comfychair's picture
Offline
Last seen: 5 years 5 months ago
Joined: 01/12/2013 - 05:39
Posts: 6198

It will be impossible to get numbers accurate enough to compare one person's readings directly with another person's, not with home-brew equipment. Accepting that the numbers don't mean anything other than when compared to numbers that came from the same test setup, and are relative, is a much better/safer situation.

I have a CT-1330B, I only use it for ceiling bounce lux, and only use it to compare two or more lights in the same session - I try whenever possible to not compare a current reading to one that was taken days or weeks before. Better to assume they won't be comparable than to assume they might be when they really aren't.

djozz
djozz's picture
Offline
Last seen: 1 hour 21 min ago
Joined: 09/07/2012 - 17:04
Posts: 17993
Location: Amsterdam

Actually I still have the ambition to get my integrating sphere numbers real, within 1% of reality. I admit that still can't guarantee anything close, but maybe one day......

(not soon though Sad )

M4D M4X
M4D M4X's picture
Offline
Last seen: 1 month 4 weeks ago
Joined: 03/19/2014 - 05:17
Posts: 8821
Location: Austria (GMT + 1)

wiki ses: Lumen = Lux·m²

so why not meassure how “enlightened” one m² is when poining our guns at it? Wink

what about using a white cardboard circle with 1 m² (1,128m Diameter) or 0,75m² (D=0.977m)

mount that “under the sky” so no reflections from somewhere else (dark ground or gras will do)

have a given distance (1 or 0,5 m ) between the Front of the light and the “ceiling”

results are now more comparable…

(excuse my drawing – right now i am carring my sleeping baby and did this walking around…)

for sure we have to find out the exact distances, but this looks cheap, easy to build and should give a BLF-Constant
-> how much light is my light

PLEASE NOTE
i do not work in "reviews, deals and codes" for the time being
maybe M4D M4X will return one day, but until then:

THANK YOU FOR YEARS OF YOUR SUPPORT AND FRIENDLY CONTACTS!

comfychair
comfychair's picture
Offline
Last seen: 5 years 5 months ago
Joined: 01/12/2013 - 05:39
Posts: 6198

Is the exact same type of white cardboard/posterboard/paint available everywhere in the world, all with the same reflectivity? If painted, even with the same brand and type of paint, the number of coats and thickness of each coat and the total paint thickness and the surface finish (brush or roller? WHICH brush, coarse bristle or fine? which roller??) will all change the readings. Still too many variables. Assuming they will be the same when they really aren't will only make things worse.

“What gets us into trouble is not what we don't know, it's what we know for sure that just ain't so.”
—Mark Twain

djozz
djozz's picture
Offline
Last seen: 1 hour 21 min ago
Joined: 09/07/2012 - 17:04
Posts: 17993
Location: Amsterdam

comfychair wrote:

Is the exact same type of white cardboard/posterboard/paint available everywhere in the world, all with the same reflectivity? If painted, even with the same brand and type of paint, the number of coats and thickness of each coat and the total paint thickness and the surface finish (brush or roller? WHICH brush, coarse bristle or fine? which roller??) will all change the readings. Still too many variables. Assuming they will be the same when they really aren't will only make things worse.

“What gets us into trouble is not what we don't know, it's what we know for sure that just ain't so.”
—Mark Twain

And does that white paint / cardboard reflect all visible wavelengths equally well? 

I really like that Mark Twain quote, it applies painfully well to these problems Undecided

RaceR86
RaceR86's picture
Offline
Last seen: 5 years 8 months ago
Joined: 02/22/2012 - 08:32
Posts: 3777
Location: Norway

Ill just start with saying that my observations that lead me to starting this thread is done over the course of about 1 and a half year. Its way more than "some guy had a light with that many lumens with those emitters, and some other guy had a fairly similar light with that many lumens and and some other emitters" and then drawing conclusions.

Im not claiming to be correct. My observations could be a off, and my thoughts based on those observations could be even more off. But I think that many of the variables will average out, and some clear patterns can be seen. For those who thinks its imposibble to see these difference due to too many variables. I could certainly be up to taking some bets on how various peoples numbers compares to each others. 


HKJ, sorry, should not have put you one the list.


Texaspyro. I pretty much agree. ANSI FL1 numbers does not mean much. They are all over the place from manufacturer to manufacturer. Some manufacturers even seem all over the place from light to light compared to others. But, if someone buys 10 different Fenix lights, and use those for calibration. And some other guy does the same. Im quite confident they will be close and "on the same page". Which makes them fairly comparable and certainly much more useful then the wide spread many have today with their calibrations.  Many have used Fenix lights as a reference. And that is an example of something that can help people to get their calibration closer.

And yes, there can 30% difference in the same cheap and similar named lux meter. Some meters does have fairly good consistency though. This is why its important for everyone to calibrate their lux meters as well. But right now, I get the impression that the lux meters with the highest possible numbers have become the benchmark for many people.


Tom E wrote:

What I don't like is the cherry picking of my posted #'s and emphasizing like it's typical.

 

I just took your number from OP in the K50 mod thread. Ryan put it there for comparisons with others. This will always happens. Someone posts a number. Others puts those numbers on a list in order to try and get useful comparisons. Comparisons will not be useful when people on a list knows that their calibration is consistently higher or lower with others. On certain lists, many readers will not be aware of who did the different numbers, and how the calibration compares.

You can always find ANSI rated flashlights that you match well with or that makes your numbers seem "conservative. But what matters is peoples averages.

Ill quote a recent post from you.

"There seems to be two camps on lumens and kcd measurments - mine, rdrfronty, and DBCStm seem to agree pretty closely, a couple of other BLF'ers also. But then there are selfbuilt's and some others that range lower."

Clearly you have noticed this as well. That is because of averages. Some people consistently have a well match. And then many other people have consistently lower numbers then you guys. When it comes to lumens especially. Who else than you, DBCstm, Richard/RMM, rdrfronty and manxbuggy have as high lumen numbers as you guys? Does anyone have a calibration that consistently puts out higher numbers?? (Personally, I have not noticed that)

Ill ask the same about lux. Does anyone, or several members have a light meter or lux reading that seems to have higher numbers then you guys, most with LX1330B? (Personally, I doubt many people with different meters are much likely to get higher numbers)

Does many people have lower nubers? (YES!)

Im not trying to prove anyone wrong. Im just saying that people are not on the same page in terms of calibration, and that changes can be made in order to get people closer.


DBCstm wrote:
Yes, I use a LX1330B. You will probably find that myself, rdrfronty, Tom E and Richard all have similar results from lights, because we're all using the same meter and same base system. rdrfronty and manxbuggy1 spent many hours, days, weeks even, testing as many verified lights as they could get their hands on. (and their hands hold a LOT of lights!) My own box was verified against theirs, and with 20 or more of their own verified lights. So I feel confident that the results are accurate enough for me to base my builds on.

Yes, I have noticed that you guys have similar results. You guys are on the same page. Ofcourse, there will be some natural minor differences from time. On average, you guys always have the highest numbers. All of you, at least you, Tom E, and Richard have measured a stock TK61 to be about 1200 lumen just to use one example (if I remember correctly). All of you will also measure higher lux compared to stock values. While many others will be close to the stock numbers from Fenix.

The point is, if you compare your numbers with several other guys who have also spent a lot of time measuring lights and getting their calibration right. Your numbers will generally be higher on average.  I expect that you guys will be roughly 10-22% higher on average when it comes to Lumen compared to say, Selfbuilt and JMpaul320. Although, there will be exceptions that will show quite similar numbers.

I cant say for sure, but selfbuilt measured Supbeam X60 to about 5100. _the_ measured it marginally higher. I would not be surprised if you measured it to 6000 lumen. I expect your measurement to show at least 5500 lumen. Selfbuilt and _the_ both measured that light to 160kcd. Spot on! I can almost guarantee that you will measure it higher.

Here your can see some of Selfbuilt work when it comes to calibration. Link 1. Link 2. JMpaul320 have a thread here. He measures lights for people, and also do a lot of measurements on vinh lights. (same light meter as you btw, with same high numbers when it comes to kcd. Certainly a difference in lumens though).

These are two examples of guys who have done calibration work. Someone correct me if im wrong, because Ill throw out some names that Im not 100% how their measurements compares as I have not studied them properly and cross checked and done lots of on paper comparison. But Relic38, JohnnyMac, _the_ in general seems to be closer to Selfbuilt and JMpaul320 in terms of lumens. Here are 5 different people who seems to be able to get not perfectly within the each other, but not that far off on average when it comes to lumen and lux. Im quite confident that these guys are closer to eachother than the "high numbers" camp.

The funny thing is that the gap between these two "camps" are so large that I on several occasions in a row have been able to put my numbers in between them. To me, that means people in the "two camps" are not on the same page. 


Where should mine and other peoples calibration be pointed towards?

The issue for me, is that I don't know where I should try to aim my calibration towards out of these two camps. Morally it makes more sense to aim towards the lumen numbers that all the different people who reviews lights and independently have come up with their own calibration that seems to be somewhat similar. I believe these numbers are realistic and certainly doable based on Cree`s own numbers. I believe I would be more in line with more people who have done independent calibration work and come to somewhat similar results. I believe my numbers on average would be more comparable with ANSI numbers from different manufacturers that way too. Which means that my modded lights can more easily be compared with manufacturer ratings (even though that will often be a bit off depending on manufacturer).

But, its more temping to aim towards the guys with the high numbers. Simply because it sucks having low numbers when the typical people I compare mods with have a higher calibration. I mod lights, and most of the numbers seen from modded lights on BLF are from the guys in the high numbers camp. And that camp is growing. Tom E always used to say, that at least my numbers are comparable with rdrfronty if questioned. And for a long time DBCstm have joined. And he shares a ton of numbers. And RMM/Richard is on that same calibration. And more people want lights and numbers from him. And those people might start using those lights for reference... It seems to be a growing camp there. And that does not help anyone...

For me personally, id like to see everyone try and meet in the "Average ANSI/Typical reviewer/conservative" call it whatever you will camp. I don't see that these people will increase their numbers based on all the calibration work many of them seem to agree on, and how close they on average are to many of the lights they review.

Im not sure what the high lumen camp have to loose in order to meet the others. Except getting results that will be more comparable with a larger number of people. Which I think would benefit everyone.

I do know that if I want my numbers to be in line with most modders who share numbers on BLF, I will have to add even more to mine despite that Im already on level or slightly above many others. The lumen gap is quite large IMO.

When it comes to lux. I know that my lowest reading meter might need around 30% increase to match the high numbers camp. My other meter might need around 15% increase. (Sure, some other guys have cheap lux meters that does match lx1330B, or are not far off.) But in general, if I increase my numbers more, ill create an even larger distance to some of the people and manufacturers that Im already above.


If someone wanted to get a reference light for calibration (lumen and lux). It would have to be several reference lights, with different beams and tints, and superlow and superhigh lumen readings in order to make it worth it.

If these lights were verified and shipped around as a calibration reference to different members. Would all the guys in the different "camps" adjust their numbers accordingly so that everyone could be as close as possible??? Unless that happens its not much use.

BLF LED database – collaboration spreadsheet and latest news about where to buy LEDs
http://budgetlightforum.com/node/19342

DB Custom
DB Custom's picture
Offline
Last seen: 2 days 21 hours ago
Joined: 01/13/2013 - 22:28
Posts: 20733
Location: Heart of Texas

Can it be as simple as the cells used? Fenix undoubtedly uses their own brand of cells when testing their lights. We (most of us) use the absolute best cell available with higher capacities and lower internal resistance. This can very easily make for the differences you’re seeing.

I’ve shown test results from 7 different cells in the same light, showing that the new FET drivers in direct drive or near it can be “tamed” with a lesser cell. Would this, then, hold true for the factory measurements as compared to our home measurements?

Just a thought. Maybe it’s the Texas air that magnifies the results down here, so good and pure. Smile

RaceR86
RaceR86's picture
Offline
Last seen: 5 years 8 months ago
Joined: 02/22/2012 - 08:32
Posts: 3777
Location: Norway

DBCstm wrote:
Can it be as simple as the cells used? Fenix undoubtedly uses their own brand of cells when testing their lights. We (most of us) use the absolute best cell available with higher capacities and lower internal resistance. This can very easily make for the differences you're seeing. I've shown test results from 7 different cells in the same light, showing that the new FET drivers in direct drive or near it can be "tamed" with a lesser cell. Would this, then, hold true for the factory measurements as compared to our home measurements? Just a thought. Maybe it's the Texas air that magnifies the results down here, so good and pure. :)

High discharge cells does not matter when it comes to peak output in the majority of properly regulated stock lights. Not that all premium lights are properly regulated. But many of the expensive lights with several cells often are. And many of those who are not regulated are not pushed as hard that you need high discharge cells in order to see their peak output. There will always be exceptions though.

Direct drive lights, and lights that rely on a certain voltage (that batteries will fall below) to maintain a certain amount of amps certainly makes comparisons much harder to do when people use different cells with different voltages. And that will happen. Its nice that you and many others state the cell, cell voltage and amp draw when sharing number for the direct drive hotrods. That really put things in context when measured that type of light.

BLF LED database – collaboration spreadsheet and latest news about where to buy LEDs
http://budgetlightforum.com/node/19342

Pages