Ripping data from a website

21 posts / 0 new
Last post
HKJ
HKJ's picture
Offline
Last seen: 10 hours 50 min ago
Joined: 05/24/2011 - 12:23
Posts: 7335
Location: Copenhagen, Denmark
Ripping data from a website

Sometimes I get strange complains, the last one I got was that my battery data was inconsistent.

What it boils down to is a guy trying to rip all the battery data from my website, but his software has problems with everything not being uniform. The “Official specifications” changes between batteries and the tags varies. The info box do not always has the same border and I use different formats for it (Depends on cells).

I must admit that I are not planning on helping him or fixing the “issues” he complains about. LOL

My website with reviews of many chargers and batteries (More than 1000): https://lygte-info.dk/

ChibiM
ChibiM's picture
Offline
Last seen: 15 hours 45 sec ago
Joined: 05/09/2011 - 10:25
Posts: 6428
Location: Holland

that's a total shame on him.. wow. 

I hope he hurts himself trying to do it, lol. 

 

Just wanted to say thank you again. You've helped thousands of people on choosing a charger and/or batteries. 

raccoon city
raccoon city's picture
Offline
Last seen: 2 hours 53 min ago
Joined: 10/06/2010 - 02:35
Posts: 14671
Location: रॅकून सिटी Palm Desert CA USA

Why won't you help me rip data from your website?!  }(

RapidLux
RapidLux's picture
Offline
Last seen: 2 hours 41 min ago
Joined: 03/24/2017 - 13:21
Posts: 464

I’m not sure what’s going on in the head of baboons with that kind of attitude. They seems to be somewhat high on themselves.

Omega_17
Offline
Last seen: 3 hours 8 min ago
Joined: 06/02/2013 - 16:56
Posts: 1032
Location: France

Why not helping him improving and sharing your data?

HKJ
HKJ's picture
Offline
Last seen: 10 hours 50 min ago
Joined: 05/24/2011 - 12:23
Posts: 7335
Location: Copenhagen, Denmark
Omega_17 wrote:
Why not helping him improving and sharing your data?

Because his request do not improve anything, except making it easier for him to rip the data.
My data is shared on my website.

My website with reviews of many chargers and batteries (More than 1000): https://lygte-info.dk/

Henk4U2
Henk4U2's picture
Offline
Last seen: 1 hour 43 min ago
Joined: 02/13/2014 - 17:52
Posts: 3641
Location: The heart of the Netherlands (GMT+1)

That’s the new today.
If a person sticks out his/her hand it is not always to shake your hand.
But he/she expects you to put something in his/her hand.

Well, if he/she wants your data, let him/her work for it.

You are a flashaholic if you are forced to come out of the closet, to make room for more flashlights.

MoreLumens
MoreLumens's picture
Offline
Last seen: 1 hour 24 min ago
Joined: 10/25/2019 - 07:08
Posts: 784
Location: Finland

Cant be mad. Makes me smile, but is it burglary anymore if they knock on your door first, ask nicely and you say come take my stuff?
Atleast they have manners. Blushing

sp5it
sp5it's picture
Offline
Last seen: 1 hour 23 min ago
Joined: 12/25/2012 - 07:51
Posts: 1126
Location: Poland

HKJ wrote:
My data is shared on my website.
We don’t know his intentions.
When your website will disappear, mirror copy will float around.
In the past I made copies of a few valuable to me websites. Information is still actual, websites and their owners are long gone. Mike

Think of how stupid the average person is, and realize half of them are stupider than that. George Carlin

 Anyone offended by my signature please fill Complaint Form. Thank you.

HKJ
HKJ's picture
Offline
Last seen: 10 hours 50 min ago
Joined: 05/24/2011 - 12:23
Posts: 7335
Location: Copenhagen, Denmark
sp5it wrote:
HKJ wrote:
My data is shared on my website.
We don’t know his intentions. When your website will disappear, mirror copy will float around. In the past I made copies of a few valuable to me websites. Information is still actual, websites and their owners are long gone. Mike

People are welcome to copy my website to their own computer, there have also been a few attempts to setup alternate views of my data, without I have complained.
But asking me to reformat my website for easier ripping is a bit much for me. It do not help by starting to tell me: “how inconsistent, contradictory information is”.
Battery tests are consistent and I will not accept blame for not receiving a full data sheet with the batteries. Some information is, of course, contradictory, because battery dealers do not always publish correct specifications.

My website with reviews of many chargers and batteries (More than 1000): https://lygte-info.dk/

CNCman
CNCman's picture
Online
Last seen: 20 sec ago
Joined: 04/07/2018 - 10:10
Posts: 705
Location: Mississippi Gulf Coast

HKJ wrote:

Battery tests are consistent and I will not accept blame for not receiving a full data sheet with the batteries. Some information is, of course, contradictory, because battery dealers do not always publish correct specifications.

.
Yes, and manufacturing tolerances give variations in product size, shape, volume etc.. which in turn gives variations in testing.
.
We greatly appreciate your diligence to accurate testing and understand these variables. Thumbs Up
.

CNC & Manual Machinist. Think outside the box too long , cannot find your way back in.

Never give up, Never surrender. Make someone Smile today.

wle
wle's picture
Offline
Last seen: 9 hours 14 min ago
Joined: 01/07/2015 - 13:49
Posts: 1960
Location: atlanta ga

i would just tell him to reinstall windows and come back then

"You never have the wind with you - it's either against you, or you're having a good day."
    Daniel Behrman, "The Man Who Loved Bicycles".
It never gets easy, you just go faster.   
-Greg Lemond.
       ,ø¤º°`°º¤ø¸,ø¤º°`°º¤ø¸,ø¤º°`°º¤ø¸

agent80
Offline
Last seen: 9 hours 33 min ago
Joined: 01/06/2013 - 02:28
Posts: 448

Feed his bot with alpha numeric gibberish values. Its possible to do that with a script. It would work only for his bot.

SammysHP
SammysHP's picture
Offline
Last seen: 4 hours 25 min ago
Joined: 06/25/2019 - 14:35
Posts: 498
Location: Germany

Am I the only one who would love to have a database with all specs, measurements and recommendations? Would make it so easy the query for specific properties.

Crawling over a website and parse information itself is not that bad. I’ve done it myself several times. It’s what you do with the data afterwards (and how much traffic it generates).

HKJ
HKJ's picture
Offline
Last seen: 10 hours 50 min ago
Joined: 05/24/2011 - 12:23
Posts: 7335
Location: Copenhagen, Denmark
agent80 wrote:
Feed his bot with alpha numeric gibberish values. Its possible to do that with a script. It would work only for his bot.

It is not that I have anything against people uses my data. Sometimes people has project where they ask me for a couple of csv files, they nearly always get that.
But ripping the all battery data and ask me to make it easier is a no.

My website with reviews of many chargers and batteries (More than 1000): https://lygte-info.dk/

BT100
Offline
Last seen: 7 hours 20 min ago
Joined: 03/26/2017 - 13:23
Posts: 119
Location: AZ USA

Thank you for all the work you do. Your website has helped me with battery and charger purchasing !

helicoil
helicoil's picture
Offline
Last seen: 3 hours 55 min ago
Joined: 06/29/2020 - 12:46
Posts: 27
Location: the Netherlands

Just see it like this …. if they ask you to make it easier for them … your work must be in high demand and that should be something of a good thing in itself wright ?

That they at the same time are only able to copy someone else’s work is a bit sad wright ?

To sum it up that’s ONE point for you and MINUS-ONE for them ! …. so basically a Win-Win for you ?

Just me thinking out loud ?

Cheers,

Martin

Parametrek
Parametrek's picture
Offline
Last seen: 5 days 3 hours ago
Joined: 02/08/2017 - 19:50
Posts: 161

I can relate to both sides. And I can tell you what happens when all of your data is neatly organized: they ask for the actual database instead.

On the data harvesting side: I have collected and organized more than 75000 pieces of information from almost 100 different sites. Nearly all of it by hand. HKJ’s PNG tables are extremely clean compared to some of the nonsense out there.

There are only 2 times when I will complain: when there is no website (Wuben and Convoy are what people ask for most) or the website is missing information that is trivial for the manufacturer to provide. (Zebralight has no candela or throw numbers!)

SammysHP wrote:
Am I the only one who would love to have a database with all specs, measurements and recommendations? Would make it so easy the query for specific properties.

Crawling over a website and parse information itself is not that bad. I’ve done it myself several times. It’s what you do with the data afterwards (and how much traffic it generates).

There is my battery database…. it doesn’t show all batteries in existence but it does show all that are for sale. What good is searching for cells that can’t be purchased? Right now the information in it is a mix of manufacturer specs and HKJ’s measurements where the manufacturers have exaggerated. What additional specs would you like to see it have?

You are in Germany so the european database should work nicely. Its got DE’s biggest battery seller Akkuteile.

Try the largest, fastest, most flexible flashlight search engine: parametrek.com

43 brands, 2375 models and still growing!

NEW: US battery database and European battery database

TheIntruder
Offline
Last seen: 1 day 23 hours ago
Joined: 03/09/2018 - 02:11
Posts: 197

You don’t owe them, or anyone else, anything.

IP theft, even casual, is rampant.

Data aggregators have their place, but the proper approach it would be ask for permission, and credit the source, not chide it for not making theft easier.

I think it’s safe to assume that the party is trying to benefit from your body of work, and neither credit nor compensate you for it. Even though that work is public, it’s still subject to certain rights, so I’d never consent to that, and respond accordingly.

xxo
Online
Last seen: 10 min 52 sec ago
Joined: 03/22/2019 - 14:35
Posts: 150
Parametrek wrote:
There are only 2 times when I will complain: when there is no website (Wuben and Convoy are what people ask for most) or the website is missing information that is trivial for the manufacturer to provide. (Zebralight has no candela or throw numbers!).

That kind of information is trivial if they just make it up, which is what it seems many (most?) do. It is expensive to have ANSI specs tested by a certified lab….much easier to leave things out or make up fake numbers.

SammysHP
SammysHP's picture
Offline
Last seen: 4 hours 25 min ago
Joined: 06/25/2019 - 14:35
Posts: 498
Location: Germany

Parametrek wrote:
There is my battery database…. it doesn’t show all batteries in existence but it does show all that are for sale. What good is searching for cells that can’t be purchased? Right now the information in it is a mix of manufacturer specs and HKJ’s measurements where the manufacturers have exaggerated. What additional specs would you like to see it have?

You are in Germany so the european database should work nicely. Its got DE’s biggest battery seller Akkuteile.


I know your website and use it regularly. But when will you include chargers? Big Smile