Convert 18650 Innogear Solar LED lights to AC 110v power source?

Hello!
First post here but long time lurker :slight_smile:

Sorry if this has been asked before but I searched and searched and still unsure.

I have 4 Innogear solar powered led lights that die off too early to my liking and would like to connect them to a constant power source.

the lights have 4 led’s each and the battery’s inside are 18650x1 per light(of 4 led’s)

from my research I need a 3.7v power supply that will transform the 110v ac to dc

the question is how many amps will I need to run the 4 lights with 4 led’s each?

using a multimeter I determined each light of 4 leds draws 0.2 amps

so do I need a power supply that has 0.8 amps then?

I found this power supply on ebay
http://www.ebay.com/itm/AC-DC-Power-Supply-Adapter-Wall-Charger-For-3-7v-li-ion-Battery-LED-18650-/262186190964?hash=item3d0b83c874:g:Y7wAAOSwwE5Waj9r

the output is :DC 4.2V±0.5V, 500mA

so would I need 2 of those to power all 4 lights? and is 4.2v too much ?

Thanks in advance!

s here are the lights on amazon
http://www.amazon.com/InnoGear-Powered-Landscape-Lighting-Waterproof/dp/B00WUD4EQG