This is could be an easy to solve issue, but it all depends. GearBest should refund you anyway, I believe. First of all, what I said before:

A possible way to fix the issue.-

Connect the battery to the circuit board via a DC/DC buck module. Like one of the following CC/CV XL4015 buck boards:

Attach the output/input battery leads from the solar light circuit board to the buck module, and tune the desired output voltage to the battery in it (left side trimpot). I recommend not going above 4V to maximize battery life in spite of only being able to attain a lesser capacity from the battery. You will not need to adjust the current potentiometer because the output voltage from the solar led board is limited to 4.7V or maybe a tad higher. This will prevent the XL4015 module to work in CC mode since it needs nearly 6V of input to regulate in constant current mode (tested by myself; I have a blue module and a few red ones), but this is no issue because the solar light circuit board will adjust output voltage to limit output current in any case. With this we have solved the battery charging problem.

Then there's another problem: to allow the solar light circuit board to drain current from the battery. This can be accomplished by attaching a set of leads from the battery to the circuit board, restricting the current direction from battery to board via schotty diode(s). This will produce a small voltage drop, though it can be limited by using multiple schottky diodes in parallel and selecting low Vf units. A handful of 1N5823 - 1N5825 or equivalent diodes would do, for example: https://www.ebay.com/itm/10pcs-10SQ045-10A-45V-Schottky-Rectifiers-good-quality/282397972819 A handful of 10SQ045 diodes, go big or go home LoL!

Finally, I'd get some new cells for it. That 1200mAh POS cell is more than probably dangerous dead meat already.

Cheers ^:)