Doing what it is supposed to do, CV (constant voltage 4.2vdc output) when you put a discharged battery the voltage will sag until the battery starts saturating on the charge.
You need to go ahead and push above 4.2vdc into the charger (4.8vdc-5vdc [standard USB voltage] nominal), it internally regulates it back down to 4.2vdc