Nope not through the phone's speaker.
This is happening for both units I have (one that makes a modulated whine when plugged into the mains and another that is quiet-no whine whatsoever when plugged into the mains)
So the hum (basically low frequency sound) is coming from both units into the headphones. As I said I immediately noticed that since for that phone I use actively 3 different 5V power supplies.
I did some further checking and it appears that while plugged into certain main sockets the hum is very low, to the point of "passable" and in others it is annoying. (So it is clear that this will totally random for many users and directly related to their hearing capabilities at some point)
That being said with others there is no hum whatsoever no matter what mains socket I plug them in.
Doing the same check for the unit that emits a high pitch noise by just having it plugged into the mains socket, it appears it has zero effect on the high pitch whine. Actually I can pick it up in my hand and it will whine while holding it in the hand, yes while not plugged into the mains, not that is abnormal to do that, I am just mentioning.