Of course it is, but I doubt anything will happen if you do.
The internal switcher converts 3-4V down to 1.5V, feeding 2V into the 1.5V output will most likely be harmless. If the open circuit voltage of the NiMH charger is 4V or more, it may damage the battery.