It usually is not a problem. Most devices that I have used that are designed to run on alkaline batteries run just fine, or even better on NiMH batteries. Just try using your device with 4 x NiMH batteries and see if it works.
There are a couple reasons for this. First of all, a historical difference in battery voltage ratings. Alkaline batteries are referred to as 1.5V batteries, but they actually only produce 1.5V when they’re almost completely full, and under light loads. They effectively got rated at their peak voltage.
In more recent years, it became common to list battery voltages based on the average voltage over the course of the discharge. NiMH cells are listed as 1.2V because of this, but they’re actually typically around 1.4V when fully charged. Alkaline and NiMH voltages are closer than the labeled voltage suggests.
The load is also important. At 10mA current draw, a load at which an AA battery would last over a week, alkaline AA battery that is 3/4 full will be at about 1.4V. An NiMH battery in the same conditions will be at about 1.3V. Very similar, but the alkaline battery has a slightly higher voltage.
Increase the load to 500mA, which is a realistic load in many flashlights or motor-driven devices, and the 3/4 full alkaline battery has had its voltage sag to 1.2V, while the NiMH battery is still very close to 1.3V - at these higher loads, the rechargeable battery actually can maintain its voltage better.
One of the implications of this is the types of devices that don’t run on NiMH batteries also tend not to run well on alkaline batteries. The alkaline battery likely has most of its charge remaining when such a device stops working, because the designer did not give it very much flexibility for the voltage it can use.
Interestingly, I get a lot of free alkaline batteries as a result of this. Any time I need batteries, I grab a handful out of the recycling bin at work. I’d say at least 25% of them are over 1.4V. Clearly there are a bunch of devices that didn’t run well at lower voltages (I think it’s the walkie-talkies the maintenance crew uses), so they throw out far more batteries than they should need to.