Seeking means of maximizing energy use from LiIon battery supplying a 3.3V load

Hello all. Thanks for taking up your time to read my post.

I have a device that operates with 3.2 V MAX, and I wish to power it from a standard 3.7 Li-Ion battery. A fully charged battery=4.2V.
I used a simple LM317AEMPX circuit to drop voltage to 3.2V and it worked perfectly.

Now the problem is as the battery starts to drain, the output voltage starts to become lower, so subsequently the LM317 output voltage will be lower as well, for example

batt=3.7V, LM317AEMPX=2.5V
In which this case the voltage is not enough to keep the device up and running.

So what I need is some sort of dynamic regulator to keep the output voltage fixed at 3.2V until the battery voltage drop to 3.2V as well. Advice on good ways to achieve this would be appreciated.

The given diagram shows as follows. I checked the wiring and found that it is correct. But I really don’t know which procedure goes wrong.

Can anyone help me ? I am very puzzled about this question. Any of your ideas would be highly appreciated.