Do it!
I replaced the battery in a Bluetooth headset adapter with an 18650 holder, works great and can run for around 3 weeks or more.
Yeah, it’s unsightly, but function over form.