Well, in the olden days, lots of high-end CR123A lights used a 3A configuration, for example Surefire's high-output lamp assemblies used in their military series and other lights. This will give you a lot of heat. You needed gloves when running them for more than a couple of minutes.

But I think the most important issue is what happens when something goes wrong and the battery is shorted. I had one of the tiny Olights that did this right out of the box. When you put the battery in and screwed on the tailcap, it became very hot within a couple of seconds. Electronic switches are not as safe as mechanical ones, they could fail at any time, in your parked car, in your backpack, in your sock drawer and so on, and a shorted CR123A does not just generate heat, it also has the potential to generate high-pressure gas. They have a safety vent and will leak hydrogen. In a well-sealed flashlight body, the pressure will build up. Tailcaps from flashlights supposedly went through a car door, wood drawer and other stuff in incidents widely discussed on another flashlight forum.

Another issue is physical damage to the battery, but I'm not all that concerned about that. Here's a video of it anyway. It gives you an idea about what could potentially happen when a CR123A is in thermal meltdown. https://www.youtube.com/watch?v=fG_UuPmLO1c