I don’t see a problem with series/parallel lithium primaries. For example the Nitecore TM26 allows 2s4p 123A lithium primary cells, although they are not wired in the way recommended by ImA4Wheelr (which I also recommend).
You probably need to change your math a little bit though. Since you have a 6P configuration, divide your current draw by 6 before using HKJ’s graphs (which were generated for individual cells). So for a 3A draw from the pack you should follow the green line on the graph (3A / 6 = 0.5A). This nets you approximately 3.15Ah * 6 = 18.9Ah. The discharge ranges from 1.45v down to about 1.25v before falling off rapidly, so you are only generating 5.8v at the beginning of the discharge and 5v at the end. The LED won’t light below 5v and at 5v it barely lights up. At 5.8v you only get a 1A drive current.
You just don’t have enough voltage to make this project work properly with a 4s config. Better do 5s5p and use a driver to regulate the current.
I’m using djozz’s MT-G2 crash test graph for reference on this as well as HKJ’s Energizer Ultimate Lithium AA review:
https://budgetlightforum.com/t/-/19600#comment-417559
http://lygte-info.dk/review/batteries2012/Energizer%20Ultimate%20Lithium%20AA%20UK.html