I’ll clarify this a bit more: once you tap after the buzz, it enters config mode and blinks with the following pattern: * ** *** **** ***** and so on.
If you tap turn off at some time within the interval containing n blinks, you end up in group n. What I would do is wait for the interval of 11 consecutive blinks to finish, have the interval of 12 blinks start, and half-tap turn off at some point during this 12-blink interval.