………as do I, I keep them stuck to a rack in my basement, but have repeatably found a D4 or EDC18 stuck not to the rack but to the $450 computer (Toyota replacement cost) in my pocket for my 2021 vehicle that they call a car fob (in lieu of a key). As I go snow skiing and even worse (was out in the bay kayak crabbing 3 days back) ZI have a key cover on the fob. There is no actual key, just this electronic device that remotely works. Have not damaged it yet, but I could see it easily occurring.
I commonly carry a light in, or clipped to my hoodie pocket. In the line of risks, I advise you be attentive. A firefly detaching itself from your oven door while in full swing of cooking, to come to rest on your right testicle is sure to remind you of your attachments and surroundings.
Many, if not all, modern car key fobs have a compartment containing a manual key for emergencies.
The manual key is usually a piece of steel and that is probably what attracts your magnet.
I have two D4V2s with magnetic tailcaps and they ride in the same pocket as my wallet. No issues so far. But I don’t carry any re-writable cards like hotel keys. Actually the corner of one card has started peeling. I guess it will fall apart physically long before the magnet does anything to it.