Right now it seems like the ‘default’ variant is set based on which variant is ‘upper leftmost’ in the variant container frame.
Instead, I suggest decoupling arrangement from precedence, and using the property ordering functionality to determine the default variant. The default variant should simply be the one with the first value for each property, regardless of where it’s placed in the variant frame. This would allow teams to set defaults directly from the properties panel, without having to move components around.
See this thread for some more context: How to set defaults / precedence of variants with multiple properties?