Instance Swap reseating icon size

Hello everyone,
I’ve created icons, all vector layers with the same name, and every icon has a variant of sizes - small (16px) medium (24)and large (32). They also have round/sharp Boolean property.
I’ve created buttons, also with size variants with matching names and sizes. There’s also variant for only label, only icon, left icon and right icon.
There’s also instance swap for every icon size.
I also have the icon as nested instance.
So I take an instance of the button and I change its size, then I change the icon - lets say from “search” to “plus” for example - but if I want to change the size of the button, the instance swap return to its default icon instance.
What should I do, or what did I do wrong?