I have 2 icon libraries of different sized icons at 24x24px and 32x32px, Icons are named as such: 32/Right Arrow 24/Right Arrow etc.... These icons are in a frame titled Icons and published as a library.

When I use these icons in a button with an Instance Swap property added for the icon, I can select from the Icon 24 folder or the Icon 32 folder:

When selecting from the 24 folder, the resulting Icon is not 24x24px, it is resized to  ~ 32x32px. The original button was created using a 32x32px icon.

Why does Figma not respect the icon size selected in the instance swap? I have tried creating the icons using the Icon Resizer plugin which uses auto layout and  manually with a frame and no auto layout. I have tried various settings in the auto layout options - fixed width (set to 24x24) and hug contents, nothing results in the selected 24x24 icon staying at 24x24. 

Should I have created icons with variants for the two sizes 32x32 and 24x24? 

Must I make two separate buttons one using a 24x24 icon and one with a 32x32 icon?

any help appreciated - hoping to understand the underlying logic at play so I can puzzle out future unexpected behavior.