Issue with swapping libraries for multibrand system

Hi guys, Im working on a multibrand design system and encountered an issue with library swapping.
I have a main file which includes base components with variants and multiple files with text/color styles and few icon-sets. I created a new file, connected libraries and I put a new design together using instances of a base component. Now I need to restyle it for different brands using available libraries. I’m using swap library feature. I can swap style libraries but can’t swap icon libraries. The “Swap library” button does not appear. I don’t think its a bug but I can’t work out the solution.
Do you have any ideas how can i make it work? I attach a picture with an architecture I described above.