Figma file with library that contains Variables is unremovable if added once in a random figma file

Hi all,

I have a question:

we are using variables in a library file. This work amazing.

When i create a new Figma file and i load that library - it shows the modes i can apply on frames and pages etc.

When i remove the library again from my Figma file - for some reason, the variables keep on beeing active in my Figma file while. Since i am working with multiple libraries , it starts to mix modes from various libraries and i have no option to “disable” or remove them.

Does someone have the solution? Thanks!