How to get rid of variable references in the document?

Here is the code I made - source-foundation/src/utils/swap-variables.ts at main · namad/source-foundation · GitHub

It does the following

1. Collect all layers within the selection with Object.keys(n.boundVariables).length > 0
2. Then I iterate through these layers and check which variables are bound to which props
3. I retrieve bound variable details with figma.variables.getVariableById(varId) and figma.variables.getVariableCollectionById(collectionId)
4. I check against local collections if the collection with the same name exists and has a variable with the same name in it, here is the code I use source-foundation/src/utils/figma-variables.ts at main · namad/source-foundation · GitHub
5. If there is a full match I use figma.variables.setBoundVariableForPaint(), figma.variables.setBoundVariableForEffect() and node.setBoundVariable() to bind local variables instead of what is bound to the layer originally

It works just fine but after each run, I can see these ghost collections added to the page modes

Video of what’s going on