Swapping components fails

Hello everyone! Have you encountered a scenario when you try to swap an icon component to another one in a button, and Figma does weird stuff (such as the icon doesn’t swap and just changes color)? This happens a lot in our project. When I try to swap the content component in a modal, it can’t swap to the right component.
I heard this is an issue with big-size files. Is there a workaround without splitting files into smaller ones? I would really appreciate any suggestion~