Once a color style is used for a fill color, you cannot toggle its visibility. The only option is to delete the fill entirely or detach from the color style.
Figma is all about linking and keeping styles up to date. The way this works actually prevents that from happening if one needs to turn off a background on a component. Anyone else? Saw one other post with no responses.