UPDATED: 01/13/2022
Much of what I wrote below is actually incorrect; or, at least, beside the point.
After a couple days of being excited about the result of my fix, I moved the group of components (all with no background color themselves), back into the frame from which they came. The next time I published, it reverted all the assets panel swatches to the color I was trying to eliminate.
It turns out @z-wang already offered the solution above about adding a frame behind the components.
Latest Hypothesis
The background swatch of an asset seems to be a calculated value based on the background of whichever container holds them. This could be the canvas or any frame on the canvas.
The ultimate solution to my issue turns out to be much simpler than I originally made it out to be.
New Solution
Simply change the background color of the frame in which the main components are contained (or move them to a frame and change the color to one that will provide the proper contrast once in the assets panel.
I have a more permanent solution that may seem a bit radical in its method, but does fix the asset background swatch completely.
Issue
The color of the asset swatch background appears to be based on the background at the time of the component creation. In our case, the components are icons that we want to swap in and out of button components. Unfortunately, the icons had been created initially inside a frame that was a color which ultimately is intended for many of the icons. The result is that these icons are entirely invisible in the Assets Panel now.
Solution
You can remove the background association by cutting the main component (+x) from the library and pasting it into a different project file/library; presumable, you would paste it onto a background color you prefer.
This pasting process effectively moves the component to a new library and severs its links to whatever structure anchored it to the offending background swatch color and assigns a new background swatch color based on its new library location.
In our case, we have a single main components project file library. We simply cut/pasted everything from the original library into the new library and republished.
Caveat
The component must remain in the new library. You can’t just copy it back to the original library over a new background. If you try this, it will reclaim its mysterious link to the original background swatch color.