No, item thumbnails cannot be shown in the Variants dropdown menu. And you don’t need to use Variants for your icons. Make each icon a separate component.
Here’s what Figma has to say about this:
https://help.figma.com/hc/en-us/articles/360056440594-Create-and-use-variants