Just found a way to do so
Some things are a bit awkward in Figma
When the font panel is highlighted, don’t use the dropdown arrow, just leave the text cursor in the panel, then keyboard arrow down or up 
1 Like