Reuben, that happens when the SVG you are importing doesn’t have set dimensions on it (viewbox / width & height).
Answering the original question, can you send the code of the original SVG before import? Note that if that SVGs are exported from Illustrator, masks in them won’t import to Figma correctly. Also defs and use are not allowed.