if it’s going to be printed, you may try following common font size used by most word-like software which is around 10-12px. If it’s going to be published on web, you shouldn’t go below 12px.
If you’re not sure about your font-size pattern, you can try using the common patterns which is multiples of 4 starting from 12, 16, 20, 24…