FS colors are a good place to start, but I think the reality is most paint manufacturers do not "scale" the colors so they wind up being darker than on the real thing. Also, weather, sunlight and other factors alter the way colors look.
i.e. in the end, a prefect matched FS color may not actually feel like the right choice anyway
For example, I swear most Ghost Greys I see fell too blue and dark. I have even gone to the naval air base museum near me and tried to match colors to the planes, and just about everyone look too dark or saturated, even when those are considered the "best" or "closest" matches to actual FS color swatches or sample panels
for example, Vallejo Dark Ghost Grey compared to what would be an FS 36320 area on this A-10
and then you get all this stuff for paint colors that are all supposed to represent Dark Ghost Grey
Funny thing is, many of these could look like closer matches depending on lighting, weathering, and other factors
For example, same plane, but a little farther away and on the shaded side it looks darker/bluer and actually matches the Vallejo color better
Same with Light Gull Grey FS 36440
Even though this is a fairly close match, I just feel like the things I have painted this color are way too dark compared to how the aircraft look sitting there at the museum or in pictures. And this I attribute to the scale factor and the need to lighten up colors the smaller you get model wise