If you are covering a a large area, you could always fill in the gaps with a few insets of city level detail.
I agree with Greg.
Or you could do what old maps did and use a texture or dark color fill for those areas.
Examples from my website:
Tavernier's Description Du Pays Armorique A Pres Bretaigne
Erwin Raisz's Map of the Lower Nile
Texture can fill in white space quite nicely.
I think that's a great solution, those maps look great with the textures. Makes them look complete!