Here is what I'm trying to do. I have the WWF ecoregions dataset ( about 14,000 polygons) I am trying to calculate some statistics about how many of these 5' cells would fall within each polygon and how many would be split by polygon boundaries. Too me it seemed like a fairly simple vector problem, but it turns out to be a very huge dataset. I'm doing it with a raster grid now, but I'm not sure if I'll be able to answer the second question easily.
Why do you want to do that? Surely the actual answer as you've described is not what you are finally after?
What is the result you want after you've obtained those summaries of your 5' cells? Perhaps there's another way to think about it? Mathematically there's really no need to generate the vector grid at all, but how to interrogate the WWF dataset with the implicit grid sequencing depends on what you're trying to achieve. As someone noted, it's really the display of the vertices that will choke some GUIs, I'd be keen to try this problem in R or the like.
Also, what about tiling the the thing into 4 (or more) parts and then doing the clip/overlays in stages? Manifold (on your 3Gb ram machine) would certainly handle that.
(Naively it sounds like the old school projects to approximate area calculations, but I'm sure there's more to it than that