I've been making maps for awhile now (since source release). The only time i have encountered the vertex buffer size limit is when i have placed far to much detail in any given scene of my maps. For instance having to many rounded pillars visible all at the same time will cause this error.
Basically thats why removen some aspects of your map fixed the issue. Im not sure this will help any but this is from personal experiance.