RMQ's new BSP format is now complete in it's first revision. We have working versions of QBSP, Light and Vis that output a BSP in the new format, and we have an engine that is capable of loading and running the format.
We're calling it "BSP2". This shouldn't be viewed as some kind of usurpation or arrogance on anyone's part - the name just fell out naturally in the course of informal discussion. An older working name was "BSP 29a" but I guess that "BSP2" is a little more snappy.
I've written up a "BSP2 for Engine Coders" document, which I guess I'll release publicly sometime soon, but don't want to give any specifications just yet as the format has only been quite lightly tested (I've compiled ID1 e3m3 - because it's so damn awesome - quite a few times with it, and one of our mappers has compiled a map with over 64k verts with it) so it may be subject to minor future revision.
The fundamentals are in place and it seems quite solid though.
Here's an extract from the document containing some contextual info:
The objective of BSP2 is to address some limits in the stock BSP29 format. While many engines have boosted limits, there remain some cases where the on-disk format enforces a hard limit of 64k. During the course of RMQ development these cases were hit on a number of occasions; it was clear that we were exceeding the capabilities of the Quake formats and that a more robust and general solution was needed.
Alternatives such as Half-Life BSP, Quake II BSP, Quake III BSP or others were considered but rejected on grounds of legal issues, not actually doing anything to fix this problem, enforcing too many changes in the content creation pipeline (such as changing your map editor) and enforcing too many (and too complex) changes in engine code.
BSP2 will coexist peacefully with BSP29 so far as is possible; minimal changes are required to engine code (the largest by far being in the loader), the .map format remains exactly the same, and - while new map compiling tools are needed - the changes to these tools are also extremely minimal and can be made in the order of 30 minutes or so per tool.
BSP2 is not a radical revision of the format. You won't find RGB lightmaps, 32-bit textures or any such major overhaul here. These are left for other already standardised ways of providing such content. It's sole objective is to overcome limits that prevent content creators from fully realising their visions. By choosing this approach it is hoped that the path to implementing BSP2 support in as many engines who's authors choose to do so is as eased as is possible, and dependencies on other features (such as RGB lightmaps) are not an obstacle.
The key points here are overcoming limits, minimal changes, easy and fast to implement, no hidden dependencies on other engine features, and peaceful coexistence. I think we achieved all of these, and I'm happy with the result (within the bounds of being happier if the problem had never had to be dealt with, of course).