Lightmaps! And Performance!

Just added 64 bit lightmaps (for hardware that supports them) to the Quake II codebase. These are advantageous for two major reasons, with reason 1 being that you can configure them to use a much higher dynamic range, so that light doesn't get clamped to the max as quickly, and reason 2 being that they provide a much finer degree of quantization between individual steps of that range. This means that lighting looks smoother, and brighter spots are able to really bright.

Now the bad news - you'll have to double your value of gl_modulate in order to use them. So if you normally use gl_modulate 1, set it to 2, and if you normally use 2, set it to 4. Because gl_modulate now operates in hardware, there is no option above 4, so if you normally use 4 you will have to compensate by doubling your value of intensity. It's not quite the same end result, but close enough. If on the other hand you use intensity 4 and gl_modulate 4 all the time, you really need to get a brighter monitor!

I could remove the requirement to do this by simply changing one occurance of 127 to 255, but doing so would lose the extra dynamic range headroom. I'd rather sacrifice the ability to set gl_modulate too high than do this.

Of course, in DirectQ it's all handled using pixel shaders, so none of this is a problem.

I've decided to not get too hung up about performance with this engine. Naturally that doesn't mean that I'll intentionally write slow code for it, but it does mean that if something worthwhile impacts on performance, then the worthwhile thing is staying in. The exception of course is if a slideshow is the result, but something in the order of 60 FPS would be acceptable on the hardware I do most of my development on (it's currently getting almost double that, by the way, so there's a long way to go).

Of Holy Wars And Righteous Indignation

I've just spend the last hour writing a code formatter. Now, enough hot air and wasted energy has been expended arguing the relative merits and demerits of each style, and I'm not going to join the fray. The sole relevant point is that the style you are used to is important. It helps make code more readable, and - as it's a known fact that reading code is harder than writing it - anything that helps make code more readable is a bonus.

My own style has changed and evolved over the years, just like anyone else, and if I was willing to put in the effort I could probably adapt to the styles used in the Quake codebases in a matter of a coupla months. But until then I'm working at a lower speed, reading code that looks strange to me, and wasting time by reformatting it as I go. Even worse, the Quake codebases can be a mish-mash of different styles, depending on who was working on them at any given time.

I'm not doing this for money and it's not my day job; if it was I would put in the effort to adapt, but that's not the case.

So after hunting down various reformatters on the web I've decided to write my own. Why? Cos the others are all rubbish. The only GOOD reformatter I've ever seen is the one built into Visual C# (and presumably VB, but I've no exposure to it) 2005 and above. Precise control over all the important things. The others are too limited, and/or impose their own idiosyncracies, and sometimes even add injury to the insult by making you pay for them. This way I can target specific things that drive me batty in the Quake codebases and adjust them to specific outputs that I feel more comfortable with. What other people like or dislike is their own business.

We Have A Fix On Target But No Lock

It's starting to take some good shape now and mosey towards release status. Still a while off, but it seems eminently stable with baseq2 functionality. Once again I'm using the Speed Demos Archive material for testing of functionality and stability.

Today I did a very worthwhile adjustment to how R_LightPoint works. I'd always smelt a rat about this function, suspecting that the colour it returned was not quite the same as the correct surface light colour, particularly with respect to dynamic lights. Then it occurred to me that - as I've shifted lighting to something more similar to Classic Quake and am therefore storing all lightdata in main memory - I could just read the data from the final generated lightmap texture instead of having to rebuild it from the raw lightdata. This means that I can skip the addition of dynamic lights in R_LightPoint and have a precise result. This is definitely something that's going into the main DirectQ codebase.

I've also added in stencil buffer shadows (if you have a stencil buffer available) for a nice improvement to the default shadows. The "gl_shadows" cvar can use floating point values between 0 and 1 to control the intensity of the shadows.

"scr_showface 1" will replace the "Red Cross" health icon on the sbar with the face picture for your chosen skin. I've hacked the sbar code slightly to shift the y position up by 4 units and also to add another 4 units between the stat icon and value. It might not be 100% mod compatible, but how many mods actually customize this in such a way that it would be affected?

"gl_modulate" and "intensity" are still hanging around, but the way they work has been changed. Only values of 1, 2 or 4 are allowed, and if you set them to anything else they will just pick the nearest of the legal values. They now take effect immediately, and the behaviour of gl_modulate with alias models has been changed slightly so that it's the same as the behaviour with the world (it was different in stock ID Quake II).

"gl_monolightmap" is still there but now has no effect. The whole R_BuildLightmap function has been cleaned out of the various optimized cases; what may have been required in order to get it running on a P133 in 1997 no longer applies in 2009, and cleaner, simpler code is definitely preferable in this case.

Anisotropic filtering ("gl_anisotropic_filter") has been added. The old gl_texturemode, gl_texturesolidmode and gl_texturealphamode cvars are still there but no longer functional. Texture filtering is now either anisotropic or trilinear.

Hardware gamma has been added, using the old "gamma" cvar to control it. It works the same as in DirectQ but without the independent red/green/blue sliders, as I want to keep things simpler for this one.

The correct video mode formats supported by your adapter are enumerated and made available. Integrating the refresh DLL with the main engine helped a LOT here. As a consequence of this, the first time you run the engine it may start up in a weird and unexpected mode (it will however always respect the value of vid_fullscreen). It makes an effort to adjust to a supported mode if your cvars are outside of the ranges. I might change the cvar names and supply a safe mode (something like 640 x 480 windowed) as a default, just to avoid this first-time-startup issue.

DirectInput has been added for the mouse. I know a lot of Quake players prefer to use DirectInput, so now it's in Quake II also. Use "m_directinput" to control it, default is 1. If it seems too fast or too sluggish, use "m_diboost" to apply an additional scale modifier (default 3).

I'm going to add in BMP/PNG/JPG screenshots, just like in DirectQ. These are so easy to code in Direct3D that it would seem crazy not to. It will be a string cvar called "scr_screenshotformat", with a default of "tga".

External texture support might be added. I'm in two minds about this one; it seems a nice feature to have, but is it really used often enough in Quake II engine mods? I think I'll do it anyway just for completeness sake.

I might also add in additional external crosshair support; seems easy enough in the Quake II codebase.

I'm also going to see about removing the hard-limit on the number of textures (I've already upped it to 2048, but I want it gone). Texture-wise, proper bilinear filtering (from the D3DX library) is used for resampling (if required), and 8 bit textures are uploaded direct to the hardware in 8 bit + palette format (for faster texture uploads, which roughly evens out the overhead from using compressed textures).

The other main thing is that I still haven't gone back to fixing up the "lost device" case. It's annoying enough to have to do it in the first place, so I'll save it for a time when I just feel like grinding out code rather than doing anything creative or productive.

Think that's all for now.

All Squad Leaders Co-ordinate With Sister Teams

I did some DirectQ work today; ported over the new lightmap uploading code from Quake II. It's ever so slightly slower than my previous method, which I guess is most likely API overhead rather than anything else, but I'm happier with it as the previous method had always given me a slight case of the heebeejeebies (it worked Ok, but was it the Right Thing to do?) I might shoot for a hybrid method to try get the best of both.

I've also decided that DirectQ will stay with HLSL and won't get "enhanced" particles. I'm really just too far down the HLSL route to change right now, and on the particle front I've decided that the original particles are too finely tuned for the game to modify them. Of course, that latter has been done by many others, but I like the more traditional look of DirectQ so there we go.

Both of these are of course subject to change if inspiration for either strikes (or if I get the itch - right now I just don't have it).

On the Quake II front it's gained a few bugfixes as well as anisotropic filtering. It's one thing saying that the port to Direct3D is "complete", which it is, but a whole other thing is sorting out rendering bugs and slowdowns, as well as other bugs that may have been brought in by changes to data structures and code structures. This exactly mirrors the DirectQ situation where the original port was done over a weekend (I think it took 2 days this time) but was then followed by 2 weeks of shaking down the code. Of course I know more about what I'm doing this time, so I was able to avoid some basic mistakes. The flipside of that is that I can spend more time implementing things that I just wouldn't have been able to last time.

A final interesting development is the use of DirectQ 1.0 as an experimental codebase. I ported LIT support to it earlier on, just for laughs, so it's starting to build up a more comprehensive feature set which may see the light of day sometime. Maybe that will compensate for retaining HLSL in the current DirectQ codebase. I wouldn't however hold any breath over this potential "DirectQ Zero" release.

Trespasser!

In order to test the old Software Quake waterwarp I fired up the original DirectQ 1.0 source code and wrote it in. Conclusion: it's ugly, and has been removed. Performance-wise however it was a nice surprise, giving me only a 1 FPS drop.

It's actually quite nice having this codebase available, as it gives me a good platform for experiments such as this without the risk of polluting any of the main codebases. I'm going to port my new lightmap updating code from Quake II over to it shortly so I can do a good (and valid) performance comparison between the two.

It's also really strange going back to it and seeing all the things that were wrong with Quake in general and GLQuake in particular - a nasty trip into the past that was just full of bad vibes.

Last Recon Data Indicates Access Through The Sewers

I've been thinking about water warps, and I think the solution is to use a method similar to software, which updates the texture every frame from a sintable lookup. Textures would update from a default 64x64 tile into a new 128x128 tile, meaning that the typical case will be no worse than a coupla lightmaps being updated per frame (better in fact as I can send 8-bit data plus palette down rather than 32 bit data, meaning a 17K upload instead of 64K), although factoring mipmapping into the equation will be an interesting thing to resolve.

Where this may get complicated and even more interesting is with high-res external textures that are not guaranteed to be powers of 2 in DirectQ, but for the Quake II engine it's at least worth exploring to see how it goes.

All Units, We Have A Disable And Destroy Alert On Slow Cinematics

I've successfully sped up cinematic handling at large resolutions by a factor of about 50. I had originally (and misguidedly, as it turned out) thought that writing the raw data directly to the backbuffer would be the optimal solution, and - while it did work OK at 800x600 - at anything higher it went to a total crawl. Using regular texturing turned out to be the Correct Thing to do, after all. I'm not certain if it was the use of a fullscreen Device or the higher resolution, in fact; but at this stage I'm not really going to bother finding out, as it now works very fine indeed.

I've optimised it beyond the GL handling in that it caches the texture and only needs to recreate it if the size changes or a new cinematic begins. This is the D3D equivalent of a single glTexImage followed by glTexSubImage until the texture params change.

Just for laughs I also changed the texture update to only happen every other frame, and the difference in animation was completely invisible, but perf was boosted even further still. I kinda suspect that default QII cinematics were playing at some crazy high frame rate.

All in all, this has been a good one to mark off the list.

Marines We're At Nought Percent Ground Force Operational Status

Not as bad as the title may lead you to believe. I finished video startup (Quake II has a proper mode list now) and started into lost device handling, but very quickly ended up getting nowhere with that. It's frustrating, and I hit the same issues with DirectQ; having to ensure that you release everything properly before resetting the device, then bringing them back on. I thought I could probably just drop the device and recreate it, same as a vid_restart, but there are a few complications there too. The really annoying thing is that this is something that everybody has to write code for, and everybody ends up writing broadly the same code, so why is it not handled automatically by the runtime?

Another issue I'm hitting is that cinematics are really really slow at higher resolutions. These were handled OK by 3DFX cards at 800 x 600 10 years ago, so I just need to find a way to do them faster. I suppose most people don't even bother with cinematics, but for the sake of completeness I want to have them.

Establish Communications, Priority Alpha

The Quake II render is now complete. No screenshot, as the previous one was 95% there and the remaining items were stuff like sprites, gl_flashblend 1 mode, the null model and so on. Trust me, you wouldn't see any difference.

I did say nothing new was going in, but the following did make it:

• Underwater warping.
• Z-Fail Sky Box.
• scr_consize adjustable 2D render scale.
• Removal of functionality of "intensity" cvar.
• Removal of functionality of "gl_picmip" cvar.
• Compressed textures.

The next steps are to implement some proper video startup (and handling of lost devices), bugfix why my conchars are sampling down (despite not being mipmapped), do the "false fullbrights" and find a better way of doing water warping.

This poses an interesting problem for DirectQ. I've satisfied myself that not only is it possible to replicate the render using the fixed pipeline, but that speeds should be comparable. What I'm now thinking is that I'm going start transitioning DirectQ back to the fixed pipeline. The main reason for me to go HLSL in the first place was to do water warping, but if I can resolve that item then the reason is gone.

All Pods Launched

The render is now virtually feature-complete. I think the only remaining items are Sprites and Screenshots, and then we will have a direct port from OpenGL to Direct3D.

As mentioned before, I got rid of gl_modulate and intensity, and instead used a 4 x modulate blend to replicate these at their default settings. I'm not overly pleased with some of the stair-step artefacts this gives however, so I may well restore gl_modulate. Both cvars have been left hanging around anyway so that this engine can play nice with existing configs.

State change optimizations are in and - as predicted - we're at 100 FPS on the Intel.

Following this I'll need to work on video startup - right now it just assumes that you're running in an 800 x 600 window and doesn't support anything else, which was very handy for development but not good enough for Real World Use. Then I want to add in the "false fullbrights" and underwater warp, so that we're feature-compatible with Quake 1. It'll be missing the scrolling sky of course, but that's all vertex shader work rather than pixel shader, so I can live without it.