I've fixed the weird bug where if you specify a +map command on the command line, you will enter the map looking down and at an angle instead of straight forward. This was caused by the mouse cursor being positioned at [0,0] instead of at the center of the window during startup, which caused the first input read to be incorrectly offset.
Mouse input should now be a lot smoother than it was. I've also been restoring the joystick code, but haven't been able to test it properly yet. The lack of a joystick doesn't help here. I probably won't restore the XBox 360 controller code as there are some heavy dependencies on which version of XInput you have installed, and not all versions are compatible. Of course, if everyone was guaranteed to be on the same version this would be a lot easier.
There are some cases where semi-corrupt TGAs can cause a crash on map startup; this generally only occurs if you have external textures or skyboxes loaded. I had noticed this before and coded a workaround, but some extra code I since added didn't take account of this and tried to reference a NULL texture object. So that's also fixed now.
The weird thing about these TGAs is that the old Quake TGA loader handled them fine, but yet they're definitely semi-corrupt and it's definitely a content bug. You can see this in the file sizes and if you open them and re-save them in an image editor.
The MDL lighting code works fine on ps2.0 hardware, branching and all, but is somewhat slower. This needs some more work to get it back up to speed, but on balance I think that the next beta can go out with it as is.
Monday, February 28, 2011
I've fixed the weird bug where if you specify a +map command on the command line, you will enter the map looking down and at an angle instead of straight forward. This was caused by the mouse cursor being positioned at [0,0] instead of at the center of the window during startup, which caused the first input read to be incorrectly offset.
Posted by mhquake at 1:00 PM
Sunday, February 27, 2011
There will probably be a beta 3 shortly with some more changes and fixes coming through. This is an unfortunate consequence of waiting so long (and changing so much) between releases, but we'll all have to live with it, and will hopefully get a better end result for everyone out of the process.
I've fixed up MDL lighting so that it more accurately mimics the software Quake formula, is consistent for all models, and doesn't introduce unwanted dark patches. Right now it uses shader branching, which is a little unfriendly - but not impossible - for ps2.0 hardware. I'll most likely eventually move this to a 1D texture lookup, but for now I'm going to suck it and see.
Client angles for ProQuake servers should now be fixed again. This was caused by differences in the rounding formula used, so I've added some extra code to revert to the old (less precise) formula if we're on a ProQuake server. Once again I find myself angry at ProQuake for stuff like this; just bumping the protocol would have made things a whole lot easier. There are a multitude of ugly and nasty hacks in the entire ProQuake messaging system that make any adoption of ProQuake features a total pain.
Speaking of client angles, there is a really weird bug where if you start a map from the command-line (or a batch file) your initial angles are all messed up. I've successfully reproduced this, but the odd thing is that every time I break into the debugger it just goes away of it's own accord. We might have to just live with this one until such a time as I can investigate further.
Depending on how other requirements pan out, this release should happen over the next few days.
Posted by mhquake at 10:06 PM
Saturday, February 26, 2011
Friday, February 25, 2011
It's confirmed that a Beta 2 will be forthcoming over the next few days as a few bugs (one serious) have surfaced.
The serious bug is related to the behaviour of ATI cards with my surface vertex caching system; at least everyone who's experienced it so far is on ATI (the specific model doesn't seem to matter). I've gone back and removed the caching system, cleaned out a lot of the code, and generally made things a lot simpler.
One thing I'm experimenting with is converting most primitives to triangle-strip layout; on some hardware this can potentially give faster performance. I'm still using indexed triangles as the basic primitive, but they are laid out in memory in strip order, so according to the documentation this should give optimal performance. I think with the likes of Quake - which doesn't really stress vertex submission - it's not going to be too big a deal, but it will be interesting nonetheless.
I have an initial cut of code written that resolves the MDL shading problems, but I've yet to transfer it to my main codebase. The one thing it doesn't do is interact well with overbrights; as it's based on software Quake it just assumes that everything is overbright always. Not sure how I'm going to approach that one. It also requires pixel shader branching, but I can work around that by using a 1D texture as a lookup table and setting the texture address mode to clamp.
I'm also curious about the FPS lock I introduced in this version. Basically, a single player game running at higher than 72FPS will fuck up physics - so badly that sometimes it can kill you. To work around this I've locked FPS to a max of 72 in single player games. This is a balancing act where I fear that I'll never get it right - people will complain about the FPS lock, but if I remove it people will complain about the screwy physics.
More next time.
Posted by mhquake at 11:40 PM
Thursday, February 24, 2011
Just been doing a play through some maps with the DirectQ beta, and some changes will be coming through for the next one. It's too early to say if this will be a full release, a beta 2 or a release candidate, but overall things are seeming quite solid enough.
Those changes I mentioned are exceptions to the "quite solid enough" rule. Some brief discussion of them.
The MDL lighting model needs some more tweaking. I've double-checked a few things with it, and while I'm certain that I've got the basic formula right for software Quake MDL lighting, I'm a little out in the overall scaling. Some MDLs are coming out too dark, some have unwanted shading in places, and so on. The fact that software Quake uses an inverse scale makes things a little trickier to match up right.
There are some places on hardware T&L kit where performance drops off. I'm suspecting that this may be on account of my vertex buffer caching (which is not used with software T&L) having too wide a spread between vertexes that have been cached and vertexes that are newly added; especially as disabling the cache prevents it. Doing so causes performance to drop a little, so I might need to investigate other areas for improvement there. One of them might involve caching an entire PVS but that would break integration of brush models with the main render. That may still be viable however; there's plenty of room for experimentation here.
I'm not 100% happy with particles; I think as the code evolved it got a little messy. If I could up the hardware minimum to ps3.0 I could resolve this, but doing so would exclude parts like the Intel 915 and 945, which I don't want to do. I might just leave it as is and come back to it later.
I don't think that sprite batching turned out as effective as it could be. With hindsight I could probably amalgamate most of the draw routines for this into one of the other quad-drawing routines; the 2D GUI stuff looks appropriate.
I'm interested especially in any reports of weird lockups and suchlike. I've got some stuff running on a second thread in this one, and it seems stable enough to me, but it's the first time I've ever released any multithreaded code so naturally I have a few small nagging concerns.
On balance though I actually am quite happy with it overall. The cleanup of the renderer was long overdue and it's been designed as a solid baseline to move forward from, rather than just getting the job done in a quick and dirty fashion.
Posted by mhquake at 10:27 PM
Get it here.
The first thing you need to know is that this is a beta release. It's purpose is to get feedback on what works well, what doesn't work well, and what doesn't work at all. Nothing in here should be viewed as the definitive final way that things are going to be, but it is a good indication.
You will need D3D9 capable hardware with pixel shaders 2.0 support at a minimum. If you don't have that, stop right now. If you're in Intel land this means that you'll need at least a 915.
If you haven't updated your DirectX in a while, it might be an idea to do so. This release was compiled against the April 2007 release of D3D9, so you will need to have at least that version.
Posted by mhquake at 12:12 PM
Wednesday, February 23, 2011
A busy day today; DirectQ got some more work done and RMQ got bumpmapping.
The latter was on request and we're just trying it out to see how it looks. It won't be an essential feature, but it's something interesting and fun to do while my mind is churning over some of the finer points of CSQC. I'm using my old trick of assuming that there's just a single light source and it's positioned at the player viewpoint, which is not as effective as real bumpmapping but is easy, fast and cheap to do.
I'm almost certain at this stage that the DirectQ 1.8.7 beta release will happen sometime tomorrow. Unless any last-minute crash bugs turn up, this is the space to watch. It's important to realise that it's a beta release, not the final product, so if your favourite feature hasn't made it or if you think you should be able to run it but can't, there's no need for panic. I would be interested in knowing about any problems encountered though. I can't promise a solution to everything, but if it fits in the general ethos of DirectQ and seems important enough, I can promise to give it my attention.
At this point I should probably remind everyone that from this release onwards DirectQ is going shaders only. You'll need fully D3D9 capable hardware to run it, so you might want to spend the price of a takeaway pizza if you don't have it. Note that this doesn't exclude Intels such as the 915, 945, 950, 960, etc; these are perfectly capable of running it and it will give great performance on them.
The upside of this is that a lot of the clunky old code paths required to support the fixed pipeline have been completely eliminated, and the whole thing is a lot more efficient than it ever was. I think that given that D3D9 is 9 years old, and capable hardware has been on the market for that amount of time, it's a fair trade.
The next post should be the release.
Posted by mhquake at 10:10 PM
OK, it's mostly working now. The lack of fullbrights means that I need to dive back into the Nehahra engine again and figure what it does (probably something ghastly), but otherwise it seems there.
No, I didn't change the status bar. That's DirectQ's "QuakeWorld"-style HUD option (cl_sbar 2) and it's been there for some time now (yes, even in 1.8.666).
Posted by mhquake at 12:08 AM
Tuesday, February 22, 2011
I've been beefing up Nehahra support a little more in the upcoming version, so for this post I'm going to concentrate on that.
Firstly, whatever about the overall quality of the content, I personally view Nehahra as a failed opportunity. Here was the chance to get a good solid set of standards from which it would be possible to move forward and instead it was a bunch of cheap and nasty hacks. In fairness to it, that was probably symptomatic of the times.
Gameplay-wise and stylistically it's not to my taste; it seems to be trying too hard to one-up Half Life for it's own good. Most definitely "not Quake" IMO.
Anyway, on to the details.
It's quite obvious that Nehahra assumes that it's going to be running in an engine that doesn't support fullbrights, so - rather than try to hack around it - I'm just disabling fullbright loading if Nehahra is detected. This will get rid of all the shining pixels that appear throughout it's maps.
Nehahra uses Quake protocol 15, but has made changes to it without bumping the version number. I think this is quite an evil thing to do, but we're stuck with it now. So far I've supported Nehahra alpha, but I'm also going to support fullbright too. These protocol 15 changes will only take effect if Nehahra is running, of course.
After playing around a little I actually got Nehahra fog working (by emulating the old cvars as commands and doing some tricksy stuff with them). It's not feature-complete yet, but it's getting there. DirectQ's sky fog is based on FitzQuake's which lightly fogs the sky, but for Nehahra things need to be done differently. The old skybox drawing method assumes that sky is drawn at a more-or-less infinite distance, so for Nehahra it needs to be fully fogged out.
That's about it for now; if any more come up I'll add them to this post.
Update: oh yes, this mod sucks when it comes to nasty hacks. In a world where fog and skyboxes are loaded from worldspawn, Nehahra has to go and be a slave to the Great Gods Of QC yet again, and instead spawns them from an info_start entity. All well and nice for it, but it does mean that they are not preserved when saving/loading a game. WTF is going on?
I'll need to examine the Nehahra Engine source code, but therein lies another problem - the author (Bengt Jardrup) has this rather bad habit of not including the full source code with his releases. Compiling his stuff and running it in a debugger is never a straightforward process.
Having this mod coexist with more modern methods of doing stuff is certainly a lot of trouble; even more recent versions of DarkPlaces don't bother with it.
Oh well, onwards and upwards.
Update 2: found it. Another QC hack. Sigh...
Posted by mhquake at 10:16 PM
I've been doing work on brush surfaces, and now have everything totally simplified, streamlined and integrated. Everything - sky, liquids, solid surfaces, alpha surfaces, world surfaces, brush model surfaces - now goes through the same code paths. This has been a long-standing goal for over a year now, and the fact that the new code is simpler, more capable and runs faster is a good sign that it's been a move in the right direction.
MDLs got another tiny speed boost.
Sprites and particles still need some work done, but - especially in the case of sprites - if it doesn't happen this week it won't be that big a deal.
Posted by mhquake at 5:06 PM
Monday, February 21, 2011
I've been undoing a lot of the experimental work I did on DirectQ late last year. At that time I was using it as a testbed for stuff that was potentially going to be in RMQ, and unfortunately I had made a slight mess in places. While it didn't actually have any negative effect, it was very untidy and would have caused trouble in the future; better to get rid of it now.
The primary changes are that the timers have gone back to floating point in the majority of places, and protocol support has been revised somewhat. I'll deal with these in turn.
Reverting the timers to floating point was an easy change to decide on; after all, QC uses floating point so at some point in time they have to be converted over. This was really the deciding factor, but a secondary one was that network time transmission (svc_time messages) also uses floating point. Changing that is a protocol change, and then we start digging up a whole heap of dependencies.
The main host still uses integer milliseconds (because it's the only way to get rock-solid timing on Windows), and time advancement and differences there are still measured in integer milliseconds, but these are converted to floating point as soon as it's been decided that we're going to run a frame.
This keeps the code cleaner but still keeps the timer steady: any floating point drift in one frame will be cancelled out by the next.
Protocol changes might be a little controversial, but I believe that it's the best decision in the longer term. Up to now DirectQ has supported a whole mess of different protocols, including the original, Fitz, the BJP protocols, an experimental enhancement of one of the BJP protocols I made, and an experimental RMQ protocol (which had become different from the real RMQ protocol, and which - unfortunately - I had set as the default in a released version - ow ow ow).
This created a hugely complex architecture around messaging, which has now been completely gutted. Protocol selection has been reduced to the original, Fitz or the real RMQ protocol. I'm not going to maintain compatibility with the others going forward.
The only impact that this has on released content is that the WarpSpasm demos will no longer play. I don't think that this is a big enough deal to worry over. However, demos recorded in previous versions of DirectQ will also no longer play, and some of these were circulated on Func. This might cause some pain in the shorter term, but I believe that it's best to do it now and get it over with rather than have to maintain the wrong RMQ protocol in addition to the real one going forward.
Speaking of the RMQ protocol, I do believe that it's now been finalised and that the upcoming release of DirectQ will be the first unveiling of the final version. This will be a Good Thing as it will let everyone see it well in advance, and those who want to implement it can do so.
This protocol is functionally identical to Fitz 666 with the exception of an additional MSG_WriteLong (on the server) and MSG_ReadLong (on the client) following the version in an svc_serverinfo message. If this value is 0 then the protocol is otherwise identical to Fitz; if it's non-zero then it's a set of flags that define what's different. Currently defined flags are:
#define PRFL_SHORTANGLE (1 << 1) #define PRFL_FLOATANGLE (1 << 2) #define PRFL_24BITCOORD (1 << 3) #define PRFL_FLOATCOORD (1 << 4) #define PRFL_EDICTSCALE (1 << 5) #define PRFL_MOREFLAGS (1 << 31)Doing it this way, all future revisions of the protocol can maintain full compatibility with each other and nothing should break any more.
That's about that for now; still on track for the Beta 1.8.7 release later on this week.
Posted by mhquake at 9:44 PM
Sunday, February 20, 2011
Following the recent flurry of activity I've gone into quiet mode for a bit. That doesn't mean that work has stopped, just that I've nothing of particular interest to report at present. Everything is still ticking along nicely and on schedule, with the most recent work revolving around minor bugfixes and rightening up/restructuring some code more than anything else.
Expect news soon.
Posted by mhquake at 11:58 PM
Wednesday, February 16, 2011
Quick one this time; just been doing some Intel bugfixing and testing. The Intel 965 seems to interpret D3DSHADE_GOURAUD and D3DSHADE_FLAT in a very odd manner indeed when combined with shaders. Fortunately I've just been able to get rid of them entirely from the engine.
More speed too, and with more optimizations to come. Particles and sprites are the last remaining items to get another working over, and there's something else with MDLs I want to try.
Then I think I'm into a final round of testing and might release the early beta. It won't happen till next week though, and that means towards the end of next week, not Monday.
The RMQ folks have the software Quake MDL lighting code now so it'll be interesting to get feedback from them on it.
Update: the experimental "something I wanted to try with MDLs" didn't work out too good. In theory it could have avoided even more vertex buffer updates by splitting lastverts and currverts into two buffers, but it ended up being slower (owing to two buffer locks per mdl, plus two walks through the in-memory copy of the verts to update the buffers) and it broke muzzleflashes. Nice idea though, and sometimes you do just have to try these things out.
Posted by mhquake at 12:03 AM
Monday, February 14, 2011
It may not look like much but this is a more or less exact replica of software Quake MDL lighting (with texturing switched off so that you can see the shading). It's done per-pixel in a shader, and of course it gets texture filtering and gouraud shading, but otherwise it's spot on; everything looks the same from all angles. There's just some CPU-side light level clamping left to be done (not that much of a big deal) and we're there.
That's another major bone of contention of mine resolved: how inferior GLQuake MDL lighting was to software Quake's.
Vertex submission has gone up to 16 bytes as we now need to submit two (compressed) normals per vertex as well as 2 (compressed) positions; two of each are needed for interpolation (last verts or currverts). I could compress this down further (back to 12 bytes) but it's probably getting into micro-optimization territory. I'll most likely end up giving it a try anyway, just for the laugh.
On the other hand the requirement to refresh vertexes when the entity angle changes has been removed, so those 16 bytes only need to be refreshed about 10 times per second (rather than 72 times). Interpolation, lighting and shading, etc are all done in shaders, and are all per-pixel and per-millisecond correct.
This is quite a major performance gain with MDLs, which have always been quite a bottleneck in Quake.
Posted by mhquake at 11:24 PM
Sunday, February 13, 2011
I've managed to get the vertex submission for MDLs down from 32 bytes per vertex to 4 bytes in common cases and 12 bytes in the worst case. Pretty neat. It remains to be seen how this behaves on ps2.0 hardware (that's for tomorrow) but it's not going to be too onerous if it doesn't; worst thing that can happen is I revert back to 32 bytes, but even if so, the advantages of the caching system will more than make up for it.
I also fixed shadows in fog while I was at it.
Coming soon to a PC near you will be an early beta release of my current code. I have yet to put it through it's paces with serious testing, but I have been doing some runs through e1 and various other maps to ensure that it at least doesn't explode when confronted with standard stuff.
Watch this space for news of dates.
Posted by mhquake at 9:07 PM
I've just moved MDL interpolation entirely to the GPU. This is a mixture of benefits and costs, with the primary cost being that the vertex buffer now needs to store position information for both frames being interpolated, which makes the vertex submission a little heavier. I believe however that the benefits far outweigh this. Number one is that some costly CPU-side calculations can now be performed in dedicated hardware which is optimized for this kind of thing (even on Intel software vertex shaders it should be faster owing to better code). Number two is that the vertex buffer now only needs to be updated whenever either of the frames change, rather than every frame. This is typically 10 times per second for most Quake animations, so that's quite a saving.
Longer term it would be possible to remove the need to update the vertex buffer at all, but doing so would mess with my hack for viewmodel muzzleflashes.
None of this would have been possible with the old code.
Posted by mhquake at 5:45 PM
Saturday, February 12, 2011
Just spent a few hours between yesterday and today tracking down a nasty shader compiler crash bug. It turned out to be a combination of my own stupidity and a documentation omission. If you're passing preprocessor definitions to the effect compiler you need to use a NULL-terminated array (rather than just an array); this is only documented in the D3DXCreateEffectFromFile functions (which I don't use), it's completely omitted from the other D3DXCreateEffect function documentation.
I say my own stupidity here because I should have been aware that the array needed to be NULL-terminated; there's no length parameter for it so how else is the shader compiler going to detect it's end? It still doesn't excuse the documentation though; having an important fact only documented in a function that someone might never even use seems a mite dangerous to me.
This one has been present since fog reappeared in DirectQ, but it wasn't a crash bug until now as I had some other variable declarations following my D3DXMACRO, which soaked up the abuse and eventually got NULLs into the right places. It was only after my recent code reworking that it started blowing up.
I've been doing some experiments with hardware geometry instancing, and this is another area where documentation is quite obtuse and confusing. It doesn't help that instead of saying straight up what the various methods and parameters actually do it instead veers off into giving some fairly foggy (and simplistic) examples and leaves you to figure out what the hell is going on for yourself.
But all's well that ends well and I've implemented it on particles, which are a good candidate for this rendering technique. I've been able to reduce the particle vertex submission overhead by a factor of over 4, and have also offloaded a lot of the billboarding calculations to my vertex shader. It had always annoyed me that D3D9 didn't get geometry shaders, but this is a good compromise and helps to reduce one bottleneck. It also removes one of the reasons for eventually porting to D3D11.
This type of instancing is only available on SM3 or higher hardware, but I've left the old codepath in for lowlier devices.
Posted by mhquake at 12:36 PM
Been working on a major restructuring of the HLSL interfaces. There is quite a serious bug here in loading shaders from executable resources that I haven't quite tracked down yet; this has been a problem for quite a few releases but it's only now that it's started becoming one I gotta solve (in other words it didn't cause crashes before).
Much happier with the new shader code despite that. It's quite shocking the extent to which things have changed over the past few days in particular; the renderer code overall is now completely unrecognisable when compared to the old.
One item that I'm still oscillating between two different approaches on is MDLs. My current code is a huge saving on bandwidth usage at the expense of some extra overhead elsewhere. It's a bit of a balancing act whether or not this is going to be a definitive saving overall and in all situations; I'm probably going to end up implementing two different code paths for MDLs and either cvar-izing them or selecting the best based on hardware capabilities. We'll see.
Posted by mhquake at 12:27 AM
Wednesday, February 9, 2011
A screenshot of the Marcher Fortress has become something of a DirectQ tradition whenever a significant milestone is achieved, and recently I mentioned that I was getting ~100 FPS on an Intel 945 when running it. So here it is.
[Update: I promised you 100 FPS...]
And yes, this was on a 945.
Posted by mhquake at 11:24 AM
Tuesday, February 8, 2011
Been doing some more benchmarking, and I'm now getting almost 100 FPS in the main canyon area of the Marcher Fortress on an Intel 945. Yeah, this one is gonna be good.
Fixed the PF_VarString crash. It only happens when you exit a map, and is caused by a NaN in the globals; adding a simple NaN check before the G_STRING resolves it nicely. And as it only happens on exiting a map, it's safe to just swallow the error harmlessly.
I guess the root cause of this is different floating point exception setting defaults in the compiler (between MSVC 6 and MSVC 2008), or possibly removal of the ASM code. But it still feels good to have finally tracked it down and understood why it was happening.
Posted by mhquake at 7:42 PM
Monday, February 7, 2011
Been having some fun with software T&L cards, MDLs and entity alpha.
First up, hardware T&L is one of those things that you tend to take for granted until you're suddenly hit by a part that only does software. In this case it was an Intel 945 (Intel is weird in that it has a few parts that are fully DirectX 9 but only have software T&L). It turned out that the vertex buffer useage pattern I had built up for world surfaces was completely inappropriate for software T&L. First thing was that it crashed, but on further investigation I decided that - seeing as how randomly hopping around in a vertex buffer is bad for software T&L performance - to make some small changes. Now if you have software T&L it doesn't cache recently used surfaces, but just streams them out to the buffer always. It's not really a performance loss as the only performance gain from caching was to have these surfaces already in driver-optimal memory, which does not apply to software T&L.
Second thing was MDLs. It occurred to me that I could implement a form of vertex caching here too, where if the current frame hasn't changed there's no need to update the vertex buffer for an entity using that MDL. There are some nasty gotchas with this, in particular relating to lighting (an important component of lighting is derived from entity angles), and interpolation also means that the current frame tends to change every frame anyway (as it's really an interpolated position), but overall it's proven effective enough.
Entity alpha is a barrel of laughs. I based my original implementation on FitzQuake's (derived when I adapted the protocol) and was never happy with what looked like a mess to me. All the ENT_ALPHA_DECODE, ENT_ALPHA_ENCODE stuff - eeewww! Now, I know why it's done that way (so that a memset to 0 will be valid for fully opaque) but it still seems inelegant, and wasn't living nicely with Nehahra alpha (another world of pain) either. So I've ripped it apart and also fixed an omission in DirectQ where static entities didn't get alpha set.
In other news the dreaded PF_VarString crash is back. This generally occurs if you change a map or exit the game at the precise moment that QuakeC is sending a console or center print with a value in it. This seems to crop up at random intervals, even in a straight port of IDQuake to MSVC 2008, and was probably always there but I just hadn't triggered it for a while.
If anyone knows what the hell is going on with it (and of a fix) I'd appreciate the heads up.
Posted by mhquake at 11:30 PM
Sunday, February 6, 2011
Just did a quick comparison of the old code vs the new on an Intel 965, and the results were quite impressive: timedemo demo1 framerates went up from 105 to 178, so obviously I'm doing the right thing here.
I'm doing some quite nifty things with caching right now that are translating into huge bandwidth savings, which are really of benefit to integrated 3D cards. It doesn't give any difference on more powerful hardware because that's more than able to handle Quake's requirements and bandwidth is not a bottleneck, but it counts in the important places.
Posted by mhquake at 10:35 PM
The base Quake render is now more or less complete. A lot of cleaning up has been done too, but there's still quite a bit of that left. Next step is to start tackling alpha surfaces (alpha water is already done).
I've been only cursorily benchmarking it, as a benchmark of work in progress is not really valid. It does serve to identify any trouble spots where things might have gone wrong though. So far there's no real speed difference, but like I said, it's work in progress. I expect that real benchmarking will turn up minimal difference with ID1 maps but quite an improvement with more complex scenes; especially as there have been a lot of bandwidth savings (this should really help integrated cards).
Brush surfaces are the big saving; in most frames these surfaces are entirely stored in a vertex buffer and no longer need to be updated at all. Other savings were had with MDLs, sprites, GUI stuff, and particles.
I've dropped occlusion queries entirely. Code complexity was one reason, but there were others, including batch breaking. The single buggest one though was realising that the kind of hardware that could benefit most from them doesn't support them, whereas the kind of hardware that does support them doesn't really need them.
Direct3D 9 class hardware is now a must. Aside from the obvious HLSL (Shader Model 2) requirement, there are also other D3D9 features (like multiple vertex streams) being used. As I've said before, this raises the baseline for integrateds to an Intel 915. Recent integrateds (like on netbooks or anything bought within the past 5 years or so) will handle it fine, but this won't run on your cruddy old GeForce 4 MX. Time to spend the price of a few beers on eBay.
That's a trade-off. By raising the minimum requirement I can make use of more advanced features that lead to more efficient and cleaner rendering. Supporting 10-year old hardware has a cost in more than just code complexity; every feature that needs to be written at that level is a barrier to further expansion and enhancement.
Posted by mhquake at 4:33 PM
Saturday, February 5, 2011
OK, so I've bitten the bullet and ripped out the fixed pipeline stuff. There was quite a bit of mess accumulated from it (especially with water and sky), so it'll take more time to fully remove it, but right now any code relating to it that's left is effectively inactive - it never gets called.
I've also finally got some proper source control for DirectQ; I can't believe I've gone over 2 years without it, and a lot of problems in the past would have been made easier if I'd had it back then, but oh well.
For now there's no public repository and no nightly builds for the very simple reason that maintaining those would generate an expectation that they would be somehow fit for purpose. They're not; while working on DirectQ I more often than not do not have a working version at the end of a day (or even a week), and this is even more true when I'm going through code and rewriting/reworking stuff. Features frequently get removed and stay removed for extended periods during a rewrite, things get left broken while dependencies elsewhere are fixed, old code and new code co-exists, and in general terms a WIP DirectQ codebase is a complete mess.
As things come together I may consider opening the repository to the public, but for now it's strictly "me only".
Posted by mhquake at 10:36 PM
Been doing some more experimental work with vertex buffers and I've now got a fairly good caching scheme implemented where I don't need to write a surface to the vertex buffer if it's already in there; this should result in a good overall performance improvement in most cases.
However, it does mean that I'm now at a stage where I need to make a decision on dropping the old fixed pipeline and saying "DirectQ is shaders only". The fundamental problem is that the caching system requires use of shaders for water and sky polygons (otherwise buffer switching overhead may wipe out any performance gains). Of course there's also the possibility of retaining the old system for the fixed pipeline, but there is a cost in code complexity associated with that (this has been mounting up over the past few versions in other areas too).
The impact of going shaders only is that older hardware will no longer be supported. I'm thinking though that it's timely to make that jump. DirectQ has never really been about "older hardware"; it's been about "integrated graphics". The original baseline of an Intel 915 remains supported under this scheme, and with Direct3D 9 now being a 9-year-old API, I think it's reasonable to say "tough luck" if something isn't supported.
There's also the code complexity element. As I said, it has been mounting up, and the whole edifice is really starting to look more and more fragile. It's only a matter of time before something brings it crashing down, and I'm thinking that the wise man recognises and addresses that problem before it happens. "If it ain't broke, don't fix it" can seem a very attractive approach, and what I have now by definition "ain't broke" (in other words it works). But it will break if I start adding more, so the other approach of "prevention is better than cure" starts coming forward.
Hmmmmm. Time to flip a coin.
Posted by mhquake at 2:18 PM
Wednesday, February 2, 2011
After doing the sequential image dump, I mentioned that a few things had pricked my interest, and one of those was working with Wave files. By odd coincidence AVI files also use the same container format, so I'm taking the opportunity to write a small standalone program to take these sequential images and sound and convert them to an (uncompressed) AVI.
The format used (RIFF) is an odd beast. I had some fun with it when working on Wave files in the past, and AVI files take that fun and turn the volume up to 11. It's not that it's difficult, it's more the case that it's quite clunky, awkward and poorly documented. Based on an original design from 1984 (which I believe was used for DeluxePaint, Adrian Carmack's weapon of choice) and filtered through the tender loving care of Microsoft and IBM's short-lived alliance, what emerged was something that definitely had four arses instead of a mouth.
Tricksy and messy to work with - oh yeah.
Anyway, the eventual end goal of this is going to be a simple standalone cross-platform C-based AVI writing library with no dependencies that should be usable in any Quake engine (or elsewhere if you wish). Let's see how far we get along the road to that goal!
Posted by mhquake at 1:45 AM
Tuesday, February 1, 2011
I've been adding AVI capture to the RMQ engine. This is something I've resisted for a long time as it's quite OS-dependent and there are some things about implementations of it which I've studied that I'm not happy about, in particular the fact that it tends to stick it's tentacles into lots of other places in the engine too.
However it is a useful feature for demoing new RMQ stuff in the RMQ engine itself, so it's a valid thing to have.
The eventual compromise we found was to just dump a stream of images (and a wave file) into a subdirectory, which you can then process into a video yourself. It's not ideal, but going further would require messing around with codecs, which brings us right back into OS-dependent land. This way we get to stay nice and platform-neutral.
So I'm really rolling my own in other words, and - I must say - it's quite a bit of fun. There are various problems to be resolved with this, and discovering them (as well as discovering solutions to them) is perking up my interest in other areas. It's been a while since I've written any code to deal with WAV files, for example.
One of the biggest problems, that's going to remain a problem, is that disk I/O is a serious bottleneck here. This has bearings on how fast you can run your capture, what resolution you run it at, what formats you use, etc. I'm currently leaning towards running at 20-25 FPS and defaulting to 320x240 resolution, which seems to give a good enough balance. The image format of choice is TGA; this does make disk I/O slightly more of a problem, but the tradeoff is that it's very light on the CPU - switching to a compressed format would most likely just swap one bottleneck for another.
Another problem was synchronising audio with video correctly, but that was easily resolved by adding another maxfps upper bound if we're currently capturing and just letting the image capture happen every frame instead of on a timer.
So all in all it's done and working nicely; no sample videos for you (they're not going to be any different to those you'll find anywhere on YouTube) but a nice clean and simple piece of work. There are a few things to tidy up with it, but overall I'm happy.
By the way, I don't see this as being a feature for DirectQ, and my recommendation there is to use FRAPS if you want to capture video. This option was floated for RMQ, but the fact that FRAPS is Windows-only ruled it out. With DirectQ being a Direct3D engine (and therefore Windows-only too) it's a non-issue.
Posted by mhquake at 5:44 PM