DirectQ Update - 27th June 2011

The ProQuake match timer and teamscores are in. I guess that'll make some people happy. One slightly sticky bit was finding a suitable piece of on-screen real-estate for the match timer; ProQuake itself just overwrites the last two positions in the frag bar but I've shifted it upwards a little so that I can retain them.

Otherwise it's just been minor nips and tucks here and there. It's working well and running fine, so we're closing in on release country.

DirectQ Update - 26th June 2011

Some folks have reported trouble with collisions and sinking into the floor in certain maps/mods - these should now be fixed.

I've added some tighter hardware validation on startup; for the most part it shouldn't affect anybody, but if you've been managing (somehow) to run DirectQ on hardware that is not fully D3D9 compatible, you may not be able to any more.

directq.cfg will now execute properly on startup. It did before, but it was out of sequence, meaning that the settings in config.cfg would stamp over it.

Player skins have been reworked slightly again and now get the proper palette colours. This is going to be a few percent slower than before, but I'd rather go consistently slightly slower than be jerking back and forth between very fast and very slow. bigass1 speeds are maybe 530FPS for me so I don't really think anyone can complain too much.

I've experimented with notifying the player that changes to r_wateralpha don't get saved to your config, but the trouble is that you get this notification for everything that doesn't get saved, including when mods stuffcmd a cvar during gameplay. That's been reverted, and will have to wait until I clean out the video startup code (at which time I'll be able to reliably implement "seta" - right now there are incompatibility problems between seta and the video cvars).

At this time I'd really love to move the video cvars to the registry. That would fix everything more cleanly, and would mean that none of these problems would even exist. I got massive negative feedback last time I suggested doing this however... sigh...

DirectQ Update - 23rd June 2011

I'd intended for a new release to be out roundabout now, but what with one thing or another this wasn't to be, and nothing much has been done for the past week or so.

Right now I'm curious about how people would like to see the next version pan out. We have two options available to us.

Option 1 is that I could roll up everything I've got ready, do a bit of a push to get it releasable over the next few days, and put it out. That would be much the same as what you've currently got with some tweaks, bugfixes and improvements present.

Option 2 is that I could start tackling the video code and beat that into the shape I want it to be in. This is long overdue, needs to be done sometime soon (especially as window resizing has exposed quite a few things in it that need fixing), but will obvious mean that you'll need to wait longer before you get anything.

In both cases there is also a major block of RMQ work coming up that I need to devote time to, so I'll be ping-ponging back and forth between the two; basically using each as an escape valve from the other for when I get annoyed/stuck/burned-out/whatever.

So, which option?

Why some things are the way they are part 1 - r_wateralpha

I regularly get a bug report to the effect that r_wateralpha doesn't get saved in DirectQ. This is actually intentional and there are some very good reasons for it.

The first reason is that GLQuake doesn't save it either. Now that's a fairly weak reason, and there are occasions where I've changed this behaviour from GLQuake (and have even gone so far as to change a default from GLQuake) where I've felt that the original way was either stupid or obsolete.

The second reason is the important one.

Not every map has translucent water. In particular, if you do a fresh install of Quake from CD and use standard ID1 content, your ID1 maps won't have translucent water. Now, if DirectQ saved your value of r_wateralpha this would break on standard default ID1 content.

Here's where priorities come into play, and priority number one with no exceptions or arguments is that if a decision involves breaking on default ID1 in order to support a new(er) feature in some way, then it's not a decision. It just doesn't happen. Supporting default ID1 correctly is the single most overriding priority and nothing else comes even close.

Unfortunately r_wateralpha is one of those features - fog is another - that got added to GLQuake without much in the way of thought going into how to handle it in any kind of robust or consistent manner, and now we're stuck with the consequences. With r_wateralpha it was never meant to be a robust feature - check your your GLQuake readme - so it's kinda forgivable and understandable. (Fog is different but that's going off-topic.)

Yes, this situation sucks, but this is one of those cases where it's just not possible to be all things to all people, and a lowest common denominator that's going to work in a predictable manner needs to be chosen instead. And in cases like this the only reasonable thing is to do what ID Quake did.

So if you want your chosen value of r_wateralpha to persist, put it in your autoexec.cfg instead.

(As a postscript: the inevitable question is "how about detecting if a map has translucency or not?" and the inevitable answer is "I tried that before, it didn't work.")

Release 1.8.8 Test 2 is Out

Here be the link: http://directq.codeplex.com/releases/view/68306.

This updates the previous test with more fixes and improvements. As it's still a test release it doesn't supersede the final 1.8.7 as the recommended version (it is hoped that the next one will).

DirectQ Update - 13th June 2011

I've changed colormapping for player skins; this is something I had worked on before (with mixed success) and had always meant to revisit now that I'm on shaders-only. Under the new method, a "colormap" texture is extracted from the original skin (as defined in the MDL) and colour changes are carried out entirely on the GPU.

So what does it look like? Roughly something like this:



The red channel is a multiplier for the base skin, the green channel for the shirt colour and the blue channel for the pants colour. This doesn't mean that you'll get a green shirt; the shader extracts an intensity value from each channel and multiplies that by the selected colours.

There are a number of interesting, and some quite cool, consequences of this change:

• Colour changes no longer require a texture upload.  This completely eliminates any hitching or stalling when somebody changes colour, and makes it impossible for a malicious player to affect your game by repeatedly changing colour.
• Drawing the player model is moderately slower owing to a slightly more complex shader; that's the tradeoff.  You shouldn't notice this at all in general gameplay though.
• Colours are subtly different owing to different gradations in shade.  Modulating a greyscale image by a fixed colour can never be the same.  You won't notice this unless you do an exact pixel-by-pixel comparison in an image editor though.  (Yeah, another part of the tradeoff.)
• External textures will now work on the player model, and will even get the correct shirt and pants colours.  Note that the mask areas for shirt and pants are still extracted from the original 256-colour skin, though.  (That's required as there are specific palette ranges used for them).
• Not needing to store a separate texture for each player has resulted in some video memory saving.
• There's an interesting content bug (or maybe it's intentional?) in the original player skin: parts of the axe, some pixels on the player's face, and the soles of his boots also get colour changes.  I've cross-checked this with the old way and confirmed that it happens there too.
• Animating skin groups will work with the player model, and it's possible to have different mask areas for each skin in the animation.
• It will eventually be possible to extend this colormapping to other models, but for this version of the code I'm leaving it as "progs/player.mdl" only.
• You can load an external texture defining custom mask areas (and even blend the colours for shirt, pants and base in some areas): the naming convention is: "progs/player.mdl_0_colormap.tga".

I know that this isn't compatible with some other engine's way of doing it, but for now it's the way I'm doing it. I'll probably revisit it again later if it becomes a problem for anyone, but right now I think that the advantages and added possibilities outweigh the other side of the tradeoff.

I'll probably put a second test build, containing this and the other recent changes, up tomorrow.

DirectQ Update - 10th June 2011

I changed the logo.

Was never too keen on the old one; it was always just a placeholder. The new one pleases me much better.

Not sure if I want to bring back a splash screen though; part of me thinks it's a nice thing to have, but another part of me thinks that it's completely unnecessary.

Update for 9th June 2011

Been doing some benchmarking with the external texture loader in DirectQ. I've a stated intention to simplify this code, but the end result is that I'm not going to just yet. The primary reason is performance. A test simplified version ended up loading maps at up to one tenth the speed that the current (quite messy) version does, and I'm not prepared to take that hit.

I'll need to fork the codebase and do some experimental work before I'm happy to proceed down this route.

I'm at the stage where I honestly can't remember the status of some of the older bugs people reported with this, so if you'd like to remind me, and if they're still present, now would be a good time to make yourselves heard.

A potential crasher in the cvar subsystem has been caught; this is so highly unlikely to happen that it's silly, but just for the sake of doing things right it's nice to have.

I need to sort out some visual glitches that can happen when crossing water surfaces. This is generally only a problem if you have a powerup activated, but it looks ugly and wrong. I fear that the root cause of it is down to client/server timer separation where one of them thinks you're still underwater but the other doesn't (or vice-versa). If this is the case then it may be unfixable.

RMQ-wise we're gearing up for a demo release shortly. On the engine side there's not really much to be done (and I don't like bigger changes this close) but I'm on standby in case something needed does come up. There are a lot of interesting things shaking out of the final stages of work which it's definitely premature to talk about right now, but over the coming months as things settle down and decisions get made I'll probably write a little about some of those.

The one that it's not premature to talk about is IQM support. I've learned that there is now an IQM version 2 which fixes some subtle bugs in version 1. Changes are minimal, but they are still breaking changes. My initial thought was to junk IQM version 1 and go direct for version 2, but I'm starting to think that supporting both would be preferable.

We're not certain yet if we're going to require IQM for the coming demo, but overall we do want to put our collective weight behind supporting the format and do want to see it becoming something that gets more widespread engine support in the future. While I have my own personal list of things I actually dislike about it - with the inability to animate on the GPU being top of that list - as skeletal animation formats go (in the context of Quake technology) it is something that it just makes sense to do.

Fun with Direct3D 11

Just for fun I'm writing a D3D11 BSP viewer. I did something similar when I was originally exploring D3D9, and it's a regret that I lost the code a long time ago. This time, assuming that I ever finish it, I'll release the wee bugger.

This isn't a serious project and shouldn't be seen as any kind of indication that a move to D3D11 may happen soon; right now it's just experimental stuff that I'm doing to relieve some minor burn-out I'm feeling from the main code bases. (The code that I eventually come up with might not be the best way of doing things either...)

Getting to grips with a new API for the first time is always an interesting experience. No doubt a lot of my initial impressions will change over time as I come to understand what's going on, and begin to appreciate the reasons why things are the way they are. For now though, my old "designed by a team of monkeys on LSD" joke is proving to be a fairly accurate description.

Let's take one example: creating textures. A pretty basic requirement of any 3D engine. I can do it in two lines of code in D3D9 if I wanted (in practice DirectQ's loader is a little bit longer because Quake - being Quake - has lots of exciting special cases that need handling). D3D11? I'm up to almost 60 lines with no sign of stopping. Declare a struct. Fill it in (every struct in D3D11 seems to have at least 10 members, all of which need a valid value). Declare another. Fill it in. Pass them as parameters to a create function. Rumour has it that you need an array of D3D11_SUBRESOURCE_DATA structures for mipmaps, but where did it say that in the documentation again? This is going to be painful. What's a ShaderResourceView anyway? And why do you need one? It doesn't seem to contain anything useful that's not already defined in the texture. Can you associate the same ShaderResourceView with multiple textures? I don't know and the documentation sure ain't telling me.

OK, so this is sounding very similar to some of my initial impressions of D3D9, but those impressions did change and I eventually gained a pretty good respect for it. Will the same happen with D3D11? I don't know, and while it might not be fun to find out, it sure won't be boring.

Going 2010

Now that Visual C++ 2010 Service Pack 1 is out, and has had a few months to shake down, I've been re-evaluating the merits of switching DirectQ from a 2008 project to a 2010 project. This was something that I'd originally looked at last year when 2010 was first released, and I had always intended looking at it again roundabout this time.

Aside from change for change's sake, there are some perfectly good reasons to consider this switch. The C++ compiler is maybe twice as fast, intellisense (badly broken in 2008) is fixed, there are bugfixes in the compiler that may be worth having, and so on.

On balance however I'm concluding for now that it's still too early to make this change. Of course it will have to happen sometime as 2008 slips into the realms of unsupported, but there were several things about 2010 that I found annoying during my initial evaluation, and while SP1 does address many of them, it's still the case that the annoying things outweigh the (otherwise obvious) advantages. Maybe wait for the next one - in the past really good releases of Visual C++ (6, 2003, 2008) have tended to alternate with quite mediocre ones (2002, 2005, possibly 2010), with the quite mediocre one generally being the first release for a new technical platform.

Another 2010 switch I'm considering is to move DirectQ to the June 2010 version of the DirectX SDK. This will be a more or less essential move if I ever do switch to Visual C++ 2010 (or the next one), and also if I ever do switch to D3D 11. However in this case I can also use it perfectly well with 2008, and there are also valid reasons for making this change.

One in particular is that it seems to get some extra performance. A few percent here, a few percent there, nothing earthshattering but it's consistently that little bit faster. That's a very nice thing to have, so it weighs in favour of this particular move.

An obvious consequence of such a move is that you will possibly need a Direct 3D upgrade. Not something I'd lightly force on you, especially after the havoc of the early days, so it's still something that's up for a final decision rather than having a final decision already made. It is something that I wouldn't be disagreeable to so long as everyone else was OK with the idea, however.

Comments?

This is probably worth repeating

I mentioned this in a comment, but it's probably worth repeating in a post because it's quite important and has some broader general applicability.

Yes, I removed the automap code a few versions ago. It will most likely be coming back, but I cannot at present say when or give any indication of what form it will take.

The reason why is to do with DirectQ's final remaining major renderer bottleneck. At present there is a HUGE GPU stall - about a third of a frame when the GPU needs to go idle while I'm filling in a dynamic vertex buffer with surface data. Obviously a more optimal strategy that lets me do something useful with that time is required, so I need to investigate it and try a few things out.

Now, it just so happened that the automap code got in the way here. A key component of working on this issue revolves around how the list of surfaces to be rendered is built, and with the automap code I had two different code paths for building this list - with the automap enabled or disabled. Now, you might think that I could have just kept the old way for the automap and tried out new stuff for the standard world refresh, but I want to go deeper than that. I want to explore the in-memory representation of surfaces at a very fundamental level, and come up with something that is just hands-down flat-out more efficient for using on more modern hardware than Quake was originally designed for.

And the automap code held me back from doing that, because every time I did something I would have to do it twice.

When a novelty feature that's not meant to be very robust conflicts or interferes at such a basic level with a core requirement, the novelty feature has to be the one to go. There shouldn't be any arguments or questions about this.

Another thing that should be instantly recognised as being The Way Things Are is that experimental work is, by definition, unbounded in time. After all, it took over a year for DirectQ to come to the beginnings of it's current form, and it took a further year and a half for that to shake out as the best approach for each requirement started to emerge. I don't expect things to take that long this time, but it's clear that another experimental phase is now needed to address deep inadequacies in the Quake architecture.

The other option - which may yet prove to be viable - is to short-circuit the whole thing and move to D3D11 now, which resolves the problem in a clean and efficient manner (but with some cost loaded on elsewhere).

------------

Where this has bearings on features in general is in terms of something that should be obvious: additional features add an exponential overhead to the workload.

Let's take a hypothetical feature, and say that we have three possible combinations: it's not supported, it's supported but disabled, and it's supported but enabled. That's three code paths that need to be tested, debugged and worked over until everything is just right. Add a second such feature, especially if it's a feature that has interactions with the first, and you don't have six paths, you have nine. Add a third and it's twenty-seven, and by then you're in territory where bugs and incompatibilities really come flying out. In just three features where some modest interaction exists.

That's a battle that needs to be fought every day, and the more time you spend fighting it, the less time you have for doing the really productive stuff. Now, if those features are really valuable or vital then it might just be worth fighting it, but you really do need to ask "do we absolutely have to have this feature?" first. Because if you're not asking that question then you're just inviting trouble on yourself.

(Addendum - in a rather quirky Pingback of Karma, this is more or less one of the key reasons why DirectQ existed in the first place: frustration at the horrific mess that OpenGL extensions - and the driver compatibility thereof - were causing for me. It's also the reason why I chucked the fixed pipeline and 16-bit modes.)

1.8.8 Test Release is Out

http://directq.codeplex.com/releases/view/67510

This is a test release which has now been made publicly available for the purposes of gathering feedback and bug reports on recent changes. A full release is intended to follow in time for QExpo.

It doesn't replace the previous 1.8.7 as the recommended release for casual players.

DirectQ Update - 1st June 2011

I made no changes at all yesterday (aside from the new version number, of course).

I'll be doing some more shaking down today, and if no major changes happen as a result of that, the release will be tomorrow.

I'll update as and when.