I think it's just about ready for release now, so there are a few minor things I need feedback on before I let it out.
Firstly, I switched the project to Visual C++ 2010 a while back; I'll do a 2008 build for the release (after testing to ensure that everything works correctly of course), but longer term I'd definitely prefer to have the release be the same build that I do the majority of my work with. Thing is that 2010 apps won't run on Windows 2000, but on the other hand 2010 is a much more productive environment for me to work in. Now, I'm not going to let that prevent you from being able to run DirectQ if you really and genuinely need Windows 2000 support, but I am interested in how you would like to see this working out for subsequent releases.
Secondly, there may be some bugs in this release that I haven't caught. The nature of the way it was developed means that I had several lengthy breaks in coding, and one or two things might have been inadvertently overlooked. I'm going to have to ask for some tolerance on that one, and will do my best to address anything critical in a follow-up release as soon as possible.
Anyway, those aside I'm currently doing what I think is the final debug build of 1.8.4, run this a few times, ensure nothing crazy happens, then do the same for a release build and we should be pretty damn close.
See you at the release party. ;)
Monday, May 31, 2010
I think it's just about ready for release now, so there are a few minor things I need feedback on before I let it out.
Posted by mhquake at 8:36 PM
Tuesday, May 25, 2010
1.8.4 is really coming together well now, but one thing that occurs to me is that before I release there is scope to make some further changes.
Currently the main render is done using a combination of HLSL and fixed functionality. Liquids and the sky are HLSL, under water warp (if present) and screen flashes (if present) are also HLSL, but everything else is fixed.
Fixed functionality is also used for absolutely everything if HLSL is not available on your 3D hardware.
Now, if I was to switch absolutely everything over to HLSL I could probably get a few percent extra performance. I'm not entirely certain how much, but I'm guessing that - as I dropped about 5% when I moved back to fixed for 1.7 - the gain would be something similar. I'd intend keeping an alternate fixed path as well, but going full HLSL where HLSL is available does provide a reasonable performance increase.
It's definite that this is going to happen sometime, but the question is when? Should I do it for 1.8.4 (and delay releasing) or should I just bring the current codebase together and fit for release, then make the move for the next one after that?
I'm leaning towards the second option right now, as I think it's important to get the 1.8.4 changes out and in use by people, but I'm definitely interested in other folks opinions.
Posted by mhquake at 4:39 PM
Sunday, May 23, 2010
OK, I've caved in and decided to add user-replacable crosshairs to DirectQ. This will actually be quite easy to do, and will most likely follow the way QRack does it, which is a subdirectory called "crosshairs" under your game directory with images called crosshair1, crosshair2, etc in it. The usual LINK, TGA, DDS, PNG, BMP, JPG and PCX formats will be supported, although for formats that don't have an alpha channel things will look a mite weird.
These will directly replace the built-in crosshair images in DirectQ, so if you want to use the classic Quake crosshair you shouldn't have any replacements stored in your game dir. I could make it work a little more flexible, but things would get messy and I don't like messy.
Speaking of messy, I've decided to remove padding from MDL skins. The solution I had for external textures was fine up to a point, but would break down if a MDL had more than one skin in use, only one (or any number less than all of them, in fact) of which was replaced by an external texture. This means that most MDL skins that are not replaced will need to go through a resampling filter, but fortunately D3D has a few higher quality filters available so that the damage can be minimized.
Ultimately we're in tradeoff country again with this one, and on balance a slight loss of visual quality (which most people won't even notice) seems preferable to a byzantine and fragile code-path. That's just the way some things go.
Crosshairs are done. Very easy as predicted, and for bonus marks I was able to generalise the drawing function for the "+" crosshairs, DirectQ's built-in additional crosshairs, and user-replacement crosshairs.
I've found another bug which occurs when the viewmodel pokes into an entity: the entity which is being poked-into vanishes. I had thought that this was an occlusion queries thing, but now think it's an unwanted side-effect of my "prevent the viewmodel poking into walls" code. Not certain what the best option to fix this might be; clearing the depth buffer before drawing the viewmodel would work for sure, but I'm terrified of the potential overhead of doing this...
I'm now 99% certain that it happens if the view is inside the entity's bounding box, and actually is an occlusion queries thing.
Confirmed and fixed.
Posted by mhquake at 10:58 PM
Replacement skins are also broken in 1.8.3; this was a consequence of the texture padding code used to avoid resampling of standard skin textures. I've - kind of - fixed this for 1.8.4, but some more work is needed to get it more robust and general.
Here's the evidence:
One thing that may potentially cause trouble is the naming convention I use. Because ID1 has both s_light.mdl and s_light.spr, and because I support replacement sprite textures (must check those too...), I use the same naming convention as DarkPlaces, so to replace skin 0 on soldier.mdl you would use a skin called soldier.mdl_0.tga (assuming TGA format). And no, I'm not going to change that...
The difference is that I'm not fussy about where you put them; put the skin in id1/textures, id1/models, id1/textures/models, id1/textures/progs, id1/progs or even id1/elephant; DirectQ doesn't care and will detect it and use it.
Overall though I must say that I still have a preference for the original ID content (this is partially by way of an excuse for these oversights - did you guess?) Somehow the crappy low-res 8-bit textures capture a part of the Quake atmosphere that all replacements I've seen to date have failed with. Maybe it's the grunginess that fits better with the overall feel of the game, I don't know.
That's not to run down the work of those who made the content, and not to say that anyone who thinks differently is wrong. We all have our preferences and that is mine.
Posted by mhquake at 12:11 AM
Saturday, May 22, 2010
I finally achieved another long-time goal by unifying the splash window with the main window. This just makes some things in the code a lot simpler than they were before and will ultimately enable me to be able to bring up D3D, sound, input, etc a lot earlier than I had previously been doing so (which is the only real point in having a splash window in a Quake engine).
I had intended to actually have 1.8.4 out by now, but there are a few other things I need to go back over and check up on. Hopefully it won't be too much longer.
Posted by mhquake at 11:29 PM
Wednesday, May 19, 2010
I've been having a quick look at some skin bugs and running some comparitive tests with other engines to see if I can get some better understanding myself of what's happening.
First up is Warpspasm. Without external textures we get something like this:
Now, I've also tested this one under both DarkPlaces and FitzQuake (latest versions in both cases) and the same happens with them (the above shot is actually Fitz). This one I'm afraid is a content bug rather than an engine bug.
The Quoth pack provides external textures for a certain amount of it's content, which I haven't yet tested (I'm not even certain if this particular model has a texture provided). More news on that one maybe tomorrow.
Second one is the old long-standing bug of player skin colours reverting to default after death. My current theory on this is that it's actually after respawn that it happens, and that the reason why is because the entity number is no longer in the range for clients (and therefore doesn't get a translated skin). If this is correct it should also happen with GLQuake, in which case it's a 50/50 toss-up between whether it's intended behaviour or whether it's a bug in GLQuake that it's desirable to fix. A cross-check with software Quake should confirm, but in any event I'm leaning towards it being desirable (for me, at least) to fix.
One difficulty I can see is that if the entity number does change, some method of saying "hey! new entity here! inherit some properties from an old one (which no longer exists) please!" will need to be devised.
Once again, software Quake may very well have an easy solution to this, otherwise there might be something I can adapt from one of the major multiplayer engines. If neither of those are the case it could well get ugly.
That will probably also get some investigation tomorrow.
====== UPDATE 1 ======
I've now tested the Warpspasm bug in Fitz, DarkPlaces and Aguirre's engine and in all of them we get the same problem. This one is definitely a content bug.
The skin change bug doesn't happen in Software Quake (I've also confirmed that it's an inherited GLQuake bug). This is good news as it means that there is a fix somewhere to be had, and also that it's intended original Quake behaviour that the skin colour stay after respawn. I hate inherited GLQuake bugs where you can never be certain if the "bug" is an intended code change or just an omission in GLQuake. Thankfully there is Software Quake to fall back on as the final arbiter of the way things should be.
Of course, those who subscribe to The Great Gods Of GLQuake will always insist that any different GLQuake behaviour is somehow "more correct", but it's a big world and there's room for all kinds of people in it.
====== UPDATE 2 ======
Player skins are now properly fixed. As a bonus I was also able to clean out a lot of legacy skin translation code (there's more to go, but not too much) and make a more robust and general version of the skin translation code. God, GLQuake really did suck badly...
Posted by mhquake at 1:11 AM
Monday, May 17, 2010
Once again I've done nothing in the past week or so. :(
Truth is that I started hitting the burn-out threshold at an inconvenient time, so a break is needed. It's also a good opportunity to take stock of my current ideas and filter things through my head in terms of finding a way forward for one or two items that I'm stuck on.
These don't really have any bearing on 1.8.4 so I'll hold off on mentioning them for now, but they will start taking shape when I move to 1.9 (whenever that is!)
One thing that does have bearing on 1.8.4 is another bug with player skins. This part of the engine has been slightly buggy for some time now, so it's getting near time to start tackling it in earnest and getting a good solid fix in.
As soon as I have info on that I'll update.
Posted by mhquake at 12:51 AM
Tuesday, May 11, 2010
If you've read portions of DirectQ's source code you might have noticed that I perform my own transforms in software rather than using hardware transforms. You could be forgiven for thinking I'm silly to be doing this, but in actual fact hardware transforms - even in heavy scenes with up to 100,000 vertexes in them - are actually somewhat slower.
This isn't the normal way of things so there must be a reason, and the reason why is batching. Let's take one of the most familiar scenes in Quake.
There are 4 zombies (2 hidden behind a wall) and 3 torches (one hidden behind a wall) that must be sent to the GPU in this scene. The batching that DirectQ does enables all 4 zombies to be sent in a single operation, and likewise all 3 torches to be sent in a single operation. With hardware T&L in use it would be necessary to reset the modelview matrix (or D3D equivalent) between each entity, thereby breaking the batches into smaller chunks, and thereby making less efficient use of your GPU. True, you do gain speed in one area by avoiding the need to transform in software, but you lose far far more speed than you gain by breaking the batches, so on balance it's slower.
For 1.8.4 I've added an alternate code path to demonstrate this. Right now it's there but completely disabled (if (0)), so anyone who wants to use it would need to recompile the engine. I might add a cvar to control it, but I really don't want to encourage anybody who might be under the impression that hardware T&L must be always faster in every situation to use this code path, so then again I might not. What I do want to do is show that - yes - I have thought about these things, considered the options, tested different approaches, and settled on what works best.
This post I guess demonstrates another area where things may not always be as you expect, and goes hand-in-hand with the post I made some time ago about disabling multitexture in GLQuake (and why it's sometimes a performance advantage) versus disabling multitexture in DirectQ (and why it's never a performance advantage).
Next time I'll talk about hardware instancing and why it's not appropriate for DirectQ.
Posted by mhquake at 10:16 PM
Monday, May 10, 2010
I haven't updated for a while because I haven't done any work for a while, but today I knuckled down and started in again on a few things.
First up was a minor rewrite of my MDL texture coord generation. What on EARTH was I THINKING when I wrote the original version of this? Quite an idiotic approach, so I fixed it.
Next I looked at my external textures, thinking that I might move them to a binary search. I generally like using built-in API/CRT functions rather than rolling my own where possible (although in some cases I do roll my own) as they are tested, debugged and working code.
Normally I don't run with external textures as I prefer the classic look, and previous tests of this code just used one or two replacements (like the conback), but this time out I dropped in a full set just for the hell of it. The upshot is that external texture loading in 1.8.3 is kinda broken. Only about half of the textures load, and it's alphabetical order based, so those that come later in alphabetical order are the ones that don't load.
Now, the problem is that when initially enumerating external textures I decided to reuse my "scratch" memory buffer as a temporary holding area. This is a 256K buffer, giving room for over 65,000 external textures (an array of pointers), but being a little on the cautious side here I thought "what if someone actually does have that amount of external textures?", so I imposed a max. And when calculating the max, what did I use?
SCRATCHBUF_SIZE / sizeof (d3d_externaltexture_t);Yup, forgot a pesky "*".
So instead of the correct maximum of 65536 an incorrect maximum of 512 was used.
The really odd thing here is the lack of bug reports. I guess it confirms my general theory that most people who download DirectQ are just regular people who want to play Quake rather than hardcore community members who care about this kind of thing.
What this means anyway for 1.8.4 is that it's going to be released sooner rather than later. There were a handful more things I wanted to do, but thankfully hadn't yet gotten to the stage of disrupting the code overmuch for them, so the current plan is to bring what I have together and release it.
The option of a 1.8.3d did occur to me, but for the sake of one bugfix it's not worth it, and I'm unwilling to take the 1.8.3 line any further by rolling in some more updates.
So watch this space sometime over the next week or so!
Posted by mhquake at 9:43 PM
Tuesday, May 4, 2010
I'm currently about half way through porting Q3A's sky clouds to DirectQ. This will be as an option rather than replacing the classic sky clouds, and can be useful for certain maps as it distorts at the bottom rather than at the sides (especially if no skybox is provided, or if you don't want to use one).
I've said before how similar DirectQ's renderer is to Q3A, but this has been quite a revelation - the code was almost a straight copy and paste, with perhaps the biggest difference being that Q3A uses tess.xyz[tess.numVertexes] whereas I use verts[numverts].xyz (which works better with D3D Vertex Buffers).
No screenshots yet because it's still unfinished, but as soon as I have something I'll post it. I'll also update with details of how to select it as an option, but it's pretty much a foregone conclusion that it will be r_skywarp 2.
OK, here it is.
Now for the bad news. There are still ripples. Nowhere near as bad a GLQuake thankfully, but there nonetheless. These most likely come from a number of operations in the texcoord generation function that produce a curve, and therefore are not amenable to linear interpolation during rendering (the exact same problem as the classic sky and water, in other words). I've solved this for classic sky and water by moving these operations to a pixel shader, and could likely do the same with this at some stage.
These ripples are also present in Q3A by the way, if you go looking for them (it just hides them better by using alpha layers).
It's slow. Not as slow as it could be, but a whole lot slower than the HLSL classic sky warp. Increasing the detail level (another option to solve the ripples) could only make it slower, and turns out to be total geometry overkill. Q3A uses 8 subdivisions which is reasonable enough; increasing this to 32 almost eliminates ripples but brings the vertex count to over 5000 and the index count to over 30,000. That's 10,000 triangles just to render the sky - nope, not gonna happen.
The texture scale is poor; textures render too large towards the top and too small towards the bottom edges. In fairness you rarely ever see the bottoms, but it could still be a problem in some scenes.
All the same it's a nice option to have, it could help improve certain scenes, and if you set r_skyalpha less than 1 you can almost suspend your disbelief so far as the ripples are concerned.
Not something to be promoted to menu option status in other words, but good enough to leave as a cvar (and yes, it's r_skywarp 2).
Some time I might come back to it and explore the pixel shader option, but for now I've satisfied my own curiosity regarding use of Q3A sky as a viable alternative, so it's a good enough result.
Posted by mhquake at 8:18 PM
Monday, May 3, 2010
I really didn't expect to be making this post, but ever since I brought back the non-HLSL water warp I've been optimizing it quite well, and now have it in a state where even with a gl_subdivide_size of 8 it runs faster than the HLSL warp. This was achieved by removing duplicate vertexes from the resulting subdivided surface, so that the cost of submitting vertexes to the renderer has been colossally reduced.
I'm still undecided. The HLSL warp is higher quality as it's per-pixel, but having two code paths for liquid surfaces is currently a little too complex for my liking (and right now the non-HLSL warp would fit better with where I want to go with the renderer in the future). At the same time, the default gl_subdivide_size of 128 does not provide good quality, and I do want to have a default behaviour that is good quality. On the other hand the non-HLSL warp would be a default that works on everything, whereas the HLSL warp isn't.
Surface subdivision also slows down map loading, although I have some memcpy and memset optimizations in that can negate much of that. It does use more memory though, which is another downside.
I think I'm going to be doing some more tests and comparisons before I come to a decision, but as things stand my preference for simpler cleaner code paths needs to be balanced against the need for a sensible default that both looks good and runs well (or well enough). It's a difficult one...
Update: I've done some further testing with this and it appears that in cases where you're not bandwidth limited the optimization is of marginal benefit. It should still be good for the GeForce FX crowd though, as DirectQ's shaders won't run too fast on those cards.
Overall though it's a good thing to have, but the HLSL water warp will be staying nonetheless.
Posted by mhquake at 3:28 PM
Saturday, May 1, 2010
Credit to Sajt on the Inside 3D Forums for spotting this one. The host_framerate cvar is, and has been for some time, terminally messed up in DirectQ. The cause of the problem is that if you change the value of host_framerate, the server (and parts of the client) won't obey the new value - they will still act as though the way it should normally be is in effect.
Fortunately this is an easy fix, but I most likely won't be releasing an update for it. The reason why is that it went about 4 months before it was even noticed, so it seems satisfactory to assume that this is not a feature that's in high demand or widespread use in the first place. I more than half suspect that it's something ID put in to simulate the engine running at different speeds for test purposes, but that got unintentionally picked up on by modders.
A further reason is that the DirectQ source code has already forked in two - the current 1.8.3c release is actually a separate fork. While this is just one small fix, the hard reality is that there are a lot of things that could be described as "one small fix", and they all mount up. I don't want to create a situation where there are two very divergent codebases which would need to be reintegrated at some point.
If there is sufficient demand for it, I might do a binary only release (with the affected code file included to satisfy the GPL), but I would really prefer to not start going down that route. It's the first step on a very slippery slope that would lead to a 1.8.4 release that contains completely different code to the current 1.8.3 releases.
I suppose the deciding factor would be someone showing me a mod that is available now and that depends on changes to host_framerate. Works in progress don't count unless it's a WIP that's going to come out within the next few weeks.
Posted by mhquake at 8:30 PM