Play in the Mud!

No, I'm not talking about the "if there's grass on the field" limerick..

Halo Wars has gone gold, officially sounding the impending closing of Ensemble Studios. And with the impending job hunt coming along I decided to sharpen my coding teeth and start putting together some various research topics that I've been waiting to find time to do; In the games industry, you get so little time to do REAL research and prototyping that's not aligned to a product time line.

Much to my chagrin however, I realized that in all my years in the industry, I did not have a code base that didn't legally belong to someone else. IE all the code I had been using and prototyping on has been the property of the companies I've worked for at the time. Hypothetically, i could continue to use snippets of code from these companies I've worked for, but legally, that's a bad idea; and on top of that, most useful code is so ingrained in common classes & libraries that extracting small portions of it is just a nightmare of pulling a spegetti noodle out of a plate. AND with MS's legal department eyeballing ES with the impending closure, I decided that it had been high time, and decided to throw together my own code base, that I owned the rights to, rather than some other corporate entity with lawyers who look at me like I'm going to steal the coffee maker...

Now, some of you may not find this interesting, or very rewarding, but for the few of us, this type of process can be very meditative. That is, creating and organizing code bases and categorizing ideas and program flow puts us in a very tranquil and zen state of programming when our tasks are usually quite chaotic. But none the less, I needed to start creating a library of code to use. Firstly, I laid down some ground rules about WHAT type of code i was putting together.

This IS a collection of code to be used for personal research and demos. I'm not going to use this to ship a game, or sell to companies.

This IS going to allow slop. I'll follow standard coding syntax & commenting, but why spend time doing useless things. If I did ever want to propagate some of this code into a mainline of a project I'm working on for a company, I will port the code to the new system and only take what's needed. Spending un-needed brainpower on specific nuances in interface design and consistency throughout name spaces is just something I don't want to burn up.

This is NOT an "engine". An "Engine" denotes some sort of pipeline for content processing and handling, and that is NOT what I'm building.

This is NOT optimized. Programmers can spend a ton of time writing the perfect implementation for SIMD systems and faster KDTrees, but that's not what i want to focus on here. You've always got plenty of time to optimize, but you don't always have time to research.

I like to call this library a "Mudbox" (Or "MudBase" like "CodeBase"), not a "Sandbox," specifically for the reasons listed above that this isn't meant to be production & main-line quality code, this is idea code; fast, dirty, and cheap. If I need it to be something more, then I'll re-write it in the context that's needed. (Your first writing of code is usually pretty crappy anyhow. It usually takes 2-3 revs of the 'concept' before it solidifies and actually results in a high quality product.)

At this point, I've got 100% of the code systems done that I originally planned for. Which includes enough functionality to port various demos and code systems that I had using other code bases to my own, and have them work to produce the originally desired results. Which is a very good thing, considering I started entirely from scratch!

To be honest, it's very liberating and free-feeling to be working in this type of code base. And I honestly encourage all of you to break free from your corporate code bases, and make your own mud-base to just hash out ideas and 'fun code' in. I think we as programmers get to focused on lots of minute details : "Well, as I write this file system, I need to take into account the fact that we may be using archives later, so that information needs to propagate into my design."
Yea, you're supposed to think that for a shipping product; After all, you're being paid to not need to re-write code. But for your personal code base, fuck it and just use fopen with no error checking. I mean who's watching, really? And is it really the point to write the most stable system, when you're just testing if the overall concept for your crazy brained idea is going to work at all?

Another great part about breaking free from corporate code bases, is that you can vulture some code from great resources on the internet. TBH I've most likely bought in a little TOO hard to the "Good programmers code, great programmers reuse" concept. I mean really, I can't recall how many times I've re-written a thread safe vector class, that I STILL don't own the rights too; That's quite a defeating feeling.. There's some great resources out there that you can directly look at when making your own mudbase, and quite frankly, you SHOULD be looking at other resource material!


Here's some great resources that you can use when looking for this type of code.

---

CatMother

Homeworld

Quake series

Unreal 1

Graphite

Intel Threaded Building Blocks

Nocturnal Initiative

EA STL

Arauna

Graphics Gems

I suppose the moral of this story is: If the corporate owned code keeps grass from being on the field, then write your own code base and play in the mud.

Ok, so I did end up referencing the limerick...

~Main

Sparse Voxel Octrees

#1 Happy New Year.

#2 My child has been born! I've now been defecated on by Bipeds as well as quadrupeds..

#3 Sparse Voxel Octrees:

I've been spending a great deal of time lately attempting to make heads / tails of the whole 'sparse voxel octree' technology that ID is touting as the 'next big thing'.
You can find a good writeup of similar technologies here at Anteru's blog and get the origional post from PC Perspective.

What piqued my interest in this topic was the latest Windows Graphics Summit, in which we discussed the concept briefly in terms of it's supposed 'silver bullet' to artist content generation.

I did a little digging on the whole 'point cloud' / 'voxel' rendering process, and drummed up some good links.

Jon Olick from ID gave a Siggraph talk on their tech, you can find a discussion on it over at OMPF; a website, IMHO that contains some pretty good resources in terms of raytracing and hardware black-magic. The link provided also contains many sub-links to other great papers on the same topic.

An excellent paper over at Pixar's vault : Point-Based Approximate Color Bleeding echo's a similar desire to work in a point-cloud space, rather than a parameterized mesh/texture space. Their voxel setup allows them to approximate various forms of lighting to be used for near / far geometry. Pixar also has a great chapter in the book Point Based Graphics, that, from what i can tell, was written entirly by the guys over at Pointshop3D (which is an interesting concept in it's own right). The chapter details their point cloud system for the RenderMan system that uses Brick maps and other fun things to access LOD in volume space for various rendering reasons.

Sylvain lefebvre (whom I will always give props to for his research into the texture synthesis relm) has a great derivation that appeared in GPU GEMS2 titled Octree Textures. It's a little different concept, but the same idea in the end;

After going through all this research, and implementing my own version of an SVO system, I came to a couple conclusions that I'd like to share.

After implementing my own version of the system, I've got some thoughts that I figure I can briefly share:

• As far as ray tracing goes, when we discuss making it accessible to the hardware, standard poly/ray ray tracing is quite difficult to map to the hardware. And by this, i mean that it needs to be simple and easy enough to create dedicated hardware set aside to actually produce the system. Like putting LZW on a chip, or a Hardware tessellation unit. Voxel grids on the other hand, are much simpler to implement in the hardware relm for obvious reasons. I can't help but to think that if the future was to design graphics hardware for it, then yes, voxel ray tracing would be easily produced in hardware, and would likely give us a much better performance throughput as far as ray tracing is concerned.


• Voxel data for an entire level takes up an epic crap ton of memory. You can consider that most geometry in videogames right now get away with a few memory tricks by re-using tiling textures, and instanced objects. But for a voxel representation, each one of those pixels (voxels) will be unique; ultimately forcing you to need an out-of-core management system in order to interact and use it properly. Now, IMHO that seems like a stretch to me; I know we're moving in the direction of multi-core systems, which enable out-of-core compression and streaming, but it seems like a step backwards to increase our memory footprint (to a MASSIVE scale) to achieve the same visuals and reduce our ability to handle dynamic environments. Now, you can do this with the proper setup, but I'm simply stating that maybe we could do something else with all that bandwidth; instead of dedicate it to streaming in the same scene representation that we can achieve now


• If you ignore the memory / streaming footprint issue, Voxels make a great deal of sense from the performance perspective. In traditional rasterization, you have the difficulty with overdraw and tessellation problems that occur when dealing with ZBUFFER style rendering. On top of that, we have to implement a great deal of extra LOD systems for geometry and textures in order to effeciently balance the payload when we get sub-pixel tris/texels. Octree Voxel tracing fixes that; As your ray is being cast through the octree, if the children of the current node have a screen space area that's less than a pixel size, return the color of this node instead of moving lower in the tree. That's a pretty interesting concept if you think about it. Hypothetically, you could increase your detail indefinitely. It also removes the need for artists to be concerned with memory footprint / geometry detail, because it's all translated into something that the programmers manage, rather than the artists. Which from my experience is a huge win from the asset creation standpoint.

With the links I've provided, you can find a great deal of other smart people talking about the same idea. To be honest, after implementing the concept myself, I can't see this technology really taking hold of the industry when stacked next to Realtime raytracing in a multi-core universe.

Now, if hardware manufactures came back and started providing dedicated hardware for this type of task, then yea, I can see it catching up, but that's a huge amount of time down the line. I have a feeling this technology is a good idea, but will fall the way of the hype of 'mega texturing.' Great idea, but what about it makes the end user really want to pay the extra money to upgrade their box to buy it?

Just like attempting to implement realtime GI for games; Is the dev time and result enough to squeeze an extra 30k product sales for your title?

If not, move on to something more important; like trying to figure out how to change a dirty diaper in 30 seconds flat..

~Main