andrew edward baileyvenompersonal website

The Drop Spider Venom Engine

Feature List

Deferred Render - A deferred render is a scene based rendering system that performs lighting (and shadows) as a whole, rather than polygon by polygon. As lights are applied separately as polygons in 2D, the number is virtually unlimited, with no hard-coded limit due to size of pixel shaders and so forth.

Multi Track Animation - A skeletonal animation system via a multi-track blender as standard, allowing runtime mixing of clips and sub-sections of clips.

C++ Game Engine - Runtime engine code is a C++ engine built on two layers, a general drop spider framework and a venom game engine specific.

Python Build System - The build pipeline is written in python, and utilises the C++ framework for common types and utilities, via python extensions. As such all tools are easily combined, and can optionally be driven by a build system such as SCons.

Render Primitives - Static mesh, layered particle effects, skinned mesh and height-map terrains. Also a static mesh instancing system for fast rendering of normally poor overhead to polygon count like eco systems.

The Venom Deferred Renderer

Venom employs (optionally but with preference) what is referred to as a 'Deferred Renderer'. This is a method of rendering that uses multiple (deferred) passes on the source geometry, rather than a single pass, on order to perform lighting effects.

While this requires multiple issuing of the geometry to the GPU, each pass is much simpler, and hence much faster, than that of a normal render. It also removes hard limits on the number of lights and shadows, so being far more scalable, both in a given game, and for future hardware.

Depth Pass

The first pass lays down the Z buffer. On some machines this may be as simple as rewriting to the Z buffer without writing to any frame buffer. On DX9 it isn't possible to sanely get the Z Buffer as a texture source, so we write it out to a frame buffer. We take the opportunity to write it out in a linear normalised format, 0-1.0 for eye to far clip plane.

Venom also supports punch through on this pass. 'Punch through' is a nemesis to modern rendering techniques, but is still required. By laying down the punch through now, we do not have to worry about it in any more passes. This is because we can use Z EQUAL compare for all other 3D passes.

cocost
Depth buffer visualized.

Material pass

The second pass lays down the information for lighting and other effects. This is nominally normals as shown here, and the specular decay (hidden in the alpha channel of this image). The normals may come from vertex normals or bump maps, and stored as RGB in viewspace orientation.

Note we use Z EQUAL, with no Z update, so we only do exactly one screens worth on pixel shading (in theory). Other material information can be laid down on this pass, including material multipliers and such. Venom, however, does not do this, but provides this information in the final composite pass. Also motion blur vectors and other special effect information may be laid down.

cocost
Normal buffer visualized.

Lighting

This is of course the main meat of the exercise, why we are doing this in the first place!. The lighting pass is often referred to as '2D' because each light is processed as a quad (or two tris!) render, rather than the geometry of the object receiving the light. But of course the actually lighting is 3D!. Small point lights are a winner here, costing no more than particles sprites, but giving proper lighting results (like specular) instead of just polygon additive effects.

So first a full screen quad pass is made to lay down the ambient colour and clear the specular buffer. This uses dual render buffers, but if this is not possible is can easily be be done in two. In this pass the Venom engine also renders global 'rim' lights. One for horizontal lighting and another for vertical. Rim lights are used to help outline objects in other wise dark scenes. Because the normals are in view space we simply look that up for each pixel and use the X and Y components accordingly.

Then for each scene light in the viewable frustum (parallel, spot etc.) a rectangle covering the extent of the light is drawn. The pixel shader extracts the normal, and uses the depth buffer with the screen position to inversely project the viewspace position. Then the standard lighting models can be used to generate the diffuse and specular colours, which are additive blended to the frame buffers. Repeat for each light. This is why there is no hard limit to the lighting, because we can simply do as many light passes as required, or GPU possible. Each light can be any lighting model we desire are well, even toon lighting or outline generation for example.

cocost
Diffuse buffer visualized.

Shadows

Shadows buffers are generated the same way as for other lighting methods, by rendering depth from the camera perspective to a buffer, or a set of buffers for hierarchical resolution. They are applied, however, as textures to the corresponding light pass, where the viewspace position is projected into light space and the z value compared with the XY look up in the shadow buffer. The comparison result determines if the light calculation is done or not.

Compositing

Then it is time to stick it all together. The usually method for a deferred renderer at this stage is to render the source geometry for real as so to speak, laying down diffuse texture, and maybe some decals. Then apply lighting as a quad pass using the diffuse and specular render outputs. This method, however, requires the lighting tint values, be either a constant or a map, to have been laided down in the material pass, using at least another buffer. It is just as easy to access these values in the diffuse render pass and blend in the diffuse and specular renders at this time, which is what the Venom engine does. And as alpha channel specular maps are one of the most common forms of map, it is actually far more efficient this way, as the alpha channel of the diffuse texture is already available and doesn't require another texture stage.

compo1
FInal image.

The Venom Animator

An eight track (can be compiled for more) animation system is available for every instanced rig. Each track can influence the whole or part of the rig, and be blended with the tracks before it. An efficient animator and blender ensures only the channels of each track that actually contribute to the final pose are processed. And because the blending works on the hierarchical key frames, and not the skinning bones, proper rotation and scalar blending occurs.

The Venom rigging and animation system makes a clear distinction between the source keyable channels, arranged in a hierarchical manner, and the flat bone set used for mesh deformation. The animation clips know nothing about deformation, and the deformation knows nothing about animation.

This allows the hierarchy to be arranged to be easier to animate for the authors, and combined with sub-channel compression actually results in smaller animation data than keyframed bones. A hierarchy is actually a form of data compression in its own right.

This separation allows the animator to be kept distinctly independent of the mesh deforming. This allows replacement animators, such as ragdoll for example, to be easily used.

C++ Game Engine

The Drop Spider Framework

namespace dsF::

This is a common C++ library for both engine and tool sets. It includes:-

A 3D maths library.

A library that provides the fundamentals for manipulating vectors, points, spheres and matrices.

File handling.

File handling is seamlessly virtualized to allow access to either the native filesystem or files contained within an EFS.

Resource management including:-

A loader for BRF files.

Binary Relocatable Files provide a single step solution for asset loading and management. Memory allocation, decompression and relocation are handled in a reboust simple manner.

An associated smart pointer class.

A 'Handle' and 'Grasp' system provide 'smart' pointers that are also resource file aware, meaning you don't even have to worry about when to unload assets.

Memory management.

Memory allocation is virtualized to provide a cross platform interface and provide richer debugging capabilities.

Compiled XML iterator.

XML files can be compiled into a more compact format, and an iterator class used to parse them. This format is provided by the BRF tool and requires a magnitude less overhead compared to a full DOM implemtation, which is not required for simple reading of XML based meta-data.

Locale management.

Time and Text localization built in at the 'kernel' level. This means all systems in the engine can be localization aware, via a single interface.

cocost
Framework - set of base and common routines.

Venom C++ Engine

namespace dsV::

This is a C++ cross platform 3D gaming engine. It is intended to be lightweight, not requiring cutting edge hardware, yet at the same time scalable to make full use of more powerful machines. It includes:-

Deferred shadow/lighting system

This is an optional, but highly preferable, scene based rendering system. An overview to this technique can be found here.

Blended skeleton animation

A multi-track animation system and efficent boneing/skinning engine.

Extendible Camera base class

Build your own camera concepts on the point of view class, which is used by the viewport system and scene culling.

'Glue-able' shader model, runtime compiled and cached.

Venom provides both a flexible shader model but also the ability to provide custom shader code as well. Thus being ready to roll but also extendable.

Render types:- 3D Static Mesh/Skeleton/Height map/Particle System, 2D image and font rendering.

The usual game primitives, all using the same shader model and sensibly sub-classed for common interfaces.

'Taskcard' meta game logic stack.

Proper game structure is provided from the ground up, dealing with the non-atomic nature of video game state changes.

XML stylesheet driven GUI/HUD built on 'Taskcard's

A stylesheet driven GUI class, built on the Taskcard API.

cocost
Engine - set of common routines targeted at games.

Drop Spider Tools

This a collection of python scripts and extensions. The drop spider tool chain is written in python for speed of development and robustness of execution. Custom extensions improve performance and memory usage where required, for example converting textures to platform native formats. As such, tools do not exist as an 'executable', but rather a Python class that can be imported at will into any python scripted build chain.

Graphic Library

Multiple format texture support (jpeg, png etc). Tool set for cropping, scaling and compositing, so texture manipulation can be easily performed in build chain.

File System Processing

Compression

General custom lossless compression tool.

EFS (embedded file system)

A meta file system for simplifying and managing files on all platforms.

Online DRM

An online encryption system for use in development. Not meant as a commerical system, but allows developers to show their wares without it becoming the latest bit torrent.

BRF (Binary relocatable file)

A general binary file format that can handle all data usages. Provides a single simple data management paradigm for targets. Includes XML complier to a easy read-only binary format.

GOM (Geometric Object model)

A hierarchical object model in python for describing 3D model information. Used as intermediate export format and as a target format for procedural generated data. Highly custom extendible, with each node taking any user attribute information required.

GOM serialises to a XML file inside a zip file. This allows other external file references (like textures) to be optionally embedded within the GOM file. It is also easily human readable and viewable using standard windows tools.

GOM explorer

A windows GUI application that sits on a python interpreter. It allows the loading and viewing of GOM files. Via the use of a Python interpret window, the GOM file can be manipulated with python commands or whole scripts. It includes a reference python driven Open GL renderer.

Art (GOM) compiler

Set of python classe's for preparing data for a target platform. This includes:-

Basic texture conversion (DXT compression for example).

GOM compiling,

GUI stylesheet preprocessing.

All Rights Reserved 2019.