Rendering and Shaders: Difference between revisions

From Archon Wiki
Jump to navigation Jump to search
No edit summary
 
(13 intermediate revisions by the same user not shown)
Line 19: Line 19:
  SPACE - toggle camera light
  SPACE - toggle camera light
  F1 - reload shaders
  F1 - reload shaders
F2 - take screenshot
  A - toggle animations
  A - toggle animations
B - toggle animation blending with >1 anim selected
I - toggle info display
  F8 - single step animation
  F8 - single step animation
  P - reseed shader random values (shield variation, etc)
  P - reseed shader random values (shield variation, etc)
0-9 - set the shader values from the config file (see below)
UP/DOWN - walk through higher numbered shader values
LEFT/RIGHT - change texture set


  SHIFT+Click on texture button to clear texture from it
  SHIFT+Click on texture button to clear texture from it
SHIFT+Click on animation list goes from one to the other


=== Animations ===
=== Animations ===
Line 38: Line 45:


  <code>COREPATH C:\development\mygame\</code>
  <code>COREPATH C:\development\mygame\</code>
==== Shader Values ====
You can set up values which will be passed to your shaders.  These are to allow mimicking of shader operations which game code may be pushing.  They are 4 values which are passed into the shader in register 30 of both the VS and PS.  These are integer values.
SHADERVALUE<N> <X> <Y> <Z> <W>
e.g.
SHADERVALUE0 1 0 3 99
The N value can be 0 to 63.  Apply a given value with the 0-9 keys, or UP/DOWN.
==== Info Display Size ====
CUBESIZE <float size>
The size of the cube and axis indicators shown when you select an object in the frame list.  Default size is 1.


==== Find Textures ====
==== Find Textures ====
Line 53: Line 71:
  PATH <texturePath>
  PATH <texturePath>
  PATH ...</code>
  PATH ...</code>
NOTE: Texture sets are assumed to be in subfolders, with naming of TEXTURE1, TEXTURE2 etc.  Note that the tool will only look for subfolders numbered [1,30].


==== Lights ====
==== Lights ====
Line 169: Line 189:


For the Tile flavour, there are special ID values which cause their Post Process to be applied.  They are
For the Tile flavour, there are special ID values which cause their Post Process to be applied.  They are
+ 99 - applies the post process globally after all UI has been rendered (does not affect the mouse)
+ 99 - applies the post process globally after all UI has been rendered (does not affect the mouse)
+ 100 - applies the post process to the battle view.
+ 100 - applies the post process to the battle view
[Note: the lighting file can include a CLUT entry with a texture filename.  If this is included, then the named texture is pushed into the last texture on the last pass for the battle view post process shader, so available on sampler s4]
CLUT <texture name> // no extension needed
+ 101 - applies to the battle view in the same way as above, BUT applies before the UI icons are rendered.


Generally flavours will keep values above 99 for system and automated use.
Generally flavours will keep values above 99 for system and automated use.


=== Shader Details ===
=== Shader Details ===
Post Process shaders are applied across the entire screen, and so their shaders will look very similar to the screenrgba.txt shader vertex shader.  When the pixel shader is applied, you will find the previous screen render contents in the the first sampler, followed by the 4 custom textures for the post process pass, if applicable.
Shaders must be in the DATA/GFX/SHADERS folder. Post Process shaders are applied across the entire screen, and so their shaders will look very similar to the screenrgba.txt shader vertex shader.  When the pixel shader is applied, you will find the previous screen render contents in the the first sampler, followed by the 4 custom textures for the post process pass, if applicable.
 
c0 contains the following information
x - screenWidth
y - screenHeight
z - the pass (from the post process file ordering)
w - unused
 
== Mesh Randomisation ==
To allow for the random display of sub-meshes, name the objects so they start with a ~ symbol, followed by an index [0,9] and then a hex character which is the mask (it allows a hex value, so it can be 0-9 or A-F, must be capital).  Each value can only be a single char.  So an example would be
 
~01somethingsomething
 
This would use the first random value, and show when it had value of 0.  The logic is actually
 
// OK, now finally see if we are visible!
if(((1<< (mRandom[index]&3))&mask)==0)
      return false ;      // not visible
 
So you can set up masks so that one object gets shown more, or indeed so that sometimes no object is shown, as well as having objects which show together (if they have the same masks on the same index).  Currently only the first 4 indexes are set with random values for units. 
 
As an example, if you name objects ~01 ~02 ~04 ~08 then it will only ever show one of the four variations when rendering.
 
= Transform Reset =
 
This is a script you can use to reset the transforms correctly in MAX.  The script was written by Dr. Charles B. Owen, but the host page is no longer up, so I duplicate it here for ease.
 
XNA Utilities MAXScript for 3DS Max
 
An annoying problem when using 3D Studio Max models in XNA is that the pivot points you set in 3DS Max don't end up being the pivot points in the XNA model. Now, you can do Reset XForm and that will set the origin of the pivot point correctly, but it resets the axis to the world X, Y, and Z, which is not what you want in many cases.
 
So, I created this script that fixes this problem. The script is XNAUtilitiesOwen.ms. Right click on the link and choose Save As.
 
Installation
 
To install you either:
 
In the Utilities Rollout, hit the MaxScript button and choose Run Script. Select the XNAUtilitiesOwen.ms script to run. This installs the utility. Then choose it from the Utilities list box to display it.
 
Alternatively, you can drop the utility into the 3D Studio Max Scripts/Startup directory and it will be automatically loaded on Startup.
 
How it works
 
A 3D Studio Max node has a transform associated with it (just called transform). This sets where the origin of the node local coordinate system is relative to the parent. This is the pivot point. Also associated with the node is an object offset transform. The mesh vertices are multiplied by the object offset transform, then the node transform then the parent transform. This is what allows you to freely move the pivot point. If you move the pivot point, it changes the node transform. To keep it from moving the model when you move the pivot point, the object offset transform is changed by an inverse amount, cancelling the effect on the vertices.
 
The problem is that what XNA is getting is not the vertices after the object offset transform, which sets the objects relative to the pivot. Instead, it is getting the actual model vertices and only the modifier stack has been applied. So, the transform for the bone in XNA is the product of the object offset transform and the node transform. For pivot points to work right, the object offset transform needs to be an identity matrix. I created a script that zeros that transform and creates an XForm modifier on the stack that does the same transformation. Since XNA gets the vertices after modifiers, it all works.

Latest revision as of 19:57, 8 July 2020

Converter Tool

NOTE: The Converter replaces the MAX script plugin for exporting units. The converter can accept FBX files from most 3D tools.

Valid map slots to be used are Diffuse, Specular Colour, Glossiness, and Bump (which is the normal map). The exporter should correctly handle baked materials. The maps are available in the game shaders in the following order in samplers [0-3].

Diffuse
Normal
Specular
Gloss

Tool Controls

Hold LMB - rotate camera
Hold RMB - pan camera
Mousewheel - zoom (+SHIFT for faster)
RETURN - reset camera
L - cycle through lights
SPACE - toggle camera light
F1 - reload shaders
F2 - take screenshot
A - toggle animations
B - toggle animation blending with >1 anim selected
I - toggle info display
F8 - single step animation
P - reseed shader random values (shield variation, etc)
0-9 - set the shader values from the config file (see below)
UP/DOWN - walk through higher numbered shader values
LEFT/RIGHT - change texture set
SHIFT+Click on texture button to clear texture from it
SHIFT+Click on animation list goes from one to the other

Animations

If there is a suitably formatted animation file in the same folder as the mesh then the animations will be shown in a list. The chosen animation will be looped. If you hold SHIFT when clicking an animation it will then loop both that animation and the last animation chosen without SHIFT down, blending between them. This is for blend testing mainly and should generally not be used - once turned on it cannot be turned off without restarting the tool.

Batch Processing

The batching processing allows for multiple files to be loaded, optionally have the current textures applied, and then saved out. Both FBX and S4F files can be processed. Once processed the files are exported to an S4F with the same name (so if processing an S4F file, then original file will be over-written). By default the currently applied textures in all 4 slots are applied to the model once loaded. If you hold down SHIFT when clicking the Batch Processing button then this will not happen and the model will simply be imported and then exported as an S4F. If you hold down CONTROL when clicking on the Batch Processing button the model will be imported, optimised (as per the Optimise button at the bottom of the panel, removing unused nodes) and re-exported as an S4F.

CONFIG File

If a CONFIG.TXT file is found in the same folder as the EXE, it can contain the following:

Shader Path

Must be at the top of the file, optional. Points to a folder where a CORE/SHADERS folder exists, allowing you to point the tool to the shaders being used by (e.g.) your game data.

COREPATH C:\development\mygame\

Shader Values

You can set up values which will be passed to your shaders. These are to allow mimicking of shader operations which game code may be pushing. They are 4 values which are passed into the shader in register 30 of both the VS and PS. These are integer values.

SHADERVALUE<N> <X> <Y> <Z> <W>

e.g.

SHADERVALUE0 1 0 3 99

The N value can be 0 to 63. Apply a given value with the 0-9 keys, or UP/DOWN.

Info Display Size

CUBESIZE <float size>

The size of the cube and axis indicators shown when you select an object in the frame list. Default size is 1.

Find Textures

For tools which do not have standard texture naming or placement you can set up automated searches for normal or specular textures. Use the following format:

FINDNORMAL <postfix>
FINDSPEC <postfix>

Where postfix would be something like _normal where textures are named (e.g.) main.dds, main_normal.dds, etc.

Texture Paths

Multiple texture paths which are searched in order for any textures

[TEXTURES]
PATH <texturePath>
PATH ...

NOTE: Texture sets are assumed to be in subfolders, with naming of TEXTURE1, TEXTURE2 etc. Note that the tool will only look for subfolders numbered [1,30].

Lights

Multiple lights can be set up, each light must be a triplet of direction, colour, and ambient

[LIGHTS]
DIRECTION <vector>
COLOUR <hex>
AMBIENT <hex>

DIRECTION ...

An example CONFIG file is

[TEXTURES]
PATH C:\DATA\BATTLE\UNITTEXTURES
PATH C:\DATA\TEXTURES

[LIGHTS]
DIRECTION 0 -1 0
COLOUR FFFF0000
AMBIENT FF003300

DIRECTION 1 0 0
COLOUR FFFFFF00
AMBIENT 0

Default Shaders

The system loads and uses some specific default shaders for specific types of rendering from CORE/SHADERS. These are:

screenrgba.txt
font.txt
grey.txt
screenrgba_solid.txt

The above shaders are used for standard UI object rendering and font drawing, and will generally not be changed by the user.

lit.txt - draw standard lit objects (lit via normals)
prelit.txt - draw prelit objects (vertex colour stored in vertex data)
normal.txt - draw objects with normal mapping (vertex data includes tangent space info)
lit_skin.txt - draw as per lit.txt but with hardware skinning
normal_skin.txt - draw as per normal.txt but with hardware skinning

These shaders are generally the ones that you will customise to your specific workflow and texture setups.

Custom Shaders

Custom shaders can be used in two ways. Either include a new shader in the base SHADERS folder and use the tool to apply it to the applicable meshes. This however can be fiddly if you have combined shader types (e.g. skinned/non-skinned) and requires you to create versions in the 1 folder for shadowing.

A simpler approach is often to use the Style system. To create a new style you add a numeric folder to the SHADERS folder (note that style 1 is always used for shadow generation). So for example, if you wanted a specific type of normal-mapped skinned shader you would create a new folder, say 5, and copy in your default normal_skin.txt shader file. You then alter the 5/ version as desired. Then you can use the numeric buttons in the tool to apply a style to a given model and save it out. Note that if a shader cannot be found in the style folder then the system will fall back to using the default version.

Shader input data

When writing shaders, the following vertex shader registers can be assumed to be filled

  • world matrix c0
  • view matrix c4
  • projection matrix c8
  • WVP combo c12
  • light data (dir, colour, ambient, fill colour) c16
  • world Normal matrix c20 [DEPRECATED: Use c0 and ignore/zero translation elements]
  • shadowmap lighting trx combo c24
  • camera position vector c28
  • camera look vector c29
  • C30 is sometimes used to store 4 pseudo random (integers as floats) numbers constant for a given instance of a mesh (e.g. per unit currently)
  • c31 fogging. xyz are the RGB values, w is start + (65536*end). Decoding example below.
  • c32 assume all past here are bones

registers 16,28,29,30,31 are duplicated into the pixel shader. As well as some others.

  • text colour when rendering fonts c0
  • shadow projection matrix c4
  • shadow texture half pixel size c20
  • modal data for pixel shaders c32 (implementation specific)



Fogging Decode Example

	

	float4 fog : register(c31) ;

	...

	float start, end ;
	float a, dist;

	end = round(fog.w/65536.0) ;
	start = fmod(fog.w, 65536.0) ;
	
	distV = In.drawPos - gEyePos.xyz ;

	dist = length(distV) ;
	a = dist ;
	a -= start ;
	a = a/(end-start) ;
	a = saturate(a) ;
	
	outputColour.xyz += a * fog.xyz ;


Post Processing

To apply post processing you need to create .txt files in DATA/GFX with the following format ID <id> // must be [1,255] zero is invalid

// first pass
[0]
SHADER <shader file>
TEX0 <texture file> // optional, allowed 4 textures

// optional additional passes [1], etc up to 8 passes allowed

The ID is used to determine where the post processing happens. It can be applied to a UI object on its root object using a POSTPROCESS <id> tag. NOTECurrently all post processing happens to the entire screen, so be aware when applying to UI objects.

For the Tile flavour, there are special ID values which cause their Post Process to be applied. They are

+ 99 - applies the post process globally after all UI has been rendered (does not affect the mouse)
+ 100 - applies the post process to the battle view
[Note: the lighting file can include a CLUT entry with a texture filename.  If this is included, then the named texture is pushed into the last texture on the last pass for the battle view post process shader, so available on sampler s4]
CLUT <texture name> // no extension needed
+ 101 - applies to the battle view in the same way as above, BUT applies before the UI icons are rendered.

Generally flavours will keep values above 99 for system and automated use.

Shader Details

Shaders must be in the DATA/GFX/SHADERS folder. Post Process shaders are applied across the entire screen, and so their shaders will look very similar to the screenrgba.txt shader vertex shader. When the pixel shader is applied, you will find the previous screen render contents in the the first sampler, followed by the 4 custom textures for the post process pass, if applicable.

c0 contains the following information

x - screenWidth
y - screenHeight
z - the pass (from the post process file ordering)
w - unused

Mesh Randomisation

To allow for the random display of sub-meshes, name the objects so they start with a ~ symbol, followed by an index [0,9] and then a hex character which is the mask (it allows a hex value, so it can be 0-9 or A-F, must be capital). Each value can only be a single char. So an example would be

~01somethingsomething

This would use the first random value, and show when it had value of 0. The logic is actually

// OK, now finally see if we are visible!
if(((1<< (mRandom[index]&3))&mask)==0)
      return false ;      // not visible

So you can set up masks so that one object gets shown more, or indeed so that sometimes no object is shown, as well as having objects which show together (if they have the same masks on the same index). Currently only the first 4 indexes are set with random values for units.

As an example, if you name objects ~01 ~02 ~04 ~08 then it will only ever show one of the four variations when rendering.

Transform Reset

This is a script you can use to reset the transforms correctly in MAX. The script was written by Dr. Charles B. Owen, but the host page is no longer up, so I duplicate it here for ease.

XNA Utilities MAXScript for 3DS Max

An annoying problem when using 3D Studio Max models in XNA is that the pivot points you set in 3DS Max don't end up being the pivot points in the XNA model. Now, you can do Reset XForm and that will set the origin of the pivot point correctly, but it resets the axis to the world X, Y, and Z, which is not what you want in many cases.

So, I created this script that fixes this problem. The script is XNAUtilitiesOwen.ms. Right click on the link and choose Save As.

Installation

To install you either:

In the Utilities Rollout, hit the MaxScript button and choose Run Script. Select the XNAUtilitiesOwen.ms script to run. This installs the utility. Then choose it from the Utilities list box to display it.

Alternatively, you can drop the utility into the 3D Studio Max Scripts/Startup directory and it will be automatically loaded on Startup.

How it works

A 3D Studio Max node has a transform associated with it (just called transform). This sets where the origin of the node local coordinate system is relative to the parent. This is the pivot point. Also associated with the node is an object offset transform. The mesh vertices are multiplied by the object offset transform, then the node transform then the parent transform. This is what allows you to freely move the pivot point. If you move the pivot point, it changes the node transform. To keep it from moving the model when you move the pivot point, the object offset transform is changed by an inverse amount, cancelling the effect on the vertices.

The problem is that what XNA is getting is not the vertices after the object offset transform, which sets the objects relative to the pivot. Instead, it is getting the actual model vertices and only the modifier stack has been applied. So, the transform for the bone in XNA is the product of the object offset transform and the node transform. For pivot points to work right, the object offset transform needs to be an identity matrix. I created a script that zeros that transform and creates an XForm modifier on the stack that does the same transformation. Since XNA gets the vertices after modifiers, it all works.