Rendering and Shaders: Difference between revisions

From Archon Wiki
Jump to navigation Jump to search
(23 intermediate revisions by the same user not shown)
Line 14: Line 14:
  Hold LMB - rotate camera
  Hold LMB - rotate camera
  Hold RMB - pan camera
  Hold RMB - pan camera
  Mousewheel - zoom
  Mousewheel - zoom (+SHIFT for faster)
  RETURN - reset camera
  RETURN - reset camera
  L - cycle through lights
  L - cycle through lights
  SPACE - toggle camera light
  SPACE - toggle camera light
  F1 - reload shaders
  F1 - reload shaders
A - toggle animations
F8 - single step animation
P - reseed shader random values (shield variation, etc)
0-9 - set the shader values from the config file (see below)
LEFT/RIGHT - change texture set


  SHIFT+Click on texture button to clear texture from it
  SHIFT+Click on texture button to clear texture from it
=== Animations ===
If there is a suitably formatted animation file in the same folder as the mesh then the animations will be shown in a list.  The chosen animation will be looped.  If you hold SHIFT when clicking an animation it will then loop both that animation and the last animation chosen without SHIFT down, blending between them.  This is for blend testing mainly and should generally not be used - once turned on it cannot be turned off without restarting the tool.
=== Batch Processing ===
The batching processing allows for multiple files to be loaded, optionally have the current textures applied, and then saved out.  Both FBX and S4F files can be processed.  Once processed the files are exported to an S4F with the same name (so if processing an S4F file, then original file will be over-written).  By default the currently applied textures in all 4 slots are applied to the model once loaded.  If you hold down SHIFT when clicking the Batch Processing button then this will not happen and the model will simply be imported and then exported as an S4F.  If you hold down CONTROL when clicking on the Batch Processing button the model will be imported, optimised (as per the Optimise button at the bottom of the panel, removing unused nodes) and re-exported as an S4F.


=== CONFIG File ===
=== CONFIG File ===
If a CONFIG.TXT file is found in the same folder as the EXE, it can contain the following:
==== Shader Path ====
Must be at the top of the file, optional.  Points to a folder where a CORE/SHADERS folder exists, allowing you to point the tool to the shaders being used by (e.g.) your game data.


If a CONFIG.TXT file is found in the same folder as the EXE, it can contain the following:
<code>COREPATH C:\development\mygame\</code>
 
==== Shader Values ====
You can set up values which will be passed to your shaders. These are to allow mimicking of shader operations which game code may be pushing.  They are 4 values which are passed into the shader in register 30 of both the VS and PS.  These are integer values.
SHADERVALUE<N> <X> <Y> <Z> <W>
e.g.
SHADERVALUE0 1 0 3 99
The N value matches the numeric key that you press to use the value.
 
==== Find Textures ====
For tools which do not have standard texture naming or placement you can set up automated searches for normal or specular textures.  Use the following format:
 
<code>FINDNORMAL <postfix>
FINDSPEC <postfix></code>
 
Where postfix would be something like _normal where textures are named (e.g.) main.dds, main_normal.dds, etc.


==== Texture Paths ====
==== Texture Paths ====
Multiple texture paths which are searched in order for any textures
Multiple texture paths which are searched in order for any textures


Line 33: Line 62:
  PATH <texturePath>
  PATH <texturePath>
  PATH ...</code>
  PATH ...</code>
NOTE: Texture sets are assumed to be in subfolders, with naming of TEXTURE1, TEXTURE2 etc.  Note that the tool will only look for subfolders numbered [1,30].


==== Lights ====
==== Lights ====
Multiple lights can be set up, each light must be a triplet of direction, colour, and ambient
Multiple lights can be set up, each light must be a triplet of direction, colour, and ambient


Line 78: Line 108:
These shaders are generally the ones that you will customise to your specific workflow and texture setups.
These shaders are generally the ones that you will customise to your specific workflow and texture setups.


==Shader input data==
== Custom Shaders ==
When writing shaders, the following registers can be assumed to be filled
Custom shaders can be used in two ways.  Either include a new shader in the base SHADERS folder and use the tool to apply it to the applicable meshes.  This however can be fiddly if you have combined shader types (e.g. skinned/non-skinned) and requires you to create versions in the 1 folder for shadowing.
 
A simpler approach is often to use the Style system.  To create a new style you add a numeric folder to the SHADERS folder (note that style 1 is always used for shadow generation).  So for example, if you wanted a specific type of normal-mapped skinned shader you would create a new folder, say 5, and copy in your default normal_skin.txt shader file.  You then alter the 5/ version as desired.  Then you can use the numeric buttons in the tool to apply a style to a given model and save it out.  Note that if a shader cannot be found in the style folder then the system will fall back to using the default version.
 
== Shader input data ==
When writing shaders, the following vertex shader registers can be assumed to be filled


*world matrix c0
*world matrix c0
Line 85: Line 120:
*projection matrix c8
*projection matrix c8
*WVP combo c12
*WVP combo c12
*light data (dir, colour, ambient)  c16
*light data (dir, colour, ambient, fill colour)  c16
*world Normal matrix c20
*world Normal matrix c20 [DEPRECATED: Use c0 and ignore/zero translation elements]
*shadowmap lighting trx combo c24
*shadowmap lighting trx combo c24
*camera position vector c28
*camera position vector c28
*camera look vector c29
*camera look vector c29
*C30 is sometimes used to store 4 pseudo random numbers constant for a given instance of a mesh (e.g. per unit, per object)
*C30 is sometimes used to store 4 pseudo random (integers as floats) numbers constant for a given instance of a mesh (e.g. per unit currently)
*c31 not used yet
*c31 fogging.  xyz are the RGB values, w is start + (65536*end).  Decoding example below.
*c32 assume all past here are bones
 
registers 16,28,29,30,31 are duplicated into the pixel shader.  As well as some others.
 
* text colour when rendering fonts c0
* shadow projection matrix c4
* shadow texture half pixel size c20
* modal data for pixel shaders c32 (implementation specific)
 
 
 
 
=== Fogging Decode Example ===
 
<nowiki>
 
float4 fog : register(c31) ;
 
...
 
float start, end ;
float a, dist;


*c32 assume all past here are bones
end = round(fog.w/65536.0) ;
start = fmod(fog.w, 65536.0) ;
distV = In.drawPos - gEyePos.xyz ;
 
dist = length(distV) ;
a = dist ;
a -= start ;
a = a/(end-start) ;
a = saturate(a) ;
outputColour.xyz += a * fog.xyz ;</nowiki>
 
 
== Post Processing ==
To apply post processing you need to create .txt files in DATA/GFX with the following format
ID <id>  // must be [1,255] zero is invalid
 
<nowiki>// first pass
[0]
SHADER <shader file>
TEX0 <texture file> // optional, allowed 4 textures
 
// optional additional passes [1], etc up to 8 passes allowed
</nowiki>
 
The ID is used to determine where the post processing happens.  It can be applied to a UI object on its root object using a POSTPROCESS <id> tag.
<b>NOTE</b>Currently all post processing happens to the entire screen, so be aware when applying to UI objects.
 
For the Tile flavour, there are special ID values which cause their Post Process to be applied.  They are
+ 99 - applies the post process globally after all UI has been rendered (does not affect the mouse)
+ 100 - applies the post process to the battle view
[Note: the lighting file can include a CLUT entry with a texture filename.  If this is included, then the named texture is pushed into the last texture on the last pass for the battle view post process shader, so available on sampler s4]
CLUT <texture name> // no extension needed
+ 101 - applies to the battle view in the same way as above, BUT applies before the UI icons are rendered.
 
Generally flavours will keep values above 99 for system and automated use.
 
=== Shader Details ===
Shaders must be in the DATA/GFX/SHADERS folder. Post Process shaders are applied across the entire screen, and so their shaders will look very similar to the screenrgba.txt shader vertex shader.  When the pixel shader is applied, you will find the previous screen render contents in the the first sampler, followed by the 4 custom textures for the post process pass, if applicable.
 
c0 contains the following information
x - screenWidth
y - screenHeight
z - the pass (from the post process file ordering)
w - unused
 
== Mesh Randomisation ==
To allow for the random display of sub-meshes, name the objects so they start with a ~ symbol, followed by an index [0,9] and then a hex character which is the mask (it allows a hex value, so it can be 0-9 or A-F, must be capital).  Each value can only be a single char.  So an example would be
 
~01somethingsomething
 
This would use the first random value, and show when it had value of 0.  The logic is actually
 
// OK, now finally see if we are visible!
if(((1<< (mRandom[index]&3))&mask)==0)
      return false ;      // not visible
 
So you can set up masks so that one object gets shown more, or indeed so that sometimes no object is shown, as well as having objects which show together (if they have the same masks on the same index).  Currently only the first 4 indexes are set with random values for units. 
 
As an example, if you name objects ~01 ~02 ~04 ~08 then it will only ever show one of the four variations when rendering.

Revision as of 16:22, 18 December 2017

Converter Tool

NOTE: The Converter replaces the MAX script plugin for exporting units. The converter can accept FBX files from most 3D tools.

Valid map slots to be used are Diffuse, Specular Colour, Glossiness, and Bump (which is the normal map). The exporter should correctly handle baked materials. The maps are available in the game shaders in the following order in samplers [0-3].

Diffuse
Normal
Specular
Gloss

Tool Controls

Hold LMB - rotate camera
Hold RMB - pan camera
Mousewheel - zoom (+SHIFT for faster)
RETURN - reset camera
L - cycle through lights
SPACE - toggle camera light
F1 - reload shaders
A - toggle animations
F8 - single step animation
P - reseed shader random values (shield variation, etc)
0-9 - set the shader values from the config file (see below)
LEFT/RIGHT - change texture set
SHIFT+Click on texture button to clear texture from it

Animations

If there is a suitably formatted animation file in the same folder as the mesh then the animations will be shown in a list. The chosen animation will be looped. If you hold SHIFT when clicking an animation it will then loop both that animation and the last animation chosen without SHIFT down, blending between them. This is for blend testing mainly and should generally not be used - once turned on it cannot be turned off without restarting the tool.

Batch Processing

The batching processing allows for multiple files to be loaded, optionally have the current textures applied, and then saved out. Both FBX and S4F files can be processed. Once processed the files are exported to an S4F with the same name (so if processing an S4F file, then original file will be over-written). By default the currently applied textures in all 4 slots are applied to the model once loaded. If you hold down SHIFT when clicking the Batch Processing button then this will not happen and the model will simply be imported and then exported as an S4F. If you hold down CONTROL when clicking on the Batch Processing button the model will be imported, optimised (as per the Optimise button at the bottom of the panel, removing unused nodes) and re-exported as an S4F.

CONFIG File

If a CONFIG.TXT file is found in the same folder as the EXE, it can contain the following:

Shader Path

Must be at the top of the file, optional. Points to a folder where a CORE/SHADERS folder exists, allowing you to point the tool to the shaders being used by (e.g.) your game data.

COREPATH C:\development\mygame\

Shader Values

You can set up values which will be passed to your shaders. These are to allow mimicking of shader operations which game code may be pushing. They are 4 values which are passed into the shader in register 30 of both the VS and PS. These are integer values.

SHADERVALUE<N> <X> <Y> <Z> <W>

e.g.

SHADERVALUE0 1 0 3 99

The N value matches the numeric key that you press to use the value.

Find Textures

For tools which do not have standard texture naming or placement you can set up automated searches for normal or specular textures. Use the following format:

FINDNORMAL <postfix>
FINDSPEC <postfix>

Where postfix would be something like _normal where textures are named (e.g.) main.dds, main_normal.dds, etc.

Texture Paths

Multiple texture paths which are searched in order for any textures

[TEXTURES]
PATH <texturePath>
PATH ...

NOTE: Texture sets are assumed to be in subfolders, with naming of TEXTURE1, TEXTURE2 etc. Note that the tool will only look for subfolders numbered [1,30].

Lights

Multiple lights can be set up, each light must be a triplet of direction, colour, and ambient

[LIGHTS]
DIRECTION <vector>
COLOUR <hex>
AMBIENT <hex>

DIRECTION ...

An example CONFIG file is

[TEXTURES]
PATH C:\DATA\BATTLE\UNITTEXTURES
PATH C:\DATA\TEXTURES

[LIGHTS]
DIRECTION 0 -1 0
COLOUR FFFF0000
AMBIENT FF003300

DIRECTION 1 0 0
COLOUR FFFFFF00
AMBIENT 0

Default Shaders

The system loads and uses some specific default shaders for specific types of rendering from CORE/SHADERS. These are:

screenrgba.txt
font.txt
grey.txt
screenrgba_solid.txt

The above shaders are used for standard UI object rendering and font drawing, and will generally not be changed by the user.

lit.txt - draw standard lit objects (lit via normals)
prelit.txt - draw prelit objects (vertex colour stored in vertex data)
normal.txt - draw objects with normal mapping (vertex data includes tangent space info)
lit_skin.txt - draw as per lit.txt but with hardware skinning
normal_skin.txt - draw as per normal.txt but with hardware skinning

These shaders are generally the ones that you will customise to your specific workflow and texture setups.

Custom Shaders

Custom shaders can be used in two ways. Either include a new shader in the base SHADERS folder and use the tool to apply it to the applicable meshes. This however can be fiddly if you have combined shader types (e.g. skinned/non-skinned) and requires you to create versions in the 1 folder for shadowing.

A simpler approach is often to use the Style system. To create a new style you add a numeric folder to the SHADERS folder (note that style 1 is always used for shadow generation). So for example, if you wanted a specific type of normal-mapped skinned shader you would create a new folder, say 5, and copy in your default normal_skin.txt shader file. You then alter the 5/ version as desired. Then you can use the numeric buttons in the tool to apply a style to a given model and save it out. Note that if a shader cannot be found in the style folder then the system will fall back to using the default version.

Shader input data

When writing shaders, the following vertex shader registers can be assumed to be filled

  • world matrix c0
  • view matrix c4
  • projection matrix c8
  • WVP combo c12
  • light data (dir, colour, ambient, fill colour) c16
  • world Normal matrix c20 [DEPRECATED: Use c0 and ignore/zero translation elements]
  • shadowmap lighting trx combo c24
  • camera position vector c28
  • camera look vector c29
  • C30 is sometimes used to store 4 pseudo random (integers as floats) numbers constant for a given instance of a mesh (e.g. per unit currently)
  • c31 fogging. xyz are the RGB values, w is start + (65536*end). Decoding example below.
  • c32 assume all past here are bones

registers 16,28,29,30,31 are duplicated into the pixel shader. As well as some others.

  • text colour when rendering fonts c0
  • shadow projection matrix c4
  • shadow texture half pixel size c20
  • modal data for pixel shaders c32 (implementation specific)



Fogging Decode Example

	

	float4 fog : register(c31) ;

	...

	float start, end ;
	float a, dist;

	end = round(fog.w/65536.0) ;
	start = fmod(fog.w, 65536.0) ;
	
	distV = In.drawPos - gEyePos.xyz ;

	dist = length(distV) ;
	a = dist ;
	a -= start ;
	a = a/(end-start) ;
	a = saturate(a) ;
	
	outputColour.xyz += a * fog.xyz ;


Post Processing

To apply post processing you need to create .txt files in DATA/GFX with the following format ID <id> // must be [1,255] zero is invalid

// first pass
[0]
SHADER <shader file>
TEX0 <texture file> // optional, allowed 4 textures

// optional additional passes [1], etc up to 8 passes allowed

The ID is used to determine where the post processing happens. It can be applied to a UI object on its root object using a POSTPROCESS <id> tag. NOTECurrently all post processing happens to the entire screen, so be aware when applying to UI objects.

For the Tile flavour, there are special ID values which cause their Post Process to be applied. They are

+ 99 - applies the post process globally after all UI has been rendered (does not affect the mouse)
+ 100 - applies the post process to the battle view
[Note: the lighting file can include a CLUT entry with a texture filename.  If this is included, then the named texture is pushed into the last texture on the last pass for the battle view post process shader, so available on sampler s4]
CLUT <texture name> // no extension needed
+ 101 - applies to the battle view in the same way as above, BUT applies before the UI icons are rendered.

Generally flavours will keep values above 99 for system and automated use.

Shader Details

Shaders must be in the DATA/GFX/SHADERS folder. Post Process shaders are applied across the entire screen, and so their shaders will look very similar to the screenrgba.txt shader vertex shader. When the pixel shader is applied, you will find the previous screen render contents in the the first sampler, followed by the 4 custom textures for the post process pass, if applicable.

c0 contains the following information

x - screenWidth
y - screenHeight
z - the pass (from the post process file ordering)
w - unused

Mesh Randomisation

To allow for the random display of sub-meshes, name the objects so they start with a ~ symbol, followed by an index [0,9] and then a hex character which is the mask (it allows a hex value, so it can be 0-9 or A-F, must be capital). Each value can only be a single char. So an example would be

~01somethingsomething

This would use the first random value, and show when it had value of 0. The logic is actually

// OK, now finally see if we are visible!
if(((1<< (mRandom[index]&3))&mask)==0)
      return false ;      // not visible

So you can set up masks so that one object gets shown more, or indeed so that sometimes no object is shown, as well as having objects which show together (if they have the same masks on the same index). Currently only the first 4 indexes are set with random values for units.

As an example, if you name objects ~01 ~02 ~04 ~08 then it will only ever show one of the four variations when rendering.