Icecache File Format
This page contains the description of the file format used for caching in Softimage v7.0 and later, version 102. The file extension is "icecache".
The data is not linked to any geometry type in particular, the data can be read off a polymesh for example and loaded onto a point cloud. All data types are supported, as are single values vs. dynamic arrays.
Cache files are created on a per object and per frame basis. Each file is a zip archive, utilizing the ZLIB library to save and load directly using a zip-stream. You can create / read the files however, by zipping / unzipping the files as a second / first step. It is still recommend to create / read the files using the ZLIB library, located at http://zlib.net/.
The contents of the files are separated in three major blocks:
- file header
- attribute definitions
- data block
See the end of this document for the data type descriptions
Notes:
- The Pointlocator type is supported by the ICECACHE file format, but currently is not exposed. Pointlocator data is to be treated as a binary block of data, which can be skipped when reading ICECACHE files with external tools.
- You can set the XSI_CACHE_ASCII environment variable to save the cache sequence in ASCII. This helps you visualize the format and write your plug-in.
| Table of contents |
Cache File Contents
file header
| type | bytes | name | description |
|---|---|---|---|
| char | 8 | headerstring | the header string identifying the file ("ICECACHE") |
| ULONG | 4 | versionnumber | version number of the file format (currently 102) |
| ULONG | 4 | objecttype | the type of object (see siICENodeObjectType) |
| ULONG | 4 | pointcount | the number of points / particles |
| ULONG | 4 | edgecount | the number of edges (0 for a non-polygonmesh) |
| ULONG | 4 | polygoncount | the number of polygons (0 for a non-polygonmesh) |
| ULONG | 4 | samplecount | the number of UV samples (0 for a non-polygonmesh) |
| ULONG | 4 | userdatablobcount | the number of user data blob stored (introduced in version 102) |
| WCHAR | length (multiple of 4) | typename | The name of the blob ( userdatablobcount times ) (introduced in version 102)
(we store always a multiple of 4 characters for performance reasons) |
| WCHAR | length (multiple of 4) | displayname | The display name of the blob ( userdatablobcount times ) (introduced in version 102)
(we store always a multiple of 4 characters for performance reasons) |
| WCHAR | length (multiple of 4) | description | The description of the blob ( userdatablobcount times ) (introduced in version 102)
(we store always a multiple of 4 characters for performance reasons) |
| ULONG | 4 | red | red ( userdatablobcount times ) (introduced in version 102) |
| ULONG | 4 | green | green ( userdatablobcount times ) (introduced in version 102) |
| ULONG | 4 | blue | blue ( userdatablobcount times ) (introduced in version 102) |
| ULONG | 4 | attributecount | the number of attributes stored |
attribute description ( attributecount times )
In this section the attributes part of the ICECACHE file are defined. The order of the attributes has to be alphabetical (by attribute-name). This is due to a current limitation and will be fixed for future version of the file format.
| type | bytes | name | description |
|---|---|---|---|
| ULONG | 4 | namelength | number of character for the attribute name |
| char | namelength (multiple of 4) | name | the name of the attribute
(we store always a multiple of 4 characters for performance reasons) |
| DWORD | 4 | datatype | the base datatype of the attribute (see siICENodeDataType) |
| DWORD | 4 | structure | single value or dynamic array (see siICENodeStructureType) |
| DWORD | 4 | context | the context the attribute is stored in (see siICENodeContextType) |
| DWORD | 4 | objDBId | the Database ID of the object (obsolete) |
| DWORD | 4 | category | the category of the attribute ( see siICEAttributeCategory) |
If we are looking at an attribute of the datatype ICEDataBlob, the attribute definition have additional data (IDs count and typename) (introduced in version 102):
| type | bytes | name | description |
|---|---|---|---|
| ULONG | 4 | namelength | number of character for the attribute name |
| char | namelength (multiple of 4) | name | the name of the attribute
(we store always a multiple of 4 characters for performance reasons) |
| DWORD | 4 | datatype | the base datatype of the attribute (see siICENodeDataType) |
| DWORD | 4 | IDs count | The number of IDs (introduced in version 102) |
| WCHAR | length (multiple of 4) | typename | The type of the ID (IDs times ) (introduced in version 102)
(we store always a multiple of 4 characters for performance reasons) |
| DWORD | 4 | structure | single value or dynamic array (see siICENodeStructureType) |
| DWORD | 4 | context | the context the attribute is stored in (see siICENodeContextType) |
| DWORD | 4 | objDBId | the Database ID of the object (obsolete) |
| DWORD | 4 | category | the category of the attribute ( see siICEAttributeCategory) |
If we are looking at an attribute of the datatype siICENodeDataLocation (pointlocator), the additional data is also part of the attribute description and can be skipped when reading ICECACHE files with external tools:
| type | bytes | name | description |
|---|---|---|---|
| ULONG | 4 | sizeofblock | the size of the following binary block in bytes |
| BYTE | sizeofblock | binary data block | the binary data persisting the location descriptor information (not exposed) |
attribute data
The attribute data has to be in the same order as the attribute descriptions.
The attribute data is stored as an array of values. The data is stored in chunks of 4000 elements. For each element we store a flag to determine if the chunk is a constant value, or if each element can be of a different value. Plus each value can be a dynamic array of values, depending on the structure type of the attribute. (The only exceptional attribute is the PointPosition. As it is always non-constant, it doesn't store a per-chunk flag, but just one flag prior to the whole PointPosition data chunk.) This means in the case of 8300 non-constant float elements the file contains:
| type | bytes | name | description |
|---|---|---|---|
| DWORD | 4 | isConstant | false (0) in this case |
| datatype | siICENodeDataSizeFloat * 4000 | data | one single value for each element as a chunk of 4000 |
| DWORD | 4 | isConstant | false (0) in this case |
| datatype | siICENodeDataSizeFloat * 4000 | data | one single value for each element as a chunk of 4000 |
| DWORD | 4 | isConstant | false (0) in this case |
| datatype | siICENodeDataSizeFloat * 300 | data | one single value for each element as a chunk of 300 |
There are five major cases how a chunk can be stored in the file
- Constant Single Value
This means that the value is constant for all elements, and it is a single value so we only store one single data.
- Constant Array Value
This means that as the value is constant for all elements, but it is a dynamic array, we store one array.
- Varying Single value
The value is different for all elements, so we store as many values as there are elements.
- Varying Array value
The value is different for all elements, and each one is an array so we store alot of values.
- Pointlocator binary data block
We don't store per element values, but a binary block containing the location information.
This is the contents for each case:
- attribute data ( attributecount times ) single value constant
| type | bytes | name | description |
|---|---|---|---|
| DWORD | 4 | isConstant | true (1) in this case |
| datatype | siICENodeDataSize | data | one single value |
- attribute data ( attributecount times ) array constant
| type | bytes | name | description |
|---|---|---|---|
| DWORD | 4 | isConstant | true (1) in this case |
| ULONG | 4 | arraysize | the number of elements in the array |
| datatype | siICENodeDataSize * arraysize | data | the values in the array |
- attribute data ( attributecount times ) single value varying
(the elementcount is either pointcount, edgecount, polygoncount or samplecount)
| type | bytes | name | description |
|---|---|---|---|
| DWORD | 4 | isConstant | false (0) in this case |
| datatype | siICENodeDataSize * elementcount | data | one single value for each element |
- attribute data ( attributecount times ) array varying
(the elementcount is either pointcount, edgecount, polygoncount or samplecount)
| type | bytes | name | description |
|---|---|---|---|
| DWORD | 4 | isConstant | false (0) in this case |
and then for each array (elementcount times)
| type | bytes | name | description |
|---|---|---|---|
| ULONG | 4 | arraysize | the number of elements in the array |
| datatype | siICENodeDataSize * arraysize | data | the values in the array |
- Pointlocator binary data block
| type | bytes | name | description |
|---|---|---|---|
| ULONG | 4 | sizeofblock | the size of the following binary block in bytes |
| BYTE | sizeofblock | binary data block | the binary data persisting the location descriptor information (not exposed) |
Example File Description
This example contains two internal attributes of a point cloud (pointposition, color) and one user specified one (radius).
(Note that as we are only looking at 14 points, this file is not using chunks of 4000 elements.)
| type | bytes | name | value |
|---|---|---|---|
| char | 8 | headerstring | "ICECACHE" |
| ULONG | 4 | versionnumber | 100 |
| ULONG | 4 | objecttype | 0 |
| ULONG | 4 | pointcount | 14 |
| ULONG | 4 | edgecount | 0 |
| ULONG | 4 | polygoncount | 0 |
| ULONG | 4 | samplecount | 0 |
| ULONG | 4 | attributecount | 3 |
| ULONG | 4 | namelen | 13 |
| char | 16 | name | "pointposition___" |
| DWORD | 4 | datatype | siICENodeDataVector3 (1 << 4) |
| DWORD | 4 | structure | siICENodeStructureSingle (1 << 0) |
| DWORD | 4 | context | siICENodeContextComponent0D (1 << 1) |
| DWORD | 4 | objDBId | 0 |
| DWORD | 4 | category | siICEAttributeCategoryBuiltin (1) |
| ULONG | 4 | namelen | 5 |
| char | 8 | name | "color___" |
| DWORD | 4 | datatype | siICENodeDataSizeColor4 (1 << 9) |
| DWORD | 4 | structure | siICENodeStructureSingle (1 << 0) |
| DWORD | 4 | context | siICENodeContextComponent0D (1 << 1) |
| DWORD | 4 | objDBId | 0 |
| DWORD | 4 | category | siICEAttributeCategoryBuiltin (1) |
| ULONG | 4 | namelen | 6 |
| char | 8 | name | "radius__" |
| DWORD | 4 | datatype | siICENodeDataSizeFloat (1 << 2) |
| DWORD | 4 | structure | siICENodeStructureSingle (1 << 0) |
| DWORD | 4 | context | siICENodeContextComponent0D (1 << 1) |
| DWORD | 4 | objDBId | 0 |
| DWORD | 4 | category | siICEAttributeCategoryCustom (2) |
| DWORD | 4 | isConstant | false (0) |
| float | siICENodeDataSizeVector3 * 14 | data | 14 Vectors containing 3 floats (all pointpositions) |
| DWORD | 4 | isConstant | true (1) |
| float | siICENodeDataSizeColor4 | data | as the color is constant, we only store one color (4 floats) |
| DWORD | 4 | isConstant | false (0) |
| float | siICENodeDataSizeFloat * 14 | data | 14 floats containing all radii |
Data Type Definitions
See the XSI SDK for a description of these constants:
- siICENodeStructureType (http://softimage.wiki.softimage.com/sdkdocs/siICENodeStructureType.htm)
- siICENodeDataType (http://softimage.wiki.softimage.com/sdkdocs/siICENodeDataType.htm)
- siICENodeContextType (http://softimage.wiki.softimage.com/sdkdocs/siICENodeContextType.htm)
- siICEAttributeCategory (http://softimage.wiki.softimage.com/sdkdocs/siICEAttributeCategory.htm)
C++ type definitions
typedef unsigned long ULONG; typedef unsigned long LONG; typedef unsigned long DWORD;
siICENodeObjectType
| pointcloud | 0 |
| polygonmesh | 1 |
| nurbssurfacemesh | 2 |
| nurbscurvelist | 3 |
siICENodeDataSize
See siICENodeDataType (http://softimage.wiki.softimage.com/sdkdocs/siICENodeDataType.htm)
| siICENodeDataSizeBool | 4 | for siICENodeDataBool we just store one bool (encoded in a DWORD) |
| siICENodeDataSizeLong | 4 | for siICENodeDataLong we just store one LONG |
| siICENodeDataSizeFloat | 4 | for siICENodeDataFloat we just store one float |
| siICENodeDataSizeVector2 | 8 | for siICENodeDataVector2 we store 2 floats (x,y) |
| siICENodeDataSizeVector3 | 12 | for siICENodeDataVector3 we store 3 floats (x,y,z) |
| siICENodeDataSizeVector4 | 16 | for siICENodeDataVector3 we store 4 floats (x,y,z,w) |
| siICENodeDataSizeQuaternion | 16 | for siICENodeDataQuaternion we store 4 floats (w,x,y,z) |
| siICENodeDataSizeMatrix33 | 36 | for siICENodeDataMatrix33 we store 9 floats |
| siICENodeDataSizeMatrix44 | 64 | for siICENodeDataMatrix44 we store 16 floats |
| siICENodeDataSizeColor4 | 16 | for siICENodeDataColor4 we store 4 floats (r,g,b,a) |
| siICENodeDataSizeRotation3 | 16 | for siICENodeDataRotation we store 4 floats (w,x,y,z) (equivalent to quaternion) |
Example Cache Writer Python Script
LogMessage = Application.LogMessage
from struct import pack
import gzip
def WriteHeader( nbParticles, nbAttributes ):
data = pack( "8s", "ICECACHE" ) #header string
data += pack( "I", 102 ) #version number
data += pack( "I", 0 ) #object type, pointcloud
data += pack( "I", nbParticles ) #point count
data += pack( "I", 0 ) #edge count
data += pack( "I", 0 ) #polygon count
data += pack( "I", 0 ) #sample count
data += pack( "I", 0 ) #don't know this one
data += pack( "I", nbAttributes ) #attribute count
return data
def WriteAttrDefs():
#particle color
data = pack( "L", 5 ) #attribute name length
data += pack( "8s", "color___" ) #attribute name
data += pack( "I", 512 ) #data type
data += pack( "I", 1 ) #structure type
data += pack( "I", 2 ) #context type
data += pack( "I", 0 ) #database ID, obsolete
data += pack( "I", 2 ) #category
#particle position
data += pack( "L", 13 ) #attribute name length
data += pack( "16s", "pointposition___" ) #attribute name
data += pack( "I", 16 ) #data type
data += pack( "I", 1 ) #structure type
data += pack( "I", 2 ) #context type
data += pack( "I", 0 ) #database ID, obsolete
data += pack( "I", 1 ) #category
#particle size
data += pack( "L", 4 ) #attribute name length
data += pack( "4s", "size" ) #attribute name
data += pack( "I", 4 ) #data type
data += pack( "I", 1 ) #structure type
data += pack( "I", 2 ) #context type
data += pack( "I", 0 ) #database ID, obsolete
data += pack( "I", 2 ) #category
return data
def WriteAttrData(colorData, posData, sizeData):
data = pack( "I", 1 ) #is constant?
data += pack( "4f", 1,0,0,1 )
data += pack( "I", 1 ) #is constant?
data += pack( "3f", 0,1,0 )
data += pack( "I", 1 ) #is constant?
data += pack( "1f", 0.5 )
return data
def WriteIceCache( filePath ):
import os
fileName, ext = os.path.splitext( filePath )
icePath = fileName + ".icecache"
nbParticles = 1
nbAttributes = 3
data = WriteHeader( nbParticles, nbAttributes )
data += WriteAttrDefs()
data += WriteAttrData(0, 0, 0)
cachefile = gzip.open( icePath, "wb" )
#cachefile = open( icePath, "w" )
cachefile.write( data )
cachefile.close()
return True
filename1 = r"C:\temp\test_take3_0.1.icecache"
filename2 = r"C:\temp\test_take3_0.2.icecache"
WriteIceCache(filename1)
WriteIceCache(filename2)
