Icecache File Format

This page contains the description of the file format used for caching in Softimage v7.0 and later, version 103. The file extension is "icecache".

The data is not linked to any geometry type in particular, the data can be read off a polymesh for example and loaded onto a point cloud. All data types are supported, as are single values vs. dynamic arrays.

Cache files are created on a per object and per frame basis. Each file is a zip archive, utilizing the ZLIB library to save and load directly using a zip-stream. You can create / read the files however, by zipping / unzipping the files as a second / first step. It is still recommend to create / read the files using the ZLIB library, located at http://zlib.net/.

The contents of the files are separated in three major blocks:

  • file header
  • attribute definitions
  • data block

See the end of this document for the data type descriptions

Notes:

  • The Pointlocator type is supported by the ICECACHE file format, but currently is not exposed. Pointlocator data is to be treated as a binary block of data, which can be skipped when reading ICECACHE files with external tools.
  • You can set the XSI_CACHE_ASCII environment variable to save the cache sequence in ASCII. This helps you visualize the format and write your plug-in.

Table of contents

Cache File Contents

file header

type bytes name description
char 8 headerstring the header string identifying the file ("ICECACHE")
ULONG 4 versionnumber version number of the file format (currently 103)
ULONG 4 objecttype the type of object (see siICENodeObjectType)
ULONG 4 pointcount the number of points / particles
ULONG 4 edgecount the number of edges (0 for a non-polygonmesh)
ULONG 4 polygoncount the number of polygons (0 for a non-polygonmesh)
ULONG 4 samplecount the number of UV samples (0 for a non-polygonmesh)
ULONG 4 substep the number of sub step for the simulation (introduced in version 103)
ULONG 4 userdatablobcount the number of user data blob stored (introduced in version 102)
WCHAR length (multiple of 4) typename The name of the blob ( userdatablobcount times ) (introduced in version 102)

(we store always a multiple of 4 characters for performance reasons)

WCHAR length (multiple of 4) displayname The display name of the blob ( userdatablobcount times ) (introduced in version 102)

(we store always a multiple of 4 characters for performance reasons)

WCHAR length (multiple of 4) description The description of the blob ( userdatablobcount times ) (introduced in version 102)

(we store always a multiple of 4 characters for performance reasons)

ULONG 4 red red ( userdatablobcount times ) (introduced in version 102)
ULONG 4 green green ( userdatablobcount times ) (introduced in version 102)
ULONG 4 blue blue ( userdatablobcount times ) (introduced in version 102)
ULONG 4 attributecount the number of attributes stored



attribute description ( attributecount times )

In this section the attributes part of the ICECACHE file are defined. The order of the attributes has to be alphabetical (by attribute-name). This is due to a current limitation and will be fixed for future version of the file format.


type bytes name description
ULONG 4 namelength number of character for the attribute name
char namelength (multiple of 4) name the name of the attribute

(we store always a multiple of 4 characters for performance reasons)

DWORD 4 datatype the base datatype of the attribute (see siICENodeDataType)
DWORD 4 structure single value or dynamic array (see siICENodeStructureType)
DWORD 4 context the context the attribute is stored in (see siICENodeContextType)
DWORD 4 objDBId the Database ID of the object (obsolete)
DWORD 4 category the category of the attribute ( see siICEAttributeCategory)


If we are looking at an attribute of the datatype ICEDataBlob, the attribute definition have additional data (IDs count and typename) (introduced in version 102):


type bytes name description
ULONG 4 namelength number of character for the attribute name
char namelength (multiple of 4) name the name of the attribute

(we store always a multiple of 4 characters for performance reasons)

DWORD 4 datatype the base datatype of the attribute (see siICENodeDataType)
DWORD 4 IDs count The number of IDs (introduced in version 102)
WCHAR length (multiple of 4) typename The type of the ID (IDs times ) (introduced in version 102)

(we store always a multiple of 4 characters for performance reasons)

DWORD 4 structure single value or dynamic array (see siICENodeStructureType)
DWORD 4 context the context the attribute is stored in (see siICENodeContextType)
DWORD 4 objDBId the Database ID of the object (obsolete)
DWORD 4 category the category of the attribute ( see siICEAttributeCategory)


If we are looking at an attribute of the datatype siICENodeDataLocation (pointlocator), the additional data is also part of the attribute description and can be skipped when reading ICECACHE files with external tools:

type bytes name description
ULONG 4 sizeofblock the size of the following binary block in bytes
BYTE sizeofblock binary data block the binary data persisting the location descriptor information (not exposed)

attribute data

The attribute data has to be in the same order as the attribute descriptions.

The attribute data is stored as an array of values. The data is stored in chunks of 4000 elements. For each element we store a flag to determine if the chunk is a constant value, or if each element can be of a different value. Plus each value can be a dynamic array of values, depending on the structure type of the attribute. (The only exceptional attribute is the PointPosition. As it is always non-constant, it doesn't store a per-chunk flag, but just one flag prior to the whole PointPosition data chunk.) This means in the case of 8300 non-constant float elements the file contains:

type bytes name description
DWORD 4 isConstant false (0) in this case
datatype siICENodeDataSizeFloat * 4000 data one single value for each element as a chunk of 4000
DWORD 4 isConstant false (0) in this case
datatype siICENodeDataSizeFloat * 4000 data one single value for each element as a chunk of 4000
DWORD 4 isConstant false (0) in this case
datatype siICENodeDataSizeFloat * 300 data one single value for each element as a chunk of 300


There are five major cases how a chunk can be stored in the file

  • Constant Single Value

This means that the value is constant for all elements, and it is a single value so we only store one single data.

  • Constant Array Value

This means that as the value is constant for all elements, but it is a dynamic array, we store one array.

  • Varying Single value

The value is different for all elements, so we store as many values as there are elements.

  • Varying Array value

The value is different for all elements, and each one is an array so we store alot of values.

  • Pointlocator binary data block

We don't store per element values, but a binary block containing the location information.


This is the contents for each case:

  • attribute data ( attributecount times ) single value constant
type bytes name description
DWORD 4 isConstant true (1) in this case
datatype siICENodeDataSize data one single value

  • attribute data ( attributecount times ) array constant
type bytes name description
DWORD 4 isConstant true (1) in this case
ULONG 4 arraysize the number of elements in the array
datatype siICENodeDataSize * arraysize data the values in the array

  • attribute data ( attributecount times ) single value varying

(the elementcount is either pointcount, edgecount, polygoncount or samplecount)

type bytes name description
DWORD 4 isConstant false (0) in this case
datatype siICENodeDataSize * elementcount data one single value for each element

  • attribute data ( attributecount times ) array varying

(the elementcount is either pointcount, edgecount, polygoncount or samplecount)

type bytes name description
DWORD 4 isConstant false (0) in this case

and then for each array (elementcount times)

type bytes name description
ULONG 4 arraysize the number of elements in the array
datatype siICENodeDataSize * arraysize data the values in the array



  • Pointlocator binary data block
type bytes name description
ULONG 4 sizeofblock the size of the following binary block in bytes
BYTE sizeofblock binary data block the binary data persisting the location descriptor information (not exposed)



Example File Description

This example contains two internal attributes of a point cloud (pointposition, color) and one user specified one (radius).

(Note that as we are only looking at 14 points, this file is not using chunks of 4000 elements.)


type bytes name value
char 8 headerstring "ICECACHE"
ULONG 4 versionnumber 100
ULONG 4 objecttype 0
ULONG 4 pointcount 14
ULONG 4 edgecount 0
ULONG 4 polygoncount 0
ULONG 4 samplecount 0
ULONG 4 attributecount 3
ULONG 4 namelen 13
char 16 name "pointposition___"
DWORD 4 datatype siICENodeDataVector3 (1 << 4)
DWORD 4 structure siICENodeStructureSingle (1 << 0)
DWORD 4 context siICENodeContextComponent0D (1 << 1)
DWORD 4 objDBId 0
DWORD 4 category siICEAttributeCategoryBuiltin (1)
ULONG 4 namelen 5
char 8 name "color___"
DWORD 4 datatype siICENodeDataSizeColor4 (1 << 9)
DWORD 4 structure siICENodeStructureSingle (1 << 0)
DWORD 4 context siICENodeContextComponent0D (1 << 1)
DWORD 4 objDBId 0
DWORD 4 category siICEAttributeCategoryBuiltin (1)
ULONG 4 namelen 6
char 8 name "radius__"
DWORD 4 datatype siICENodeDataSizeFloat (1 << 2)
DWORD 4 structure siICENodeStructureSingle (1 << 0)
DWORD 4 context siICENodeContextComponent0D (1 << 1)
DWORD 4 objDBId 0
DWORD 4 category siICEAttributeCategoryCustom (2)
DWORD 4 isConstant false (0)
float siICENodeDataSizeVector3 * 14 data 14 Vectors containing 3 floats (all pointpositions)
DWORD 4 isConstant true (1)
float siICENodeDataSizeColor4 data as the color is constant, we only store one color (4 floats)
DWORD 4 isConstant false (0)
float siICENodeDataSizeFloat * 14 data 14 floats containing all radii


Data Type Definitions

See the XSI SDK for a description of these constants:


C++ type definitions

typedef unsigned long ULONG;
typedef unsigned long LONG;
typedef unsigned long DWORD;


siICENodeObjectType

pointcloud 0
polygonmesh 1
nurbssurfacemesh 2
nurbscurvelist 3


siICENodeDataSize

See siICENodeDataType (http://softimage.wiki.softimage.com/sdkdocs/siICENodeDataType.htm)


siICENodeDataSizeBool 4 for siICENodeDataBool we just store one bool (encoded in a DWORD)
siICENodeDataSizeLong 4 for siICENodeDataLong we just store one LONG
siICENodeDataSizeFloat 4 for siICENodeDataFloat we just store one float
siICENodeDataSizeVector2 8 for siICENodeDataVector2 we store 2 floats (x,y)
siICENodeDataSizeVector3 12 for siICENodeDataVector3 we store 3 floats (x,y,z)
siICENodeDataSizeVector4 16 for siICENodeDataVector3 we store 4 floats (x,y,z,w)
siICENodeDataSizeQuaternion 16 for siICENodeDataQuaternion we store 4 floats (w,x,y,z)
siICENodeDataSizeMatrix33 36 for siICENodeDataMatrix33 we store 9 floats
siICENodeDataSizeMatrix44 64 for siICENodeDataMatrix44 we store 16 floats
siICENodeDataSizeColor4 16 for siICENodeDataColor4 we store 4 floats (r,g,b,a)
siICENodeDataSizeRotation3 16 for siICENodeDataRotation we store 4 floats (w,x,y,z) (equivalent to quaternion)


Example Cache Writer Python Script

LogMessage = Application.LogMessage
from struct import pack
import gzip

def WriteHeader( nbParticles, nbAttributes ):
     data  = pack( "8s", "ICECACHE" )    #header string
     data += pack( "I", 102 )     #version number
     data += pack( "I", 0 )     #object type, pointcloud
     data += pack( "I", nbParticles )     #point count
     data += pack( "I", 0 )    #edge count
     data += pack( "I", 0 )     #polygon count
     data += pack( "I", 0 )     #sample count
     data += pack( "I", 0 )     #don't know this one
     data += pack( "I", nbAttributes )     #attribute count
     return data
    
def WriteAttrDefs():
     
     #particle color
     data  = pack( "L", 5 )    #attribute name length
     data += pack( "8s", "color___" )    #attribute name
     data += pack( "I", 512 )    #data type
     data += pack( "I", 1 )    #structure type
     data += pack( "I", 2 )    #context type
     data += pack( "I", 0 )    #database ID, obsolete
     data += pack( "I", 2 )    #category
          
     #particle position
     data += pack( "L", 13 )    #attribute name length
     data += pack( "16s", "pointposition___" )    #attribute name
     data += pack( "I", 16 )    #data type
     data += pack( "I", 1 )    #structure type
     data += pack( "I", 2 )    #context type
     data += pack( "I", 0 )    #database ID, obsolete
     data += pack( "I", 1 )    #category

     #particle size
     data += pack( "L", 4 )    #attribute name length
     data += pack( "4s", "size" )    #attribute name
     data += pack( "I", 4 )    #data type
     data += pack( "I", 1 )    #structure type
     data += pack( "I", 2 )    #context type
     data += pack( "I", 0 )    #database ID, obsolete
     data += pack( "I", 2 )    #category
     
     return data
    
def WriteAttrData(colorData, posData, sizeData):

    data  = pack( "I", 1 )    #is constant?
    data += pack( "4f", 1,0,0,1 )

    data += pack( "I", 1 )    #is constant?
    data += pack( "3f", 0,1,0 )
    
    data += pack( "I", 1 )    #is constant?
    data += pack( "1f", 0.5 )

    return data

def WriteIceCache( filePath ):
    import os 
    
    fileName, ext = os.path.splitext( filePath )
    icePath = fileName + ".icecache"

    nbParticles = 1
    nbAttributes = 3
    
    data = WriteHeader( nbParticles, nbAttributes )
    data += WriteAttrDefs()
    data += WriteAttrData(0, 0, 0)

    cachefile = gzip.open( icePath, "wb" )
    #cachefile = open( icePath, "w" )
    cachefile.write( data )
    cachefile.close()
   
    return True

filename1 = r"C:\temp\test_take3_0.1.icecache"
filename2 = r"C:\temp\test_take3_0.2.icecache"
WriteIceCache(filename1)
WriteIceCache(filename2)

Example

icecache.rar (http://softimage.wiki.softimage.com/support/icecache.rar)

This page was last modified 14:39, 2 May 2011.
This page has been accessed 21778 times.

© Copyright 2009 Autodesk Inc. All Rights Reserved. Privacy Policy | Legal Notices and Trademarks | Report Piracy