File Formats, The Miscellaneous Formats (De-L, MAD, ID, MXF, Config, KeyCFG)

[Karybdis 34]

This thread deals with specifications relating to the miscellaneous file formats found in the game. These formats currently include : Delocalizers, MAD video files, ID radio station lists, MXF fonts, KEYCFG's and Configuration files.

-----------------

Delocalizers are needless to say really wierd. They appear to all be the same file as far as I can tell and I really cant figure out what their purpose is. You can find them in simcitylocale, and in plugins.

Here's as much of a spec as I can write for them

----

-HEADER

DWORD - Size of file minus this Dword

16 Bytes - "Don't localize. " In plain Text

-ID CHUNKS (128 bytes total)

Starting Chunk with an "x" at the beginning

(126 bytes)

Ending Chunk with an "x" at the end

----

The ID chunks appear to all be 8 byte Text IDs with an x on the first and last one. The significance of this I really can't figure though.

------------------

ID files are found in the mp3 directory in the installed SC4 dir just before you go into the region or mayor dirs. There's two of them. One for Mayor and one for the region. Unfortunately they're always blank for me since I dont use custom MP3's so I cant say anything for a specification

------------------

MXF fonts are the main SC4 ingame fonts that you see onscreen. They're stored in the fonts directory and accessed by an INI file in simcity_1.dat

I dont have a spec for them currently but they seem to use some type of format similar to True Type Fonts. Anyone who knows how these work might know more about this format than I do

------------------

Configuration files, or rather file, are found solely in the Simcity 4.cfg file in the my documents simcity directory. They seem to contain data relative to the ingame regions though I cant be sure about that. They are a completely binary format with the exception of the few strings describing region names. I currently have no spec for them

------------------

MAD files are the files which contain the Intro videos for simcity 4. They are dual with both video and audio. The audio involved is the EA sound format EACS.

The video segments appear to be designated by MAD identifier tags.

Audio is listenable to using the GAP Player made by a russian audio decoder.

I have no spec for this format.

Here however is Valery V. Anisimovsky's EACS spec. At least I think its his EACS spec

----

=======================

1. ASF/AS4 Music Files

=======================

The music in many Electronic Arts games is in .ASF/.AS4 stand-alone files.

These files have the block structure analoguous to RIFF. Namely, these files

are divided into blocks (without any global file header like RIFFs have).

Each block has the following header:

struct ASFBlockHeader

{

char szBlockID[4];

DWORD dwSize;

};

szBlockID -- string ID for the block.

dwSize -- size of the block (in bytes) INCLUDING this header.

Further I'll describe the contents of blocks of all block types in ASF/AS4

files. When I say "block begins with..." that means "the contents of that

block (which begin just after ASFBlockHeader) begin with...".

Quoted strings are block IDs.

"1SNh": header block. This is the first block in ASF/AS4.

This block begins with the structure describing the audio stream:

struct EACSHeader

{

char szID[4];

DWORD dwSampleRate;

BYTE bBits;

BYTE bChannels;

BYTE bCompression;

BYTE bType;

DWORD dwNumSamples;

DWORD dwLoopStart;

DWORD dwLoopLength;

DWORD dwDataStart;

DWORD dwUnknown;

};

szID -- ID string, always "EACS".

dwSampleRate -- sample rate for the file.

bBits -- if multiplied by 8 gives the resolution of (decompressed) sound

data, that is 1 means 8-bit and 2 means 16-bit.

bChannels -- channels number: 1 for mono, 2 for stereo.

bCompression -- if 0x00, the data in the file is not compressed: signed 8-bit

PCM or signed 16-bit PCM. If this byte is 0x02, the audio data is compressed

with IMA ADPCM. Note that non-compressed 8-bit files use SIGNED format!

Signed 16-bit data may be sent to the wave output without any additional

conversions, while signed 8-bit data should be converted to unsigned format.

For example you can do that so: unsigned8Bit=signed8Bit+0x80

or, just the same: unsigned8Bit=signed8Bit^0x80 (this's a bit faster).

bType -- type of file: always 0x00 for ASF/AS4 (multi-block) files.

dwNumSamples -- number of samples in the file. May be used for song length

(in seconds) calculation.

dwLoopStart -- beginning of the repeat loop (in samples). 0xFFFFFFFF means

no loop.

dwLoopLength -- length of the repeat loop (in samples). Zero for no loop.

dwDataStart -- in ASF/AS4 files this is not used (equal to 0).

After the EACSHeader the first chunk of sound data comes. If the data isn't

compressed, it's just signed 8/16-bit PCM. If the data is compressed, it

starts with a small chunk header:

struct ASFChunkHeader

{

DWORD dwOutSize;

LONG lIndexLeft;

LONG lIndexRight;

LONG lCurSampleLeft;

LONG lCurSampleRight;

};

dwOutSize -- size of uncompressed audio data in this chunk (in samples).

lIndexLeft, lIndexRight, lCurSampleLeft, lCurSampleRight are initial

values for IMA ADPCM decompression routine for this chunk (for left and right

channels respectively). I'll describe the usage of these further when I get to

IMA ADPCM decompression scheme.

Note that the structure above is ONLY for stereo files. For mono there're

just no lIndexRight and lCurSampleRight fields.

After this chunk header the compressed data comes. You may find IMA ADPCM

decompression scheme description further in this document.

Hereafter by "chunk" I mean the audio data in the "1SNd" data block, that is,

compressed data which starts after ASFChunkHeader.

"1SNd": data block. If no compression is used these blocks contain just

signed 8/16-bit PCM audio data. Otherwise the data in each of these blocks

begins with the same ASFChunkHeader described above and after that comes

compressed data.

Note that the first chunk of audio data is in "1SNh" block, along with the

global EACS header!

"1SNl": loop block. This block defines looping point for the song. It

contains only DWORD value, which is the looping jump position (in samples)

relative to the start of the song. Note that you should make the jump NOT

when you encounter this block but when you come across the "1SNe" block

which may appear some "1SNd" data blocks after this block!

"1SNe": end block. This block indicate the end of audio stream. Make looping

jump when you encounter it. It contains no data and its size is 8 bytes that

is the size of ASFBlockHeader.

Interesting that some AS4 files contain audio data beyond this block. This

should be considered as non-standard feature not worth to support.

----

I presume that these blocks of audio are intermixed with the video blocks, hence the repeating MAD (video) segments.

------------------

KeYCFG's are the files that link the specific buttons on the keyboard to the game. Technically their specification should be fairly simple, however a problem resides in that we dont know what the data and descriptions areas of the definition do. Spec follows

----

; Format of these keys are:

; < KeyCombination> = < MsgID> [[[< data1>] < data2>] < data3>] [< Description>]

;

; The KeyCombination can be any of the following:

; control -- Keyboard modifiers. Must be combined with a key.

; shift -- Keyboard modifiers. Must be combined with a key.

; alt -- Keyboard modifiers. Must be combined with a key.

; up or down -- Action happens on key up or key down. Down is default if not present. Must be combined with a key.

; 0xXX -- A literal hexadicmal key number 0x4B is the same as K.

; NNN -- A literal decimal key number 75 is the same as K.

; A-Z -- Keyboard letter.

; 0-9 -- Keyboard (not numpad) number.

; F1-F12 -- F keys.

; Numpad0-Numpad9 -- Numpad number.

; Numpad/ -- Numpad keys.

; Numpad* -- Numpad keys.

; Numpad- -- Numpad keys.

; Numpad+ -- Numpad keys.

; `~-_=+[{]}\|;:',< .>/? -- Keyboard keys.

; Escape -- Keyboard keys.

; Backspace

; Enter or Return

; ScrollLock

; NumLock

; CapsLock

; Tab

; Pause

; PageUp

; PageDown

; Space

; Home

; End

; Left

; Up

; Right

; Down

; PrintScreen

; Insert

; Delete

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

F1 = 0x6a935c34 "God Mode"

F2 = 0x6a935ca4 "Mayor Mode"

F3 = 0x6a935ca8 "My Sim Mode"

F4 = 0x6a935cab "Options Panel"

Control Shift R = 0x6a935dc9 "Build Bus Stop"

----

The Key combinations are presented above.

The MsgID is the 4 byte ID seen above. I'm not sure if this is a link to an instance ID or not. More research is needed.

Data and Description purposes are unfortunately unknown. There are ingame descriptions for each key combo in the Locale.dat file so they arent the ingame descriptions.

Help if you can hehe. I'm stumped for the moment.

--------------------

All of these files are not really specific to anything as of yet but they're still interesting formats which are worth decoding to maximize our ability to customize the game in any way we want

[DarkMatter 21]

I know a little bit about video encoding. If I can find the time, maybe I can help decode the MAD video. Anyway, here is a bit of info on modern methods of video encoding.

 

Videos are a sequence of images, called frames. Each frame is usually compressed, in multiple ways. To start, images are compressed using some derivative of JPEG compression, which takes 16x16 blocks of pixels and does quantization, RLE encoding, then some form of LZ compression on them. Quantization is a method of counting how many pixels of a certain small color range exist in the 16x16 block. Those colors are then averaged out, and replaced with a single color that uses the averaged value. RLE encoding, or run length encoding, just counts up how many pixels of the same color exist one after the other. The LZ compression is like the QFS compression used in SC4 .dat files.

 

Since videos are a sequence of frames, space can be saved even more by determining how much information in a given frame is the same as the information in the frame before it. If a 16x16 compressed block is similar to the same block in the previous frame, then the block from the previous frame is used, rather than storing a new block into the file. Just a pointer to the previous frames block is stored. This is called a P frame, or Prior Data frame. Some frames can also check the previous and following frame for similar blocks. This is called a B frame, or Both Data frame, since it checks both the previous and following frames for similar blocks, and uses the blocks from both where possible. The third type of frame is called an I frame, or Independant Data frame. Independant frames do not reference any blocks from previous or following frames, and are just JPEG compressed images. Moidern MPEG video formats like MPEG2 and MPEG4 use this method, and have highly accurate heuristical algorithms that determine where to read a video block from. Depending on the value of the heuristic, you'll have more or less P and B frames. The greater the compression, the more P and B frames you'll get, and the lower the quality since image information is repeated more frequently. The lower the compression, the fewer P and B frames you'll get, and the higher the quality since image information is repeated less frequently.

 

I figure that Maxis would probably use some sort of MPEG compression. I'm guessing this because MAD, or maybe Mpeg & Audio Data, seems a plausible definition of the name. I have a lot of stuff on my plate right now, so I don't know if I can decode the MAD video format, but maybe this info will help someone else decode them.

[Karybdis 34]

A couple of things that I have noticed.

MXF fonts are referred to along with photoshop documents in the Fonts.ini file in simcity_1.dat. Perhaps its possible that these are a modification of some photoshop type?

De-L I have an update for

DWORD - Size of file minus this DWORD
14 Bytes - "Don't Localize" in plain text
BYTE - Unknown with value 46
BYTE - Unknown with value that is divisable into the main ID string. This could be some kind of property length?

-ID CHUNKS (128 bytes total)
Starting Chunk with an "x" at the beginning
(126 bytes)
Ending Chunk with an "x" at the end

As for exactly what these files do, I'm still researching it. I'll try and find more info

[Karybdis 34]

MAD Data update.

Structure

MADk - ( Identifier with the type of data for this section) (Mad K and Mad M are known types)
DWORD - Length of this MAD chunk including the MADk header
SCH1 - (Identifier for this section) (SC H1, C1 and D1 are known values for this segment header)
DWORD - Length for this SCH1 section
MADm - (Identifier for this MAD section) (Mad K and Mad M are known types)

This structure repeates until the end of the MAD file. I'm assuming that these are intermixed parts of the Sound and Video data

EDIT: It appears that SC(Val)1 Blocks are Sound Data of the EACS type.

[DarkMatter 21]

HEhehe, Karybdis, I think I just cracked the MAD format. Check this out:

 

==================================================
4. .WVE/.DCT/.MAD/.TGQ/.UV/.UV2 Movie Soundtracks
==================================================

.WVE/.DCT/.MAD/.TGQ/.UV/.UV2 movies have the block structure analoguous to
that of .ASF. Video-related data is in "pIQT", "mTCD", "MADk", "MADm", "MADe",
"pQGT", etc. blocks and sound-related data is just in the same blocks as in
.ASF: "SCHl", "SCCl", "SCDl", "SCLl", "SCEl".
So, to play .WVE/.DCT/.MAD/.TGQ/.UV/.UV2 movie soundtrack, just walk blocks
chain, skip video blocks and process sound blocks. Note that in some games
video files (as well as audio files) are contained within game resources. As
a rule, such resources are not compressed/encrypted, so you may just search
for ASF file signature (e.g. "SCHl") and this will mark the beginning of audio
stream, while "SCEl" block marks the end of that stream.

 

 

And then I found this:

 

=============================================================================
       Electronic Arts Audio File Formats Description     5-01-2002
=============================================================================

By Valery V. Anisimovsky (samael@avn.mccme.ru)

In this document I'll try to describe audio file formats used in many (older)
Electronic Arts games. Described are formats for music, sound effects, speech
and movie soundtracks.

The games using these formats include: NBA Live'96, NHL'96, FIFA'96, The Need
For Speed, NHL'97. Maybe many more, e.g.: NHL'95.

The files this document deals with have extensions: .ASF, .AS4, .KSF, .EAS,
.SPH, .BNK, .CRD, .TGV. Note that the files described here may have other
extensions (and the same structure!): Electronic Arts tends to change
extensions from game to game.

Throughout this document I use C-like notation.

All numbers in all structures described in this document are stored in files
using little-endian (Intel) byte order.

=======================
1. ASF/AS4 Music Files
=======================

The music in many Electronic Arts games is in .ASF/.AS4 stand-alone files.
These files have the block structure analoguous to RIFF. Namely, these files
are divided into blocks (without any global file header like RIFFs have).
Each block has the following header:

struct ASFBlockHeader
{
  char szBlockID[4];
  DWORD dwSize;
};

szBlockID -- string ID for the block.

dwSize -- size of the block (in bytes) INCLUDING this header.

Further I'll describe the contents of blocks of all block types in ASF/AS4
files. When I say "block begins with..." that means "the contents of that
block (which begin just after ASFBlockHeader) begin with...".
Quoted strings are block IDs.

"1SNh": header block. This is the first block in ASF/AS4.
This block begins with the structure describing the audio stream:

struct EACSHeader
{
  char szID[4];
  DWORD dwSampleRate;
  BYTE bBits;
  BYTE bChannels;
  BYTE bCompression;
  BYTE bType;
  DWORD dwNumSamples;
  DWORD dwLoopStart;
  DWORD dwLoopLength;
  DWORD dwDataStart;
  DWORD dwUnknown;
};

szID -- ID string, always "EACS".

dwSampleRate -- sample rate for the file.

bBits -- if multiplied by 8 gives the resolution of (decompressed) sound
data, that is 1 means 8-bit and 2 means 16-bit.

bChannels -- channels number: 1 for mono, 2 for stereo.

bCompression -- if 0x00, the data in the file is not compressed: signed 8-bit
PCM or signed 16-bit PCM. If this byte is 0x02, the audio data is compressed
with IMA ADPCM. Note that non-compressed 8-bit files use SIGNED format!
Signed 16-bit data may be sent to the wave output without any additional
conversions, while signed 8-bit data should be converted to unsigned format.
For example you can do that so: unsigned8Bit=signed8Bit+0x80
or, just the same: unsigned8Bit=signed8Bit^0x80 (this's a bit faster).

bType -- type of file: always 0x00 for ASF/AS4 (multi-block) files.

dwNumSamples -- number of samples in the file. May be used for song length
(in seconds) calculation.

dwLoopStart -- beginning of the repeat loop (in samples). 0xFFFFFFFF means
no loop.

dwLoopLength -- length of the repeat loop (in samples). Zero for no loop.

dwDataStart -- in ASF/AS4 files this is not used (equal to 0).

After the EACSHeader the first chunk of sound data comes. If the data isn't
compressed, it's just signed 8/16-bit PCM. If the data is compressed, it
starts with a small chunk header:

struct ASFChunkHeader
{
  DWORD dwOutSize;
  LONG lIndexLeft;
  LONG lIndexRight;
  LONG lCurSampleLeft;
  LONG lCurSampleRight;
};

dwOutSize -- size of uncompressed audio data in this chunk (in samples).

lIndexLeft, lIndexRight, lCurSampleLeft, lCurSampleRight are initial
values for IMA ADPCM decompression routine for this chunk (for left and right
channels respectively). I'll describe the usage of these further when I get to
IMA ADPCM decompression scheme.

Note that the structure above is ONLY for stereo files. For mono there're
just no lIndexRight and lCurSampleRight fields.

After this chunk header the compressed data comes. You may find IMA ADPCM
decompression scheme description further in this document.

Hereafter by "chunk" I mean the audio data in the "1SNd" data block, that is,
compressed data which starts after ASFChunkHeader.

"1SNd": data block. If no compression is used these blocks contain just
signed 8/16-bit PCM audio data. Otherwise the data in each of these blocks
begins with the same ASFChunkHeader described above and after that comes
compressed data.
Note that the first chunk of audio data is in "1SNh" block, along with the
global EACS header!

"1SNl": loop block. This block defines looping point for the song. It
contains only DWORD value, which is the looping jump position (in samples)
relative to the start of the song. Note that you should make the jump NOT
when you encounter this block but when you come across the "1SNe" block
which may appear some "1SNd" data blocks after this block!

"1SNe": end block. This block indicate the end of audio stream. Make looping
jump when you encounter it. It contains no data and its size is 8 bytes that
is the size of ASFBlockHeader.
Interesting that some AS4 files contain audio data beyond this block. This
should be considered as non-standard feature not worth to support.

===================
2. KSF Music Files
===================

Some EA games use other format for music/speech files: .KSF.
These files begin with "KWK`" ID string. Following this ID, comes PATl header.
It begins with "PATl" ID string and its size is 56 bytes (always?) including
its ID string. After PATl header comes TMpl header:

struct TMplHeader
{
  char szID[4];
  BYTE bUnknown1;
  BYTE bBits;
  BYTE bChannels;
  BYTE bCompression;
  WORD wUnknown2;
  WORD wSampleRate;
  DWORD dwNumSamples; // ???
  BYTE bUnknown3[20];
};

szID -- string ID, always "TMpl".

bBits -- resolution of sound data (0x10 for 16-bit, 0x8 for 8-bit).

bChannels -- channels number: 1 for mono, 2 for stereo.

bCompression -- if 0x00, the data in the file is not compressed: signed 8-bit
PCM or signed 16-bit PCM. If this byte is 0x02, the audio data is compressed
with IMA ADPCM. See the note for EACS header above.

wSampleRate -- sample rate for the file.

dwNumSamples -- number of samples in the file. May be used for song length
(in seconds) calculation. Should be divided by 2 for mono sound.

After TMpl header comes sound data. For compressed files, IMA ADPCM compression
is used (see below).

=====================================
3. IMA ADPCM Decompression Algorithm
=====================================

During the decompression four LONG variables must be maintained for stereo
stream: lIndexLeft, lIndexRight, lCurSampleLeft, lCurSampleRight and two --
for mono stream: lIndex, lCurSample. At the beginning of each "1SNd" data
block and at the beginning of the file -- when processing "1SNh" block --
you must initialize these variables using the values in ASFChunkHeader.
Note that LONG here is signed.

Here's the code which decompresses one byte of IMA ADPCM compressed
stereo stream. Other bytes are processed in the same way.

BYTE Input; // current byte of compressed data
BYTE Code;
LONG Delta;

Code=HINIBBLE(Input); // get HIGHER 4-bit nibble

Delta=StepTable[lIndexLeft]>>3;
if (Code & 4)
   Delta+=StepTable[lIndexLeft];
if (Code & 2)
   Delta+=StepTable[lIndexLeft]>>1;
if (Code & 1)
   Delta+=StepTable[lIndexLeft]>>2;
if (Code & // sign bit
   lCurSampleLeft-=Delta;
else
   lCurSampleLeft+=Delta;

// clip sample
if (lCurSampleLeft>32767)
   lCurSampleLeft=32767;
else if (lCurSampleLeft<-32768)
   lCurSampleLeft=-32768;

lIndexLeft+=IndexAdjust

; // adjust index
// clip index
if (lIndexLeft<0)
   lIndexLeft=0;
else if (lIndexLeft>88)
   lIndexLeft=88;
Code=LONIBBLE(Input); // get LOWER 4-bit nibble
Delta=StepTable[lIndexRight]>>3;
if (Code & 4)
   Delta+=StepTable[lIndexRight];
if (Code & 2)
   Delta+=StepTable[lIndexRight]>>1;
if (Code & 1)
   Delta+=StepTable[lIndexRight]>>2;
if (Code &  // sign bit
   lCurSampleRight-=Delta;
else
   lCurSampleRight+=Delta;
// clip sample
if (lCurSampleRight>32767)
   lCurSampleRight=32767;
else if (lCurSampleRight<-32768)
   lCurSampleRight=-32768;
lIndexRight+=IndexAdjust[Code]; // adjust index
// clip index
if (lIndexRight<0)
   lIndexRight=0;
else if (lIndexRight>88)
   lIndexRight=88;
// Now we've got lCurSampleLeft and lCurSampleRight which form one stereo
// sample and all is set for the next input byte...
Output((SHORT)lCurSampleLeft,(SHORT)lCurSampleRight); // send the sample to output
HINIBBLE and LONIBBLE are higher and lower 4-bit nibbles:
#define HINIBBLE(byte) ((byte) >> 4)
#define LONIBBLE(byte) ((byte) & 0x0F)
Note that depending on your compiler you may need to use additional nibble
separation in these defines, e.g. (((byte) >> 4) & 0x0F).
StepTable and IndexAdjust are the tables given in the next section of
this document.
Output() is just a placeholder for any action you would like to perform for
decompressed sample value.
Of course, this decompression routine may be greatly optimized.
As to mono sound, it's just analoguous:
Code=HINIBBLE(Input); // get HIGHER 4-bit nibble
Delta=StepTable[lIndex]>>3;
if (Code & 4)
   Delta+=StepTable[lIndex];
if (Code & 2)
   Delta+=StepTable[lIndex]>>1;
if (Code & 1)
   Delta+=StepTable[lIndex]>>2;
if (Code &  // sign bit
   lCurSample-=Delta;
else
   lCurSample+=Delta;
// clip sample
if (lCurSample>32767)
   lCurSample=32767;
else if (lCurSample<-32768)
   lCurSample=-32768;
lIndex+=IndexAdjust[Code]; // adjust index
// clip index
if (lIndex<0)
   lIndex=0;
else if (lIndex>88)
   lIndex=88;
Output((SHORT)lCurSample); // send the sample to output
Code=LONIBBLE(Input); // get LOWER 4-bit nibble
// ...just the same as above for lower nibble
Note that HIGHER nibble is processed first for mono sound and corresponds to
LEFT channel for stereo.
====================
4. IMA ADPCM Tables
====================
LONG IndexAdjust[]=
{
    -1,
    -1,
    -1,
    -1,
     2,
     4,
     6,
     8,
    -1,
    -1,
    -1,
    -1,
     2,
     4,
     6,
     8
};
LONG StepTable[]=
{
    7,    8,   9,  10, 11,    12,     13,    14,    16,
    17,    19,   21,  23, 25,    28,     31,    34,    37,
    41,    45,   50,  55, 60,    66,     73,    80,    88,
    97,    107,   118,  130, 143,   157,    173,   190,   209,
    230,   253,   279,  307, 337,   371,    408,   449,   494,
    544,   598,   658,  724, 796,   876,    963,   1060,  1166,
    1282,  1411,  1552,  1707, 1878,  2066,   2272,  2499,  2749,
    3024,  3327,  3660,  4026, 4428,  4871,   5358,  5894,  6484,
    7132,  7845,  8630,  9493, 10442, 11487,  12635, 13899, 15289,
    16818, 18500, 20350, 22385, 24623, 27086,  29794, 32767
};
=========================
5. TGV Movie Soundtracks
=========================
.TGV movies have the block structure analoguous to that of ASF/AS4.
Video-related data is in "kVGT" and "fVGT" (or "TGVk" and "TGVf") blocks and
sound-related data is just in the same blocks as in ASF/AS4: "1SNh", "1SNd",
"1SNl", "1SNe".
So, to play TGV movie soundtrack, just walk blocks chain, skip video blocks
and process sound blocks.
==================================
6. Sound/Speech Files: .EAS, .SPH
==================================
Some sounds and all speech are usually in .EAS and .SPH files.
These files have the header which is just the same as EACSHeader structure
described above with two additions:
(bType) is always 0xFF for sound/speech files,
(dwDataStart) is the starting position of audio data relative to the beginning
of the file.
After the header, starting at (dwDataStart), comes audio data, up to the end of
the file. The data is either non-compressed or IMA ADPCM compressed depending
on the (bCompression) byte in the header. If it's IMA ADPCM compressed, there're
no initial values for samples and indices at the beginning of the audio data.
Just initialize them all to zeroes and start decompression at (dwDataStart).
====================================
7. Sound Effects in .BNK/.CRD Files
====================================
Most of sound effects are stored in .BNK and .CRD resource files. Those .BNKs
and .CRDs may contain several sounds. They begin with some seemingly
meaningless data, but after some junk of that data (typically starting at
position 0x228, but not necessarily) come several EACS headers describing
all sounds in .BNK/.CRD. Each EACS header has almost the same format as
described above with some minor changes (some fields have different placement):
struct EACSHeader
{
  char szID[4];
  DWORD dwSampleRate;
  BYTE bBits;
  BYTE bChannels;
  BYTE bCompression;
  BYTE bType;
  DWORD dwLoopStart;
  DWORD dwLoopLength;
  DWORD dwNumSamples;
  DWORD dwDataStart;
  DWORD dwUnknown;
};
and with the same two additions just as for .EAS/.SPH speech/sound:
(bType) is always 0xFF,
(dwDataStart) is the starting position of sound data relative to the beginning
of the .BNK/.CRD file containing that sound.
So, what you need to do is just search in .BNK/.CRD for "EACS" ID string and
read EACSHeader from the position where you found "EACS". And the same for
all sounds contained within .BNK/.CRD.
The sound data itself (for each EACS header describing it) starts at
(dwDataStart) and its size may be computed using (dwNumSamples) EACSHeader
field (for example) with the following formula:
Size=dwNumSamples*SampleSize/CompressionRatio,
where:
CompressionRatio=1 for non-compressed sounds,
   2 for 8-bit IMA ADPCM compressed sounds,
   4 for 16-bit IMA ADPCM compressed sounds,
SampleSize=bChannels*bBits (1 for mono 8-bit, 2 for mono 16-bit, etc.).
So, starting at (dwDataStart) comes just either PCM audio data (as described
above for .EAS/.SPH files) or IMA ADPCM compressed data (without initial
sample/index values, just as in .EAS/.SPH). Set CurSample(Left/Right) and
Index(Left/Right) to zeroes and start the decompression.
===========
8. Credits
===========
Vladan Bato (bat22@geocities.com)
http://www.geocities.com/SiliconValley/8682
The IMA ADPCM decompression scheme above is based on that described
in his AUD3.TXT document.
Peter Pawlowski (peterpw666@hotmail.com, piotrpw@polbox.com)
http://members.fortunecity.com/pp666/
http://pp666.cjb.net/
http://www.geocities.com/pp_666/
Pointed out corrections in IMA ADPCM decoder.
Toni Wilen (nhlinfo@nhl-online.com)
http://www.nhl-online.com/nhlinfo/
Provided me with info on KSF files, PATl and TMpl headers. Toni Wilen is the
author of SNDVIEW utility (available on his pages) which decompresses
Electronic Arts audio files and compresses WAVs back into EA formats.
Denis AUROUX (MXK) (auroux@clipper.ens.fr)
http://www.eleves.ens.fr:8080/home/auroux/nfsspecs.txt
Additional info on EACS headers. The author of The Unofficial NFS File Format
Specifications.
-------------------------------------------
Valery V. Anisimovsky (samael@avn.mccme.ru)
http://bim.km.ru/gap/
http://www.anxsoft.newmail.ru
http://anx.da.ru
On these sites you can find my GAP program which can search for audio files
in .BNK/.CRD resources, and play back .ASF/.AS4 songs, .BNK/.CRD sounds,
.EAS/.SPH speech and soundtracks of .TGV movies.
There's also complete source code of GAP and all its plug-ins there,
including ASF plug-in, which could be used for further details on how you
can deal with these formats.

[Karybdis 34]

Yep. That should be the sound format there

I posted a bit of the same spec at the very top of this thread. Valerie's tool for sound listening from MAD files was based on the ASF algorithyms so using this should allow you to play the sound blocks

The problem is actually processing the MAD video blocks correctly. I've done researching into AVI and MPEG formats and it didnt seem to fit the specifications for any of them. Then again I know very little about AVI so this is possibly just me reading the hex wrong.

The audio streams can indeed be extracted using that though. Tis why I thought that MAD was EACS audio originally before I found out it was the opening movies.

[DarkMatter 21]

No...LOL....thats what I just posted. MAD video blocks ARE ASF blocks. The whole format just use Microsofts ASF block format, it just changes the header for each block a bit. The video isn't AVI or MPEG, its ASF.

[Karybdis 34]

whooooooops. Thought that was relevant to the audio only haha

oh well, guess that means a MAD viewer is incoming then soon? XD

[DarkMatter 21]

I don't know, actually. I havn't been able to find the format of the MAD? headers. It might be really simple, like MADk or MADm, then the length of the block. It seems that way. But I can't tell for sure. I'm updating the hex editor in DatGen to perform search and highlight on patterns. This will highlight certain keys in the raw data in a color you specify, letting you see where blocks begin and end and such. I think it will help decipher the format.

Once we know the headers, Ilive could probably easily add video playing capability to the reader. Microsoft has a full API for the ASF format that comes with windows medial There are a ton of IWMAWhatever interfaces that let you load, play, and save ASF files. I'm sure he could use those to play MAD video blocks. For me, though, I'll have to write something from scratch. One of the downfalls of .NET is its a baby compared to the windows API. Only a fraction of the stuff supported in the windows API is supported in .NET, so I have to write video viewers and audio players from scratch at the lowest level. It will be a while before DatGen plays audio or video files.

[DarkMatter 21]

I think I've figured out the size of MAD headers. I wrote a structure highlighting feature for my hex editor, and the highlight below marks the different parts of a MAD header as I have deciphered it. I don't yet know what each part does, as the 3rd through 6th DWORDS always seem to be the same throughout a file.

 

 

I've also found a PowerPoint Spreadsheet that has lots of info on video encoding. It talks about MADk in it, surprisingly. Here is a link: http://yusoz.sdsu.edu/compe565/VideoCodingIntro.ppt

[Karybdis 34]

I doubt its actually this MAD. that seems more like a terminology for something else

As far as the DwORDS go, the dword after madk is the size of the audio/video block

The other 16 bytes should be the type of audio/video data for the segment, like number of channels, frequency etc. it should follow a standard ASF format though possibly in a different order if they're holding true to the previous data we found.

[DarkMatter 21]

That PPT file describes the theory behind the algorithms used for encoding ASF MPEG. The minmad term (I forget exactly what it means, but it has to do with encoding), is what gives ride to the MAD word beeing used in the header. I figure the slides will help us decipher the format properly. The MAD headers are actually quite different from ASF headers. I downloaded Microsofts ASF development kit, and the structure of an ASF is very different, although the raw video blocks themselves store the same data. We are on our own with deciphering the values in the header (aside from size, which was obvious). I can't even be sure that the header 6 DWORDS long or not...I just sorta guessed so far. I tried sending the MAD? blocks directly through microsofts ASF interfaces, and they either exceptioned or crashed the system. Its going to be fairly difficult figuring these out with the information we have, which is largely only related to audio (I don't know why the guy who wrote them ignored the video....he even said when decoding, "walk blocks chain and skip video, read audio blocks". He just passed right over all the video data and focused only on audio data.) I've spend the last couple days trying to find info on MAD? headers, but to no avail.

[Karybdis 34]

This is just a guess, but maybe they're using standard MPEG headers or slightly modified MPEG headers. MPEG allows for Just video and has a simple and small form to it.

As an example here's a copy of a simple MPEG header structure from Wotsit.org
Byte# Data Details
===================================================================
1-4 Sequence header In Hex 000001B3
code
12 bits Horizontal size In pixels
12 bits Vertical size In pixels
4 bits Pel aspect ratio See below
18 bits Picture rate See below
1 bit Marker bit Always 1
10 bits VBV buffer size Minimum buffer needed to decode this
sequence of pictures; in 16KB units
1 bit Constrained
parameter flag
1 bit Load intra 0: false; 1: true (matrix follows)
quantizer matrix
64 bytes Intra quantizer Optional
matrix
1 bit Load nonintra 0: false; 1: true (matrix follows)
quantizer matrix
64 bytes Nonintra quantizer Optional
matrix
- Squence extension Optional
Data
- User data Optional application-dependent data
===================================================================

I'm thinking the first 4 bytes of 00's are the sequence header. Therefore I think in the next couple of bytes after that we should start looking for the width and height. And then we can work with that to figure out the number of frames etc.

[DarkMatter 21]

Sounds like a good starting point to me. EA seems famous for reducing information. They probably have default values for most of those things, except width/height, and frame count.

Interestingly, that marker bit value you noted. The second byte of the blue DWORD is ALWAYS 0x01. I wonder if thats the marker bit, or something else.

Any idea of the significance of the k, m, and e values in the MAD identifier? There are MADk, MADm, and MADe identifiers, I'm curious what the different ones mean.

[Karybdis 34]

I dont think that's a marker bit but you never know. seems more like a flag to me given the circumstances since its the very end of the header.

Your testings were correct in the header for the video being 20 bytes each.

How does 768x528 sound for an aspect ratio. That's the value for each of the two bytes before the 00 01 00 00

[DarkMatter 21]

Yeah, my first guess was that that last DWORD was a set of flags. Kind of curious what, though...its always the same.

 

As for 768x528, thats a perfect size. Interestingly enough, its the exact size of NTSC coded video frames. Every NTSC (non-HDTV) TV has 528 lines of resolution, 768 pixels wide.

[Karybdis 34]

Well then its

DWORD - MAD? - Identifier of Block where ? is a unique block type identifier (k,m or e)
DWORD - Size of the block
DWORD - 00
4 Bytes - Unknown
WORD - Pixel width of the block
WORD - Pixel height of the block
4 Bytes - Unknown

That's all I got so far. When I get home from work I'll look some more.

Remaining bytes should be any of the following:

Sequence header
Pel aspect ratio
Picture rate
Marker bit Always 1
VBV buffer size Minimum buffer needed to decode this sequence of pictures; in 16KB units
Constrained parameter flag

I'm going to be looking at the block types identifier when I get home. I'd suggest having a look at the 4th dword if you get the chance. Looks like it may be broken up into various small chunks

[Karybdis 34]

KeyCFG hotkey format complete. To quote maxis here's the spec with descriptions

= [[[] ] ] []

example:

Control M = 0xec3e82f8 0x8bd88057 "Build Passenger Ferry Terminal"

KeyCombination is any combination of the keys that you want to activate this function

MsgID is the coded ID in the games exe that activates this command. Kinda wierd that they would call this a message ID, but oh well. Perhaps it calls an ingame message code.

data1,2,3 are specific data IDs (which are usually exemplars) that are used in conjunction with this action. The above example has the data1 as the exemplar for the ferries.

Description is a simple description of the function of the key combination

----------

These can be used in pretty much any way you want. Adding a new key combo is reliant on knowing the MsgID for it as far as I know.

[DarkMatter 21]

Ah! Thanks Karybdis. I needed that tidbit of information about data1/2/3. Now I can finish my hotkey editor.

[Karybdis 34]

Not sure if this is the correct format or not but here's a link to a document on the "MXF Material Exchange" format. I was hoping this is the same as our MXF fonts

http://www.ist-nuggets.tv/mxf/docs/parked%20until%20september/S380M-mxf-DMS1-20030331.trk.pdf

[ilive 36]

I have seen the same inforlation about MXF but I doubt it's the good format? Material eXchange Fromat seems to be oriented for media streaming. I downloaded one example of this mfx file and the format was not same thant our font (no MXF string found for example in the beginning of the file)

[Karybdis 34]

ack. well thanks for confirming that at least ilive

Maybe its just a modified TrueType Font or something along that line. Oh well. Back to the drawing board

[DarkMatter 21]

I have been pattern comparing MFX with TTF, and so far I havn't found many similarities. TTF fonts seem to have a much larger header area, with a fair amount of human-readable strings. Once again, in classic EA fasion, they seem to have reduced the information down to a small header and raw data. I've tried using numerous font format specifications I've found on the web, but so far, no luck.

One thing I had thought of was zipping up compatible TTF fonts that match the names of the MXF fonts. That way, for UI viewers, we could at least have the right look to fonts, even if we don't use MXF's.

[Teirusu 38]

As has been mentioned before, the Fonts.ini files contains references to Photoshop... and at the same time, the Font Fusion engine. I looked up what Font Fusion supports and CFF (compact font format)/Type 2 caught my eye.

Surprise surprise, a quick lookup on Google dug up a few links back to Adobe.

CFF (compact font format)/Type 2

Could this be it?

[Tropod 144]

Just some info for the SimCity 4.cfg file.

This is for the one Unknown file in SC4.cfg (TGI Mem Address: A9DD6E06 49DD6E08 00000001).

 

The following is done according to the Hex address of the file (that is, using the Hex Editor [decoded file]. Beats the heck out of trying to count how many bits n bytes these are down).

 

Hex Address: Followed by what it controls & some info (if any).

 

00000EFE to 00000F05 (Inclusive) Region Coordinates. It seems that this can can be further broken down, with vertical, horizontal (or north/south, east/west) & maybe at least one or two other axis.

 

Graphics Settings;

Low (1) < High (3)

00000F0F Visual Effects

00000F10 # of Cars/Sims

00000F11 Shadows

00000F12 Texture Quality

00000F18 City Detail

00000F2A Building Redraw Speed

 

Off (0), or On (1)

00000F13 Clouds/Fog

00000F14 Waves

00000F16 Color Cursor

00000F17 UI Translucency

00000F1A Variable Speed Automata

 

Color Quality

00000F26 16-bit (0x10) or 32-bit (0x20)

Renderer:

00000F15 Software (0), Hardware (1)

00000F0E User settings (0) or Default Settings (1)

00000F1E to 00000F23 Game Screen Resolution Off (0), On (1)

000004EE Auto Edge Reconciliation

00000F06 Show City Boundaries

00000F07 Show City Names

00000F08 Satelitte View (0), Transportation Map (1)

00000F0A Auto Go to Disaster

00000F0B Edge Scrolling

00000F0C Expanded Tooltips

00000F0D Urgent Advice Dialogs

 

 

& this is the same data/info, but (re)arranged according to their hex address (hopefully this'll be of some help perhaps):

 

000004EE Auto Edge Reconciliation

00000EFE to 00000F05 (Inclusive) Region Coordinates.

00000F06 Show City Boundaries

00000F07 Show City Names

00000F08 Satelitte View (0), Transportation Map (1)

00000F0A Auto Go to Disaster

00000F0B Edge Scrolling

00000F0C Expanded Tooltips

00000F0D Urgent Advice Dialogs

00000F0E User settings (0) or Default Settings (1)

00000F0F Visual Effects

00000F10 # of Cars/Sims

00000F11 Shadows

00000F12 Texture Quality

00000F13 Clouds/Fog

00000F14 Waves

00000F15 Software (0), Hardware (1)

00000F16 Color Cursor

00000F17 UI Translucency

00000F18 City Detail

00000F1A Variable Speed Automata

00000F1E to 00000F23 Game Screen Resolution

00000F26 16-bit (0x10) or 32-bit (0x20)

00000F2A Building Redraw Speed

 

 

Now (hex address) 00000F1E to 00000F23 seems to have something to do with the Screen Resolution of the game. But at this time, I'm a bit uncertain for this particular item really or whether it actually includes the whole (hex address) range here. EDIT: It is-see above.

 

Anything (that's in between or otherwise) that's not included, is because I'm not aware of its function [or it's been covered already].

[Karybdis 34]

O_o....uhh.... wow.... that is...well amazing work

I'll try and get to writing a proper spec up for it as soon as I can.

thanks a million

[Karybdis 34]

Here's the copy of the Byte Spec I just made for it. Thanks again man
-----------------------

This is the Simcity4 Config format. There is only one file for this. It is a file contained in Simcity4.cfg in the Mydocuments sc4 dir. It controls the features of the system. One system built on another. (Keymaker Humour)

Anyways it has a simple byte by byte format for the most part. Here's what we know. (Much thanks to Tropod for all this great info)
--------------

1262 Bytes - Unknown
BYTE - Auto Edge Reconciliation?
2575 Bytes - Unknown
QWORD - Current Region Coordinates
BYTE - Show City Boundries?
BYTE - Show City Names?
BYTE - Current Region Map View
1 Byte - Unknown
BYTE - Auto Go to Disaster?
BYTE - Edge Scrolling?
BYTE - Expanded Tooltips?
BYTE - Urgent Advice Dialogs
BYTE - Visual Setting Type (0 - User, 1 - Default)
BYTE - Visual Effect Quality
BYTE - Number of Cars/Sims
BYTE - Shadow Quality
BYTE - Texture Quality
BYTE - Clouds/Fog?
BYTE - Waves?
BYTE - Render Mode (Software - 0, Hardware - 1)
BYTE - Colour Cursor?
BYTE - UI Transparency?
BYTE - City Detail
1 Byte - Unknown
BYTE - Variable Speed Automata?
3 Bytes - Unknown
DWORD - Game Resolution - Horizontal
WORD - Game Resolution - Vertical
2 Bytes - Unknown
BYTE - Game Texture Bit Type (16-bit (0x10) or 32-bit (0x20))
3 Bytes - Unknown
BYTE - Building Redraw Speed
2059 Bytes - Unknown

[Guinnessmonkey 0]

Date: 11/2/2003 2:48:51 AM
Author: Karybdis
KeyCFG hotkey format complete. To quote maxis here's the spec with descriptions


= [[[] ] ] []


example:


Control M = 0xec3e82f8 0x8bd88057 'Build Passenger Ferry Terminal'


KeyCombination is any combination of the keys that you want to activate this function


MsgID is the coded ID in the games exe that activates this command. Kinda wierd that they would call this a message ID, but oh well. Perhaps it calls an ingame message code.


data1,2,3 are specific data IDs (which are usually exemplars) that are used in conjunction with this action. The above example has the data1 as the exemplar for the ferries.


Description is a simple description of the function of the key combination


----------


These can be used in pretty much any way you want. Adding a new key combo is reliant on knowing the MsgID for it as far as I know.

quote>
Strange that my searching didn't find this page before...

Anyway, that info is very useful. Thing is, does anyone know a good way to find out what the various MSGID's are? I'm tryin' to find a way to create a hotkey for labels and signs (would make labeling roads less of a chore)... I can't find anyone posting a way to reference the label tool...

[jlemen 0]

I know this is an old topic, but it is getting close to what I'm looking for....

How can the custom music be controled in the game??

Like with unused keys mapping.

Any help would be appreciated...