SANiK's notes, Issue 2 (Saturn Format)

ItsEasyActually · **Posted:** Fri Jul 09, 2010 6:30 pm

<font size=7><b>ISSUE 2</b></font>

(Updated with more comments for better understanding)

In hopes of getting more beginners interested in next gen hacking, I am going to be releasing my notes on the model formats and later on other formats. Do note, the point of these issues is to hopefully show that SEGA used the SAME format from game to game. The format first appeared in the game NiGHTs, then it got updated and was used in Sonic Adventure, to be updated and used again in the Dreamcast BIOS, then in SA2, then in PSO, etc. Due bear in mind that some notes may have errors and some are incomplete. Keep in mind if you find silly mistakes to tell me about it.

Issues that I will be releasing:

Issue 1: Sonic Adventure PC and DC model formats (version 2 of format) (already released)
Issue 2: Nights model format (version 1 of format) (you're reading it)
Issue 3: Sonic Adventure 2 model format (version 3 of format) (written by my friend =D)

Memory sizes that I use:

byte = 1 byte
word = 2 bytes
dword = 4 bytes
----------------
sbyte = 1 byte (signed)
sword = 2 bytes (signed)
sdword = 4 bytes (signed)
----------------
float = 4 bytes

Quote:

//Sega Saturn + Nights model macros
//Do note: The Sega Saturn has no FPU, meaning there is no "float" format. Also, bytes are ordered in Motorola format, which need to be flipped on the PC.
/*Anyway, since there is no FPU, decimal numbers are stored in this format: <16 bits>.<16 bits>
16 bits is 0xFFFF. Let's say we wanted to convert 3.1415 to the FIXED format
The 3 is stored in the upper 2 bytes, leaving us the fractional part.
The 0.1415 is multiplyed by 0xFFFF giving us 0x2439 (9273 decimal)
And this is how the final result would look like: 0x00032439
*/
/*
Also, Sega Saturn models are compiled into stand alone programs. So one needs to use "tricks" to actually find where the model is in the binary. Also, the models use pointers to point to where data is, but the pointers are based on RAM offsets when the file is loaded by the Sega Saturn. So, one will need to find the "KEY" (the difference between the file offset and RAM offset)
*/

//These 2 macros do the math that I described up for us.
//Usage: toFIXED(3.1415) would return 0x00032439, etc.

#define toFIXED(x) ((sdword)(65536.0 * (x))) //returns a signed long
#define toFLOAT(x) ((float)(((sdword)(x)) / 65536.0)) //returns a float

//Sega Saturn + Nights model structures
typedef struct VERTEX {
float x; //Yet again, Saturn "decimal" numbers need to be converted back to floats first *read start/top for more info*
float y;
float z;
} VERTEX;

//What's wierd of the poly structure in the Nights format is that to get the ACTUAL vertex reference, one must divide by 8 (or shift right 3)
//So, if a=24, that doesn't mean vertex #24, but you do, 24/8=3, so you use vertex #3 instead.
//Now that I think about it - the lower 3 bits might have been just bit flags
typedef struct POLY { //The Poly's are quads, 4 sided
VERTEX normal; //ONLY IN THE SDK MODELS!!! Nights models have NORMALs per VERTEX (XP STRUCTs)
word a;
word b;
word c;
word d;
} POLY;

//New stuff kids, added just for the year 2007 (MAY BE INCORRECT/INVALID)
typedef struct ATTRIBUTE {
byte doubleSided; //1 = true, 0 = false
byte flagsDrawHigh;
/*Bits 0 to 1: Zsorting
00=Sort closer than last drawn polygon,
01=Sort by closest point in polygon
10=Sort by farthest point in polygon
11=Sort by the average of the 4 points in the polygon

Bit 2: 1=Use texture, 0=No texture
Bit 3: 1=Use light, 0=No light
Bit 4: 1=Use depth, 0=No depth
Bit 5: 1=Use palette, 0=No palette
Bit 6: 1=Use near clipping, 0=No near clipping
Bit 7: 1=Use gourad, 0=No gourad
*/
word material; //0=No texture, all else = texture ID number
word flagsDecode;
/*Bits 0 to 3: Quad render options
000 = Draw (Normal)
001 = Draw with shadow
010 = Draw with half luminance
011= Draw with "transparency"
100 = Draw with gourad shading
Bits 4 to 7: Unknown/Not needed
*/
word color; //16 bit color format, (5551 (sign bit unused))
word gourad; //Bank address of gourad table (ignore)
word flagsDrawLow;
/*Bits 0 to 3: Quad surface type
001=Sprite
010=Texture
100=Polygon
101=PolyLine
110=Line

Bit 4: If Sprite type: HFlip
Bit 5: If Sprite type: VFlip
Bits 6 to 7: Unknown
*/
dword padding; //Unkown padding needed to make the struct 16 bytes
} ATTRIBUTE;

typedef struct ATTACH {
VERTEX *vertex; //Points to the vertex buffer
dword vertex_total;
POLY *poly; //Points to the polygon buffer
dword poly_total;
ATTRIBUTE *attribute; //Points to the attribute buffer (Not used, it's for polygon shading info, etc)
VERTEX *normal; //Only exists if it is an XP struct, read more about it later on
} ATTACH;

Quote:

Ripping models from files
Things to know:

The address of the vertex buffer plus the result of vertex_total times 12 equals the address of the poly buffer.
Why 12? Simple, like I said before, models are just compiled C arrays. 90% of the time, the POLY BUFFER struct follows right after the VERTEX BUFFER. We can use this to our advantage when scanning files to see if a valid ATTACH struct has been found. Each VERTEX struct takes 12 bytes, well 12 bytes times the number of verteces + the address of the VERTEX BUFFER (gotten from pointer VERTEX *vertex) should = the address of the POLY BUFFER (gotten from pointer POLY *poly). Don't forget, these pointers are RAM ADDRESSES and NOT FILE ADDRESSES. So a value must be subtracted to convert them to FILE RELATIVE ADDRESSES.
The Sega Saturn uses the Motorola byte ordering.
All pointers must have a "KEY" subtracted from them to find file relative address.
To add support to the SDK (Sega Saturn dev kit) based 3D model format, just use (the poly total times 20) instead of (poly_total times 8) and you're done. Why this difference? The SDK version adds NORMALs per POLYs (So no use for for the XPDATA struct)... and each NORMAL is 12 bytes so it ends up looking like this: POLY { Vector Normal; word index[4];}

To rip a 3D object:

We load a file and start scanning the file from offset 0
Remember, all this is guess work + pattern guessing to see IF and WHERE the model data might be within the file
(Yet again I want to stress this, SEGA compiled their models, so there's no header. Models must be brute-force read)
We save the current file offset to a variable.
We read an ATTACH struct from the file (an fread should do)
If poly_total or vertex_total is zero, then the model is a false alarm meaning: jump to the saved file position + 1, and reload a new ATTACH struct!
If we got this far, we apply a check to see if it IS really a model or just some bogus data. The check was already mentioned but here it is again:
The address of the vertex buffer + (the vertex total times 12) has to equal the address of the poly buffer. If not, it's a false alarm.
Now, there's one more check to do to really make sure we've found an ATTACH struct!
We assume since 90% of the time that a POLY BUFFER follows a VERTEX BUFFER. Well, we can also safely assume that 90% an ATTRIBUTE BUFFER (used for texture info + shading) follows a POLY BUFFER [color=red]UNLESS if the ATTACH struct is an UPGRADED ATTACH struct (XP struct) then the ATTRIBUTE BUFFER should follow the NORMAL BUFFER itself which should follow the POLY BUFFER. The most important assumption is that *drum roll*:
An ATTACH STRUCT follows an ATTRIBUTE BUFFER. So by using the RAM ADDRESS of the ATTRIBUTE BUFFER + the # of poly entries it has TIMES the size of each entry (16), we can get the position of the ATTACH STRUCT in RAM. Subtract the FILE OFFSET of the ATTACH STRUCT and we've got ourselves a KEY... which is a value we can subtract from every POINTER to get the file relative offsets they're pointing to, cool huh!? =o[/color]
Case 1 (If it's a regular struct, it has no NORMALs so no NORMAL buffer): If the address of the poly buffer + (the poly total times 8) equals the address of the ATTRIBUTE buffer, then we subtract ((address of attribute buffer + (poly_total times 16))-(value of the current offset in the file that we saved in step #1)) from EVERY POINTER to generate the KEY;

Case 2 (If it's an XP struct, it has NORMALs): If the address of the poly buffer + (the poly total times 8) equals the address of the NORMAL buffer, then we subtract ((address of attribute buffer + (poly_total times 16))-(value of the current offset in the file that we saved in step #1)) from EVERY POINTER to generate the KEY;

The above might look confusing, but it let's us 1) detect if the ATTACH struct is the EXPANDED VERSION with NORMALS or not, and 2) read the blue note under step #8 to understand the process
Now, to read the vertex data, we go to the address that the vertex pointer points to in the file. Be sure you have subtracted the value that we have gotten in the previous step (the KEY) from the pointer to get the file relative address!!
Then read a dword (4 bytes), flip the bytes (0x10203040 would become 0x40302010), and do for example X=toFLOAT(0x40302010); Keep doing this for X, Y, Z and use the vertex total to help you.
To read the poly data, we go to the address that the poly pointer points to in the file. Be sure you have subtracted the value from one of the previous steps!!
Then read a word (2 bytes), flip the bytes (0x1020 would become 0x2010), and then do A=absolute_value_of(0x2010) / 8. That will give you the correct vertex reference that this poly uses.
YOU ONLY DO THIS STEP IF YOU DETECTED AN XP STRUCT!!

Finally, to read the normal data, we go to the address that the normal pointer points to in the file. Be sure you have subtracted the value from one of the previous steps!!
Then read a dword (4 bytes), flip the bytes (0x10203040 would become 0x40302010), and do for example X=toFLOAT(0x40302010); Keep doing this for X, Y, Z and use the vertex total to help you. (Normals are per vertex)
Finally, you are done reading a body part. To go to the next object, goto the current offset in the file that you saved in step 1, and add 20 if it was a REGULAR STRUCT and 24 if it was an XP STRUCT. Finally, start reading at that location.
If you managed to get any false alarms, goto the current offset in the file that you saved in step 1 and add 2, then start reading at that location.

//New stuff yet again
typedef struct OBJECT{
ATTACH *attach;
float position[3]; //Yet again, Saturn "decimal" numbers need to be converted back to floats first *read start/top for more info*
word angle[3];
float scale[3];
struct OBJECT *child;
struct OBJECT *sibling;
} OBJECT;

Look familiar? It should! Like the SADX and SA2 formats, each ATTACH struct belongs to an OBJECT. Each OBJECT helps link up the models through the use of children/sibling + the position/angle/scale matrix data.
These structures are very hard to rip/scan for since most of the time, they're not even in the same file as the ATTACH struct!
Although, one trick is to make a list of RAM addresses (not the file relative offsets but the RAM offsets, with no KEY subtracted) of ATTACH structures and then search for those values.

Well that's it for Issue #2, good luck

SANiK's notes, Issue 2 (Saturn Format)

Who is online