Ghost Data (File Format)

From MK8
Revision as of 14:48, 18 June 2022 by Tock (talk | contribs) (→‎Filename)
Jump to navigation Jump to search
Under Construction
This article is not finished. Help improve it by adding accurate information or correcting grammar and spelling.
Platform Notice
This article is about Mario Kart 8 for Wii U. Mario Kart 8 Deluxe uses a similar file format, but that is still a WIP and the offsets are different.

In Mario Kart 8, Ghost Data is stored in DAT-files. This includes player ghosts, staff ghosts, and downloaded ghosts. Although MKTV replays are also some sort of replay and also stored in DAT-files, their file format is entirely different. Each Ghost File is stored in a separate DAT-file.

Player Ghosts, Downloaded Ghosts, and MKTV Replays are stored in the user's save folder. Staff Ghosts are stored among the game files.

Differences with Mario Kart Wii

In Mario Kart Wii, as well as earlier titles, ghost files most notably store controller inputs. When a ghost is viewed, the game replays these controller inputs. If the game is perfectly deterministic, the same controller inputs should always yield the same finishing time. In practice however, Mario Kart Wii is not quite able to recreate the circumstances in which the ghost was recorded leading to the rare Wiggler Glitch. This causes the ghost to descynchronise from the original Time Trial, often failing to finish.

In Mario Kart 8, this is different. Ghost files must also contain kart positions in some form, possibly in addition to controller inputs (unknown). In practise, this means that a ghost is always able to finish a race in the exact same way as it was recorded in. This can be verified by replacing the file contents of a ghost file with that of another. When doing so, the ghost will ignore any collisions of the track (e.g. drive through walls) and follow the layout of the course it was initially recorded in.

As a result, this means that the Wiggler Glitch cannot occur in Mario Kart 8.

Filename

Some ghost data information is encoded into the filename of the ghost file. The first two characters indicate what type of ghost data is inside the file. The remaining characters should be interpreted as hexadecimal values.

Ghost Data Filename
Offset (#characters) Size (#characters) Description
0 2 Ghost Type. "sg" for Staff Ghost, "gs" for Player Ghost (possibly a shorthand for "ghost save"), "dg" for Downloaded Ghost, "rp" for MKTV Replay.
2 2 Ghost Number. For staff ghosts and player ghosts this is simply the track ID minus 16.

For downloaded ghosts it is a value from 0x00-0x04. This is likely because the game only lets you download 4 ghosts per track. This also allows the game to differentiate between ghosts which have otherwise identical data in the filename.

4 2 Track ID. Each track has a unique value here. This does not correspond to the order in which the apear in the game's cups. E.g. Mario Circuit has a value of 16.
6 2 Character ID. E.g. Yoshi has a value of 4.
8 2 Character Variant ID. Only used fro Yoshi, Shy Guy, and Mii (standard and Amiibo suits)
10 2 Mii Weight Class. Only used for Mii to signify whether it is a light (0x00), medium (0x01), or heavy (0x02) character. Defaults to 0x00 for all other characters.
12 2 Kart ID. E.g. Buggybud has a value of 9.
14 2 Wheels ID. E.g. Button Wheels have a value of 6.
16 2 Glider ID. E.g. Cloud Glider has a value of 1.
18 1 Minutes of total finishing time.
19 2 Seconds of total finishing time.
21 3 Milliseconds of total finishing time.
24 1 Minutes of lap 1 finishing time.
25 2 Seconds of lap 1 finishing time.
27 3 Milliseconds of lap 1 finishing time.
30 1 Minutes of lap 2 finishing time.
31 2 Seconds of lap 2 finishing time.
33 3 Milliseconds of lap 2 finishing time.
36 1 Minutes of lap 3 finishing time.
37 2 Seconds of lap 3 finishing time.
39 3 Milliseconds of lap 3 finishing time.
42 1 Minutes of lap 4 finishing time. "9" if there is no 4th lap.
43 2 Seconds of lap 4 finishing time. 3b (i.e., "59") if there is no 4th lap.
45 3 Milliseconds of lap 4 finishing time. 3e7 (i.e., "999") if there is no 4th lap.
48 1 Minutes of lap 5 finishing time.
49 2 Seconds of lap 5 finishing time.
51 3 Milliseconds of lap 5 finishing time.
54 40 Player Name in UTF-16 (big Endian).
94 2 Flag ID. E.g. The Dutch flag has a value of 5e.
96 2 Motion Control Flag. 0x01 if motion controls are used. 0x00 otherwise.
98 2/4 Unknown. Probably padding (Always 0). Prior to v4 of the game, this could be 2 characters instead if the ghost was a player ghost. The cause for this is unknown.
102 1 Minutes of lap 6 finishing time.
103 2 Seconds of lap 6 finishing time.
105 3 Milliseconds of lap 6 finishing time.
108 1 Minutes of lap 7 finishing time.
109 2 Seconds of lap 7 finishing time.
111 3 Milliseconds of lap 7 finishing time.
114 End of filename; ".dat" extension here.

Game Versions

Lap 4 and 5 finishing times are unused before version 4 of the game. It's unknown if these values are properly read or written to if modifying the lap count of an existing track. Lap 6 and 7 finishing times were only introduced in version 4 of the game. Before that, these values were absent from the filename entirely. Attempting to load Ghost Data from this version of the game in an earlier version will make the game unable to recognise the file. However, renaming the file by removing these extra values will make the Ghost Data load properly. Furthermore, Ghost Data created before v4 of the game is recognised in v4 (and beyond), even though the extra lap finishing times are missing.

This change is a direct result of the introduction of GCN Baby Park in the second DLC pack, as it is the only track that has more than three laps.

Effect In-Game

Changing these values by renaming the file affects how a ghost is displayed. Changing any of the values will result in the game displaying the modified information on the ghost summary (the additional info accessed by pressing "+" on the controller). Furthermore, changing the Character ID, Kart ID, Wheels ID, or Glider ID will result in the game using these during the ghost race itself, regardless of the values for these inside the file itself.

Furthermore, a character's animations can break if the Character ID in the filename does not match the one inside the file contents.

If the Track Number and Track ID do not correspond to the same track, then the game will be unable to recognise the file.

If multiple Ghost files for the same track are present in the game's files, then the game will only show one. It's unverified if it picks the fastest time, the one recorded earliest, or based on something else.

Player Ghost File Header

The following header is absent for Staff Ghosts. However, a Player or Downloaded Ghost can easily be converted into a Staff Ghost by removing this header and renaming the first two characters of the filename from "gs"/"dg" to "sg". This header is identical to the header of a save file. Ghosts downloaded through the Nintendo Clients package do not have this header.

Extra Ghost Header
Offset Size Description
0x00 4 "CTG0" in ASCII. Unknown abbreviation.
0x04 4 Unknown. Likely game version.
0x08 4 File Size (in bytes).
0x0c 44 Probably Padding (always 0)
0x38 4 CRC-32 Checksum over all bytes after this header until the end of the file.
0x3c 12 Probably Padding (always 0)
0x48 End of extra ghost header

Data

The remaining offset values will assume that the additional 0x48 bytes of header data are not present in the ghost file, so if you're using ghost data that was either set in-game or downloaded in-game, then add an additional 0x48 to each of the offsets below.

General Information
Offset Size Description
0x00 6 Likely file magic. The values are always 00 00 04 00 03 A0
0x06 2 Unknown. Possibly game version.
0x08 4 Size of payload (everything after the 0x48-byte header, if present).

Timestamp

When a new ghost is created, the timestamp from the user's local time is used and included in the ghost data.

Timestamp
Offset Size Description
0x0c 4 Year.
0x10 4 Month.
0x14 4 Day.
0x18 4 Unknown. Not a timezone. Values seem to range from 0x00-0x06.
0x1c 4 Possibly Hour.
0x20 4 Possibly Minute.
0x24 4 Possibly Second.

Driver & Course Info

These values do not actuallly seem to be used. Instead, this data is read from the ghost's filename.

Character and Vehicle Combination
Offset Size Description
0x2c 4 Vehicle Body ID.
0x30 4 Tires ID.
0x34 4 Glider ID.
0x38 4 Character ID.
0x3c 1 Character Variant ID. See filename format.
0x3d 1 Mii Weight Class. See filename format.
Course Info
Offset Size Description
0x17c 4 Track ID.

Mii Info

In every single ghost file there is embedded Mii data so that the game can reconstruct the Mii in the case that a Mii was used in the run. Below is the start offset and size of the Mii data. If a Mii was not used during the race, this data will match the Mii of the user profile. Furthermore, this data is used to display the driver name when replaying a ghost.

Mii Info
Offset Size Description
0x244 92 Mii Data. See full description on 3dsbrew. Note that the embedded Mii name is in UTF-16 (little Endian).
0x2a0 2 Probably Padding.
0x2a2 2 CRC-16 XMODEM Checksum.
0x2a4 End of Mii data.

Location

This data is used to display the driver's flag at the end of a race when replaying a ghost.

Location
Offset Size Description
0x2a4 1 Country ID.
0x2a5 1 Sub-region ID.
0x2a6 2 Probably Padding (always 0).
0x2a8 1 Country ID repeat (purpose unknown).
0x2a9 1 Sub-region ID repeat (purpose unknown).

Player Info

Player Info
Offset Size Description
0x304 20 Mii Name in UTF-16 big Endian. Purpose Unknown.

Total Time & Lap Splits

This data is on the overview at the end of a ghost replay. If a track does not have a lap, its lap time is set to 9:59:999 just like in the ghost filename. The exception to this are the lap splits for lap 6 and 7, which are instead 0:00:000 if left unused.

Total Time & Lap Splits
Offset Size Description
Lap 1 Time
0x330 2 Minutes.
0x332 1 Seconds.
0x333 1 Padding (always 0).
0x334 2 Milliseconds.
0x336 6 Probably Padding (always 0).
Lap 2 Time
0x33c 2 Minutes.
0x33e 1 Seconds.
0x33f 1 Padding (always 0).
0x340 2 Milliseconds.
0x342 6 Probably Padding (always 0).
Lap 3 Time
0x348 2 Minutes.
0x34a 1 Seconds.
0x34b 1 Padding (always 0).
0x34c 2 Milliseconds.
0x34e 6 Probably Padding (always 0).
Lap 4 Time
0x354 2 Minutes.
0x356 1 Seconds.
0x357 1 Padding (always 0).
0x358 2 Milliseconds.
0x35a 6 Probably Padding (always 0).
Lap 5 Time
0x360 2 Minutes.
0x362 1 Seconds.
0x363 1 Padding (always 0).
0x364 2 Milliseconds.
0x366 6 Probably Padding (always 0).
Total Time
0x36c 2 Minutes.
0x36e 1 Seconds.
0x36f 1 Padding (always 0).
0x370 2 Milliseconds.
0x372 10 Probably Padding (always 0).
0x37c 4 Unknown.
0x380 4 Probably Padding (always 0).
Lap 6 Time
0x384 2 Minutes.
0x386 1 Seconds.
0x387 1 Padding (always 0).
0x388 2 Milliseconds.
0x38a 6 Probably Padding (always 0).
Lap 7 Time
0x390 2 Minutes.
0x392 1 Seconds.
0x393 1 Padding (always 0).
0x394 2 Milliseconds.
0x396 6 Probably Padding (always 0).

Additionally, (staff) ghosts recorded in v4 of the game store additional 9:59:999 data, which might be placeholders for laps 8, 9, and 10. In earlier versions of the game this data was always 0. It is unknown if this data is properly written to if the lap count of an existing track is increased. Likewise, the behaviour of the UI nor the ghost filename is known if this is done.

Total Time & Lap Splits
Offset Size Description
Lap 8 Time Placeholder?
0x39c 2 Minutes.
0x39e 1 Seconds.
0x39f 1 Padding (always 0).
0x3a0 2 Milliseconds.
0x3a2 6 Probably Padding (always 0).
Lap 9 Time Placeholder?
0x3a8 2 Minutes.
0x3aa 1 Seconds.
0x3ab 1 Padding (always 0).
0x3ac 2 Milliseconds.
0x3ae 6 Probably Padding (always 0).
Lap 10 Time Placeholder?
0x3b4 2 Minutes.
0x3b6 1 Seconds.
0x3b7 1 Padding (always 0).
0x3b8 2 Milliseconds.
0x3ba 6 Probably Padding (always 0).
0x3c0 48 Probably Padding (always 0).
0x400 End of total time & lap splits.

Additional Data

The game must also store the following information somewhere:

  • Player position
  • Player rotation
  • Player scale (E.g., when smashed by a thwomp)
  • Drift usage
  • Tricks performed
  • Horn usage
  • Item usage. When hacking in bullet bills or stars, the game properly displays this.
  • Lakitu saves

It's possible that (some of) these are stored in the Yaz0-compressed sections starting at 0x15c0.

Tools

The following tools can handle Ghost Data (DAT) files:

  • MK8Leaderboards: Fetches and formats data from the time trial leaderboards. Includes a Discord bot.