User talk:LordNed

From MK8
Jump to navigation Jump to search

BFRES-Extractor Tool

Hi LordNed, welcome to the wiki. I couldn't help but notice your tool for dealing with BFRES files, very nice to see the research being applied! Having read your Readme on GitHub, I had a few comments. Do you think the name 'sub file' is misleading on this wiki? This is a throwback to the BRRES format on the Wii (which had contiguous sub files), and our documentation of that. Topologically, it is an archive. No data overlaps between two FMDLs or an FSHP and an FVTX say, so each bit of data belongs to exactly one sub file. As with BRRES on the Wii, you can't simply extract these anyway as the string table offsets would be wrong. For BRRES, we chose not to preserve the string table offsets, so that the files could be extracted. This meant each sub file had it's own string table when extracted. I think a similar scheme could be used here; when you extract the data you regroup it into a contiguous file, with the understanding that a BFRES creator would rearange the data again. It's helpful for understanding, but splitting the BFRES up may not be the best plan anyway. Perhaps a tool to convert a BFRES to other formats and back again makes more sense; the BFRES is designed to be an end use format, not an editing and viewing format. Good luck with the program!

Chadderz (talk) 08:01, 20 January 2015 (UTC)

With regard to sub-file naming: I'm not sure. They're definitely specific types of data, but yeah they're not individual files (for the most part that we could tell, not entirely sure without all of the formats being better reverse engineered). I'm not sure what you'd call it instead, though in contrast to "Embedded File" (which is a whole complete linear file) sub-file does sound out of place. I don't think the data is ever shared here either - it's just not written linearly in the file (ie: you do not have a full FMDL in the space between two FMDL headers in the file).
I think a BFRES file does need to be split out into individual files (with possible repeat data) - it's just not clear where exactly yet. Obviously embedded files work fine as is, but it'll become a game of "Can you have X type of sub-file alone in an archive?" ie: Can skeletal animations be included in archives with nothing else? If so they could stand alone as a file. What about FTEX headers, is there anything there that we can use the FTEX for or is it just meta data for the actual binary data that actually contains the texture data.
LordNed (talk) 00:35, 24 January 2015 (UTC)
Perhaps we should choose a name like section then, but I would argue even that implies linearity. In truth, I'm not sure we have a good word for it. Perhaps we should edit the article just to make explicit that the files are not continuous.
On the topic of splitting, the trurth is that NO files other than embedded files can be split simply. ALL formats reference the string table (if nothing else). You will never be able to split the file and preserve the original format, some conversion will be required. Since conversion is required, data must be rearranged regardless. It's just a difficulty in BFRES software implementation really. What we have done in our work is write programs to convert the BFRES to a .obj model (FMDLs) and a series of .png images (FTEX) and also tools to replace a given FTEX or FMDL with a .png or .obj, which allows us editing. This allows us to avoid these issues as we never export a FMDL as a standalone file.
Chadderz (talk) 21:59, 24 January 2015 (UTC)
This would be fine by me. It is true that you can't extract continuous files out regardless due to the string table, but the re-creation/splitting of a string table is relatively simple compared to all of the sub-formats. Maybe the sub-file types should be replaced with sub-categories? Ie: "This file contains this type of data: MDL, TEX, SKEL", without implying that they're separate or unique files.
Certainly worth clarifying in the documentation, and it is likely that early BFRES tools will have to do a file replacements instead of complete unpacking and repacking. Unpacking and Repacking would require almost complete format documentation so it's a long ways off - perhaps it is my tool that is wrongly named. Fortunately the tool and source code should give a pretty good idea about how to work with the format so there's that at least. LordNed (talk) 22:10, 24 January 2015 (UTC)