ESO Mod:EsoExtractData

Main Page

EsoExtractData is a Windows command line utility program used to extract data from ESO's MNF/DAT files. It is also available at ESOUI.

Installation

• Download File:EsoExtractData.zip or at ESOUI.
• Unzip into a directory of your choice (preferably a new directory).
• Run the program from the Windows command line (ex: Start::Run::cmd) or similar shell.

Usage

IMPORTANT NOTE -- Exporting all files from the three ESO MNF files will take several hours and require over 100GB of free disk space.

EsoExtractData --help or EsoExtractData -h

View basic program usage and command options:

EsoExtractData \path\to\game.mnf \export\path\

Extracts the given MNF file and outputs to the given path.

EsoExtractData \path\to\game.mnf \export\path\ -z \export\path\zosft.txt -m \export\path\mnf.txt

Extracts the MNF file and outputs a ZOSFT and MNF directory listing to the specified files.

ConvertDDS \export\path\

Converts all DDS files recursively in the given path.

Command Line Options

-h or --help

Show basic help.

-m [filename] or --mnfft [filename]

Save the MNF file directory to the given filename.

-z [filename] or --zosft [filename]

Save the ZOSFT file directory to the given filename.

-s [fileindex] or --startindex [fileindex]

Start exporting from the given file index.

-e [fileindex] or --endindex [fileindex]

Stop exporting at the given file index.

-f [fileindex] or --fileindex [fileindex]

Only export the file with the given fileindex.

-a [index] or --archive [index]

Only export the given MNF index.

-b [index] or --beginarchive [index]

Start exporting at the given MNF index.

-c or --convertdds

Convert all DDS files to PNG. Note that this currently crashes on some DDS files.

-k or --skipsubfiles

Don't extract/save files from the MNF/DAT.

-n [filename] or --filename [filename]

Only extract filenames that match the given string or number. If a number is used it will match the fileindex of the subfile with no extension.

Filenames can include paths however the names are checked both with and without the subfile path. For example, "testfile.DDS" will match both "testfile.dds"

and "path/to/testfile.dds". Match is case insensitive. Wildcards in the filename are not supported. Added in version 0.33.

-l [filename] or --lang [filename]

Convert the given LANG file to "filename.CSV". Also outputs an ID file with the name "filename.id.txt" which is needed when converting a TXT file back to LANG.

-x [filename] or --createlang [filename]

Convert the given CSV file to a LANG file (ex: text.csv to test.lang). The CSV file should be in the same format as one created from the "-l" option including:

• One header row: ID,Unknown,Index,Offset,Text
• Column order is currently fixed.
• Offset column is recomputed when saving.
• Note that the values \r, \n and \" in texts are converted to their respective characters.
• Text column must be quoted to preserve commas in texts.
• Resulting LANG file will be larger than the original due to duplicate texts not being merged.

-o [filename] or --outputfile [filename]

Manually specify the output file for -l and -x commands.

-p or --pocsv

Output a LANG CSV file in a PO (Pootle) compatible format. The CSV will have 3 columns instead of 5:

• Location: "ID-UNKNOWN-INDEX"
• Source: TEXT
• Target: Blank

-t or --langtext

Save the LANG file in a plain text format (one text per line, LFs and CRs will be escaped to \r and \n). If combined with -p and extra blank line will be added between the texts to be compatible with Pootle's txt2po utility.

-i or --idfile

Specifies the ID file to use when converting a TXT file to LANG. An ID file is created when converting a LANG to TXT/CSV and must match the file used when converting the TXT file back to LANG. It is a simple text file with one column containing the unique ID for each text in the file.

-d [file1] [file2] or --difflang [file1] [file2]

Compares two LANG/TXT/CSV files for differences. Will output files XXX.added.csv, XXX.removed.csv, XXX.changed.csv, and XXX.lang (or XXX.txt and XXX.id.txt if the -t option is used). For column format for the changed CSV file is [id columns...], [new], [old], [translated]. The two input files don't have to be the same type. If using TXT files use the -i1 and -i2 parameters to specify the associated ID files.

       EsoExtractData -d file1.lang file2.lang
       EsoExtractData -d file1.lang file2.lang -p
       EsoExtractData -d file1.lang file2.lang -p -t
       EsoExtractData -d file1.lang file2.csv
       EsoExtractData -d file1.csv file2.lang
       EsoExtractData -d file1.txt file2.lang -i1 file1.id.txt
       EsoExtractData -d file1.lang file2.txt -i2 file2.id.txt
       EsoExtractData -d file1.txt file2.txt -i1 file1.id.txt -i2 file2.id.txt

Note: If the ID file used with -i and -i1 is the same you can omit one or the other parameter on the command line. For example, the following commands would be identical:

       EsoExtractData -g tr.txt -i id.txt -d old.txt new.txt -i1 id.txt -i2 new.id.txt
       EsoExtractData -g tr.txt -i id.txt -d old.txt new.txt            -i2 new.id.txt
       EsoExtractData -g tr.txt           -d old.txt new.txt -i1 id.txt -i2 new.id.txt

-i1 [idfile] or --idfile1 [idfile]

Use this to specify the associated ID file if using a TXT file for the first file in a -d parameter.

-i2 [idfile] or --idfile2 [idfile]

Use this to specify the associated ID file if using a TXT file for the second file in a -d parameter.

-g [file] or --origlang [file]

Use this with the -d command to specify the source for text for unchanged entries. If a text entry has not been removed or changed the text will come from this file. This can be a LANG, CSV or TXT file. If using a TXT file use the -i parameter to specify the associated ID file.

       EsoExtractData -g orig.lang -d file1.lang file2.lang
       EsoExtractData -g orig.csv -d file1.lang file2.lang
       EsoExtractData -g orig.txt -i orig.id.txt -d file1.lang file2.lang

--noparsegr2

Don't parse any GR2 files for their original filenames. By default all recognized GR2 files are loaded and parsed by the Granny DLL in order to extract and output the file to its original filename.

       EsoExtractData eso.mnf --noparsegr2

--extractsubfile combined

Files that contain compressed record/subfile data are uncompressed and their data output to a single file. The combined file format is output in the following format:

       Header (same 16 bytes as original compressed file)
               dword MagicBytes
               dword Unknown1
               dword NumRecords
               dword Unknown2
       x(0...N) Record Data (variable sized)
               dword MagicBytes = "####"
               dword Index
               dword UncompressedSize1
               dword UncompressedSize2
               dword CompressedSize
               dword Index                      (from index file if found)
               dword OrigFileOffset            (from index file if found)
               dword UncompressedSize
               byte UncompressedData[UncompressedSize]
       
       EsoExtractData eso.mnf --extractsubfile combined

--extractsubfile seperate

Files that contain compressed record/subfile data are uncompressed and their data output into individual files within a new subdirectory. Warning: This creates over 1 million files and adds several hours to the extraction.

       EsoExtractData eso.mnf --extractsubfile seperate

--luafilelist [filename]

If this option is specified a list of all files with a ZOS file table entry is output to the given LUA file in the format:

      [Index] = "Filename",

The Index is a number starting at 1 and incrementing for each file. This option also filters filenames based on the optional --fileext parameter.

--luastartindex [index]

Specify the initial file index for the LUA array output by --luafilelist. Default value is 1.

--fileext [ext]

Only extract files matching the given extension. This only applies to files that have a filename specified in the ZOS file table. You don't need to include the period. Technically this just checks if the filename ends with the given string, so TXT would match files File.TXT and FileTXT.

Example

• Convert MNF Files

    EsoExtractData c:\input\path\file.mnf c:\output\path\

• Convert MNF Files With Directory Files

    EsoExtractData c:\input\path\file.mnf c:\output\path\ -z c:\output\path\zosft.txt -m c:\output\path\mnf.txt

• Convert a Specific LANG File to CSV

    EsoExtractData -l file.lang 

• Convert a Specific LANG File to CSV With Custom Name

    EsoExtractData -l file.lang -o newfile.csv

• Convert a Specific LANG File to CSV in a PO Compatible Format

    EsoExtractData -l file.lang -p

• Convert a Specific LANG File to Text

    EsoExtractData -l file.lang -t

• Convert a Specific LANG File to Text in a PO Compatible Format

    EsoExtractData -l file.lang -t -p 

• Convert a CSV File to a LANG

    EsoExtractData -x file.csv

• Convert a CSV File to a Custom LANG File

    EsoExtractData -x file.csv -o newfile.lang

• Convert a PO CSV File to a LANG

    EsoExtractData -x file.csv -p

• Convert a Text File to a LANG

    EsoExtractData -x file.txt -i idfile.txt -t

• Convert a PO Text File to a LANG

    EsoExtractData -x file.txt -i idfile.txt -t -p

• Convert a PO Text File to a Custom LANG File

    EsoExtractData -x file.txt -i idfile.txt -t -p -o newfile.lang

Version History

• v0.53 -- 30 April 2024

• Fixed loading of ZOSFT from update 42pts patch 2.
• Fixed the export of ZOSFT text files.

• v0.52 -- 25 August 2022

• Added the --debugoutput/-v option for outputting additional messages to the log and output. Default is off (so less messages). Set a bunch of error/warning messages to debug only. Note that all messages are still saved to the log file.
• Fixed the format of the subfile index which prevents a bunch of error messages from the ESO00000.DAT file.
• Adjusted tab levels of text output for a little neater visual output.
• Introduced the EsoExtractDataGui program (Windows graphical program for basic extraction tasks).
• Fixed a memory leak when converting RIFF files to WAV.

• v0.51 -- 28 April 2022

• Added the --matchfilename option for only outputting files that contain the given sub-string.
• Added the --baselangfile option that uses the given LANG file as for base/missing entries when creating a new LANG file.

• v0.50 -- 28 June 2021

• RIFF files are automatically converted to OGG/WAV files. Use the "--noriffconvert" command line option to disable.

• v0.42 -- 30 April 2021

• Removed extra "." in DDS files exported from XV4 files.
• Model/animation names from GR2 Granny files are used to output named files if possible.

• v0.41 -- 17 September 2020

• Removed the -c/--convertdds option.
• Added the --luafilelist option.
• Added the --luastartindex option.
• Added the -y/--fileext option.

• v0.40 -- 6 February 2020

• Changed to a 64 bit build (to support Oodle decompression).
• Supports the new DAT format introduced in update 25 PTS.
• Skips empty DAT files which reduces the number of error messages.
• Added the "--oodleraw" option which saves subfiles in their original Oodle compression format.

• v0.34 -- 30 May 2019

• Fix default for the "-n" / "--filename" option.

• v0.33 -- 19 April 2019

• Added the "-n" / "--filename" option.

• v0.32 -- 23 October 2018

• Fixed crash from null pointer reference received from Granny API.
• Updated Granny2.dll file which permits more original GR2 filenames to be extracted.

• v0.31 -- 17 April 2017

• Fixed extraction of the ZOSFT from ESO MNF in update 14 PTS files. Unsure how this will work with prior file versions however (is fine with update 13 at least).

• v0.30 -- 18 January 2017

• Added more matching magic bytes for recognizing GR2 (Granny) files.
• GR2 model/animation files are output according to their internal original path/filename in addition to their ZOSFT filename (if it exists) and numeric file index. The original path will be created under the "Granny" path in the base export path.
• Recognize binary Havok files and assign them with the HKX extension.
• Recognize file data containing compressed sub-files and assign them the EsoFileData extensions.
• Recognize file data containing unknown ID data and assign them the EsoIdData extensions.
• Recognize file data for the PSB2 format (unsure exactly what it is though).
• Recognize file data containing more text data (including books) and give it the "TextData" extension.
• Recognize "FFX" file data.
• Recognize UTF-8 text files starting with the byte order mark EF BB BF and give the extension "TXT".
• Recognize the unknown XV4 file which seems to be just a DDS with 12 bytes of extra header data and 4 bytes of extra footer. The original file and a DDS version without the extra header is saved.
• Include the "--noparsegr2" parameter to prevent the parsing of GR2 files to extract their original filename.
• Added the "--extractsubfile" option for extracting compressed data from some file types:

         --extractsubfile none : Default, does nothing
         --extractsubfile combined : Outputs all compressed files in one large file.
         --extractsubfile seperate : Outputs files individually (Warning: This creates over 1 million files and adds several hours to the extraction).

• v0.29 -- 5 September 2016

• Fixed an infinite loop due to a truncated compressed file in file eso0002.dat from the update 12 PTS.

• v0.28 -- 8 March 2016

• Fixed a crash when extracting Game.Mnf data from the Thieves Guild DLC release.

• v0.27 -- 4 February 2016

• Added support for the 1.9 update on PTS (Thieves Guild DLC) for finding the ZOSFT in the ESO.MNF file. File index for the ZOSFT was changed from 0xFFFFF to 0xFFFFFF.

• v0.26 -- 19 August 2015

• When using -d the changed CSV file contains the original translation text supplied with -g if it exists in the last column ([id columns], [new], [old], [translated]).
• If the ID file used with -i and -i1 is the same you can omit one or the other parameter on the command line. For example, the following commands would be identical:

        EsoExtractData -g tr.txt -i id.txt -d old.txt new.txt -i1 id.txt -i2 new.id.txt
        EsoExtractData -g tr.txt -i id.txt -d old.txt new.txt            -i2 new.id.txt
        EsoExtractData -g tr.txt           -d old.txt new.txt -i1 id.txt -i2 new.id.txt

• v0.25 -- 28 August 2015

• Added the "-d" option for comparing LANG/CSV/TXT files.
• Added the "-i2" option for specifying the second ID file when comparing files.
• Added the "-g" option for specifying a source text for unchanged entries.

• v0.24 -- 9 June 2015

• Added the "-i" option to input an ID text.
• An ID file (.id.txt) is output when converting a LANG file.
• Convert a text file along with an ID file to a LANG file.

• v0.23 -- 11 April 2015

• Added the -t option for saving LANG files in a plain text format.

• v0.22 -- 9 April 2015

• Added the "--posourcetext" to use the source text column (2) in a PO-CSV file when converting it to a LANG file.
• Assume a PO-CSV file (3 columns) when the -p option is used with -x.
• Fix the location column (1) when creating a PO-CSV file (offset was used instead of index).

• v0.21 -- 9 April 2015

• Escape quotes in text as double-quotes ("") instead of \" in CSV files to import correctly.
• Added the "-o" option for specifying the output filename for -l/-x commands.
• Added the "-p" option for outputting LANG CSV files in a PO compatible format.

• v0.20 -- 9 April 2015

• All cells are quoted when saving a LANG file as a CSV.
• Translate DOS linefeeds in LANG file texts as "\r" (was "\n" as previously).
• Convert a language CSV file back to a LANG file using the "-x" option:

• First row must be a header: ID,Unknown,Index,Offset,Text
• Column order is currently fixed.
• Convert \r, \n and \" to their respective characters.
• Text column must be quoted to preserve commas in texts.
• Resulting LANG file will be larger than the original due to duplicate texts not being merged.
• Output filename will be the same filename with ".CSV" replaced with ".LANG".

• v0.18 - 23 November 2014

• Fixed output of Game.Mnf with filenames.

• v0.17 - 7 November 2014

• Name changed from EsoExportMnf to EsoExtractData.
• Now has better, automatic loading of ZOSFT entries for filename correlation.
• Fixed incorrect filename assignment to files with the same ID.
• Added missing , to MNF file table CSV export.
• Added the "UserData" column to the ZOSFT file table CSV export. Currently is the number of file entries found in the MNF data.
• Converts any .lang file to a CSV format if it can.
• Language file now exported in a normal CSV format (with commas and internal double-quotes escaped to \").
• Added the -b/--beginarchiveoption to start at a specific DAT file index.
• Added the -l/--lang option to convert a .LANG file to a CSV.

• v0.16 - March 2014

• Updated to support the patch for the 14 Mar 2014 beta.

• v0.15 - February 2014

• Updated to support the patch for the 8 Feb 2014 beta.

Known Bugs

• Errors Extracting from ESO0000.DAT -- There are many files from ESO0000.DAT are in an unknown/invalid format and do not extract correctly.
• Errors Extracting from ESO0204.DAT and Greater -- DAT files with index 204 and above are empty (just the DAT header) even though there are file entries in the MNF file. This results is errors when trying to extract file data from the DAT files that doesn't exist.
• DDS Conversion With -c -- There are some DDS files that will crash the built-in file loader for an unknown reason. Use the ConvertDDS.bat file included instead.

Converting LANG Files

This is a more in-depth explanation of how to convert LANG files to/from TXT/CSV files related to translating the texts to different languages.

• Create the Initial Export

        EsoExtractData.exe -l en.lang -o en.orig -p
             Creates: en.orig.csv
        EsoExtractData.exe -l en.lang -o en.orig -t -p
             Creates: en.orig.txt, en.orig.id.txt

Save these files (LANG, CSV and two TXT) for later use as you will need them when re-importing the translated text. Note that the -p option is for importing the file to [pootle.translatehouse.org Pootle]. You can omit the -p for all commands but make sure to be consistent (a file exported with -p has to be imported with -p).

• Translate the File

Import the en.orig.txt or en.orig.csv into your translation program/utility and translate strings as desired. When finished export the translated strings to a new.txt or new.csv file.

• Create the New LANG File

        EsoExtractData.exe -x new.txt -i en.orig.id.txt -t -p -o new.lang
             or
        EsoExtractData.exe -x new.csv -p -o new.lang
             Creates: new.lang

The new.lang file can be used in an add-on to "overwrite" the existing LANG file of the game.

• Update a Partially Translated File

The game continually updates its text over time so you may wish to update a previously exported TXT/CSV file with all the new/updated text:

        EsoExtractData.exe -g new.txt -i en.orig.id.txt -p -d en.lang en.new.lang -o new.updated
             or
        EsoExtractData.exe -g new.txt -i en.orig.id.txt -p -d en.orig.lang.txt en.new.lang.txt -i1 en.orig.id.txt -i2 en.new.lang.txt -o new.updated
            Creates Files:   new.updated.lang
                             new.updated.lang.txt
                             new.updated.lang.id.txt
                             new.updated.added.csv
                             new.updated.removed.csv
                             new.updated.changed.csv

The new LANG/TXT/CSV file will have all the translated lines from new.txt that haven't been changed or removed in the new LANG file and have all the new texts added. The changed CSV file will contain the new text followed by the original text in the last column. Be sure to keep the 3 new files new.updated.lang, new.updated.lang.txt, and new.updated.lang.id.txt as you will need them for future translations and updates.

Other

• Source code is available on GitHub.