News

I released a new version of MPQDraft yesterday. MPQDraft is a modding tool that allows in-memory patching of game data files, added features through plugins, and creation of self-executing mods.

This summer, jzy-chitong56 made some Chinese (Simplified) translations for the program and suggested ways in which to make the user experience nicer for those using the translations. While it took me a while to sit down and look at it, I was happy to merge the translations, and jzy-chitong56's idea of a dynamic language switcher was much nicer than the convoluted and technical way it was handled before. I thus developed a little menu button on the Main GUI to change languages. Hopefully Joel Steudler, the graphical artist who made the interface back in the day, won't be too mad at me for slightly deviating from his 2002 work :)

Apart from the new translations and the translations menu, I made it easier to build the application under Arch Linux, and there were improvements to the CI pipeline.

For those who are happy with English and don't intend to build the software yourself, this version unfortunately doesn't bring much excitement. But perhaps that is also a testament to the fact that the original author, Quantam (Justin Olbrantz), made a pretty solid piece of software back in the day.

See the MPQDraft page here, for a download link.

In December, I wrote an MPQDraft plugin, MpqFileLister that lists all the filenames that a games accesses in its MPQ archives — or at least that was the intention. A peculiarity of MPQs are that they don't contain the file names of the content of the archives. But when accessing them, the game will call functions in Storm.dll to open those files, and by hooking into those calls, we can intercept the file it is attempting to open, thus construct a list of all files that are accessed.

I had thought that hooking into the SFileOpenFile and SFileOpenFileEx calls would be enough, but when luauser32167 pointed out on GitHub that the resulting list files were not containing all assets that the game loaded, I realized those two calls were not enough.

I have just made a new release of MpqFileLister that in addition to SFileOpenFile and SFileOpenFileEx and also hook into SVidPlayBegin, SFileLoadFile, SFileLoadFileEx, SBmpLoadImage and SBmpAllocLoadImage. I thus believe all Storm.dll functions that load files by their file names are covered.

In addition, this new release allows the user to freely specify the log format and timestamps, some odd bugs have been fixed and the code has been refactored.

You can find version 1.2.0 of MpqFileLister on GitHub.

This is still rather untested on Diablo I. I tried to test, but Diablo I is just too buggy under Wine on my machine, and I don't have access to a Windows computer. I might try to see if I can contribute to Wine and fix it eventually, but I'm not going to prioritize that for a while yet.

By the way, you might be wondering what the need then is for namebreak-cuda, which I wrote about in the previous entry. The answer is simple: not all files are accessed, but finding and naming them is still of interest.

For many years, Blizzard used MPQ files as their archive files for many of their games. MPQ files are basically like ZIP files; they store many other files in a compressed format. One quirk of MPQ files is that they don't contain the filenames of their own content. Some MPQ files contain a listfile consisting of the filenames of their content, but other MPQ files do not. For these, an external listfile is needed.

The War2Dat.mpq archive of WarCraft II: Battle.net Edition has had a 99% complete listfile for years, but there has been one single file that the community has not known the name of.

Around a year ago, I made a tool, namebreak-cuda, which bruteforces filenames on Nvidia GPUs. While I put the heavy lifting in place, I never quite finished it then. So now, after tweaking it for a bit, I ran it for less than two hours, and it returned the missing filename:

ART\CHAT_BNE_PLAIN.PCX

The War2Dat.mpq now has complete listfiles after all these years. I was happily surprised that the search was so quick.

There are other archives that also don't have complete listfiles, to which I turn my attention next. I plan to make the tool more polished, create a server which can coordinate the efforts of finding more file names, and publish the code on GitHub.

As I wrote about back in May, I've been working on scraping forums and retrieving long gone content from the Internet Archives. It has turned out to be way more fiddly than I expected, mostly because I decided to turn the HTML forum posts into valid Markdown format, and people have used and abused HTML in all sorts of ways. I hope to host the scraped content of dead sites here in not too long.

The scraper is available here for those who want to make their own backups of sites, but please remember that it takes a long time to run, and every page of every thread puts load on the webservers, so please be considerate. We don't want to push any sites further towards unmaintainability.

Some might have noticed that Campaign Creations went down recently. It was once a hugely popular site, but has for a long time been reduced to a forum (with no other content available) that was neither active nor possible to register accounts at. They went dark a few months ago, which I figured was the deathknell for the site. However, I am happy to see that they once again are alive and kicking, and at least some of the content that they used to host seems to be accessible again. And once more, you are now able to register at the forum!

Over the past few days, there's been two releases of mpqcli, and StormLib has also had a release. mpqcli is a command-line tool to create, add and remove files from MPQ archives, and StormLib is the underlying library powering it. MPQ archives were used for many years by Blizzard games such as StarCraft I and Warcraft II, and contain the game assets that game modifications change.

Changes in mpqcli 0.10.1:

Added

• Command completion for common shells (bash, zsh, fish and powershell)

Fixed

• Memory leak in read subcommand
• Inconsistencies in documentation

Updated

• Code style to conform to Google CPP style with slight variations
• StormLib dependency for a variety of fixes

Changes in mpqcli 0.10.2:

Fixed

• verify incorrectly reporting success for a tampered archive
• verify failure messages now clarify whether a signature is missing or invalid
• add overwrite flag not always working as expected
• Archive size incorrectly reported for large archives
• File sizes shown as negative for large files

Updated

• Simplified how strong archive signatures are read internally

Changes in StormLib 9.40:

The changelog for StormLib is brief and only states that it "Added functions SFileOpenFileArchive and SFilegetFileArchive", but the release is actually the accumulation of changes since November 2024.

Download:

Find version 0.10.2 of mpqcli here, and version 9.40 of StormLib here.

For StarCraft I modders, one of the best set of tools has for a long time been PyMS, a graphical tool suite that allows several file types to be modded.

The author, poiuy_qwert, has been hard at work converting the source from Python 2 to Python 3, and writes:

Hey all, I've been putting a lot of work into the python3 version of PyMS recently, but again I only have time to work on the tools not to actually use them, so I need to lean on others to use/test the programs and report any issues. If anyone is able to play with them please let me know!

See here for how to raise any issues.

Also, while the source of PyMS has been available for a long time, it has so far been without any explicit license, but poiuy_qwert just now released it under the MIT license.

Find the Python 3 branch of PyMS on GitHub.

StarCraft I and Warcraft II use TBL files to store game strings for things such as unit names, dialog strings, game tips, mission dialogs etc. Roughly a year ago, I wrote a little utility, IronTbl to convert these TBL files to TXT files and back.

I recently decided to give IronTBL some love and fixed some minor bugs and issues, and spent some time hardening it. So with that, version 2.0.0 is now out.

The previous --mode flag is now replaced by proper subcommands — since this is a breaking change for the interface, I bumped the version to 2.0.0 even though the release doesn't offer any new features.

IronTBL 2.0.0 is available on GitHub.

Thomas Laurenson seems to have figured that yesterday's 0.9.10 release of mpqcli merited a version bump, since it is a rather big release.

He also included a contribution by me to add the compact subcommand which defragments an MPQ archive, thus shrinking it in size if it contains any gaps from previous adds or removes.

So, version 0.9.10 is not available anymore, but instead version 0.10.0 of mpqcli is out!

Thomas Laurenson just released version 0.9.10 of mpqcli two hours ago. As previously covered here, mpqcli is a tool for creating and modding MPQ files, used for loading custom data into games.

Both Thomas Laurenson and myself have made several contributions to the release, which is rather large:

Added

• Support passing multiple files to the add and remove subcommands
• Support reading file paths from stdin in the add and remove subcommands
• Support adding entire directories with the add subcommand
• Add an --update flag to the add subcommand. When given, files whose archived size matches the on-disk size won't be added to save time.

Fixed

• extract subcommand now reports an error when the output directory cannot be created
• Path traversal guard in extract uses fully resolved paths, closing a potential bypass
• Crash when reading strong signatures from malformed or truncated archives
• Docker glibc image updated to ubuntu:24.04
• Adding files is now ordered and operating system agnostic
• add subcommand now returns non-zero exit codes on failure
• Error messages now include StormLib error codes
• Duplicate error messages removed
• Makefile build targets corrected

Changed

• The --path flag on add now acts as an archive path prefix when a directory is given, and the full name when a file is given.

Removed

• --filename-in-archive and --directory-in-archive. They are now replaced by a unified --path flag

Find version 0.9.10 of mpqcli here.

Yesterday, TheNitesWhoSay released version 2.12 of the StarCraft I map editor Chkdraft.

It is a small bugfix with a single change:

• Fix a trigger action display-text issue where the move unit to/from locations were swapped

Version 2.12 is available here.