News

I hacked together a script for the site that will automatically take all news items, divide them by year and month and create a news archive. It will also put the news items into an RSS feed.

The pages are available in the navigation menu, as well as here:

• News archive
• News feed

Powering this site is incunable — a hackish templating engine for static site generation that I started a few years back. I have also made some updates to it the past few days, so that is at least indirectly related to this site.

Things are a bit slower now compared to a few months ago. I have a big, long trip coming up, for which planning has taken a lot of time. And right now I'm in my home town where I try to meet friends and family.

Some format specifications have been published, from the Camelot Systems site. For now, it is only four DAT files from StarCraft, and the Unit data DAT file from WarCraft II.

The plan is to let the collection of formats grow, hopefully to be of pretty much all modding-relevant files. The specifications are available on the Format specifications page.

Digging through my files, I found some modding software from the old Camelot Systems days, some 25 years ago. While CamSys was mainly focused on StarCraft and WarCraft II, they also made some software for other, non-Blizzard games:

• Tiberian Sun
• Descent 3
• Gender Wars

While these are not games I myself ever played, I suppose they might as well be uploaded here as they once briefly were on the CamSys website. The respective modding communities of these games (of which I know nothing) have presumably long since superceded these with other tools, but I thought it would be fun to upload these here anyway as a little digital library.

I finally made some finishing touches on IronTBL, the TBL to TXT converter, so I decided to release it as version 1.0.

Not much has changed since the last release (apart from some logging improvements), it mostly has some external integrations: comes with ImHex pattern language definitions and yazi integration, and it can now generate shell completions.

IronTBL is available on GitHub.

IronGRP is a GRP to PNG converter for WarCraft I, WarCraft II and StarCraft.

IronGRP version 0.5 was just released with the following features:

• Added ImHex pattern language definitions for Normal GRPs, Uncompressed GRPs and WarCraft I style GRPs.
• Added yazi integration.
• Included fallback greyscale palette.
• Added a Readme file.
• Added a logo.
• Moved the PNG handling to an external library, palpngrs.
• Better logging: Introduced logging library and the log level 'trace'.
• Removed some Optimisation schemes from CompressionType Optimised. This makes it slightly less optimised but identical to how Blizzard did it, and the code is less complex.

IronGRP is available on GitHub.

After creating palpngrs (GitHub) out of code from IronGRP, I decided to create a little utility that can convert between Palettized images and PNGs. I called it IronImage.

It can convert between PNGs and Palettized images from WarCraft I and WarCraft II. It can also convert between PNGs and the cursor format of WarCraft I and WarCraft II (which has the identical format to Palettized images, but with two extra bytes in the header — one for the x-hotspot and one for the y-hotspot).

The code is available at GitHub.

I would like to add support for PCX images, which are used in StarCraft. Image support in palpngrs is provided by image-rs, and there seems to be an experimental image-rs module, image-extras where PCX support is being considered (see discussion here). However, it does not yet seem to be in a stable state, and GIMP supports the PCX format anyway. I will probably revisit PCX support at a later time. For now, IMG and Cursor support will have to do.

From Diablo I and onward, up until fairly recently, Blizzard has used MPQ files to store their game data. The MPQ format is a container format that can hold multiple files and directories, like ZIP files. While the MPQ files store files, they do not store file names. To know the names of the files, an external listfile must be used. Internally, the MPQ files instead store two hashes of the file names to identify the files.

Almost all file names of all MPQ files are known, but there are a few unknown file names.

Back in the days, some 20 years ago, I tried to figure out all the names of the MPQ files of Blizzard. At university, I wrote a distributed brute-force program that would try to find one of the remaining portrait files, but it never found it.

I just created a little CUDA program to make use of the GPU to continue to brute-force where I left off many years ago. Just a little something to run in the background, in the hopes that some of the last unknown properties of the games will be revealed.

I have started to work on an image converter for WarCraft I and WarCraft II. I have made a library called palpngrs out of the code I had for IronGRP, which can handle conversions to and from Palettized images and PNG files. Very few lines of code then allowed me to slap together a little utility that can convert raw IMG files (used for backgrounds and dialog elements in WarCraft I and WarCraft II) as well as Cursor files (used for the mouse cursor in WarCraft I and WarCraft II) to PNGs and back.

I have some finishing touches to do and I can't decide on a name I like for it. But with this utility, IronGRP and IronTBL, a fair amount of the game files of WarCraft I and WarCraft II can now be converted to and from more accessible formats.

IronGRP 0.4 has just been released and is avaliable at GitHub. Some of the features have been mentioned before: It has support for Extended Uncompressed GRPs and WarCraft I GRPs, and it can now correctly differentiate between different GRP types. I have added some boundary checks for width, height, offsets and frame count, and added more unit tests. It can now also generate shell completions, to make the tool more ergonomic to work with.

When converting from GRP to PNGs, IronGRP will now name the output files "uncompressed_frame_xxx.png" or "war1_frame_xxx.png", respectively. This is to make it easier to identify the type of GRP that was used when converting back to GRP. When converting from PNGs to GRP, if no CompressionType is given (or if CompressionType Auto is given), IronGRP will create an Uncompressed GRP if any of the input filenames contains "uncompressed", and create a WarCraft I style GRP if any of the input filenames contains "war1".

A fair amount of refactoring has been done, so the code is now a bit better structured, and I've prepared to move a chunk of the code to handle the conversion between PNGs and Palettized images into its own library. This will allow me to use the same code in upcomming tools that I have in mind. The cursor formats of WarCraft I and WarCraft II uses the same principle of Palettized Images, as does the Parallax SPK files of StarCraft. I suspect the tilesets of all these games do too, but I haven't looked into that yet.

I extracted the GRPs from WarCraft I: Orcs and Humans from its data archive files. The format is nearly identical to Uncompressed GRPs, except that the header is four bytes instead of six (in WarCraft I there is first a two byte frame count, and then a byte with the maximum width followed by a byte of the maximum height - In StarCraft and WarCraft II the max width and max height are both two bytes each).

This caused a surprising amount of code changes in IronGRP, but nothing too difficult. There is still work to be done in automatically detecting whether the header is four or six bytes, though. I implemented a primitive check, but it fails to differentiate between WarCraft I GRPs and Extended Uncompressed GRPs. Some more work is needed to make it correct and more robust.