Saturday, January 2, 2021

Adventures in Old-School Gaming on Linux Mint 20

I had some free time recently, so I finally did a clean install of Linux Mint 20 on my PC, which seemed preferable to attempting an upgrade from Linux Mint 19.3. As usual, after getting all of the basic programs installed, I started reinstalling some essential games: Wolfenstein 3D, Doom, etc.

I did things a bit differently from last time — better, and less hacky, I hope — which required educating myself a bit. I also found a bug which doesn't seem to have been documented anywhere, and I honestly cannot be bothered to figure out where or how to submit a bug report, so I'm just going to document it here. Maybe it will help someone.

ECWolf and GZDoom

As I had done before, I started by installing some source ports: ECWolf for Wolfenstein 3D and Wolfenstein 3D: Spear of Destiny; and GZDoom for The Ultimate Doom, Doom II: Hell on Earth, Final Doom, Heretic: Shadow of the Serpent Riders, Hexen: Beyond Heretic, and Hexen: Deathkings of the Dark Citadel. Unfortunately, these source ports are not provided in Linux Mint 20's default package repository. Perhaps the easiest way to get ECWolf and GZDoom is to download .deb files from here and here respectively, but for the sake of automatic updates, I think it's best to grab them from the DRD Team repository:

wget -O- https://debian.drdteam.org/drdteam.gpg | sudo apt-key add -
sudo apt-add-repository 'deb https://debian.drdteam.org/ stable multiverse'
sudo apt-get update
sudo apt install ecwolf gzdoom

These don't actually come with the game files, though, so I downloaded the relevant games from Steam. (Although they could easily be acquired by... other means... they happen to be in my account thanks to bundles and gifts, and it's just the easiest way for me to get the data.) The Steam versions of the aforementioned games run in DOSBox by default, and the classic Doom games now come with the Unity ports as mentioned in an earlier post, but I would rather take the files provided by Steam and use them with Linux-native source ports.

Manual file management

What I did next, I now realize you probably shouldn't do. I manually copied the necessary game data into ~/.config/ecwolf/ and ~/.config/gzdoom/ as appropriate:

cp -t ~/.config/ecwolf/ -- \
    ~/.steam/steam/steamapps/common/Wolfenstein\ 3D/base/*.WL6 \
    ~/.steam/steam/steamapps/common/Spear\ of\ Destiny/base/*.SOD \
    ~/.steam/steam/steamapps/common/Spear\ of\ Destiny/base/*.SD1 \
    ~/.steam/steam/steamapps/common/Spear\ of\ Destiny/base/*.SD2 \
    ~/.steam/steam/steamapps/common/Spear\ of\ Destiny/base/*.SD3
for sd1 in ~/.config/ecwolf/*.SD1; do mv -f $sd1 $(echo $sd1 | sed s/SD1/SOD/); done
cp -t ~/.config/gzdoom/ -- \
    ~/.steam/steam/steamapps/common/Ultimate\ Doom/base/Doom.WAD \
    ~/.steam/steam/steamapps/common/Doom\ 2/base/Doom2.WAD \
    ~/.steam/steam/steamapps/common/Final\ Doom/base/PLUTONIA.WAD \
    ~/.steam/steam/steamapps/common/Final\ Doom/base/TNT.WAD \
    ~/.steam/steam/steamapps/common/Heretic\ Shadow\ of\ the\ Serpent\ Riders/base/HERETIC.WAD \
    ~/.steam/steam/steamapps/common/Hexen/base/HEXEN.WAD \
    ~/.steam/steam/steamapps/common/Hexen\ Deathkings\ of\ the\ Dark\ Citadel/base/HEXDD.WAD

For the most part, this is pretty straightforward, but things get a bit weird when we look at the Wolfenstein 3D: Spear of Destiny files. That ugly file renaming in the for loop ensures that the *.SD1 files that come with the Steam version of Spear of Destiny have the proper *.SOD names. I think Steam has these *.SD1 files because, when running the game with DOSBox, it needs to rename the *.SD1, *.SD2, or *.SD3 files to the appropriate *.SOD names depending on which mission pack is being played. ECWolf, meanwhile, just wants the Mission 2: Return to Danger and Mission 3: Ultimate Challenge files to be named with *.SD2 and *.SD3 respectively. At least, I think that's how it works. In any case, the manual renaming above resulted in ECWolf running each mission pack correcty with no complaints. So this manual file copying and renaming is an option if you just want to run the Steam versions of all these games with ECWolf and GZDoom.

However, I ended up undoing all of this manual file management in favor of experimenting with a program called game-data-packager, which took some work to understand but seems to work quite nicely most of the time.

Game Data Packager

I became aware of game-data-packager not because I went looking for such a tool, but simply because it was installed automatically as a dependency when I installed some Linux-native engines for Hexen II and the Quake trilogy (more on that later). If you don't already have it installed automatically for similar reasons, you should be able to install it on Linux Mint 20 as follows:

sudo apt install game-data-packager

Once installed, the game-data-packager program can be used to identify the required files for various games, and install them for use with various source ports.

The basic syntax is

game-data-packager [-d OUTDIR] [-i] GAME [PATH ...]

where GAME is the name of a supported game, and PATH tells the program where to look for the game's files (if it can't find them on its own). The game's files are packaged into one or more .deb files, which are written to the current working directory by default but can be sent to some other directory OUTDIR using the optional -d argument. The optional -i flag tells game-data-packager to go ahead and install the new .deb file(s) using the system's package manager. I think it's safe to delete the .deb files after installation unless you want to keep them. At this point, however, you'll need to use sudo apt remove with the name of the locally installed package (e.g. sudo apt remove doom-wad) in order to uninstall the game data from your system.

When executed with the -i flag, game-data-packager also attempts to install dependencies, including source ports and native engines, if they're not already installed. Basically, it should give you everything you need in order to play the game, as long as you supply the copyrighted data.

Wolfenstein 3D

For Wolfenstein 3D and Spear of Destiny, my usage of game-data-packager was as follows:

mkdir -p ~/deb/
game-data-packager -d ~/deb/ -i wolf3d
game-data-packager -d ~/deb/ -i spear-of-destiny

Additional positional arguments would specify folders in which to search for each game's files, but game-data-packager is smart enough to look in Steam's directories. Since I had already downloaded the games via Steam, the above commands were enough to locate the games on my system, package the necessary data into .deb files (which I saved to a ~/deb/ folder I had created), and install the games from those .deb files. In this case, the Wolfenstein 3D and Spear of Destiny data files get installed to /usr/share/games/wolf3d/. The package manager tracks these games' data as the locally installed packages wolf3d-v14-id-data, spear-of-destiny-data, spear-of-destiny-mission1-data, spear-of-destiny-mission2-data, and spear-of-destiny-mission3-data.

However, it assumes you want to use Wolf4SDL for Wolfenstein 3D; it did not care that I already had ECWolf installed, so it installed Wolf4SDL for me. If you want to use Wolf4SDL, you're in luck. If you're like me and you want to use ECWolf instead, you can either edit the BaseDataPaths field in ~/.config/ecwolf/ecwolf.cfg to include /usr/share/games/wolf3d/, or you can make some links:

ln -s -t ~/.config/ecwolf/ -- \
    /usr/share/games/wolf3d/*.wl6 /usr/share/games/wolf3d/*.sod \
    /usr/share/games/wolf3d/*.sd2 /usr/share/games/wolf3d/*.sd3

This got ECWolf to see all of the files it needs on my system. Wolf4SDL is still installed, which is a bit redundant, but I don't mind.

Doom

Next I used game-data-packager to package up the classic Doom games — The Ultimate Doom, Doom II: Hell on Earth, Master Levels for Doom II, and Final Doom — which I had also downloaded via Steam:

mkdir -p ~/deb/
game-data-packager -d ~/deb/ -i doom
game-data-packager -d ~/deb/ -i doom2
game-data-packager -d ~/deb/ -i final-doom

This installs all of the relevant .wad files to /usr/share/games/doom/, tracked by the local packages doom-wad, doom2-wad, doom2-masterlevels-wad, plutonia-wad, and tnt-wad. As with the Wolfenstein games, game-data-packager also automatically provided me with a source port, Chocolate Doom. I probably won't it use very much, because I already have GZDoom installed, but it doesn't hurt. Meanwhile, no tweaks were required to get GZDoom to find the data files; its default configuration included /usr/share/games/doom/ in the search path.

Master Levels

In addition to Chocolate Doom, the automatically installed dependencies also included a program called doom2-masterlevels which provides a simple GUI for selecting maps from the Master Levels pack. (Note that the command ending in doom2, above, took care of packaging the Master Levels for Doom II data as well as the Doom II: Hell on Earth data.) Here's where things get weird, though: I didn't even notice this doom2-masterlevels program until I found it by accident. There should have been a shortcut created in the Linux Mint menu, but it wasn't there. This is because the desktop entry file at /usr/share/applications/doom2-masterlevels.desktop contained a mistake:

TryExec=/usr/share/games/doom2-masterlevels-tryexec

The file referenced in that TryExec field, /usr/share/games/doom2-masterlevels-tryexec, does not exist. As a result, the entry was ignored and did not show up in the menu. However, the file /usr/share/games/doom/doom2-masterlevels-tryexec does exist. This discrepancy in the .desktop file is obviously a bug. I honestly cannot be bothered to figure out how to report this bug, to whom it should be reported, or whether it has perhaps already been fixed in a newer version of game-data-packager than my package manager provides. (Linux Mint does not always have the latest version of everything.) However, it's easy enough to fix the desktop file by changing the TryExec line:

TryExec=/usr/share/games/doom/doom2-masterlevels-tryexec

If you don't want to use a text editor, this can also be done from the command line using sed:

sed s/\\/usr\\/share\\/games\\doom2-masterlevels-tryexec/\\/usr\\/share\\/games\\doom\\/doom2-masterlevels-tryexec/ -i /usr/share/applications/doom2-masterlevels.desktop

With that fixed, the "Doom 2: Master Levels" entry now shows up in the menu.

Heretic and Hexen

I ran into some more minor trouble when I tried to package the Heretic and Hexen files. As with the previous games, I had downloaded them from Steam and tried running game-data-packager with the -i, and it failed with errors about engines not being available. Specifically, for Heretic:

ERROR:game_data_packager.build:Engine "chocolate-heretic" is not (yet) available, aborting

And for Hexen:

ERROR:game_data_packager.build:Engine "chocolate-hexen" is not (yet) available, aborting

However, these errors are nonsense. When the Chocolate Doom source port was automatically installed to /usr/games/chocolate-doom upon packaging and installation of the classic Doom games, the executables /usr/games/chocolate-heretic and /usr/games/chocolate-hexen were installed as well. These actually appear to be Chocolate Doom under the hood, but I quickly confirmed that they were able to run Heretic and Hexen respectively if executed with the appropriate -iwad arguments. Perhaps game-data-packager gets confused because chocolate-heretic and chocolate-hexen are virtual packages. In any case, to get past these bogus errors, we can omit the -i flags so that game-data-packager just creates the .deb files, and then install from those .deb files using apt:

mkdir -p ~/deb/
game-data-packager -d ~/deb/ heretic
game-data-packager -d ~/deb/ hexen
sudo apt install ~/deb/heretic-wad_64_all.deb
sudo apt install ~/deb/hexen-wad_64_all.deb
sudo apt install ~/deb/hexen-deathkings-wad_64_all.deb

I've probably discovered an actual bug here as well, but again, I really don't know how and to whom it should be formally reported, and the workaround was easy enough.

Hexen II

Hexen II is playable on Linux with the Hammer of Thyrion source port, which is called uhexen2 in Linux Mint 20's package repository. Hammer of Thyrion recognizes game data placed in ~/.hexen2/data1/, so we can copy the appropriate .pak files into that directory manually after installing uhexen2. Alternatively, we can use game-data-packager to install the files to /usr/share/games/hexen2/data1/, tracked by the local package hexen2-data. Again, I had downloaded Hexen II via Steam, so I didn't need to tell game-data-packager where to look for the files:

mkdir -p ~/deb
game-data-packager -d ~/deb/ -i hexen2

I think this would have resulted in the automatic installation of uhexen2, if not for the fact that I had already installed it beforehand using apt:

sudo apt install uhexen2

So that probably could have been skipped.

Quake

Linux Mint 20 also provides native engines for Quake, Quake II, and Quake III Arena, and all of these games are supported by game-data-packager. All three games are also on Steam, but I don't have the Steam versions. I have them on a set of CDs, in a box set called The Ultimate Quake, which I had purchased before I even created a Steam account. Fortunately, my outdated PC still has an optical disc drive, and game-data-packager can read from CDs (although it apparently needs to be told explicitly where to look).

The funny thing about Quake and Quake II is that the discs contain both game data and CD audio. In other words, each CD-ROM is also a music CD. Not surprisingly, game-data-packager creates separate packages for game data and music, hence the separate game-data-packager commands below. (Maybe the game files and music packages for a game can be created with a single command, but I used separate commands anyway because I don't think combining them would make things less confusing.) Meanwhile, ripping the CD audio required some extra dependencies which were not made known to me until game-data-packager started complaining that it could not find cdparanoia or oggenc. I assume these were not automatically installed as required dependencies because they are only sometimes needed. In any case, installing cdparanoia is trivial because that's the exact package name, and oggenc is made available for Linux Mint 20 in the package vorbis-tools, so I installed both:

sudo apt install cdparanoia vorbis-tools

Now, to install Quake, I ran the following commands with the CD inserted:

mkdir -p ~/deb
game-data-packager -d ~/deb/ -i quake /media/michael/Winquake/
game-data-packager -d ~/deb/ -i quake -p quake-music /dev/cdrom
game-data-packager -d ~/deb/ -i quake -p quake-aopfm

In the command to package and install the game data, /media/michael/Winquake/ is the location to which Linux Mint automatically mounted the CD when it was inserted, so that part will vary. The game's music doesn't show up in /media/michael/Winquake/ (because, I guess, that's just not how CD audio works), so in the command to rip the music, I used /dev/cdrom (which is a reference to the optical disc drive itself, and is actually a symbolic link to /dev/sr0). The last command installs the package quake-aopfm which is a free expansion. I can't vouch for whether this expansion is any good, because I haven't played it yet — but I noticed after installing the base game that a shortcut for the expansion had already been created in my menu, and that expansion shortcut wouldn't work without the expansion's package installed, so I figured I might as well install it instead of deleting the broken shortcut.

Installing Quake II was similar. I swapped out the Quake CD for my Quake II CD and ran the following:

mkdir -p ~/deb
game-data-packager -d ~/deb/ -i quake2 /media/michael/Quake2/
game-data-packager -d ~/deb/ -i quake2 -p quake2-music /dev/cdrom

Finally, Quake III Arena had no CD audio portion on the disc, so it required only this:

mkdir -p ~/deb
game-data-packager -d ~/deb/ -i quake3 /media/michael/Quake3/

I think these commands would have triggered automatic installations of these games' native engines, but I had already installed them manually beforehand:

sudo apt install quake quake2 quake3

Other Notes

I think much of what's written above could also be applied to other versions of Linux Mint, but I've been clear about the fact that I'm using Linux Mint 20 just in case I'm wrong about that. Meanwhile, as of the time of this post, I'm using gzdoom version 4.5.0, ecwolf version 1.3.3, game-data-packager version 64, chocolate-doom version 3.0.0-5, and uhexen version 1.5.9+dfsg-1.

Also note that game-data-packager gave me some warnings about being unable to unpack certain files due to missing programs, so I installed a couple of additional packages just to make the warnings shut up:

sudo apt install innoextract lhasa

However, these don't appear to be strictly necessary, and I'm not sure if it really made a difference. At least for the Doom games, game-data-packager was already finding all of the data needed for playing the games, despite the warnings.

Saturday, September 12, 2020

Classic Doom Unity Ports and Linux

This seems to be turning into a Linux gaming blog, and there's probably no stopping it, so I might as well not even try.

So, hey, the classic Doom games on Steam got updated to include those official Unity engine ports. That's cool. I mean, I guess it's cool. I'll defer to the video by GmanLives on YouTube for what's actually cool about it:

Update: It seems GmanLives has made the above video private, for some reason. Uh, okay...? Whatever. You don't need to watch this video before reading anything that follows. It was just a general overview of the Unity ports of the classic Doom games, and apparently some people disagreed with the opinions therein, but I don't remember any part of it being so controversial as to warrant removal.

It was thanks to GmanLives that I even found out about the updates to my Steam copies of these games. I go back to Doom and Doom II somewhat frequently, and in fact I do play the copies I bought from Steam, but I don't typically launch them through Steam. Last time I installed them, I immediately copied the WAD files out of the Steam folders to use them with GZDoom and then uninstalled the Steam copies. Therefore I don't often look at these games in my Steam library, and therefore didn't see the big "MAJOR UPDATE" notices, which I now see have been there since September 3rd.

So I was a bit excited when I heard that the Unity ports had come to Steam a week prior, because I thought I might be able to play the No Rest for the Living expansion for Doom II, which had previously been available on Steam only via Doom 3: BFG Edition, which I'm not going to buy because I already have regular old Doom 3. However, once I launched the game, I saw that the Unity version of Doom II includes only Hell on Earth and Master Levels by default. I think you can get No Rest for the Living as an official add-on, but that seems to require a Bethesda account. Maybe I'll create one. Maybe I'll download the WAD from elsewhere. I would rather play any add-on WADs in GZDoom anyway.

The real disappointment — and I'm fully aware that I'm nitpicking here — is that the inclusion of the Unity versions technically breaks the out-of-the-box Proton compatibility of Steam's The Ultimate Doom and Doom II: Hell on Earth on Linux. That's not a tragedy, because they can still be made to run on Linux quite easily. However, it's not a good look for Proton, because both games were whitelisted by Valve for Proton 3.7-8, and that whitelisting is now incorrectly applied to the Unity ports because they were added as updates to the existing games instead of showing up in users' libraries as separate items. Steam for Linux will launch the Unity ports with Proton 3.7-8 by default, and it won't work. Valve whitelisted these games only on the basis of their DOSBox-powered versions running well with that version of Proton.

Of course, using Proton to run the classic Doom games in the Windows version of DOSBox is a bit of a joke. You'll have a better experience running them in a natively Linux-compatible source port like GZDoom. If you really want to run them in DOSBox, there's a native Linux version of that too (and you can make Steam use it for DOS games by installing the compatibility tool Boxtron), so there's no good reason to run the Windows version of DOSBox in Proton. Nonetheless, Valve whitelisted the games for Proton 3.7-8 based on the fact that their old DOSBox-powered versions, in spite of themselves, did work perfectly in Proton 3.7-8 (and, in theory, still should). The new Unity versions don't run with Proton 3.7-8, nor do they work with the current version, Proton 5.0-9. They do seem to work perfectly with Proton 5.9-GE-5-ST, a fork of Proton which is not included in Steam by default but which is easy to install. So if you want to play the Unity ports on Linux, it requires only minimal effort.

The real problem here is that someone is bound to see "Proton 3.7-8 selected by Valve testing" on the newly updated classic Doom games and understandably assume the default versions of the games will run with Proton 3.7-8 out of the box. Now that the default versions are the Unity ports, that's not the case. When you launch either The Ultimate Doom or Doom II: Hell on Earth in Steam now, you will be presented with the option to run either the Unity version (which is the default) or the DOS version, and the default Proton version selected by Valve itself is good only for the DOS version which is now relegated to a secondary or extra option.

The other problem is that, because the Unity versions don't show up as separate games in the Steam library, you can't select different compatibility tools for the DOS and Unity versions. These versions being combined is good in general for those who don't want cluttered libraries, but preliminary testing on my own machine suggests that the DOSBox-powered version of The Ultimate Doom doesn't want to run using Proton 5.9-GE-5-ST which is the only way I know how to run the Unity version. If this is true and it's not just me, alternating between the two versions would require changing the game's compatibility settings each time. I suppose most people, upon deciding which version of the game they prefer, would just stick with it. Still, this bugs me.

Even if there is a Proton version which runs both DOSBox and the Unity ports, I still maintain that using Proton is sub-optimal for the DOSBox-powered versions anyway, and one should really be using Luxtorpeda as an alternative to Proton. Luxtorpeda runs the Steam versions of the classic Doom games in GZDoom automatically, and I probably would have used Luxtorpeda if I hadn't already set up GZDoom manually before I knew Luxtorpeda existed. However, selecting Luxtorpeda as a compatibility tool for The Ultimate Doom would work only for the classic DOS version; it wouldn't be able to run the Unity port. You can't select a single compatibility tool which runs both versions optimally, and you can't select separate compatibility tools for different modes of running a single entry in the Steam library.

In short, I'm slightly annoyed. I'm still just going to keep using GZDoom to run the original WADs anyway, so I guess you could say I'm annoyed on behalf of anyone who hasn't already figured this out. But I guess it's reasonable to assume that anyone who can't figure it out probably lacks the patience for playing games on Linux in the first place.

Saturday, July 25, 2020

Pinstripe and Broken Achievements

I wanted to write this post at the end of December, but the winter holiday season is a busy time, so that didn't happen. Then, by the time the holidays were over, I was already back at work, and forgot about it. Later on, I had an idea about writing this as a follow-up to How to Cheat in Spec Ops: The Line, but that didn't happen either, mostly due to laziness. Now I have a moment and I can only hope that I remember accurately the details of what I had meant to document in December.

So anyway, there's this game called Pinstripe. I have a hefty collection of random indie stuff in my Steam library, mostly from Humble Bundle and from Fanatical (formerly Bundle Stars), so I probably wouldn't remember exactly where I got this one in particular if there weren't records of it. Cross-referencing my Steam account's history of product key activations with IsThereAnyDeal's list of bundles featuring the game, I see that I must have gotten it from the $1 tier of Humble Jumbo Bundle 12, in late November of 2018. My achievement history shows that this is around the time I first played the game as well, so it didn't languish in my backlog for an extended period of time as many indie games from bundles tend to do. Late November into early December was a nice time to play the game as well, given the snowy winter setting.

Overall, it's a cute game. In any mini-review of any kind of game, I feel morally obligated to judge the gameplay first and foremost; but, in this kind of indie game, it would probably be a lie to say that the game and its mechanics are much more than a vehicle for the developer's artistic expression, which could have been channeled into a short film instead if it could have survived becoming that much less engaging. I won't claim to know the developer's intentions but I suspect the goal was to tell a story in an interactive medium and not to get the player hooked on addictive gameplay. So I'll just say that the gameplay is all right; it gets the job done, and manages to be neither frustrating nor entirely boring. But if you don't care for the story or the music or the art direction, you won't care for the game itself.

What stands out to me in hindsight is the audio. The soundtrack is really nice. The voice acting is also pretty great by video game standards. By the end, the story is perhaps a bit clichéd — well, perhaps incredibly clichéd, if I'm being perfectly honest — but its presentation is not as unbearably pretentious as so many indie games are. In short, they did a good job. It's a nice game. I should feel bad for having paid so little for it.

After having played through the game once or twice, I had unlocked all but one achievement:


Beating Pinstripe in an hour is not a remarkable feat. It's a short game. So, I thought, I might as well try it. There's no visible in-game timer, so I took note of the clock on my computer when I started, and played through the game, already knowing where to go and the solutions to the puzzles. I finished in less than an hour. The achievement didn't unlock. I figured the problem was that I had essentially started a "New Game Plus" so I deleted my save and even reinstalled the game to ensure I was starting with a clean slate. I finished the game in less than an hour again. The achievement didn't unlock.

A trip to the game's discussion forum on the Steam Community site revealed that I wasn't the only one having trouble with unlocking the achievement. It was apparently bugged. I checked the forum again today and it's apparently still bugged. There are workarounds which have solved the problem for some players. At least one guy claimed to have done it just by reinstalling the game, but that hadn't worked for me. In mid-December of 2018, someone came up with another fix which seemingly solved the problem for a lot of other players.

This fix involved editing the Windows registry. Specifically, according to this Steam forum post, one must clear a TotalPlayTime entry from HKEY_CURRENT_USER\Software\Atmos Games, LLC\Pinstripe. Yes, the game apparently tracks total play time in the registry. That just boggles my mind. I don't understand it. Perhaps I just don't understand game development or Windows software development well enough to see why Atmos Games would do this, but it seems insane to me. In any case, for whatever reason, this workaround didn't work for me. I assume I must have done it incorrectly, because it had worked for others. However, having failed once with this attempted registry hack, I had already failed the speedrun achievement at least three times in total, and I was fed up. I uninstalled the game.

However, this was not before I had seen another post claiming to have a fix for the Linux version of the game. At the time, I wasn't using Linux on my personal computer yet, but I figured I might try it someday. Sure enough, I started using Linux the following year. Attempting again to get the last achievement in Pinstripe wasn't really my top priority as a new Linux user, but I was bound to come back to it eventually.

So at the end of 2019, I reinstalled Pinstripe on Linux Mint. Now there was no longer a Windows registry claiming I had played for hours, so if I had still remembered how to finish the game quickly, I probably could have just unlocked the achievement fair and square in one attempt. Unfortunately, I hadn't played the game in about a year. I would most likely have had to play through the game once to remember what to do, and then a second time for the speedrun achievement. I had already messed around with this game enough, and I had already beaten the game in under an hour multiple times as verified by my own clock, so I didn't feel like playing fair. If this Linux workaround documented on Steam could be used to unbreak the achievement, it could also be used to cheat. So I decided to remove any chance of failure, and cheat.

I mean, in my defense, it was barely cheating. The game had already cheated me.

I played through most of the game, and sure enough, as I neared the end, I was not on track to beat the game in under 60 minutes. So I closed the game and, following the advice found in the Steam forum, I opened up the file ~/.config/unity3d/Atmos\ Games\,\ LLC/Pinstripe/prefs. It's a plaintext file in some XML-like format with a handful of tags specifying various preferences such as screen resolution height and width. As promised, there's also a line of the form
<pref name="TotalPlayTime" type="float">xxx</pref>
where xxx is the elapsed play time in seconds. Note that, for the sake of verifying what units are used here (for it could have been milliseconds for all I remember of December 2019), I just reinstalled the game and played it for about a minute, and this line in my current copy of this file now reads as follows:
<pref name="TotalPlayTime" type="float">62.1948</pref>
At the time I first found this file, I suppose it must have read something closer to 3600, as I was approaching the one-hour mark with not enough time to spare. So I opened up the file in vim, decreased the value by a few hundred seconds, opened the game back up, and played to the end. The achievement finally unlocked.

Now I've got my 100% completion in Pinstripe. Was it worth it? Well, no and yes. The time I spent trying to unlock the achievement on Windows was a waste of time. However, I still liked the game and wanted to play through it again last December; I also wanted to test drive the Linux version; and, while I was doing that, I figured I might as well get the achievement I was owed. Editing a text file, by the way, was a lot easier than playing around with the Windows registry. Linux wins again, I guess.

There are people who pay absolutely no attention to achievements. For the sake of my mental health, perhaps I should try to be one of them. For the most part, I guess I am. I have far too many games to be caring about unlocking all of the achievements in all of them, considering how many games have achievements which are just arbitrarily time-consuming, seemingly for no reason other than to decrease the number of players willing and able to unlock them and thus to increase their perceived value among self-proclaimed achievement hunters.

If I've already unlocked over 90% of a game's achievements, though, it's hard not to go for 100%. If there's only one achievement left, it's almost impossible not to try, or to be annoyed by it forever if I know it's unattainable. I have other games which, like Pinstripe, have some broken achievements, but there are not always easy workarounds to unbreak them. Do achievements really matter? Not really. But they're a part of the game, and you shouldn't sell broken stuff.

Dear game developers,
Stop selling broken stuff.

Thursday, April 16, 2020

Decoding the Level Passwords in Pushover

Pushover is a puzzle game about an ant pushing over dominos. I played it when I was a kid, and more recently I got a Steam copy of it from a cheap bundle of old games. I didn't really start playing this Steam copy until this week, at which point I remembered that, like many games of the time, Pushover has no save system. Each time the player reaches a new level, the game provides a password which can be used to return to that point.

As I played the first few levels, I noticed something interesting. The first four level passwords were as follows:

Level   Password
01      00512
02      01536
03      01024
04      03072

Two of the passwords being powers of two immediately caught my attention. If we convert each password to binary (using sixteen bits just because bits usually come in groups of eight), we get this:

Level   Password   Binary
01      00512      0000001000000000
02      01536      0000011000000000
03      01024      0000010000000000
04      03072      0000110000000000

Following this pattern, I expected the password for level 05 to be 0000100000000000, or 02048, but using this password brought me to level 07 instead. Going back to level 04, I played up to level 08 and found that, while there was still clearly some kind of pattern, it was a bit more complicated than I had thought. Converting the first eight level passwords to binary, we have:

Level   Password   Binary
01      00512      0000001000000000
02      01536      0000011000000000
03      01024      0000010000000000
04      03072      0000110000000000
05      03584      0000111000000000
06      02560      0000101000000000
07      02048      0000100000000000
08      06144      0001100000000000

At this point, we can already start to see what's going on, if we look at the binary in columns. There seems to be a pattern emerging:

  • The 29 bit column (10th from the right) has 2 ones, 2 zeros, 2 ones, and 2 zeros; we might expect this to continue indefinitely.
  • The 210 bit column (11th from the right), after a single zero, has 4 ones in a row, followed by some zeros; we might expect this column to have a pattern of 4 ones, 4 zeros, and so on, albeit offset by a single zero at the beginning.

However, I didn't really see it until I had played through another eight levels. Converting the first sixteen level passwords to binary, we have:

Level   Password   Binary
01      00512      0000001000000000
02      01536      0000011000000000
03      01024      0000010000000000
04      03072      0000110000000000
05      03584      0000111000000000
06      02560      0000101000000000
07      02048      0000100000000000
08      06144      0001100000000000
09      06656      0001101000000000
10      07680      0001111000000000
11      07168      0001110000000000
12      05122      0001010000000010
13      05634      0001011000000010
14      04610      0001001000000010
15      04098      0001000000000010
16      12290      0011000000000010

Focusing only on the higher-order (leftmost) bits, we can now clearly see that:

  • The 29 bit, starting at level 01, has a pattern of 2 ones, 2 zeros, and so on.
  • The 210 bit, starting at level 02, seems to have a pattern of 4 ones, 4 zeros, and so on.
  • The 211 bit, starting at level 04, has 8 ones in a row, and has presumably started a pattern of 8 ones, 8 zeros, and so on.
  • The 212 bit, starting at level 08, is all ones all the way through level 16, and may have started a pattern of 16 ones, 16 zeros, and so on.
  • The 213 bit first becomes one at level 16.

But something is also happening in the 21 bit (second from the right), where a one suddenly appears starting at level 12. This doesn't seem related to the rest of the pattern, and I think I know why.

It was after completing level 11 that the little ant protagonist found... a bag of Quavers...? (Did I mention that Pushover was sponsored by a snack food company? I've never had Quavers; apparently they're British.) On the menu screen which appears between levels, I could see after completing level 11 that a little sprite representing a bag of Quavers had been added to the bottom of the screen where there appear to be nine spaces available. So it's probably safe to assume that nine virtual bags of Quavers are pointlessly awarded throughout the game.

The fact that the 21 bit changed after level 11 suggests that some of the lower-order bits are used for tracking the bags collected, independently of the higher-order bits which indicate the level number. As an experiment, I took the password for level 12 and flipped the 21 bit to zero, resulting in 0001010000000000 or 05120. Sure enough, entering the password 05120 took me to level 12, and upon completing the level, the inter-level menu screen did not show the bag that was there when I had beaten levels 11 through 15 before. Moreover, the password it gave me for level 13 this time was 05632, which is the previously obtained level 13 password with the 21 bit flipped to zero. However, taking the level 02 password and flipping the 21 bit to one did not result in a valid password. So I cannot cheat my way into already having a bag prior to level 12, but it seems I can enter levels 12 or later without one.

If some bits are only for tracking bags collected, and if any level can be entered without any bags, then we should be able to get a complete set of level passwords just by predicting the pattern of the bits indicating level number. The passwords of levels 01 through 16 suggested that only the higher-order bits (starting with 29) encoded the level number. So at this point, I predicted that the full pattern (if it continued indefinitely) would be as follows:

  • The 29 bit will start with one at level 01 and then change every 2 levels.
  • The 210 bit will become one at level 02 and then change every 4 levels.
  • The 211 bit will become one at level 04 and then change every 8 levels.
  • The 212 bit will become one at level 08 and then change every 16 levels.
  • The 213 bit will become one at level 16 and then change every 32 levels.
  • The 214 bit will become one at level 32 and then change every 64 levels.
  • The 215 bit will become one at level 64 and then change every 128 levels.

This algorithm does predicts the values of the seven highest-order bits for the first sixteen level passwords. Unfortunately, I realized later that it's not quite correct. While looking around in the game's files for goodies, I found that the Steam version of Pushover comes with a file called PUSHCODE.TXT — which, bizarrely, contains most but not all level passwords, and seems to have been written not by the developer or publisher but by someone from a demogroup known as Tristar & Red Sector Incorporated. The file begins:

                            TRISTAR & RED SECTOR
                                              
                                  PRESENTS

                            PUSHOVER (LEVELCODES)


LEVEL 1    00512
LEVEL 2    01536
LEVEL 3    01024
...

It continues all the way through level 67:

...
LEVEL 65   17023
LEVEL 66   18047
LEVEL 67   17535

NOTE: THERE ARE MISSING A FEW CODES!!!!

TIME BANDIT/TRSI

Pushover apparently has 100 levels, so I guess the player who wrote this file gave up about two-thirds of the way through the game. However, they got far enough to show me that the 215 bit does not get flipped to one at level 64. Any number with a one in the 215 place would be at least 32768, and none of the passwords in PUSHCODE.TXT are that high!

So I wrote a few lines of Python to convert all the passwords in PUSHCODE.TXT to binary. The results are as follows:

Level   Password   Binary
01      00512      0000001000000000
02      01536      0000011000000000
03      01024      0000010000000000
04      03072      0000110000000000
05      03584      0000111000000000
06      02560      0000101000000000
07      02048      0000100000000000
08      06144      0001100000000000
09      06656      0001101000000000
10      07680      0001111000000000
11      07168      0001110000000000
12      05122      0001010000000010
13      05634      0001011000000010
14      04610      0001001000000010
15      04098      0001000000000010
16      12290      0011000000000010
17      12802      0011001000000010
18      13826      0011011000000010
19      13314      0011010000000010
20      15362      0011110000000010
21      15878      0011111000000110
22      14854      0011101000000110
23      14342      0011100000000110
24      10246      0010100000000110
25      10758      0010101000000110
26      11782      0010111000000110
27      11270      0010110000000110
28      09222      0010010000000110
29      09734      0010011000000110
30      08718      0010001000001110
31      08206      0010000000001110
32      24590      0110000000001110
33      25102      0110001000001110
34      26126      0110011000001110
35      25614      0110010000001110
36      27662      0110110000001110
37      28174      0110111000001110
38      27150      0110101000001110
39      26638      0110100000001110
40      30734      0111100000001110
41      31246      0111101000001110
42      32270      0111111000001110
43      31758      0111110000001110
44      29726      0111010000011110
45      30238      0111011000011110
46      29214      0111001000011110
47      28702      0111000000011110
48      20510      0101000000011110
49      21022      0101001000011110
50      22046      0101011000011110
51      21534      0101010000011110
52      23582      0101110000011110
53      24094      0101111000011110
54      23070      0101101000011110
55      22558      0101100000011110
56      18494      0100100000111110
57      19006      0100101000111110
58      20030      0100111000111110
59      19518      0100110000111110
60      17470      0100010000111110
61      17982      0100011000111110
62      16958      0100001000111110
63      16510      0100000001111110
64      16511      0100000001111111
65      17023      0100001001111111
66      18047      0100011001111111
67      17535      0100010001111111

This is all consistent with my predictions about bits 29 through 214, but it's the 20 bit — not the 215 bit — which gets flipped to one at level 64. So we can revise the earlier prediction:

  • The 29 bit starts with one at level 01 and then changes every 2 levels.
  • The 210 bit becomes one at level 02 and then changes every 4 levels.
  • The 211 bit becomes one at level 04 and then changes every 8 levels.
  • The 212 bit becomes one at level 08 and then changes every 16 levels.
  • The 213 bit becomes one at level 16 and then changes every 32 levels.
  • The 214 bit becomes one at level 32 and then changes every 64 levels.
  • The 20 bit becomes one at level 64 and then changes every 128 levels.

This isn't quite as elegant as what I had initially predicted, because the "level bits" are no longer contiguous, but it appears to be correct. The last rule is already unlike the others anyway, in that we would never actually see the 20 bit change from one back to zero after 128 levels because there are only 100 levels in the game. I predicted that bit to flip at a frequency of once per 128 levels to continue the power-of-two pattern. So that last rule could just be written as:

  • The 20 bit is one at level 64 and up.

The passwords in PUSHCODE.TXT also give us a hint about how bags are tracked. It seems that each one has its own dedicated bit. If we enter the level 67 password above (17535 or 0100010001111111) and then quit back to the menu screen, we see that we have the first six bags. But if we flip a few bits to get 0100010001010101, and enter the corresponding decimal password 17493, quitting back to the menu shows that we have only the second, fourth, and sixth bags.

If there are nine bags, and if each bag has its own bit, then we need nine bits to represent them all. Meanwhile, if bits 20 and 29 through 214 and are used to determine the level, then the nine bits remaining to represent bags are 21 through 28 and 215.

From another source (the description on this YouTube video), I've found that level 100 is actually a bonus level, and that its password is 44543 (or, in binary, 1010110111111111). This is is still consistent with the above rules regarding bits 29, 210, 211, 212, 213, 214, and 20; additionally, bits 21, 22, 23, 24, 25, 26, 27, 28, and 215 are all flipped to one.

By starting with the level 100 password, individually setting the 21, 22, 23, 24, 25, 26, 27, 28, and 215 bits back to zero, entering the resulting passwords to start level 100, and then quitting to the menu to see which bags are shown, I've confirmed that each of these bits corresponds to a particular bag. Interestingly, setting 215 to zero in particular, resulting in the password 11775, has the effect of making level 100 easier by revealing the special domino types which are normally hidden on this bonus level. More importantly, though, I'm pretty sure we have now fully decoded the game's level passwords.

  • The 20 bit is one at level 64 and up.
  • The 21 bit is one after the first bag is collected (level ≥ 12).
  • The 22 bit is one after the second bag is collected (level ≥ 21).
  • The 23 bit is one after the third bag is collected (level ≥ 30).
  • The 24 bit is one after the fourth bag is collected (level ≥ 44).
  • The 25 bit is one after the fifth bag is collected (level ≥ 56).
  • The 26 bit is one after the sixth bag is collected (level ≥ 63).
  • The 27 bit is one after the seventh bag is collected (level ≥ 78).
  • The 28 bit is one after the eighth bag is collected (level ≥ 89).
  • The 29 bit starts with one at level 01 and then changes every 2 levels.
  • The 210 bit becomes one at level 02 and then changes every 4 levels.
  • The 211 bit becomes one at level 04 and then changes every 8 levels.
  • The 212 bit becomes one at level 08 and then changes every 16 levels.
  • The 213 bit becomes one at level 16 and then changes every 32 levels.
  • The 214 bit becomes one at level 32 and then changes every 64 levels.
  • The 215 bit is one after the ninth bag is collected (level = 100).

The above assumes that bags are awarded for completion of levels 11, 20, 29, 43, 55, 62, 77, and 88, based on the binary representations of the first 67 passwords passwords found in PUSHCODE.TXT and of the remaining passwords found on this page. I don't feel inclined to play through the entire game to confirm it myself before publishing this post. Given that we can enter any level just by setting the "level bits" (29, 210, 211, 212, 213, 214, and 20) — without turning on any of the "bag bits" — there's actually more than enough information here to allow me to amuse myself by writing a little Python script that gives me a valid password for any level I want to play:

import sys


def get_level_bit(frequency, level):
    offset = frequency // 2
    return (level + offset) // frequency % 2


def generate_password(level):
    bits = 0
    for index in reversed(range(16)):
        bits = bits << 1
        if 9 <= index <= 14:
            bits += get_level_bit(2 ** (index - 8), level)
        if index == 0:
            bits += get_level_bit(128, level)
    return bits


if __name__ == "__main__":
    print(generate_password(int(sys.argv[1])))

Is this really useful when I could just look up the passwords on the internet? Not really. But reverse engineering the game's passwords was more fun.