Dolphin Progress Report: April and May 2018

Users of Dolphin may have noticed that things look a little bit different in the GUI. That's right, mid-April DolphinQt was unleashed to the masses as the default GUI! It hasn't been without some expected headaches and growing pains, but, overall most of the features are working and the transition is going along as smoothly as we could have hoped. For those having problems, the DolphinWx.exe is still included and will be updated with all the core changes.

Considering that we spent a whole monthly article on Qt, let's get into some actual emulator changes. In between the mountain of Qt changes has been an explosion of progress and fixes hitting everything from CPU edge-cases to Vertex Loader fixes! With that, please enjoy this month's notable changes!

Notable Changes

5.0-6860 - Remove Support for Projection Hacks by JosJuice

Back in the days of D3D9 and OpenGL 2, it wasn't possible for Dolphin to accurately emulate all kinds of GameCube and Wii GPU behaviors. Depth bugs, clipping errors, and other issues were just not solvable in correct ways. Developers had to tackle these bugs in a different, more customizable manner. The end feature, Projection Hacks, allowed users and developers to finely tune elements of how a specific game renders in Dolphin, effectively working around bugs caused by unemulated and incorrectly emulated features. But as Dolphin and graphics APIs have advanced, these GPU behaviors can now be emulated properly, and projection hacks have seen their usefulness dwindle to nearly nothing.

In fact, leaving projection hacks on alongside correct emulation leads to games being unexpectedly broken. In the case of Skies of Arcadia Legends, users would constantly report shadows being broken long after the issue was fixed. Why? The most common culprit was old INI files with the projection hack still activated by default.

5.0-6860 finally removes projection hacks completely from Dolphin's codebase, putting the remaining issues to rest. Alongside its removal, the final four games that still had projection hacks embedded into their INI files have had them removed. Surprisingly, some of these games actually have graphical issues (unrelated to projection) that were clipped away by the projection hack! Don King Boxing and Rockstar Table Tennis have strange issues with shadows sticking to the screen longer than they do on console. Previously, these issues were harder to spot as their projection hacks worked around bugs by completely culling out the shadows altogether!

Fixing the black half of the screen with projection hacks hid other issues.

The other casualty of removing projection hacks is actually some user made content. Some of our more clever users have been using projection hacks as a way to bypass some undesirable effects, such as depth of field or bloom. In this case, there isn't a great answer to give except that there are other ways to accomplish the same effect, such as game patches, Action Replay codes, or Gecko Codes. Plus by using these instead, they will also work on console rather than relying on features embedded in an emulator.

5.0-6899 - Fix Rumble in Various Games by spycrab0

The GameCube Controller's rumble motor (src: ifixit)

In games like Luigi's Mansion, Pokémon Colosseum, and Pokémon XD: Gale of Darkness, rumble just wouldn't work in Dolphin. It didn't matter whether you were using a Native GameCube Controller Support or a standard PC controller with rumble configured.

Strangely enough, there were reports that this was actually a regression. If you go way back, and we mean way back, to when Dolphin first went open source in 2008, rumble actually worked in these titles. Annoyed that Luigi's Mansion had no rumble, spycrab0 took a step away from his GUI work in order to finally solve this issue. After all, what's the use of a good GUI if the games he wanted to play had annoying bugs! After an epic bisect that went back all the way to 2008, he found this line.

if (type == 1 && strength > 2)

The condition looks reasonable, which is how it managed to sneak by all these years. It turns out Strength isn't talking about rumble strength or anything related to rumble at all! Rather, it relates to controller analog mode for whatever reason. Thus, in the title that use an analog mode that is less than two, rumble would never function. Removing this erroneous check restores rumble without causing any issues, making developers wonder exactly why this check was ever added in the first place. Whoops.

5.0-6907 - Emulate IOS File System Timings by leoetlino

The reason why this change was necessary was ridiculously silly. The MLB Power Pros series had an issue in Dolphin where their banner wouldn't save due to a race condition that passed on console, but failed on Dolphin. This condition was actually fairly problematic to figure out. Blindly slowing down the timings didn't fix the issue and we were already too fast compared to the Wii, so it seemed like there may not be a happy medium in emulation. Despite the initial roadblock, leoetlino made a test suite that could be run on Wii consoles to get accurate timings for many file write and read commands. Using this data, leoetlino modified Dolphin's IOS File System Timings to match that of a top condition Wii. Even if it didn't fix MLB Power Pros, it's always good to have correct timings. In this case? Only when given accurate timings does MLB Power Pros manage to save its banner in Dolphin.

Before the timing fixes, the banner really wasn't getting saved. Props to Nintendo for handling this case in the system menu, though!
Console correctly passes the race condition. Transferring the console save to Dolphin would cause the banner to show up in both Wii System Menu and gamelist.
All of this work was spurred by the fact that MLB Power Pro's banner didn't save!

The work that leoetlino did resulted in a lot of games having their timings changed and a few ancient hacks getting removed from the emulator. An ancient hack was embedded into Dolphin's boot code to help the System Menu boot because read/write cache timings didn't exist at all. Without them, every single 4 byte write done by the System Menu would take an eternity to complete compared to on the actual Wii. In general, though, reads and writes will complete a bit slower than they did before. The most noticeable case of this is in Super Smash Bros. Brawl, where the initial NAND check now takes about as long as console instead of completing several seconds faster. While slower read/write times probably sound like a downer, users have already started to report some very important benefits of being correct.

Silent Hill: Shattered Memories - A Problem Child

Silent Hill: Shattered Memories is one of those games that seems to keep popping up with weird issues. It hates being rendered at higher resolutions, it uses EFB Copies to detect how much snow is on screen, and it even managed to hit a weird bottleneck that we were forced to optimize because it was just that slow.

With all of these issues sorted out, the main game seemed to run well enough... until you got further in. Another of the promiment features in Silent Hill: Shattered Memories is a cell-phone you unlock very early on. And, much like modern cell-phones, you can use this phone to take pictures and save them to the Wii NAND. While most of the time everything would work fine, there was a chance that the game would hang when saving or loading photos. While you could simply avoid taking pictures most of the game, a puzzle later on required taking and saving multiple photos. In Dolphin? This could become an exercise in frustration as there was a very real chance of hanging when spending any meaningful amount of time in the game's photo gallery.

We hope you like this screen, because it could end up being the last thing you see!

Users returning to play Silent Hill: Shattered Memories after this build need fear not any longer. With proper IOS File Read/Write Timings, these hangs are no more and the puzzle can be completed without needing to rely on savestates.

5.0-7038 - Remove Hybrid Wii Remote by JosJuice

This month, the honorary role of feature executioner has been played by JosJuice. Wherever he is, we know neobrain nodded in approval.

Hybrid Wii Remotes have always been something of a troubled feature. The idea of combining other control methods with Wii Remotes and its extensions (such as the Nunchuk) is a promising idea, one that could lead to better ways to play games that had sometimes problematic controls on the Wii. Plus not everyone has easy access to a sensor bar, so wouldn't it be great if you could use a mouse or joystick in games where the pointer is only needed for menus? That's just one situation, but there are others where working Hybrid Wii Remotes would be amazing.

Too many users tried to use Hybrid Wii Remotes expecting working behaviors like the one above. Instead, they encountered random disconnects, the feature suddenly refusing to accept any input, failure to detect extensions or Dolphin crashes when connecting one, and working configurations just randomly not working with no explanation. Hybrid Wii Remotes even blocked other Wii Remote changes due to the weird ways it hooked into the Emulated Wii Remote code!

Hybrid Wii Remotes was an unstable mess, one that was so bad devs wouldn't dare touch it. After years of neglect, it was decided that in order for good Hybrid Wii Remotes to ever happen, the first step was to rip out the old Hybrid Wii Remote code.

And so it was done.

With the first step of fixing Hybrid Wii Remotes now complete, we're hopeful that the feature will rise from the ashes in a better form.

5.0-7088 - Enable Auto-Updater by Default by spycrab0 and delroth

There have been hundreds of Qt changes over the month and we'll be doing a special feature (not a story, we promise) at the end of this Progress Report highlighting all of the changes. This one, though, has been requested for years, so, we've given it is own progress report entry.

The Auto-Updater has been embedded into Dolphin since late march, when delroth pushed the initial version of the auto-updater and spycrab0 helped with the GUI interface for it. This feature is only in the Qt GUI which was made the default on April 23rd, so users stuck on DolphinWx.exe will not have these options. Now that Windows version of this feature is fully live, let's take some time to talk about how it works.

Currently, there are three update tracks you can follow.

  • Stable - If you are ok with us only releasing stable builds roughly once in a blue moon, this is the track to follow. Don't expect any miracles, though!
  • Beta - Sync up your update with the Progress Reports. We do some testing prior to Progress Reports just as part of making them, so by choosing this track you'll get a reasonably frequent update schedule and you shouldn't encounter any catastrophic issues. Probably.
  • Dev - Every time you open Dolphin, a summary of the new changes will popup and you can choose to update to the latest development build!
Update controls are in Config > General.
While Dev builds come with auto-generated summaries, Beta updates come with hand written updates that point to the latest Progress Report!

If you don't want to deal with auto-updates, this feature can be fully disabled in the main configuration menu. The help menu also gives you the option to manually update if you would prefer to keep auto-update off while being able to update at your leisure. We hope this allows users to follow development more easily going forward. Enjoy!

5.0-7151 - Implement Copy Filter (Deflicker/Brightness) by stenzek

Emulating the GameCube/Wii Copy Filter was seen for years as a silly luxury feature that no one would really want to use. There were two major features that we knew used the copy filter before. The first we'll get into is gamma adjustments. A user could adjust their monitor or graphics drivers settings to get the desired effect that gamma would provide. Secondly was the deflicker filter, which was seen as even more pointless than gamma. Deflickering reduced jitter on interlaced video by essentially blurring the image vertically to smooth the interlaced lines. Dolphin has no support for interlaced output in the first place, so having the deflicker filter would only make things needlessly blurry.

However... emulating the copy filter was the oldest remaining open bug on Dolphin's issue tracker, so it kept getting attention. With Dolphin being a much more complete emulator nearly a decade after the issue was opened, it seemed to be about time to put the issue to rest. And with "Disable Copy Filter" enabled by default, the blurring will not happen unless you want it to.

At 1x native with deflicker off, the game is as sharp as it has always been.
But now if you want to make Melee needlessly blurry, you can!
The higher the internal resolution, the less deflicker will influence the image.

Despite some initial skepticism toward the feature being useful, it turns out there are a lot of interesting quirks and uses of the copy filter. Upon hearing plans to emulate the copy filter, one user was passionate enough to investigate and write up a blog post with excellent demonstrations about uses and some curious decisions some games made regarding deflickering. Thanks to PPLToast's article, we have a few more examples than even we knew about!

Examples of the Copy Filter in Action

Fades and Transitions

Star Wars Rogue Squadron III: Rebel Strike and Soul Calibur II are two titles we know use the copy filter in order to fade out the screen. These games more or less accomplish this by blurring lines with an increasing amount of black rather than the line below it.

Rogue Squadron III is particularly fun with this issue because, as seen in the video above, some camera work and loading is done while the screen is blacked out by this feature.

Gamma

Tons of games use the copy filter for gamma and we can't list them all. A few notable examples include the Metroid Prime games and the Resident Evil series. In these games and others you can change how much black the pixels on the screen are blended with.

Deflickering

Pretty much every game deflickers itself on interlaced output and very few games actually give you the option to disable it. When outputting in progressive scan, games will disable the deflickering while leaving other behaviors of the copy filter unaffected assuming the developers are sane. Unfortunately, mistakes can happen, and in the case of Twilight Princess's PAL version, you ended up with a game that even had the deflicker filter on in progressive mode, making the game much blurrier than it's NTSC counterpart.

While some games give the option to disable deflickering even when they think they're outputting interlaced, most games do not. As such, the Disable Copy Filter option is on by default, so there is no chance that games will become needlessly blurry without user interaction. However that option is lying a little bit, since it doesn't entirely disabling the copy filter - fade and gamma effects will still work even with this option checked. So you can leave this option checked and most copy filter effects (aside from deflicker) will still work.

5.0-7505 - Fix Issues with JIT Paired Singles Instruction in Full-MMU Titles by booto

When there are regressions that break a game's ability to run consistently, there's always the danger of other things breaking while that game is broken. In the case of Star Wars Rogue Squadron III: Rebel Strike, it hasn't been consistently booting since shortly after 5.0 as the very basic GPU timings added to single core cause the game to freeze soon after booting. Given that Dolphin's goals are to run every game, no one is thrilled that the game is unplayable, and there have since been efforts to figure out what's breaking RS3.

A WIP branch that modifies how FIFO interrupts are handled has shown some promise, but after getting RS3 to boot again testers were greeted by an even more frightening proposition - the game was randomly crashing. Thankfully, Dolphin itself wasn't crashing, so the game was able to print out its register dumps into the logs. From there, booto was able to reconstruct what was happening when the game crashed. By reading the game's assembly, setting breakpoints, and figuring out what the game was doing, he was finally able to lock down the mistake. A quick hack allowed the crash to be bypassed, and a few hours later he figured out why it was crashing.

It turns out that Dolphin's emulation of two GameCube quirks were combining to crash Rogue Leader!

The DSP's ARAM chip.
Image Credit: Anandtech

The first quirk is a technique that Factor 5 borrowed from operating systems. When Windows runs out of memory, it maps a page file to the local HDD or SSD and uses it as essentially very slow "extra ram". The page file isn't actually memory, as the CPU can't see it as memory, but by swapping data back and forth between the page file in storage and system memory, the CPU is able to utilize the page file as "extra ram" in emergencies. Factor 5, naturally, wanted more memory than the 24MB of system memory the GameCube has, so used this technique to tap into the DSP's 16MB of ARAM.

Here's how it works on the GameCube. Rogue Squadron allocates memory and gets a virtual memory address back. If the game mapped more virtual memory than there is physical memory, when the game tries to use it, the CPU faults and sends a data storage interrupt (DSI). At this point, Rogue Squadron's custom memory allocator catches the DSI and finds a suitable location in main memory, and swaps that with whatever is in the DSP's ARAM. The game can then overwrite the bit of main memory with contents that were in ARAM, and now there is unique game data in both sides of the swap, and the game has "more ram". With the MMU updated to bypass the CPU faults, the memory allocator can swap back and forth to let the game access either memory address.

The GameCube's CPU. Credit: Wikimedia / CC by-SA 4.0

The second quirk is in how the PowerPC processor of the GC handles SIMD (Single Instruction, Multiple Data). Or to be more accurate, how it doesn't. SIMD places two floating point values into one register, allowing two math operations to be performed with only one instruction. For example, two 32-bit values can be placed into one 64-bit register, and both edited with a Paired Single instruction. This allows for more math per cycle, and thus, a faster, more efficient processor. Except the PowerPC 7 line of CPUs that the GameCube and Wii's CPU is based on don't support SIMD at all; that functionality only came to these processors with PowerPC 9. But apparently Nintendo asked for some form of SIMD during the GameCube's development, and IBM hacked together a solution.

The GameCube's CPU cannot place two 32-bit values into one 64-bit register, so to allow Paired Singles to work, IBM instead uses two registers. So one 32-bit value is stored in one 32-bit register, and one 32-bit value is stored in a 64-bit register. Modern CPUs have proper SIMD, but they can't support the GameCube's weird pseudo SIMD. So to emulate this, Dolphin has helper routines prepared ahead of time to map these to instructions the host CPU can run. With the type conversion that can occur, this can bloat out to many instructions, so the JIT has an optimization to pre-compile these paired single helper routines ahead of time.

That optimization, combined with the trick to get more memory, is what broke Rebel Strike. The helper routines are generated before the game boots, and they have baked in assumptions on how the memory will be handled. So when the game's memory allocator detected a DSI and tried to set up the swap to borrow the DSP ARAM, the helper functions would feed it incorrect values, and the game would crash. booto rectified the issue by making sure that when a DSI occurs, those default values are stripped away.

Now you might be thinking, "how did the game ever work at all?" Well, previously, whenever the MMU was used, Paired Singles load/stores would fall back to the interpreter by default, avoiding this issue. But at some point, someone assumed it would work in the JIT and removed the interpreter fallback, but Rebel Strike was no longer booting at that time so no one could know that it was now crashing. Whoops.

Either way, the end result is that a crash is fixed before most users will even get the chance to encounter it!

One day, Rogue Squadron III will return to the realm of the playable.

Unfortunately, we're still a ways off until FIFO Interrupts are fixed up to the point that Rogue Squadron III will be playable again. The game does not get past the main menus in Dolphin master at this time.

5.0-7765 - Implement WASAPI with Exclusive Mode by spycrab0

Audio and video latency have become a bit of a priority in recent years. Since the Dolphin 4.0 release, latency has decreased dramatically in Dolphin as features have been added to help get both audio and video to the player faster. In games like Mario Golf: Toadstool Tour, timing is everything, and it's really hard to go back to older builds because of how bad latency was before features like exclusive fullscreen. Since then, audio and video have seen dedicated features for lowering latency further, such as Immediately Present XFB copies on the video end. On the audio end, things have improved in much subtler ways. Features like TimingVariance allow users to sacrifice performance in order to lower audio latency.

Unfortunately, despite our best efforts, audio latency has lagged behind video latency at this point. Most of the problem isn't in Dolphin, but in getting the audio to the users. For Windows users, we now have a solution to this problem with WASAPI Exclusive Mode. This output mode on Windows is designed specifically for low latency, and in tests showed to make an extremely large difference without having to modify TimingVariance!

On a fast enough computer with TimingVariance set low, it's possible that Dolphin can match console audio latency! The downside to exclusive mode audio output is that no other applications will be able to push audio while it's in use. Please keep that in mind before you use the WASAPI audio backend. Another thing to keep in mind is that, on Windows, Cubeb also uses the WASAPI interface. In theory, Cubeb could be just as low latency! Unfortunately, it doesn't support exclusive mode yet. If it adds support in the future, we'll likely drop the standalone WASAPI backend and just add a setting to the Cubeb backend for exclusive mode.

5.0-7783 - AArch64 Vertex Loader - Fix 565 Color Format by degasus

When debugging issues on Android devices, it can be very easy to fall into the trap of always blaming graphical issues on the drivers, especially those that don't show up on desktop! In this case, the issue originally reported was that the Smash Ball aura from Super Smash Bros. Brawl was completely missing on an Adreno device. Given the fact that the aura effect works fine on desktop and Adreno devices are sketchy on tons of EFB effects, the issue was initially brushed off.

The smash ball's lack of glow effects certainly looks like a driver bug at first glance.

It wasn't until another tester connected a missing graphics bug to an old Vertex Loader JIT bug that the issue was more interesting. And, this time the bug wasn't reproduced on adreno, but rather the NVIDIA Shield T.V., which featured the same NVIDIA drivers we use on desktops. With that information on hand, the tester was able to narrow down both bugs to being related to the AArch64 version of the Vertex Loader JIT!

"Open Concept"
With the bug fixed, the barn raising is done!

degasus easily tracked down the bug, which turned out to be a similar bug to the one that once broke the very same games on the x86-64 Vertex Loader JIT! With a simple copy-paste failure fixed, both games, and likely others, now render correctly on AArch64 devices.

The Hundreds of GUI Updates for DolphinQt

The changeover to Qt has come with a thundrous aftershock. Hundreds of changes have hit the new front-end in just a month as missing features are added, existing ones are fleshed out, and bugs are stomped out. While spycrab0 has led the charge with Qt additions, we've also seen contributions from aldelaro, sepalani and others. A wide variety of issues have been worked on by people as we attempt to renovate our GUI.

New Features

The biggest new feature to Qt that didn't exist when the change-over started is exclusive fullscreen mode. This fullscreen mode is essential to get low latency, sometimes shaving off up to three frames of input latency in Dolphin! It was actually a bit of a surprise that exclusive fullscreen wasn't working in the initial merge as initial testing said otherwise due to an erroneous result. Once spycrab0 realized it was missing, it only took him a few days to get it implemented. While there were some growing pains, exclusive fullscreen should be working properly in latest master. Custom themes on the other hand are purely cosmetic, but can look quite nice when the theme is designed by the right hands.

Custom styles were also added over the month. We could talk more about it, but a picture is worth a thousand words.

RPCS3's styles can even be used on Dolphin with minor modifications. Click here for our user style forum thread!

Some other parts of Dolphin saw some notable improvements. The netplay menu has also been improved with a cleaner look and customizable boxes to give users more control over the size of the chat window. Dolphin's debug GUI has also seen a lot of improvements and is fairly usable in latest master. While this may not be big news for most users, it being finished is an important part of being able to completely drop DolphinWx. The same can be said about the FIFO Analyzer, which is another tool used for debugging. This tool was initially thought to be unneeded in DolphinQt, but, when developers ended up needing to go back to DolphinWx for testing, spycrab0 was pressed into porting it over.

Another long requested feature debuting in Qt this month is the All Devices option in the Controller Configuration Menu. This allows users to easily configure multiple devices to the same configured controller. If you're playing a Wii game and want the mouse to control the Infrared (pointer) and a controller to do everything else, you can now easily configure it!

Everyone who has ever mapped multiple devices to one emulated controller is joyfully screaming right now.

Bug Fixes

There've also been a lot of bug fixes to Qt that don't have to do with new features. The most dangerous of these issues is a bug with the Memory Card Manager implementation that could potentially corrupt a memory card. We highly recommend updating to the latest builds if you're planning on using the DolphinQt Memory Card Manager. The addition of exclusive fullscreen lead to a lot of issues transitioning between fullscreen and windowed mode, which were promptly introduced and fixed during the course of the month.

On top of those major fixes were a lot of minor issue fixes. DolphinQt would sometimes not update settings until after you closed Dolphin, resulting in some frustration. The same is said for turning on a feature, meaning that it was possible for the GUI to show one setting when you're actually running another! While these issues have mostly been fixed, if you run into any issues with settings not taking effect at the proper time, please report this on the issue tracker or forums.

While DolphinQt hasn't radically changed in look since its merger, tons of features, small additions, and bug fixes have made it much easier to use. As time goes on, we hope that Qt continues to improve as it evolves far beyond what wxWidgets was ever capable of!

Last Month's Contributors...

Special thanks to all of the contributors that incremented Dolphin from 5.0-6805 through to 5.0-7953!


Pues siguir el discutiniu nel filu d'esti artículu, nel foru.

Entrada siguiente

Entrada postrera

Entraes paecíes