July and August have been busy on the blog. On top of working on this Mega Progress Report, two feature length articles had to be made after developers selfishly merged several huge, highly anticipated features without considering that it makes work for us here at the blog! Well, the extra work-load is well worth having the incredible Ubershaders and support for Dragon Quest X!
But even beyond those two massive articles, we've been hard at work keeping up with everything else going on because this is going to be a big one. Adding to that someone on the blog staff thought this month of all months would be a good idea to try and harness the power of the blog to get extra testing on some old issues. So buckle up, this is going to be a long read!
Notable Changes¶
5.0-4647 - Add IOS 4v3 Constants by AwesomeMarioFan¶
Accuracy is always good, but sometimes inaccuracy leads to a more desirable behavior. In the case of titles that used the oldest known IOS for the Wii, they worked fine on Dolphin because IOSv9's constants were close enough that the titles did not have problems. After merging a bunch of IOS-HLE fixes, including using different constants for various IOSes to fix games like Majora's Mask (VC), the Wii Startup Disc stopped functioning.
The reason for this is simple: because we did not have access to IOS 4v3, we had no entry for it in IOS-HLE. Thankfully, someone with access to the NAND created by the Wii Startup Disc showed up, disassembled the IOS, and got the proper values to add to our IOS-HLE implementation. While users may not be clamoring to run the Wii Startup Disc in Dolphin, it's nice to have this piece of Wii history functional for all time.
Correction: earlier version of this article stated that the person who implemented the IOS 4v3 constants had access to the Wii Startup Disc. That was incorrect - they only had access to a NAND requiring it.
5.0-4692 - Remove Hardcoded File System Stuff by leoetlino¶
If you're like many users and have used Dolphin across the years with many Wii titles, you may have updated to latest master recently and been greeted by this annoying little message.
What gives? Well, it turns out that Dolphin was incorrectly calculating how much storage space was left. This was causing a plethora of bugs, including one that showed up when installing titles from the Wii Shop! When installing titles from the Wii Shop, it would correctly calculate how much space was left before the download, but after the download, it would only take up one block!
It turns out Dolphin's nonsensical FS code was straight up copied from SNEEK. There was no obvious reasoning to return the incorrect block information that we can see outside of it being used to get around FS limits within SNEEK. For an emulator, this behavior is bad, and has been causing hangs along with invalid NAND issues since the beginning of time.
Fixing this may seem like a bad thing - your Dolphin NAND is now constrained to the same limits as a Wii. Some users may have to set up multiple NANDs to store all their saves or channels, but this is a necessary step going forward. Fixes like these throughout the years have greatly increased our compatibility with games, and helps us when implementing new features down the road. Besides, we're implementing every accuracy thing we can in hopes of getting Red Steel working someday. Maybe we can finally get past our nemesis: the lady in the red dress!
As an aside, isn't it funny that games designed to not boot in Dolphin (Disney Infinity/Cars 2) and even other Ubisoft games loaded with anti-piracy (The Adventures of Tintin: The Game, The Smurfs: Dance Party) all run in Dolphin, while this odd launch title doesn't?
5.0-4703 - Basic TMEM Support by mimimi¶
Dolphin has been around a very long time, so you may be surprised that until this point, it's been able to completely ignore GameCube/Wii Texture Memory emulation without consequences. TMEM is the GameCube/Wii's own cache for textures and features eight slots for textures. For the most part, games don't rely on any specific behavior of TMEM, so Dolphin was able to pretend it doesn't exist without issues.
...Until we ran into a particular issue in Spyro: A Hero's Tail. For some reason, the bloom was glitching out in strange ways.
After some investigation, phire discovered it was relying on TMEM caching in order to render the bloom! It was creating an EFB copy, and then creating a second EFB copy, overwriting the first. But it didn't invalidate the texture cache, so it was able to pull out the first EFB copy, then invalidate the cache, and finally use the second EFB copy. While phire did a lot of the legwork to get implement the feature roughly two years ago, other projects and some complications caused the feature to be put on the backburner.
Earlier this year, mimimi revived the effort, fixed the remaining bugs. While it is pretty obvious that emulating TMEM is more accurate, it was discovered that emulating TMEM was also a huge performance boost when it comes to hashing textures, especially in safe texture cache titles. Oddly enough this is because we can skip hashing textures that we're certain are in the GameCube/Wii TMEM.
5.0-4869 - Ubershaders 2.0 by stenzek¶
The big one. Ubershaders is one of the most, if not the most anticipated feature for Dolphin merged in years.
Everyone hates shader compilation stuttering, but, solving it just didn't seem possible. The GameCube/Wii doesn't use shaders and needs no warning before switching its proto-programmable pipeline to new configurations. We went through our feature article on why it was so hard, and all of the secondary solutions that mend but do not completely solve the issue. The end-game came down to that it just took too long to compile shaders, and we had no real way of knowing what shaders we'd need right up until the game decided to use a particular configuration. How do you completely solve a situation like this?
Ubershaders is as insane as it sounds. It implements an interpreter for the GameCube/Wii rendering pipeline within shaders, meaning that these ubershaders are powerful enough to emulate the whole GameCube/Wii proto-programmable pipeline without the need to generate new shaders for new configurations.
The downside to Ubershaders is that they are absolutely massive. While modern gaming GPUs are powerful enough to crunch Ubershaders, they can still bring a GTX 1080Ti to its knees at 5x native internal resolution or higher. Considering almost half of Dolphin's users are on integrated graphics and struggle to reach 2x IR without Ubershaders, we didn't see this as a full solution without a path to help them.
Enter Hybrid Mode Ubershaders. Instead of trying to always run these gigantic Ubershaders on these weak integrated cards, we only use them in place of shaders that aren't generated yet. This means that the Ubershaders are usually only running for fractions of frames, so even an integrated card can get the full benefits of Ubershaders. This also applies to everyone else - a GTX 760 may only get 2x IR on Exclusive Mode Ubershaders, but, with Hybrid Mode Ubershaders, performance is identical to that of Ubershaders being off!
For more information on Ubershaders and how they work, feel free to check out our feature article on the subject.
5.0-4894 - Support loading BC7 (BPTC) textures from DDS files by stenzek¶
For a while now, custom texture creators have had to make a choice between PNG and S3 texture compression (S3TC, in the DDS file container).
PNG offered lossless, exceptionally high quality textures, but PNGs have to be decompressed into 8-bit per channel bitmaps (essentially bmp files) before they can be loaded onto the GPU. This consumes obscene amounts of system memory, as high as 16GB of RAM in some texture packs! If users didn't have enough RAM to hold the decompressed textures, they won't be able to prefetch custom textures in order to prevent stuttering upon loading and decompressing textures.
Custom texture creators could also choose S3TC, which didn't have to be decompressed since S3TC is supported by GPUs directly. While this allowed everyone to prefetch custom textures ...its quality left something to be desired.
This split in custom texture support is frustrating and annoying, especially when the primary use for custom textures is to get higher quality textures into your favorite games. Now, they have a new choice! BPTC, a newer GPU supported compression format (also using DDS as a container), offers the best of both worlds. It is able to run on the GPU so it doesn't have to be decompressed, but it is also a high quality format. So now you can have nice quality textures with prefetching to avoid stuttering, all without needing ridiculous amounts of RAM!
The only downside to BPTC is that it requires D3D11 and/or OpenGL4 compatible GPUs. Specifically, that is the Nvidia GTX 400 series and newer, Intel 7th generation graphics and newer (depending on drivers), and AMD Radeon HD 5000 series and newer. Those GPUs are five to seven years old at this point, so it probably won't be a problem for anyone on desktop PCs. As for Android... Mercifully, our Android UIs don't even expose custom texture support. One bullet dodged.
Because BPTC and the older S3TC both use DDS files as their container, don't be alarmed if you see new texture packs still using ".dds" files. Just because you see DDS doesn't mean you're sacrificing quality going forward! Look for BPTC or BC7 in texture pack descriptions going forward.
5.0-4950 - Extracted Disc Rewrite by JosJuice¶
When making mods, wouldn't be nice if you could just extract the ISO into a folder a loose files and load it into Dolphin? Dolphin has had that ability for a long time, but it was a bit limited. After nearly falling into disrepair, JosJuice has gone through and done a long needed rewrite to extracted ISO support that makes it usable for almost every game! With these changes, Extracted ISOs:
- Now show up in the Gamelist
- Support GameINIs
- Correctly detect region information
- Have higher compatibility
- No longer need manually set file and apploader paths
The only thing users need to do to get all of these new features is re-extract their ISOs with the newest build of Dolphin. Part of the problem is the old extracted discs won't even have all the information needed to work with the new features.
5.0-4896 - Config: Reload game INI on title change by leoetlino¶
On the GameCube, it's very, very rare to boot multiple games without turning the console off and on. Unless you're using some kind of loader, you're pretty much locked to playing one game until you turn off the console. So, it was a fair enough assumption for Dolphin developers to assume that users would only be playing one game when launching Dolphin back in the day.
In comes the Wii, with all kinds of extra features that now make it possible to boot many games without ever turning off the console. Recently, we began to emulate the Wii's ability to track titles, meaning that Dolphin is aware of the GameID of the currently launched game. While we originally tackled this because it was required for some Wii functionality, it also has given us the ability to improve the user experience greatly when using Dolphin more like a Wii.
Dolphin can now automatically set the correct settings for a game, even when it is not the original title launched. This is incredibly important for Wii games as the Wii Menu enforces Safe Texture Cache which is fairly intensive but not always needed. Now, Dolphin will have it enabled for the System Menu, but when you launch a game like New Super Mario Bros. Wii, it will know you changed titles and no longer force Safe Texture Cache on, and revert to your default setting!
While this sounds great, the feature isn't foolproof. Some settings can't be set during execution. MMU games will still manually need MMU enabled before you start up emulation, DSP-HLE/LLE cannot currently be swapped (though, this may be possible to handle in the future.) Also, any settings not swapped over to the new config system will not attempt to update on title change.
Possibly the coolest use of this is in Super Smash Bros. Brawl, where the Masterpieces have unique GameIDs, and thus we were able to enforce the settings necessary to run them correctly without dropping the performance in main game. So you can finally enjoy your 30 second demo of Donkey Kong guilt free.
5.0-4982 - Send rumble data to the adapter on a separate thread by HannesMann¶
Dolphin boasts extremely low latency controller input thanks to USB Passthrough support. But, passthrough can sometimes have drawbacks. Some very adept players noticed that rumble output was causing framelag in Super Smash Bros. Melee. It turns out that a flaw in the way Dolphin was sending rumble on USB Passthrough mode was able to cause delays if rumble commands were sent too rapidly.
Now rumble is sent out on a separate thread, preventing the lag. This actually affects any game that uses rumble on a GameCube controller, so if you're using USB Passthrough and noticed weird lag at the oddest of times, this could be your magic bullet.
5.0-4968 - Synchronize Wii Remote Disconnects/Reconnects on Netplay by mimimi¶
One of the most entertaining things about playing Dokapon Kingdom on Netplay with GameCube controllers is that it requires a Wii Remote connected to get through the main menu. Once you're in-game, it could just disconnect harmlessly. But if you weren't able to complete setup in the 5 minutes it takes for a Wii Remote to time out, Netplay would hang because Wii Remote disconnects were not synced on Netplay. When the player attempted to reconnected it, it'd only be reconnected on one instance. breaking the Netplay session.
Previous attempts to synchronize Wii Remote disconnects/reconnects on Netplay had failed to this point, even though this code has existed for quite a while. It turns out that reconnecting Wii Remotes was running through the UI thread of all things, which was not deterministic. After fixing that, reconnecting Wii Remotes on Netplay worked like a charm. While this helps in Dokapon Kingdom for setting up, it also helps in any other game where you run into significant downtime. It's even possible to reconnect Wii Remotes in a different order to switch player order in some games!
5.0-4986 - Fix Time in Forecast Channel by AwesomeMarioFan¶
So if you thought fixing the Wii Shop Channel was niche, just wait until you try out the Wii Forecast Channel in the latest builds of Dolphin! That's right, thanks to AwesomeMarioFan the Wii Forecast Channel now functions in Dolphin.
The problem was that there were poor assumptions made in how the Wii calculated the time, resulting in the channel rejecting an invalid timestamp. After a hardware test, we confirmed our prior behavior was incorrect and now use the typical UNIX_EPOCH instead. This appears to also help out the Wii News Channel a bit, but that still does not work correctly.
5.0-5044 - Add support for SDHC by TBoshoven¶
As more of the Wii NAND is emulated more accurately, space is actually becoming a concern. The days of installing an infinite number of channels without adverse effects is over, so this change comes just in time to help ease people into this new, more accurate world. Supporting the SDHC protocol allows Dolphin to use up to 32GB SD cards, rather than the previous limit of 2GB.
Note that because Nintendo made some poor decisions, using a 32GB SD Card will cause noticeable delays when accessing it in the SD card menu, just like on real Wii. Nintendo designed the Wii to read SD cards block by block, which may have been reasonable on their recommended 128MB cards in 2006, but on a 32GB card it can take minutes in Dolphin! It does take quite on a while on a Wii with bigger cards, but we are aware that SD card reading appears to be even slower on Dolphin currently. This shouldn't cause any major issues, but, some Super Smash Bros. Brawl mods, especially those with streaming audio from SD card, may malfunction.
By default Dolphin only creates a 128MB file on your hard drive. To make a larger virtual SD Card and use SDHC support, you'll need to use an external program to create a blank file then format at it as a FAT32 partition.
5.0-5097 - Remove Fractional Internal Resolutions by JosJuice¶
This is something we've wanted to do for a long time, but knew there would be some blowback from users. Let's get this out of the way: higher internal resolutions are a hack, a hack we gladly try to support. But, we also must take into consideration keeping Dolphin as stable as possible while providing the features that users want. In the case of 1.5x and 2.5x internal resolution, it's been on the chopping block multiple times. This time, an issue so damaging turned up that we really had no choice but to remove it. But why do these internal resolutions cause problems?
The problem with non-integer resolutions is that there's a lot more rounding involved. 1x, to 2x, to 3x is a lot simpler than 1x to 1.5x, or 2.5x internal resolution. Using a fractional internal resolution, like 1.5x, means that Dolphin cannot cleanly multiply the native resolution. With Auto (Window Size) you could even get different horizontal and vertical internal resolutions - it was a mess. The GameCube and Wii can read and write directly to their framebuffers; so a rounding error could cause visual errors or even crashes! In the case of Paper Mario: The Thousand Year Door, it'd do something more insidious.
During certain effects/transitions, you'd get invalid reads/writes and unknown pointers, but with the default settings the game would seemingly go on as per normal. You'd save, close your game, go back to continue tomorrow and your save doesn't exist! What happened? By using fractional internal resolutions, the game managed to write into memory it shouldn't have and overwrote the GameID. Your save is safe and sound, but associated with a GameID that doesn't match!
While an experienced dev could fix this, most users would be forced to restart or continue from their last backup. But then again experienced developers may not even get the opportunity as they'd probably be using full MMU emulation. Under that, the GameCube isn't allowed to write just anywhere, and instead of allowing it to corrupt memory, the game will crash. Considering how bad memory corruption can be, crashing is definitely the behavior we'd prefer here.
At this point it was obvious that fractional internal resolutions should be removed, but the reason we hadn't removed this feature years ago made us hesitate once again - users with very weak GPUs, such as older Intel iGPUs, use non-integer IRs to maximize their resolution. A very large portion of our userbase uses Intel graphics, so we were quite concerned about how would be inconvenienced if we removed this feature. Turning to our optional usage statistics reporting data, we were able to determine that a large majority of users using fractional internal resolutions were more than powerful enough to go up to the next integer resolution. Only 1.38% of our users (and 8% of the people using the problematic settings) may have to drop down to the nearest integer internal resolution instead. Considering fractional internal resolutions can literally corrupt your savedata and cause crashes, we felt it wasn't asking too much for some users to take a slight dip in visual quality.
Surely enough, there were some complaints once we removed this feature. Apparently the difference between 2.5x IR and 2x IR is enough that the graphics are too bad to play for some users. For people wishing to needlessly expose themselves to dangerous behaviors, we host older versions of the emulator on our download page!
5.0-5179 - Add support for installing disc updates from the game list by leoetlino¶
In some cases, having Wii Updates are required now for some functionality within Dolphin. While this mostly is for optional things such as using the Wii System Menu for now, it's nice to be able to install the updates without having to go through any complicated steps. While we provide users in the tools menu the ability to create a fresh, updated NAND along with all IOSes from Nintendo's Update Servers directly, there's no guarantee those servers will be around forever.
While we've been able to install Wii Updates from the System Menu for a while, what happens if you don't have a System Menu or can't connect to NUS? Now all you need to do is right click a disc, and install Wii System Update! This also gives you access to older versions of the System Menu that may not be easily accessible in modern times without manually searching NUS.
This functionality may have even more importance in the future. While Dolphin employs a hack to allow users to boot games without updates as long as they're booting them directly from the gamelist, there's no guarantee that hack will be maintained forever. If some game, feature, or update breaks the hack, there's no guarantee the team will be inclined to maintain it or make a new one.
5.0-5205 - Ignore capabilities reported by an XInput device by ToadKing¶
Years ago when Xinput support was first implemented in Dolphin, Dolphin tried to do things correctly and implemented the full specifications of this input library. However, it turns out that a majority of programs that use XInput take a bit of a shortcut and ignore non-essential parts of the spec. Naturally, this laziness has dripped down toward manufacturers, as many controllers are now improperly implementing the specification because the games they use to test aren't enforcing it correctly.
Specifically, within the specifications is "XInputGetCapabilities", a feature that allows a program to retrieve the functionality of a controller. Dolphin implemented this because not only was it in the spec, but it could be used for controllers with additional functionality, like the PS4 controller. Unfortunately, no one else seems to be using this and it's fallen into disrepair. Some controllers report corrupt values when this function is used, leading to buttons not working and even crashes!
Fortunately the solution to this was really easy. All we had to do was just ignore the capabilities a controller reports and treat every XInput device as a 360 controller, just like every other program does. Unfortunately this means that the PS4 controller's touchpad and motion controls will continue to require custom drivers and complex button mapping software - unless you are on Linux of course!
5.0-5261 - mtmsr PIE fix for Resident Evil by JMC47¶
Sometimes all that needs to be done for an issue is follow directions. While merry cleaned up all the major issues with the JIT64 and PIE, this one managed to slip through in an odd way. PIE support caused the codesize to get bigger in some cases, and mtmsr relied on a jump to be within 127 bytes for a nifty optimization. After PIE, it was hitting exactly 128 bytes in Resident Evil, which meant the optimization for mtmsr had to be disabled.
Thanks to the panicalert (and degasus) more or less giving exact instructions on what to do, even someone with little coding knowledge could fix this issue. This allows the Resident Evil games (particularly the older GameCube ones) to work properly on JIT64 again.
5.0-5291 - WFS updates by delroth¶
The month of August had an unexpected surprise when delroth's addition for WFS support alongside leoetlino's many IOS-HLE fixes did the impossible and got Dolphin onto Dragon Quest X's official MMO servers! delroth and JMC4789 (who of course already had Dragon Quest X ready for this moment,) hopped on to show off this unique Wii game from Dolphin's perspective.
This was accomplished by tons of IOS accuracy fixes alongside support for IOS59's unique feature, Wii File System! Wii File System is a full fledged file system for the Wii that is installed onto a USB Stick for Dragon Quest X. We could go on and on about the many fortuitous events that had to happen for Dolphin to fully support Dragon Quest X before the servers went down, but, we already have a feature length article on the subject.
Let's just say this: supporting Dragon Quest X pushes Dolphin's ability to handle the unique pieces of the Wii to the absolute limit, harder than any other game on the console. There is simply nothing else like it on the Wii.
5.0-5320 - OpenGL: Fix EFB Pokes by stenzek¶
Quick mention on an ubershaders regression. OpenGL pokes are now fixed again. This regression was causing loading screens to not appear in various games, especially throughout the Tony Hawk series. It happened regardless of whether Ubershaders were enabled or not due to a small typo. It is fixed now and things should return to normal in those titles.
Issue Tracking¶
Throughout all of the many changes we've made, quite a few bugs have snuck into the emulators. The issue tracker has become really hard to use due to fairly relaxed standards for submitted bugs. A lot of users ask if they can donate to the project to help make it better, but, money isn't everything. If you want to help make Dolphin better, donate your time, help us track down some of these difficult to figure out bugs so that future versions of the emulator are improved even further.
One of the key ways to make a bug easier to solve is to find out what revision of Dolphin originally broke it. Regressions make up a lot of the current bugs on the tracker and are frustrating for users running into those issues as old builds used to work. By finding out the exact revision that broke it, it greatly narrows down what could cause the bug and oftentimes will lead us right to the culprit.
Helios has worked on a tool to make bisecting painless even on Windows. You input the last known working build and the most recent build tested, and it'll give you the next build to check, allowing to search through tens of thousands of revisions with only dozens of actual builds needing to be tested. If it's not a regression, sometimes going through the huge number of options in Dolphin can narrow down what's going wrong. For example, the ever present Fire Emblem: Radiant Drawn cutscene crash can be avoided by setting the emulated CPU clock to 6%, which heavily points toward that problem being a timing issue. Other common settings that can affect games include MMU, Dual Core vs Single Core, Interpreter vs Cached Interpreter vs JIT, video backend, enabling and disabling cheats, DSP-LLE vs DSP-HLE, and many more. Trying as much as possible, no matter how insane, is how a lot of the scariest bugs in Dolphin have been tracked down.
If all of that fails, it can still be helpful to test the game thoroughly and track down exactly how to trigger the bug. With MMU enabled, if a game crashes, it shouldn't crash Dolphin, and thus the game will spit out it's crash report if logging is enabled. That crash report can sometimes be useful for tracking down what went wrong, and should be included in any issue report that involves a game crashing within Dolphin.
If your testing manages to narrow down some of these issues, you could find yourself and your findings featured in future Progress Reports. Happy testing!
Issue #9875 - Silent Hill: Shattered Memories - Mirrors Bug Out by Chromestalker¶
What is needed: Console Verification
Difficulty: Easy
Someone who is familiar with Silent Hill: Shattered Memories could help close or confirm an old bug regarding mirrors. The Wii was not exactly a powerhouse, so devs often took shortcuts with mirrors that can behave a little oddly. The only question is whether Dolphin is the culprit or the game itself.
Issue #10284 - Zapper: One Wicked Cricket freezes after first stage by Nicholasleejy¶
What is needed: Investigation on a strange game behavior
Difficulty: Hard/Unknown
Zapper: One Wicked Cricket is some kind of Frogger like game for the GameCube. This is more of a task for someone who's looking to really deep dive into the game, as just about every option in Dolphin has been exhausted, even down to the point of taking saves to and from console to try and get past it. Every CPU backend, every GPU Backend, MMU, various audio settings and more have been thrown at the game. It just seems like nothing in Dolphin can affect this.
After the first stage of Zapper: One Wicked Cricket, the game will freeze and spam the logging... with DOLPHINSLOTA... in hexidecimal.
Is Dolphin triggering some anti-piracy? Is it detecting Dolphin as a debug console and stopping things? JMC4789 has taken a look into this issue, and found that the saves from console don't work on emulator and the saves from emulator don't work on console. Because SLOTA likely means the memory card slot, this is likely an issue there, but even when unplugging memory cards it happens. But what could that issue be?
Issue #9499 - Starfox Assault May No Longer Need FPRF by JMC4789¶
What is needed: Investigation of a game
Difficulty: Easy
FPRF is a setting that trades a slight performance down for much better floating point accuracy. But, considering how much performance can be at a premium, it's best to only use it in the games that actually need it. Starfox Assault used to crash at the end of the first stage without this setting on. But, that no longer seems to be true with other accuracy fixes in the JIT.
If you have Starfox Assault, try disabling FPRF from the game properties page for Starfox Assault and report back if it crashes or not. That's all you need to do.
Issue #9144 - JITARM - Can't Pluck Pikmin in Project M by oOSuperMarioOo¶
What is needed: Verification and Possible Bisect
Difficulty: Medium
You don't need to be super technical to test this one, but, it's a bit more than just loading a game and playing it. First thing to do would be to verify that this bug is still happening. Because Project M can be a bit of a pain to load on Android, this could be problematic and may require pushing extra files or using patching a Brawl ISO with Project M, rather than using the SD card method.
If the bug is still happening, things get a bit more complicated and whoever tackles it will likely need special instruction on how to figure out which CPU instruction is causing the bug. Still, we're hopeful that previous CPU fixes have already taken care of this bug.
Note: This issue can only be tested on ARM devices. This mostly means Android phones, tablets, and the NVIDIA Shield T.V.
Last Month's Contributors...¶
Special thanks to all of the contributors that incremented Dolphin from 5.0-4603 through to 5.0-5328!