Index

1. Background
2. Supporting arm64 on our CPU LLVM Recompilers
3. The 16K memory page-size problem
4. Pull requests
5. Challenging the limits of PlayStation 3 emulation
6. Platform showcases
7. Available emulator downloads
8. A note about other mobile platforms
9. Special thanks to the developers

Background

Initial support for arm64 devices started in 2021 after the Apple M1 series launched. We started exploring running rpcs3 on arm64. Thanks to the efforts of RPCS3 core developer Nekotekina, we got a basic running build compiled for arm64 in December of 2021 that wasn’t able to run anything. After some cleanup from kd-11 to get RSX working on arm64 and more enhancements from Nekotekina, we finally got samples running and rendering by Jan 2022. Only PPU/SPU interpreters worked correctly, resulting in very poor performance, but it was the first step to getting RPCS3 running correctly on future arm64 devices. Initial efforts were limited to linux support as it is widely available and accessible to developers without hardware lock-in. A few months later, a contributor Jeff Guo submitted a series of patches bringing the x64 macOS branch up to speed. Some major strides for macOS were made at this stage, such as understanding how the new JIT mechanism works. An attempt was made to get LLVM working but by late 2023 we still only had working builds on arm64 without LLVM support.

Supporting arm64 on our CPU LLVM Recompilers

RPCS3 graphics developer kd-11 had been working behind the scenes on improving arm64 support on Asahi Linux since 2021. Initially, there was no access to graphics drivers so performance did not matter but things kept improving on that front over the years to the point where Asahi could actually be a daily driver. The arm64 LLVM integration work was revisited in late 2023. The initial target was linux as any issues could be tracked down to source level for debugging. kd-11 quickly uncovered the reason LLVM did not work with RPCS3 JIT was due to how the emulator handles guest code. Since RPCS3 was initially written with x86 architectures in mind, some architectural decisions hinged on x86-specific behavior. One of them was that every few blocks, the guest would exit back to the host to handle state checks as well as unwind the stack. RPCS3 uses tail-call-optimized frames for all the JIT blocks but x86 consumes stack during any calls that need to be reclaimed through a ret chain or reloading the stack pointer. This is different than arm64 which uses a dedicated register for “linking” calls. A ret-chain does not exist with stackless frames on arm64. When the JIT code executes the first return, the CPU will enter an infinite loop jumping to it’s current position and hang because the link register is not updated with the previous position.

Naturally, there are plenty of ways around this problem as with all things in computer science. We explored some of these approaches in early 2024 mainly to achieve a proof of concept. The first one was easy – if we provide a minimal stack for every function call and mimic x86 behavior. We keep the frames themselves stackless but track the previous call address and reload it to unwind. Unfortunately, this doesn’t work reliably since the basic blocks assume all stack space is scratch and will clobber the return address every once in a while leading to random crashes. Several other variations of this were attempted including reclaiming some registers for a shadow call stack pointer. They all work, but have different upsides and downsides. For example, some of them required modifying LLVM to accommodate RPCS3’s JIT architecture. Which wasn’t ideal as it required us maintaining a fork of LLVM. Another quick solution is to just replace all returns with a far jump to some host gateway that fixes up the register state and we can exit. This also proved challenging since our JIT blocks can loop between blocks and do recursion. The performance hit became too high for such a strategy to work.

By late February 2024, kd-11 had some samples running with a customized LLVM fork. And made further improvements over March resulting in homebrew running under arm64. After a brief hiatus, kd-11 made a breakthrough, and some commercial titles were able to run on arm64 with PPU LLVM by late June 2024! The proof-of-concept stage was now over. It was time to pick a winning strategy and ship the product.

In the end, kd-11 settled on a IR transformer that analyzes the JIT IR generated for x86-64 and makes tweaks to accommodate arm64’s requirements. The idea is simple – generate IR once for a reference architecture (in this case amd64) and then transform it to target other architectures as required. LLVM makes this kind of work possible since everything is represented as an IR before compilation. LLVM basic blocks have some constraints with regards to flow control and since our JIT blocks are always callable from a gateway, there is an ABI of sorts in place with regards to which registers will hold what value – e.g the guest thread context always lives in the first register passed to the block. This setup was extended to provide extra information at compile time that the transformer can piggy back on to correctly identify the location of objects. We don’t need to store the entire call chain for example, we just need to know the location of the closest host gateway. This approach worked really well and by late July, most commercial games were booting and running fairly well but there were more problems that required addressing.

The 16K memory page-size problem

By default, x86-based operating systems provide page granularity of 4KiB. The PS3 natively also supports 4KiB granularity and does expect that the operating system will correctly align memory allocations as such. However, most arm64 platforms today come with a minimum page granularity of 16KiB. This usually means better memory performance as modern era devices use more memory with larger objects.

Support for 16K pages was added to the emulator in 2021 and was fairly well tested before using interpreters. Most games don’t notice that the page sizes are 4x larger than what they requested and this made the porting efforts much easier than we expected on that front. However, when we finally got LLVM running, it became clear that the performance impact can be quite massive. Emulators like RPCS3 use dirty page tracking to handle cache evictions for textures. While modern games have large textures that take up a lot of space, the PS3 is a different beast. Not only are many textures tiny, all user-facing GPU objects are often represented as textures. This includes texel buffers, uniform buffers, shader storage buffers, etc. The sizes can vary wildly here and evicting a single 16K page can cost us a lot in re-uploading resources every draw call. Fortunately, not many games seem to be affected by this, but there are some unfortunate outliers with very poor performance on 16K platforms.

Pull requests

Early work

#11315 – Initial Linux aarch64 port (PPU only)
This is the original work from Nekotekina that brought basic ARM support to RPCS3, on early 2022. Only PPU was initially supported and LLVM was not supported at all. This work is mentioned here for historical purposes as it is the pillar of the current ARM port. This wasn’t the only pull request of course and there were many follow-ups polishing compatibility on ARM devices.

• Update asmjit dependency (aarch64 branch)
• Disable USE_DISCORD_RPC by default
• Dump some JIT objects in the rpcs3 cache directory
• Add SIGILL handler for all platforms
• Fix resetting zeroing denormals in thread pool
• Refactor most v128:: utils into global gv_** functions
• Refactor PPU interpreter (incomplete), remove “precise”
  • Instruction specialisations with multiple accuracy flags
  • Adjust calling convention for speed
  • Removed precise/fast setting, replaced with static
  • Started refactoring interpreters for building at runtime JIT (due to poor compiler optimisations)
  • Expose some accuracy settings (SAT, NJ, VNAN, FPCC)
  • Add exec_bytes PPU thread variable (akin to cycle count)
• PPU LLVM: fix VCTUXS+VCTSXS instruction NaN results
• SPU interpreter: remove “precise” for now (extremely non-portable)
  • As with PPU, settings changed to static/dynamic for interpreters.
  • Precise options will be implemented later
• Fix termination after fatal error dialog

LLVM bring-up pull requests

#15182 – Minor arm64 improvements
This was the first arm64 pull request of 2024, merged on February 11st, and allowed Arkedo games to run fine on arm64 with PPU Interpreter and SPU Recompiler LLVM. Almost all other games still did not work properly. SPU LLVM would hang very early after booting up the games and audio did not work as a result.

#15869 – gl: Fixes for wayland (asahi linux, aarch64)
Fast forward several months of internal work, kd-11 starts wrapping up months of internal work and prepares to open a series of pull requests to upstream arm64 support, with a second pull request opened on August 1st. This is a smaller one allowed OpenGL to start working on Asahi Linux with the arm64 Linux builds of RPCS3.

#15904 – aarch64/cpu: Add LLVM support
This PR introduced the initial draft of the LLVM transforms that allowed JIT to work on arm64. The work was developed and tested entirely on Asahi Linux for testing.

#15915 – rsx: Fix fragment constants decoding for non-x86 platforms
Some RSX functionality was broken when running on non-x86 CPUs and needed touching up. Since this was high performance code, it had been rewritten in SSE and AVX for x86 and the cross-platform fallback was untested.

#15925 – aarch64/llvm: Handle processing of leaf nodes
This PR improved the LLVM transformer to handle “leaf nodes”. A leaf node is a compute-only basic block that does not make any external tail calls. These types of blocks will attempt to return to the caller after executing and were not correctly handled in the original work. They are surprisingly rare as most normal blocks will end up calling a syscall or library function at some point while normally breaks the traversal chain by starting a gateway exit.

#15962 – aarch64/llvm: Improve compatibility
This PR reworked the transformer a fair bit. Initially, some optimizations were done by eliminating tail calls and replacing them with static branches. This worked very well and performance was good, but LLVM could sometimes clobber it’s own registers even when clobber information was provided. We restored tail calls to some capacity here and a micro-assembler was introduced to write proper inline assembly blocks for injecting into the IR with proper dependency resolution.

#15971 – aarch64 fixes
One challenge we faced with x64 support was decoding the reason for a segfault. A table was written that basically had instruction patterns for matching whenever a segfault is received so that we can determine in a cross-platform manner why the fault was reported. This information was very incomplete for arm64 and many instructions were being miscategorized.

#15974 – Rework aarch64 signal handling
Replaced the instruction decoding tables from the previous PR with a proper parsing of the machine context passed in to Unix signal handlers. This is a lot more reliable as we can check the CPU register state at the time of the fault and know for sure why a fault occurred.

#15981 – aarch64: CPU branding info and misc improvements
This PR introduced some work to help identify arm64 processors. Unlike x86 where you can use CPUID, no textual representation of the CPU name and details exists at all and we have to infer the details from some register values.

#15987 – aarch64: Support for apple exceptions
This PR started the porting of the arm64 work to macOS. One advantage of starting with Linux is that the changes are compatible with other unix-like OSes including macOS. On the first build, things just worked, but exceptions needed to be reworked since the machine context is stored in a different format than Linux.

#15992 – macOS: Implement remaining portions for native arm64
Some minor alterations were needed to get the JIT running correctly on macOS. This PR implemented reserving the x18 register on macOS as it is always reserved for Rosetta2 use and is not preserved during syscalls. That solved random crashes where the x18 register would magically change to 0 in the middle of execution and crash. Some extra code was also added to help read the CPU name, number of cores, etc.

#16011 – aarch64/gl: Misc fixes
Maintenance PR fixing some compiler warning and fixing a bug in our OpenGL backend that caused shadows to break in some titles.

#16022 – aarch64: Support calloc patch blocks
With most of the base functionality done, it was time to start supporting extra features such as game patches. This PR fixed a crash that occurred when a “calloc” style patch instruction was used. These instructions force-inject leaves into the code and add extra jumps instead of returns. This was causing the transformer to generate invalid sequences that led to crashes.

#16035 – aarch64: Fix compilation for windows-on-arm (msys2)
This was the initial windows on arm support PR. RPCS3 was successfully compiled for windows arm64 using msys2 and clang. The visual studio portions of RPCS3 are not compatible with Microsoft’s ARM64 API as it is missing too many intrinsics that we need otherwise.

#16058 – arm64: Fix remaining issues for Windows on ARM
This PR finally got us working windows-on-arm builds that could successfully run demos. Not much testing was done due to lack of test hardware. While windows-on-arm has been around for over a decade (previously windows RT) their devices are largely ignored by the masses and it is difficult to find any testers who run compatible hardware. We expect this to become less of an issue as the new Snapdragon X laptops and tablets become more common.

#16070 – arm64: macOS CI
These changes from nas introduce the automatically compiled macOS builds for arm64, with auto-updater support.

#16148 – arm64: Linux CI
These changes from kd-11 introduce the automatically compiled Linux builds for arm64.

Challenging the limits of PlayStation 3 emulation

And by now you might be thinking, if there’s a Linux arm64 build for RPCS3, which requires an armv8.2-a CPU, at least 8GB of RAM, and an OpenGL 4.3 or Vulkan capable GPU/drivers, what’s stopping it from running on an Raspberry Pi 5 device? How far can we challenge the limits of emulating the console known for being the most resource demanding to emulate still 18 years after its release?

To put this to the test and support arm64 development, AniLeo acquired a Raspberry Pi 5 device. This low-cost arm64 device costs around £68/€93/$85, plus £4/€6/$5 for an Active Cooler.

The device was setup with Arch Linux ARM to make use of all the latest packages, in order to be able to compile RPCS3 on device. The device was also overclocked to squeeze out more performance, the CPU was overclocked to 2900MHz (+400MHz) and the GPU was overclocked to 1060MHz (+100MHz).

#15978 – vk: Support v3dv, allow creating device without textureCompressionBC
We started by trying to running games on the default Vulkan render, but this failed as the mesa v3dv driver for Broadcom GPUs is missing full textureCompressionBC support, due to hardware limitations. However, not all was lost, since the BC1-BC3 formats are supported, and that’s all that RPCS3 needs.
We then added a workaround on RPCS3 to allow the Vulkan render to boot even without this feature reported as supported by the v3dv driver. We could now boot games through Vulkan.

Unfortunately the Vulkan render would only work for a small amount of time, before hanging the entire system and requiring the device to be power cycled. This meant that we had to shift to testing with the OpenGL render. Fortunately, the results were great, with the mesa’s v3d OpenGL for Broadcom GPUs being able to render all tested games correctly with no visual bugs.

However, initial results from testing dozens of games were not very promising, games were displaying very low performance overall and no game seemed to run well, even simple ones. After some investigation, we realised the Broadcom VideoCore VII GPU in the Raspberry Pi 5 is not only unbelievably weak, but was also several times weaker than the PlayStation 3’s own GPU – the RSX. This means that Raspberry Pi 5 is not capable of rendering these games at 720p.

#15977 – config: Set minimum allowed resolution scale to 25%
After testing with different rendering resolutions, we saw a great performance boost as the rendering resolution was lower, as the bottleneck shifted away from the GPU back to the CPU.
Unfortunately, even 360p rendering of these 3D games proved too much for this GPU. We then decided to settle on rendering games at the PlayStation Portable screen’s resolution, 272p, by setting the resolution scale at 38%.

Here we can see God of War 1 tested at both 720p (PS3) and 273p (PSP) resolutions. Running the game at its PS3 resolution of 720p, we can see it struggles to render at around 10 FPS. With PSP resolution, however, it runs at a smooth 30 FPS. With this new piece of data, we were now seeing several games running at Playable performance, which are showcased in the next section.

Platform showcases

macOS arm64 on Apple M1 (macOS Sequoia)

RPCS3 x64 build running under Apple’s Rosetta2 x64 -> arm64 translation layer on the left;
RPCS3 arm64 build running natively on Apple Silicon on the right, showing a huge performance improvement!

Linux arm64 on Apple M1 (Asahi Linux)

Unsurprisingly, RPCS3 runs very well on the M1 when using Asahi Linux. While the Vulkan driver for Asahi is still not ready, we can make use of OpenGL which can render games with decent performance. Unfortunately OpenGL is still not as performant as Vulkan so games still run faster under macOS when using the MoltenVK layer.

Linux arm64 on Raspberry Pi 5 (Arch Linux)

As what might come as a surprise for many, several PS3 games are be Playable on Raspberry Pi 5, albeit for the lower resolution. A lot of Playable games will not perform well on Raspberry Pi 5 due to its hardware performance limitations, we can still push it to its limit to run several games at 30 FPS, rendered at PSP resolution.

Windows arm64 on Windows 10 for ARM

This was the last platform to be supported, for one main reason: kd-11 only had access to macOS and Linux on arm64, and no one else in our team owns any kind of arm64 device that can run Windows. In order to be able to develop this platform, an arm64 virtual machine was temporarily acquired, as physical devices that can run Windows on ARM are expensive.

At first, support was attempted through the msvc compiler, but due to many encountered issues, it was deemed that this was only going to be possible by using msys2 and compiling RPCS3 with clang.

A lot of debugging and two pull requests later, we had RPCS3 running on arm64. Due to the lack of testing hardware, only samples were tested. Homebrew games should work without issue, but commercial titles are likely to run into problems due to Windows-on-ARM having a hard requirement for ASLR which doesn’t play too nicely with some aspects of RPCS3’s JIT engine.

Available emulator downloads

Without further ado: you can now download Linux arm64 and macOS arm64 binaries from the Download page on our website. These two versions now join our existing Windows x64, Linux x64, macOS x64 and FreeBSD x64 builds.

Windows arm64 binaries are not distributed at this point. This is due to the low availability of hardware to develop and test with, as well as missing automatic compilation and deployment through GitHub CI. Users on Windows must compile their own binaries locally until we start distributing them.

FreeBSD arm64 binaries are not available. FreeBSD support and the FreeBSD Ports x64 builds are maintained by FreeBSD contributors. We have not worked on or tested any support for this platform, as we do not use it ourselves, but we invite any contributors to add FreeBSD arm64 support if they wish to do so. The work done on the other platforms should make this endeavour a much easier task now.

A note about other mobile platforms

Adding arm64 architectural support is a key step to ensure long term preservation of the PlayStation 3 console, as arm64 CPUs make their way into the conventional desktop and laptop market.

We are aware it comes with its own disadvantages, such as having to deal with toxic users that have harassed other emulator developers in the past, like the brilliant developers behind AetherSX2, or redistribution of modified builds of an emulator, often violating their FOSS license, while claiming credit for work they didn’t do themselves. As it has already happened with other emulator projects being re-distributed in these malicious ways.

We are also aware of the increase of scam applications that portray themselves as a PlayStation 3 emulator while being malware, or that redistribute RPCS3 under a different name and claim it is a new emulator. Some of them are even vetted in Google’s Play Store. Unfortunately Google haven’t acted on the reports we have made, and these scam apps continue to be exposed to users.

We are thus reminding once again that there are no other PS3 emulators that can run PS3 games. Any footage you see is either completely fake, or recorded on a real console, or through RPCS3 itself and distributed under the claim it’s another emulator with a different name.

For these reasons, we are disallowing Android and iOS discussion in our communities. We have no intention of porting RPCS3 to these platforms at this time, so no discussion on these topics is needed. Unfortunately, this blanket decision is ultimately needed due to the high toxicity of several individuals in the Android community, who refuse to take no for an answer, some of which have already been banned from our communities.

Special thanks to the developers

We would like to thank the specific contributions to this big feature development. As a reminder, RPCS3 as a whole is made possible thanks to the contributions of many current and past developers and contributors at Team RPCS3 since 2011.

kd-11
– Lead developer for Linux, macOS and Windows arm64 support
– Lead graphics developer for OpenGL and Vulkan support on arm64
– Linux arm64 automated build deployment CI

Nekotekina
– Initial Linux arm64 support, laying the groundwork for the full development of this feature

nastys
– macOS arm64 automated build deployment CI
– macOS arm64 build testing

AniLeo
– Raspberry Pi 5 development support for Linux arm64
– Raspberry Pi 5 / Linux arm64 extensive game testing
– Website back-end development for the arm64 builds

DAGINATSUKO
– Website front-end development for the arm64 builds
– Design of the promotional arm64, Raspberry Pi 5 and Apple Silicon banners

Megamouse
– Bug fixes related to the macOS arm64 build

schm1dtmac
– Bug fixes related to the macOS arm64 build

elad335
– CPU emulation performance optimisations for lower end hardware

This blog post was written by kd-11 and AniLeo.

Demon’s Souls has long been one of the most popular titles for RPCS3. When it went ingame for the first time it made gaming headlines everywhere. I had been following RPCS3 pretty loosely at the time, but when Demon’s Souls went ingame I finally tried out the emulator. Back then the graphics were quite broken, and I was lucky if the framerate went over 20FPS. But despite all that it was still amazing to me. I thought to myself “Wow, it will be so incredible when the game is playable at 30FPS… maybe in 2 years or so”. The game ended up being very playable much earlier than I expected, all the way back in 2017. Since performance was so solid I thought that it would only be a matter of time before someone out there would make a 60FPS patch.

Fast forward 2 years and there still was no (proper) 60FPS patch. And I had already started to contribute to RPCS3, and created a patch to unlock the framerate in NieR. I assumed that a framerate patch for Demon’s Souls must be impossibly difficult since the only 60FPS patch that was made broke the gamespeed and caused everything to run twice as fast. On a complete whim, I decided to try running Demon’s Souls with the new generic FPS unlocking methods by developer eladash. I knew that since the game used a fixed timestep instead of delta time, the game wouldn’t be playable at proper game speed but I was curious about just how much faster Demon’s Souls was compared to two years ago. Since I’ve had the same quad core desktop PC since the start of 2017, I would be able to see just how much the software has improved.

“Wow, RPCS3 is really fast now. But it’s midnight now, I’ll try investigating a patch for proper game speed at 60FPS tomorrow.” I said to myself. Next morning, before I got breakfast I had the game running at proper speed with high framerates.

The patch simply changes the amount of time that’s advanced each frame from 33.3ms to 16.7ms. Since the patch is meant to be used with Vblank at 120 which doubles the max framerate, this lets us get a correct game speed at 60FPS. After that, it is then necessary to set Clocks scale to 200 to re-enable the games internal frameskipping at 60FPS, without which you will drop game speed every time your computer can’t hold 60FPS. I’ve also included another line that’s commented out by default (has the symbol # at the start of the line) which sets the timestep to 8.3 ms. This is meant to be used with Vblank at 240 and Clocks scale at 400 for proper gamespeed at 120FPS.

Warning: The instructions mentioned below are outdated. Click here to visit the Patch repository on our Wiki for the latest patch.

I’m sure you’re itching to play Demon’s Souls at 60FPS by now so let’s go over all the steps you’ll need to take to get it running on RPCS3:

1. First of all, you’ll need the EU or US disc version of Demon’s Souls. Only the disc versions will currently work.
2. Next, you’ll need to right click on Demon’s Souls in the game list and click Configure, this will create a custom config for the game so all future changes to settings must be made here instead of on the normal settings window/button.
   On the CPU tab, ensure that SPU block size is set to Safe (which is the default setting).
   On the GPU tab, ensure that Write color buffers is enabled, the Renderer is set to Vulkan and MT RSX is OFF (it can cause some deadlocks currently).
   On the Advanced tab (only available on new RPCS3 builds) change the following settings:

Clocks scale: 200
Vblank Rate: 120
And then click the save button.

3. Now, download the patch here and place it in your RPCS3 folder. If you are on Linux, then place it in ~/.config/rpcs3/.
4. Finally, there is one last optional step to reduce log spam which can help with stability issues. Right click on Demon’s Souls again and click on “Open custom config folder” and open the corresponding config file. In This file, scroll down to the bottom and enter the following text within the {braces}:
   Log: {sceNp: Fatal}
   Please note that this is case sensitive, when finished hit save.

Be sure to check for the ideal settings for this game in our wiki. RPCS3 advances fast and the ideal settings may change from week to week. With the patch enabled at 60FPS, I’m able to get a locked 60FPS 95% of the time on my i7 7700K. If you have a Ryzen 3000 or i7 8000 series desktop CPU then you can reach 60FPS 99% of the time. Of course as the emulator continues to improve further, the system requirements will continue to drop lower and lower.

Users should also note that the FPS counters are inaccurate with Demon’s Souls as the game will automatically output duplicate frames when the framerate drops below its FPS cap. It is for this reason that FPS counters, even third party software like Afterburner will improperly report 60FPS at all times with the patch enabled because the game is outputting duplicate frames. This is not a bug, its just an unfortunate result of how the game works. To properly measure frame-rates you will need to use tools to analyse video footage, or enable the debug overlay and look out for when the draw calls change.

I hope you enjoy the patch. It’s been killing me since I’ve had to keep it secret for a few days so that we could make this blog post and the YouTube video. If you’re happy with the patch then please consider making a donation to our Patreon. https://www.patreon.com/Nekotekina While RPCS3 is already stable and performant enough to run games such as Demon’s Souls at 60FPS, there’s no end when it comes to things that can be improved when talking about emulation of a system as complex as the PlayStation 3.

Closing Words

If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.

Also, come check out our YouTube channel and Discord server to stay up-to-date with any big news.

Table of Contents

SPU Improvements by Jarves
RSX Improvements by kd-11
List of known Improved Games
Closing Words

SPU Improvements by Jarves

There were two main changes to the SPU emulation that brought us to this point of allowing so many newer titles to progress past ‘Intro.’ Let’s take a quick look at both individually.

SPU Interrupt Fix

Many titles in RPCS3 ‘hang’, but in the case of the titles mentioned above, they do not actually crash, and the fps counter would still change with just a black screen. This normally would be mistaken for RPCS3 just being slow and the game taking a bit to load, but opening up the debugger in RPCS3 tells a different story. The games would loop over the same code on both the PPU side and SPU side. In this case, they are waiting on something, but what?

SPU MFC Queue Stall Fix

This commit first requires a quick tutorial on how the Cell processor actually works:
For design reasons relating to performance and speed (explaining this more could be a blog post on its own!), the SPU, unlike the PPU core, doesn’t actually have direct access to main memory, and instead each core has their own tiny 256kb of Local Storage (LS) memory. In order for the SPU to access main memory, and get data in and out of their respective LS, memory transfers are ‘queued’ to what’s known as the Memory Flow Controller (MFC). The MFC is then responsible for the actual copying and transfer of memory. On top of that, there are multiple different types of transfers and the MFC can not only choose to execute transfers out-of-order, the SPU can request certain transfers are held, forcing something that would normally be in-order, to not be.

That last statement from above is key; the emulation of the MFC in RPCS3 has only ever supported in-order executing. Up until these new titles started booting did we finally see SPU code actually request a transfer be held, or stalled, so there was never a reason to change or implement it. I’ve not only fixed the emulation of MFC so it stalled, or held, certain requests by the SPU, I also added out-of-order execution when encountering these special transfer requests. This ended up bringing a slightly more accurate MFC emulation, along with killing some random crashes in these new titles.

After these needed SPU changes were made to get games booting, kd-11 tackled some of the graphic issues in them.

RSX Improvements by kd-11

It has been an exciting month since it was decided earlier on to finally present the SPU improvements code by Jarves. While this means that more games successfully boot, it also means rpcs3 graphics back-end is exposed to games utilizing more advanced techniques and new bugs were inevitably going to get introduced.

Before, I tackled any new games that would be bootable, I decided to take a look at games that were already relatively stable that presented similar looking bugs based on early game screenshots of God of War 3. It is usually a lot easier to debug a game that boots successfully since you can make progress a lot faster if the code doesn’t randomly crash every few seconds. For this task, I decided to look into Dark Souls 2 which had multiple problems from tone-mapping to depth precision.

1. Dark Souls 2

Earlier in the month, hcorion fixed a mouse bug that was keeping Dark Souls 2 from booting. The game was relatively stable but had graphical artifacts that I identified as being related to the HDR pipeline.
After pouring through traces for a few days, it became apparent that some instruction to clamp negative values was missing. Searching for this led nowhere until I found that instructions in the target shader differed very slightly with those from other shaders in that 3 extra unused bits were being manipulated. Quick tests showed that these bits could add extra precision and clamping modifiers and implementing this into the emulator fixed the random bright spots.
The next challenge was the missing shadows. Looking through the traces showed that a color texture was being bound for use instead of a depth texture which made no sense. It was then when I discovered there was a problem with render target address aliasing. It is not possible to determine via addresses or MRT configuration if a memory block will be used for depth or color if set to both. It is also valid to do so on the PS3. The value to be written is determined by other settings such as depth test status or color write mask. After tuning the checks to only always allow one type of target to exist, shadows were working, but flames and collectibles were visible through walls.
Looking through again showed the game was writing to addresses as color buffers in RGBA then binding the memory as the depth buffer. This is allowed since its all just bits to the RSX. This prompted the creation of an overlay pass system to handle data casts between incompatible types such as color to depth. With the depth working properly it was time to fix the final issue – depth was not working correctly. In fact it looked as though only 8 bits of precision were available which is very poor.
I recognized this error from previous work handling data casts between depth textures and RGBA color and it was reported in other titles such as God of War collection and Castlevania: LOS2. The issue came down to respecting texture channel swizzles where the components are shuffled around during readback. After setting this up, Dark Souls 2 was more or less looking great.

2. God of War: HD Collection

As a consequence of properly fixing the depth casting/interpretation as RGBA color, games like God of War Collection also got partially fixed and I embarked to fix the remaining glitches.
The major issue with flickering around some objects like flames or the blades of chaos were fixed by fixing the depth, but shadows were not working on OpenGL. Quick testing showed that face winding on OpenGL was broken and was promptly fixed.

3. Demon’s Souls and Vulkan ZCull

This took a few days to implement and tune for accuracy and it works very well, albeit a little slow and very demanding on system resources. Tuning in the near future will fix that issue as well. This finally resolved the “sun shining indoors” bug when playing Demon’s Souls using Vulkan.
Other fixes applied to Dark Souls 2 such as implementing precision modifiers also fixed the last remaining visual bugs on the game such as the Valley of Defilement being overly bright to the point where geometry can be seen popping into view and also the dark tone on the character creation and profile loading screens.

4. Uncharted

Testers had reportedly gotten Uncharted 1 to boot as well as its demo. There were many problems with it, including spamming the infamous “Texture upload requested but texture not found” error as well as broken HDR pipeline and misaligned screen-space effects. Luckily this one was not too difficult since most of the work was already done when investigating Dark Souls 2. A few texture cache fixes later and the game was graphically okay, with the exception of broken shadows, but that could wait since there were more pressing issues.

5. Ratchet & Clank

Ratchet & Clank HD I, II, III and IV games were also booting with the SPU fixes, but there was a minor problem where there was no display output.
To be more accurate, sampled clamped edge artifacts were all over the screen meaning that a texture was accessed outside its boundary in the vertical axis. The issue came down to a bad interpretation of the s1 data type when inserted via immediate commands.
Another issue was missing loading screens which were fixed by aligning data writes in the float3 format to 16 bytes (essentially float4 configuration).

6. God of War 3

This one hit the existing RSX emulation pipeline hard and highlighted so many bugs. First was to get bugs related to texture readback (Write Color Buffers) out of the way. With this fixed, nothing improved, but investigations led to the reimplementation of the pack and unpack fragment shader instructions.
This game uses a lot of tricks to achieve scaling on floating point textures which was not possible natively on DX9-class hardware. This requires packing bits into 8 bit values, writing to RGBA8, scaling down, then reconstructing the floating point textures again, averaging, then the cycle repeats until the proper value is written to the final 1×1 target. This still does not fix the final problem with the game which is that it exploits register aliasing, which is difficult to implement. As such, brightness calibration is still off.
On the PS3, one 128-bit register can hold 8 half-floats or 4 full single precision floats. Writing to a half value will modify some bits in the float value that occupies the same memory region.
Fixes for this will be coming soon to rpcs3 once the implementation is ready.

List of known Improved Games

Note: This is far from being a complete list of improved games. There are surely many other games that also improved from the aforementioned changes. Obviously, there’s no way for us to test even half of the huge library that is the PS3’s. It’s up to you to retest your games and find out if and how they improved.

God of War I & II

Huge improvements made it so both God of War I and II are now Playable. Jarves’ SPU fixes fixed freezing cutscenes and finally brought us working Audio (previously the game was completely dead silent).
On the graphical side, kd-11 fixed broken lighting and shadows. This allows the game’s beauty to be finally fully enjoyed on RPCS3, as graphics are now perfect.

God of War III

Oh boy. The game everyone kept asking endlessly for us to get fixed. It seems the time has finally come, as God of War III finally not only started properly booting with Jarves’ SPU fixes, it actually now goes Ingame!
But as nothing is a sea of roses, when we went Ingame, we found ourselves looking to an incredibly slow and graphically broken game (as we were already expecting from such an early emulator like RPCS3).
The game was running at ~1fps on battles, but the important detail here was that we were using SPU Interpreter, as SPU Recompiler wasn’t working due to an unimplemented instruction.
Jarves promptly implemented the missing instruction, and back ingame we went, this time with ~3fps instead of ~1fps. Although it may not seem like a lot, it is a ~3x increase on speed. Not bad as a first step towards achieving good framerates.

God of War: Ascension

There’s still a weird looking bug with clipping on this one. Nevertheless, it also now goes Ingame!

Uncharted 1

Uncharted 1 now goes Ingame! Performance is not ideal yet, but progress on graphics has been done already as described by kd-11 in his above writeup.

Uncharted 2

Uncharted 2 previously crashed after the loading icon (spinning dagger). Now it shows intro and a few videos, and will crash after playing them.

Uncharted 3

Uncharted 3 Beta now shows title screen, Uncharted 3 Demo shows menus and Uncharted 3 (Disc) shows the Naughty Dog intro screen, but unfortunately none go Ingame yet.

Ratchet & Clank I, II, III, IV (Deadlocked/Gladiator)

Jarves’ fixes yet again saving the day, making all four mentioned Ratchet & Clank HD Remasters now go Ingame. After kd-11 fixed the R&C graphical issue (mentioned in RSX Improvements), all games started rendering proper graphics.
Sadly though, these games currently suffer from heavy flickering in textures, which is something we plan to debug and hopefully fix soon.

Ratchet & Clank: Tools of Destruction

Ratchet & Clank: ToD now shows a couple of Intro screens before freezing.

LittleBigPlanet II

LittleBigPlanet II now goes Ingame, although for now it needs existing save data utility from a console (dev_hdd0/game/BCUS98245_USER1) in order to work.

LittleBigPlanet III

LittleBigPlanet III also goes Ingame, also needing existing save data utility from a console (dev_hdd0/game/BCUS98362_USER1).

WipEout HD

WipEout HD now goes past menus and goes Ingame, ready to race! Though do note it may be unstable, but despite that, the framerate is quite good.

inFamous 1

As most of you figured out from our progress report.. Yes, the last screenshot that was on it was inFamous 1. This game now boots and goes Ingame as well, however as stated in the video the full game gets stuck during the tutorial section and you can’t control your character. To get around this you can either load a save or play the demo version. Graphics are sadly heavily broken right now.

inFamous 2

And just like inFamous 1, inFamous 2 now boots to get ingame you need to play the demo version, as it currently requires the SPU interpreter option and doesn’t work with ASMJIT performance is lower than Infamous 1. Graphics are broken just like its predecessor.

Playstation All-Stars Battle Royale

Playstation All-Stars now goes Ingame while also rendering correct graphics. Sadly, the game is unstable with Vulkan and performance isn’t the best.

Heavenly Sword

Albeit a bit unstable at times, Heavenly Sword goes Ingame with good graphics.

Gran Turismo HD Concept

This title now also goes Ingame, and you can play through races. Graphics look good, but as of now they occasionally flicker. It also runs a bit slow, as one would expect.

Gran Turismo 5 Prologue

Gran Turismo 5 Prologue now goes to Menu, unfortunately with broken graphics.

Gran Turismo 5

Gran Turismo 5 now shows the copyright screen.

Gran Turismo 6

Gran Turismo 6 now shows… something.

The Last of Us

Yet another step towards getting the big boi to run in RPCS3. The Last of Us now goes further than the initial loading screen to display the Health Disclaimer and play some audio before dying.

Killzone 3

Killzone 3, which previously showed only a black screen, now shows Intro screens.

Motorstorm RC

Motorstorm RC now goes Ingame with a few graphical bugs.

Closing Words

These changes are a huge step in PS3 emulation, and now allow for huge AAA titles which whose emulation was eagerly awaited to start functioning.
While the games shown here aren’t in a Playable state yet, they’re now closer to reach such status than ever, as now most go Ingame and can be further debugged within RPCS3.
Lastly, do note that we haven’t extensively tested these changes with a broad range of games, so it’s up to you to find out which other games also improved, as surely many did!

This blog post was written by Ani, ssshadow, Jarves and kd-11.

As you know lead core developer Nekotekina, who has worked on RPCS3 since 2013, launched his highly successful Patreon in January this year. Thanks to the overwhelming amount of support, he has been able to work on RPCS3 full time for 5 months now and the progress he has done is staggering. The RPCS3 team has discussed the best course of action for the road forward and have decided that the time is now right for kd-11 to join Nekotekina’s Patreon in order for him to obtain hardware necessary for development and testing.

The Purpose

As kd-11 works on the graphics side of things, especially the renderers using OpenGL, Vulkan, and D3D12 (Direct3D 12), modern hardware is required for development and testing. Ideally, he needs a modern AMD, Nvidia, and Intel GPU to easier identify bugs specific to one platform. Furthermore, kd-11 works as a consultant and therefore has the possibility to accept or turn down certain jobs. If support on Patreon reaches a certain level, he may be able to spend more time on the project.

As of now the goal is obtaining necessary hardware for development and testing:

• – A modern AMD GPU that supports DirectX12 and Vulkan.
• – A modern Nvidia GPU that supports DirectX12 and Vulkan.
• – A modern Intel CPU where the iGPU supports DirectX12 and Vulkan

Roadmap

With the support of generous patrons kd-11 will be able to work on the following tasks:

Short term

• – Investigate game specific bugs like the overly bright bloom in [redacted] or the overly dark lighting in Demon’s Souls.
• – Improve performance and compatibility of all renderers.
• – Enable Vulkan on Linux, which will bring a tremendous performance improvement for some games.
• – Implement missing functionality that will fix various bugs in different games. For example, the broken shadows in many games including Demon’s Souls depend on this task.

Mid term

• – Properly support some PlayStation 3 features like MSAA that are lacking at the moment.
• – Implement RSX reports which are missing at the moment.

Long term

• – Enable higher rendering resolution to play games at any resolution, for example 4K instead of 720p in Demon’s Souls or [redacted].
• – Enable extra graphics options to support extra rendering features, such as custom anti-aliasing modes.

Closing words

kd-11 has recently done some significant performance improvements in graphics emulation. Check out the video below where many games are twice as fast, and some even more than that. Also do note that those aren’t all the games that received a boost from the mentioned changes as we couldn’t test all working games to find out, so you can find other games that will receive some boost on Vulkan.

You can support Lead Core Developer Nekotekina and Lead Graphics Developer kd-11 on Patreon with the link below. You can also find Nekotekina’s roadmap here.

Don’t forget to join our Discord server to chat with the developers and follow the latest progress on RPCS3.

It finally happened! The PlayStation 3 emulator RPCS3 has launched one of the most iconic games on the platform: Demon’s Souls. Are you curious to know how it all happened? What the developers went through? Welcome to this first of hopefully many to come developer logs where we will now tell you everything.

It all began in our Discord channel where a user going by the nickname, Numan approached us with some interesting news. He claimed to have conducted a detailed investigation of what Demon’s Souls was trying to do when it was crashing one second after booting. He drew the conclusion that the game was trying to allocate memory pages of different sizes, access levels, and different attributes. When the game then tried to access a specific memory address, it crashed with an access violation. Consequently, these discoveries lead to a more accurate implementation of the function sys_memory_get_page_attribute() which was responsible for this particular crash. Suddenly the game booted up in RPCS3 for the first time ever and showed a loading animation!

The Beginning Hours

After loading for a while, the loading screen faded out and we faced a black screen and nothing more. Analysis showed that the game was trying to communicate with PSN and connect to the servers for online play. Some changes in the sceNp module were made to tell the game that “the console” is offline and no connections should be made. The game now loaded a bit further, started a few more thread… and we faced even more disappointment: still a black screen and nothing unusual in the logs. A dead end. For another day or two, no progress was made.

You know what they say about not being able to see what is right under one’s nose. A very common thing was happening; the game was loading encrypted .sdat files. However, the emulator was not decrypting these on the fly at this point in time (last week that is, now it does thanks to Jarveson). So the game loaded encrypted files which could not be processed and the game got stuck. When Numan went through the logs again and noticed this oversight, he made a big facepalm and then decrypted the 26 .sdat files manually. Then the miracle happened, the game started to show some introduction screens!

The game then complained about a lack of space on the hard disk. This problem was quickly circumvented by creating some empty directories that it failed to create itself. This will be fixed properly later on with some changes in the cellGame module.

The opening video now started to play, although one could only hear it as the display was black. The video was skipped and the game reached the main menu:

With much anticipation a new game was started!

And… we are ingame!

But the graphics are not very impressive. Let’s try again.

Much better. So what happened here? ssshadow channeled the power of the soul arts and managed to get the game to display the ingame graphics. However, it happened by accident, he played around with the ingame brightness slider while reloading the save. When setting brightness to zero, reloading, and then setting it to the maximum value of 10 graphics would sometimes randomly display. Most often however, they would still be black. This lucky person ran through the first tutorial level, killed all the enemies and was generally pleased by the performance.

Meanwhile it became obvious that much more work was still needed. The game makes use of many of the features the platform provides, and thanks to this many imperfections in the graphics implementation surfaced. Lead graphics developer kd-11 immediately started debugging and soon enough fixed the videos and the main menu:

The most difficult task still remained, to find the reason why the ingame graphics were almost always completely black. Kd-11 rolled up his sleeves and started to rewrite parts of the shader cache and image buffers. It took another full week of experimenting, debugging, and retesting but the actual reason for the missing graphic were finally found. The value of a uniform fragment shader variable responsible for overall scene brightness was found to be calculated to negative zero. The resulting vector was therefore nullified and the image was not visible (although still drawn correctly in the dark). During the search of a solution dozens of other bugs were detected and fixed in among many parts of the emulator the shader decompiler (https://github.com/RPCS3/rpcs3/pull/2482) which fixed a lot of issues in many other games too. Finally, at the end of the week the game consequently looked like this:

With a few more fixes the graphics and performance were improved even further. The game is mostly correctly displayed, special effects such as fog, motion blur, bloom, particle effects and much more are all there:

Last but not least it was time for Nekotekina to work his magic as well. Almost three weeks in the making he finally committed his reservations re-rewrite which not only increases performance in this game by some 50%, but also gives a nice boost to the overall performance of RPCS3 in general. For example, [redacted] is almost ten times faster than it was before, now reaching 10 – 30FPS on a fast CPU. This commit finally made it possible to reach a “smooth” 30FPS in many parts of Demon’s Souls:

At the moment, there are still several known problems with Demon’s Souls:

• Performance. The game is very demanding of resources, and while Ryzen @ 4 ghz or 6 cores of Haswell-E can run some areas of the game at 30 fps, it can also drop down to as little as 10 fps in big open areas such as in front of The Boletarian Castle. One really wants a stable 30 fps (native frame rate, maybe someone will mod it to 60 fps), and preferably also on “normal” CPU’s. But there are both short and long term plans to improve performance, for example by making the LLVM recompiler not crash in this game.

• The player character shadow is missing, and there are other minor graphical bugs here and there.

• The sound is a bit unstable. It can work fine for a while, but after some 5-10 minutes it can suddenly stop. There are also problems where sound effects are repeated before they are finished or they sound very loud and strange.

• The game can randomly get stuck on loading screens. Not critical as it auto-saves continuously.

RPCS3 v0.0.2

As you probably noticed by the title already, we are entering a new version (and no more Pre-Alpha now, just Alpha!). It has been a really long time since the last version bump (v0.0.1 Pre-Alpha on 17 Jun 2016). Since then, a lot of improvements were made, improvements big enough that we finally decided to finally bump the version again. The version will be officially bumped on GitHub as soon as the last pull requests related to Demon’s Souls are posted.

Resumed Changelog since v0.0.1 Pre-Alpha:

PPU/SPU
– PPU LLVM AOT recompiler reimplemented
– PPU LLVM caching implemented
– PPU thread scheduler implemented
– PPU breakpoints reimplemented
– PPU analyzer improved
– Various LV2 system improvements
– IdManager improvements
– Debugger fixed and enhanced
– PPU/SPU accuracy improvements

Memory
– Memory system cleanup
– sys_memory_get_page_attribute improvements

Graphics
– A LOT of bugfixes and accuracy improvements for all Graphic backends. (There were really a lot of improvements here and we can’t list them all!)
– Shader decompiler improvements
– Several RSX bugfixes and accuracy improvements

HLE
– Major sys_net improvements
– cellOsk implemented
– cellVDec fixes and improvements
– Several other small improvements

Input
– MMJoystick implemented

Linux
– Several Linux specific bugfixes were made. Some are still being worked on as there are still a few Linux specific bugs.

Vita
– Initial package unpacking implemented
– Small Interpreter improvements

Logging
– Logging system cleanup
– TLS bugfixes and improvements

Misc
– Automatic LLE loading
– MSELF, SDAT, EDAT, SPRX: Decryption on the fly!
– Firmware update file (PS3UPDAT.PUP) installer
– XAudio fixes

Many other unlisted bugfixes and improvements

The Conclusion

Regardless, the fact that Demon’s Souls goes ingame and is playable to some extent is certainly a huge milestone for RPCS3. I would like to thank every developer who made this possible. Numan that did the initial investigations and got the first boot, Oil who helped work on the same issues, kd-11 that fixed countless of graphical issues, and Nekotekina who improved the overall performance. Of course even only tangentially related improvements by a lot of other people have helped a lot too. Jarvesson implemented automatic .sdat decryption that increases ease of use and ssshadow implemented cellOsk (on screen keyboard) earlier this year which this game uses for player name input in the beginning.

Do you want to try it out? It seems like only the US and EU disc versions of the game currently work. Simply download the latest RPCS3 build here and turn on the setting “write color buffers”. Also read the quickstart guide, check the forum thread, and ask for help on our Discord channel where all of this started.

You can support lead developer Nekotekina on Patreon. Your support will help speed up the development of the emulator so that one day this game will be perfectly playable from start to finish with good performance and accurate graphics.