Backwards compatibility has always been a minefield of assorted topics. There’s straight emulation, ports, remasters, remakes, and even full-on reimaginings. Even with the launch of the PS4 Pro, developers had to manually code updates to improve their games for the more powerful console. Real-time AI upscaling could completely redefine how PS5 backwards compatibility works though. Instead of next-gen “versions” of the game, we could see games automatically upscaled to meet the power that’s possible under the new consoles.
Now this news comes out of Microsoft (bear with me, I’ll tie it back into PlayStation). In 2018, Microsoft acquired Playfab. Playfab develops various tools that utilize the cloud infrastructure to assist developers. Playfab uses Microsoft’s Azure network to provide various tools to developers that wouldn’t be possible without the power of the cloud.
Playfab founder James Gwertzman revealed in an interview that an Xbox Game Studio is attempting to use machine learning and cloud computing to upscale textures in realtime.
It’s working scarily well. To the point where we’re looking at shipping really low-res textures and having ML models uprez the textures in real-time. You can’t tell the difference between the hand-authored high-res texture and the machine-scaled-up low-res texture, to the point that you may as well ship the low-res texture and let the machine do it.
The technology could drastically reduce game file sizes as they could ship with lower resolution textures and let the AI upscaling take care of the rest. Gwertzman calls it “magical compression technology.” Pairing this technology with the existing AI-based ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) model (the upscaling effect seen in the header image above) could have a drastic effect on upscaling older games without much effort from the developer. Numerous other companies, from NVIDIA to AMD are reported to be working with similar technologies.
How does this improve PS5 backwards compatibility?
But if that’s all Microsoft, how does it affect PlayStation? Remember that Sony Microsoft cloud deal that happened last year, one that many people assumed was all about cloud services and game streaming? Reading back on the wording of that deal hints that perhaps it wasn’t just about finding ways to improve PlayStation Now. “I believe that our joint development of future cloud solutions will contribute greatly to the advancement of interactive content. Additionally, I hope that in the areas of semiconductors and AI, leveraging each company’s cutting-edge technology in a mutually complementary way will lead to the creation of new value for society.” It could very well involve utilizing these cloud-based solutions that take machine learning and AI tools to another level.
It’s already been rumored that PS5 backwards compatibility will extend beyond the PS4. What if there was realtime tech in place that could upscale any older game’s textures without the need for a ton of work from the developers? We’re not talking a simple resolution bump here. This is actually extrapolating texture details from low-resolution images. This is the magic of some guy shouting “Enhance!” at a blurry photograph in a network crime drama and somehow getting a high-resolution image of his face. The value proposition for backwards compatibility suddenly skyrockets. The PS5 becomes a machine that not only plays older games but natively enhances them.
It should be stated that the technology for auto upscaling without development input isn’t quite here yet. The technology referenced above is for developers to use to ship low-res textures with their game and have the console up-res them in realtime. ESRGAN likewise is a mod that requires development and can’t be done passively on the fly just yet. Gwertzman notes that there are some limitations to the technology right now, but that machine learning is a new discipline of video game development for developers to master.
In this case, it only works by training the models on very specific sets. One genre of game. There’s no universal texture map. That would be kind of magical. It’s more like if you train it on specific textures it works with those, but it wouldn’t work with a whole different set.
It’s especially good for photorealism, because that adds tons of data. It may not work so well for a fantasy art style. But my point is that I think the fact that that’s a technology now — game development has always been hard in terms of the sheer number of disciplines you have to master.
This opens up a new realm of possibilities on what could be going on behind the Sony Microsoft cloud partnership and how it will impact the PS5. AI-based technologies could enable developers to focus on other aspects of game creation and while leaving other areas up to machine learning. Developers are only beginning to explore what’s possible. Don’t worry, it’s not likely that computers are just going to be creating all video games for us in the future, but the cloud is going to open up a lot of areas of power and creation that will assist developers in ways yet unknown.
[Source: WCCFTech]
