Sony's AI Image Tech Patent Revealed

Sony Interactive Entertainment has filed a patent application for an AI-generated image technology named "Generation Super Sampling." The patent, published on October 15 in the United States, was uncovered by IGN. It outlines a method for AI to predict and generate images in real-time, potentially improving gaming performance.

Predicting the Next Frame

The technology aims to reduce input lag by predicting the next frame of a game before it is fully calculated. This approach differs from current methods used by PC graphics cards, such as Nvidia's DLSS and AMD's FSR, which generate images between calculated frames.

How the System Works

The system uses two neural networks: an encoder that summarizes an image into a small data packet and a decoder that uses this data to generate the next image. The model can also use motion vectors and player inputs to make more accurate predictions.

Reducing Latency

Sony's method aims to reduce the delay between a player's action and the game's response. Unlike current methods that wait for the next real image to be calculated, Sony’s system displays a predicted image while the console is still working on the actual one.

AI-Free Backup and Learning

The patent also includes an AI-free backup method that moves pixel blocks based on motion. Additionally, the model improves over time by comparing its predictions with the actual images generated by the console.

Sony's Broader AI Strategy

The patent aligns with Sony's public plans for AI-based graphics. In March, Mark Cerny, the architect of the PS5, mentioned that an image generation library would be available on PlayStation platforms "at some point." This follows a partnership with AMD on AI graphics tools called the Amethyst project.

Current and Future Applications

Sony is already using AI-based upscaling with PSSR 2.0 on the PS5 Pro and QSSR on the standard PS5. A new version of PSSR, incorporating AI image generation, is expected for the PS6, though no official release date has been given.

Challenges and Competition

Despite the potential benefits, there are concerns that developers might rely too heavily on AI to artificially boost frame rates, leading to visual issues and latency. Microsoft is also developing its own AI-based image generation for its upcoming Xbox Project Helix, set to launch in 2027.

Cost and Delay Concerns

Sony has delayed the PS6's release due to a global component shortage, which could also increase the console's price. The rising cost of memory and storage might push the PS6's price above 1,000 euros.