How GPU hardware acceleration works with Linux

The processor of current PCs is usually powerful enough to work smoothly with all types of content. However, some processes put a higher load on the processor — for example, when you watch videos.

Still, playback is usually smooth because streaming providers automatically reduce the video quality if the internet connection is not fast enough or the PC’s performance is insufficient.

Nevertheless, it makes sense to reduce the load on the CPU because this reduces power consumption and the battery lasts longer on laptops.

The graphics unit (GPU) of the PC is located in the main processor, in a separate graphics chip, or on a dedicated graphics card. The GPU often doesn’t have much to do apart from displaying the desktop. But the graphics chip is often more powerful than the CPU when it comes to playing (decoding) videos.

The GPU can also help when converting videos into other formats (encoding). The overall power consumption is reduced, videos can be played in higher quality levels or converted faster.

GPU utilization under Linux

Whether and how hardware acceleration can be utilized by the graphics chip depends on several factors under Linux:

• The model of the graphics chip and its capabilities
• The driver used
• The display protocol (X11 or Wayland)
• The respective software (browser, video player, video editor)

IDG

There are also several interfaces for different application areas that can be used for hardware acceleration:

• Vulkan is mainly used by games and can also be used when playing videos.
• Open GL is the predecessor of Vulkan and offers support for 2D/3D vector graphics on the desktop and in games.
• Vdpau offers hardware acceleration for video playback, comes from Nvidia, and is also supported by AMD graphics drivers.
• CUDA is a programming interface (API) from Nvidia with which program parts can be processed by the graphics processor (GPU). The API is used, for example, in AI applications and when playing (NVDEC) and converting (NVENC) videos.
• VA-API is an interface for decoding and encoding videos with AMD and Intel GPUs.
• Quick Sync Video is the name of the hardware acceleration of Intel GPUs, which is accessed via the VA-API.

Media players usually recognize the available interfaces automatically and use hardware acceleration by default. Web browsers, on the other hand, usually do not offer this function with common graphics chips.

Current graphics chips provide hardware acceleration for playback with the codecs H264 (AVCHD), H265 (HEVC), and VP9. This means you are well equipped because YouTube, for example, usually uses H264 or VP9.

You can find out what a chip can do from Nvidia, Intel and AMD.

IDG

In the foreseeable future, YouTube will increasingly switch to the license fee-free AV1 codec, the improved successor to VP9. Netflix will also be using AV1 in the future.

However, only newer graphics chips such as Intel Tiger Lake (11th generation) or Nvidia GeForce from RTX 2050 offer hardware acceleration (decoder) for AV1. If you want to use the AV1 encoder for conversion, you need a graphics card from GeForce RTX 4050, AMD Radeon RX 7000, or Intel Arc.

Even if the graphics chip supports a format, drivers and interfaces must also be compatible. VLC, for example, can use Vdpau for hardware acceleration if the Nvidia driver is installed.

However, browsers such as Mozilla Firefox or Google Chrome rely on VA-API, which the Nvidia driver does not support. A suitable driver with VA-API (“nvidia-vaapi-driver”) is only officially available in the distribution repository from Ubuntu 23.04 onwards. Support for VA-API is included as standard for Intel graphics; AMD drivers support VA-API and Vdpau.

The open source driver Nouveau for Nvidia chips also supports VA-API and VDPAU, but only for older chipsets such as GeForce 700 or 600.

Check support for Intel chips

The graphics units from Intel are usually found in the CPU of the PC. Quick Sync Video has been included since around 2011, starting with the Sandy Bridge processors (Intel HD Graphics 2000 and 3000). Linux distributions contain the necessary drivers as standard.

IDG

You can find out what the chipset has to offer in the terminal with the vainfo tool, which can be installed via the package of the same name. It displays the version numbers of the VA-API and the Intel driver.

You can also see the supported codec profiles, such as “VAProfileH264Main” and “VAProfileVP8Version0_3”. The addition “VAEntrypointEncSliceLP” indicates that encoding, i.e. accelerated conversion to this format, is also offered for this codec.

You can check whether a program actually uses the VA-API with another tool, which you can install with

sudo apt install intel-gpu-tools

in the terminal. Then start

sudo intel_gpu_top

and in another terminal call the media player VLC in the form

vlc 

If the video is encoded with a codec such as VP9 or H264, VLC provides information on the use of VA-API and the associated driver, intel_gpu_top shows the utilization of the GPU.

Note: If a dedicated graphics card from Nvidia is installed in the PC, the functions of the Intel graphics unit cannot be used. For laptops with hybrid graphics, you can switch to the Intel chip via the Nvidia X Server Settings tool under “PRIME profiles.” If the option is not available, use the following in the terminal

sudo prime-select intel

You must then log out and log in again. Replace “intel” with “nvidia” or “on-demand” (if required) to switch back to the more powerful Nvidia chip.

Play videos in Firefox with VA-API

Firefox offers support for the VA-API under X11 (Linux Mint) and under Wayland (Ubuntu) by default.

Call up the internal address “about:support” to check the function. Below the “Graphics” section, you will see “WebRenderer” after “Composing.” If “WebRenderer (software)” is displayed here, hardware acceleration is not active. Under “Media” you can find out for which codecs hardware decoding is offered.

You can call up a list of YouTube example videos in high resolution here.

Show “Statistics for interested parties” via the context menu of the video. If the line after “Codec” begins with “av01,” hardware acceleration is only available if the graphics chip already supports the format.

Try out different resolutions after clicking on the cogwheel symbol behind “Quality” until “vp09” appears behind “Codec.”

You can also install the “Your Codecs” extension in Firefox and use it to block the VP8, VP9, or AV1 formats. If all are blocked, YouTube will deliver videos with the AVC1 codec (H.264), but usually only in a lower resolution of up to 1080p.

In the YouTube settings, which you can access via the hamburger menu (three horizontal lines) at the top left, the behavior can be influenced under “Playback and performance.” If the graphics chip does not support AV1, activate the option “Prefer AV1 for SD” under “AV1 settings.”

YouTube then only uses AV1 at low resolutions, which reduces the CPU load. Otherwise it will switch to VP9 or another format. In the terminal you can use

sudo intel_gpu_top

to see how much support from the GPU reduces the load on the CPU.

IDG

AMD graphics chips: Linux automatically sets up the necessary drivers and the functions can be checked with vainfo. If the tool does not display anything, the kernel modules probably do not support the chipset.

For newer AMD graphics cards, you should use kernel versions 6.2 or higher. Ubuntu 22.04 automatically installs this version through updates. Users of Linux Mint 21 go to “System management > Update management, and in the menu to “View > Linux kernel.” Select a more recent kernel version and click on “Install.”

If you want to check the GPU utilization, install the Radeon-Profile tool with these three lines:

sudo add-apt-repository ppa:radeon-profile/stable
sudo apt update
sudo apt install radeon-profile

The value behind “GPU usage” increases as soon as you play a video.

Google Chrome and Chromium

There are numerous tips circulating on the internet on how to activate hardware acceleration in Chromium-based browsers. Depending on the program version and desktop environment, you may be successful.

In our tests, we were unable to persuade Google Chrome or Chromium to cooperate. One reason for this is the outdated version of VA-API in Ubuntu and Linux Mint. The browsers do not currently work with AMD graphics chips, regardless of which version is installed. The description therefore only applies to Intel graphics.

IDG

If you are not afraid of the effort, you can update the libraries and drivers. To do this, use the script “build_intel_ media.sh” (only for Intel chips). It installs the program packages required for development, downloads the source code to the “$HOME/src” folder, and compiles the files. Then restart Linux.

In the terminal, vainfo now displays VA-API version 1.21 (or higher) (previously: 1.14). Under Linux Mint, set up Chromium as a system package via the application management. Google Chrome does not yet work with the new VA-API, nor does the Snap package from Ubuntu.

Start the browser and play a test video. Open the internal address “chrome://gpu” in another tab. Behind “Video Decode:” it says “Hardware accelerated.” However, this means nothing because Chromium also provides this information if hardware acceleration is only theoretically possible. Detailed information can be found at “chrome://media-internals.”

Click on the link with the addition “(kPlay)” while the video is playing. The “kVideoDecoderName” is followed by “VaapiVideo- Decoder” if GPU acceleration is active.

Retrofitting VA-API for Nvidia

You can already use the VA-API driver under Ubuntu 22.04 or Linux Mint 21. However, you must create the driver yourself from the source code.

The prerequisite is that the proprietary Nvidia driver in version 470, 500 or higher is activated. The VA-API driver is primarily intended for hardware acceleration in Firefox; the function is not guaranteed for other software.

In addition, the driver does not yet work optimally under Wayland and certainly not with Firefox installed as Snap or Flatpak. Linux Mint fulfils these requirements, Ubuntu users follow these instructions.

IDG

To install the Nvidia driver — if not already present — Ubuntu users search for “Drivers” via “Activities” and click on “Additional drivers.”

With Linux Mint, the path leads via “Driver management.” Select the driver with the highest version number and the addition “Proprietary, tested” (Linux Mint “recommended”). Click on “Apply changes” and restart Linux after completing the installation.

To be able to create the VA-API driver, install some packages in the terminal:

sudo apt install build-essential git meson gstreamer1.0-plugins-bad libffmpeg-nvenc-dev libva-dev libegl-dev cmake pkg-config libdrm-dev libgstreamer-plugins-bad1.0-dev

Then create a working directory and download the source code (four lines):

mkdir ~/src && cd ~/src
wget https://github.com/elFarto/nvidia-vaapi-driver/archive/refs/tags/v0.0.11.tar.gz
tar xvf v0.0.11.tar.gz
cd nvidia-vaapi-driver-0.0.11

Adjust the version numbers if necessary. Then compile and install with

meson setup build
sudo meson install -C build

to compile and install the driver.

Activating the Nvidia VA API driver for Firefox

In order for Linux to load the driver, you must set some environment variables. Open the file “/etc/environment” as administrator in a text editor and add the three lines

export LIBVA_DRIVER_NAME=nvidia
export MOZ_DISABLE_RDD_SANDBOX=1
export NVD_BACKEND=direct

is added. The last line is currently required for Nvidia drivers from version 525. The Linux kernel must also be informed. Open the text file “/etc/default/ grub” with root authorization and add this option to the line “GRUB_CMDLINE_LINUX_DEFAULT”

nvidia-drm.modeset=1

Then execute in the terminal

sudo update-grub

and restart Linux for the changes to take effect. You can check the correct function with vainfo. The tool outputs “vainfo: Driver version: VA-API NVDEC driver [direct backend].”

You need to change some settings so that Firefox takes the new driver into account. To be on the safe side, use a new user profile to test the settings first.

In the terminal, start

firefox -P

and create and start a new profile. Call up the “about:config” address and set the following three options

media.ffmpeg.vaapi.enabled
media.rdd-ffmpeg.enabled
widget.dmabuf.force-enabled

to “true” in each case. Then restart Firefox.

To test the GPU utilization, start Nvidia X Server Settings and click on “GPU 0″ ([Nvidia model]).” The values behind “GPU Utilization” and “Video Engine Utilization” increase when hardware acceleration is used.