Better, faster screen capture with kmsgrab

Those of you that have tried to use x11grab are probably aware that it is terrible at high resolutions. To be fair, it's not really FFMPEGs fault either to capture an aging X11 window (yes I remain hopeful for Wayland). The user experience isn't great though, and not only does it drop frames, it tends to lag the entire X server causing everything on screen to stutter as well. By the way, this isn't limited to FFMPEG, full XSHM screen capture on OBS has the same issue (and probably uses similar underlying logic.

There are a few alternatives:

• However Xcomposite works on OBS doesn't seem to be prone to this issue
• OpenGL capture also seems to work fine at higher resolutions

Anyways, here, I'm going to describe kmsgrab. Its another capture source for ffmpeg that works much better then x11grab (with some limitations). It still drops some frames but overall seems better then x11grab. It is quirky sometimes though and isn't as flexible. Also, it doesn't capture the cursor (not sure why tbh, I guess something about compositing?)

Information on the input is available here: https://ffmpeg.org/ffmpeg-devices.html#kmsgrab

To use it, you first need to enable CAP_SYS_ADMIN as follows

sudo setcap cap_sys_admin+ep /path/to/ffmpeg

Credit to http://lists.ffmpeg.org/pipermail/ffmpeg-user/2018-June/040328.html that helped me figure that part out.

You can run `which ffmpeg` to find out your binary location, but it is typically /usr/bin/ffmpeg. So, you would set it by running

sudo setcap cap_sys_admin+ep /usr/bin/ffmpeg

Then, to capture, you can use the following

ffmpeg -device /dev/dri/card0 -f kmsgrab -i - -vf 'hwmap=derive_device=vaapi,scale_vaapi=w=1920:h=1080:format=nv12' -c:v h264_vaapi -b:v 2500k -maxrate 4000k -y output-file.mkv

This also captures with VAAPI to use hardware encoding. You need vaapi setup to use it, otherwise, use libx264. See https://wiki.archlinux.org/title/Hardware_video_acceleration and https://trac.ffmpeg.org/wiki/Hardware/VAAPI on using VAAPI.

Adding -b:v and -maxrate will use VBR (in theory). Use the same value for both to get CBR instead (useful if you're going to stream straight to Twitch/Owncast). VAAPI also supports -qp for constant quantization parameter, but not CRF which would be superior since it can do more analysis to set the qp. Also, don't forget to add audio capture with ALSA/Pulse/Jack.

Note, you may need to change /dev/dri/card0 to a different DRI if you have 2 GPUs (eg: integrated and dedicated). I also found that kmsgrab didn't work on my desktop with an RX 480. No idea why. It works fine, I just had to pick the correct card and plane. It was defaulting to the iGPU instead of the dGPU, and since there's nothing in that buffer, it would fail.

As always, I recommend never recording to mp4 files (or anything else w/ metadata at the end) since you will corrupt the file if the recording crashes. Use mkv (or ts) instead.

Looks like the h264_vaapi has a problem initializing and saving in the mkv file. A workaround is to use CPU encoding, but note that on AMD video cards, you'll need to keep the vaapi section, just not the encoder. For this, swap to normal scaling, and bgr0 format. Here is an example using the mpeg2 encoder, since it sucks, and is fast enough on slow computers.

ffmpeg -re -f alsa -i pulse -framerate 15 -device /dev/dri/card0 -re -f kmsgrab -i - -vf 'hwmap=derive_device=vaapi,hwdownload,format=bgr0' -s 1280x720 -c:v mpeg2video -q:v 10 -g 240 -c:a libmp3lame -b:a 64k -y ~/Recordings/screencap-(date +%Y-%m-%d--%H-%M).avi