Try to make gstreamer x11imagesrc bearable

OBS has x11 screen capture (terrible perf) or window capture with xcomposite (good). FFMPEG has x11grab (terrible) or kmsgrab (good).

As far as I know, gstreamer can't do kms capture, can't do GL image capture, nor has an xcomposite equivalent. I haven't found an equivalent high performance capture with gstreamer short of doing something like just grabbing the texture and pushing it through an appsrc.

    /* Create the source */
    ximagesrc = gst_element_factory_make("ximagesrc", "ximagesrc");
    /* If you're running over SSH, equivalent to setting DISPLAY=:0 */
    g_object_set(ximagesrc, "display-name", ":0", NULL);

    /* This is maybe a secret to getting usable screen capture performance with ximagesrc for this use case.
    If remote=FALSE, I've found that this drops a lot of frames, and may end up needing have use-damage=true.
    If remote=TRUE, it appears to be significantly more efficient, and you can use use-damage=false if you want. */
    g_object_set(ximagesrc, "remote", TRUE, NULL);

    /* This one depends on the use case. If not a lot is changing, true is better.
    If a lot of things are going to change on each frame, false is better. */
    g_object_set(ximagesrc, "use-damage", TRUE, NULL);
    /* Create the caps filter. If you try capturing from the ximagesrc at too high of
    a framerate, you will run into massive stuttering and dropped frames. At that point,
    it may be better to not limit it at all. */
    caps = gst_caps_new_simple("video/x-raw",
                               "framerate", GST_TYPE_FRACTION, 20, 1,
  • scripts/ffmpeg/gstreamer_ximagesrc_slightly_better_performance.txt
  • Last modified: 2024-05-14 21:59
  • by Tony