Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect A/V out of sync. #1909

Open
changxiangzhong opened this issue Nov 21, 2024 · 1 comment
Open

Detect A/V out of sync. #1909

changxiangzhong opened this issue Nov 21, 2024 · 1 comment
Assignees
Labels

Comments

@changxiangzhong
Copy link
Contributor

We have a few reports regarding A/V out of sync issue, it's rarely reproducible, but we want to capture it in our analytics system, to understand how bad it is. I've looked into EventLogger and ExoPlayer's internal Listener, I couldn't find anything.

Could you please shred some light on out of sync detection?

@icbaker
Copy link
Collaborator

icbaker commented Nov 25, 2024

Does the player know the audio and video are out of sync?

Generally ExoPlayer runs the playback clock based on timestamps from the audio track, and aims to release video frames to the screen so they are in sync with the corresponding audio timestamp.

If you see A/V out of sync some possible causes are:

  1. Video decoding is too slow, so the frames aren't ready when they need to be released to the screen, so they are released late. In this case the player knows it's out of sync. The demo app displays 'video frame processing offset' (vfpo`) in its debug text at the top of the playback screen which measures this. See more info here: https://medium.com/google-exoplayer/improved-rendering-performance-operating-mediacodec-in-asynchronous-mode-and-asynchronous-buffer-3026207850b2
  2. The media has incorrect/mismatched timestamps, so the player does everything "right" but the user perceives the audio and video are not in sync.
  3. There's a bug in the player logic which is aiming to keep audio and video in sync (and so by definition, the player doesn't know the audio and video are out sync).

In case 1, you can measure vfpo from DecoderCounters in the same way as the demo app.

In case 2 and 3, there's not really a way to measure this from within ExoPlayer, because if the player was able to measure a sync problem (and we're not performance-constrained, i.e. case (1)), it would just fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants