Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: KeyError 'channel' in playlist.py line 96, tubearchivist stopped downloading new videos / all playlists deactivated #835

Open
2 tasks done
gebi opened this issue Nov 16, 2024 · 3 comments
Open
2 tasks done

[Bug]: KeyError 'channel' in playlist.py line 96, tubearchivist stopped downloading new videos / all playlists deactivated #835

gebi opened this issue Nov 16, 2024 · 3 comments
Labels
question Further information is requested

Comments

@gebi
Copy link

gebi commented Nov 16, 2024

I've read the documentation

Operating System

Ubuntu 22.04 with docker compose

Your Bug Report

Describe the bug

Adding playlist PLbFspXL_XHc0Vk1MoObY9XLa1i6c-L1Pb on 0.4.11 creates a deactived playlist (see debug output)
When the playlist was initially synced with 0.4.10, both 0.4.10 and 0.4.11 stop downloading new videso completely.

Synced with tubearchivist 0.4.10 created a playlist that is shown as containing no videos, but has the thumbnail of the first video shown as playlist thumbnail.
after playlist -> delete -> all, and resync with 0.4.11 playlist is shown as 'deactivated' (deleting only metadata was not enough, interestingly enough, all videos are downloaded, just not shown in the playlist)

logs for both problems below

  1. tubearchivist stops downloading new videos completely (when playlist is synced with 0.4.10
  2. playlist is deactivated when synced with 0.4.11, but download of everything else still works, actually currently all playlists are shown as 'deactivated', and all playlist log the same error as the problematic playlist

Steps To Reproduce

adding playlist and trying to sync it

Expected behavior

not stopping the whole tubearchivist instance to stop syncing (for playlist initially added with 0.4.10)
and syncing the playlist when playlist is added on 0.4.11

Relevant log output

# ad 1 - tubearchivist stops downloading new videos completely (when playlist is synced with `0.4.10`

tubearchivist    | [2024-11-16 18:06:29,655: ERROR/ForkPoolWorker-16] Task update_subscribed[e718e3c8-4add-4e2b-8d87-85cb556361b9] raised unexpected: KeyError('channel')
tubearchivist    | Traceback (most recent call last):
tubearchivist    |   File "/root/.local/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
tubearchivist    |     R = retval = fun(*args, **kwargs)
tubearchivist    |                  ^^^^^^^^^^^^^^^^^^^^
tubearchivist    |   File "/root/.local/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
tubearchivist    |     return self.run(*args, **kwargs)
tubearchivist    |            ^^^^^^^^^^^^^^^^^^^^^^^^^
tubearchivist    |   File "/app/home/tasks.py", line 107, in update_subscribed
tubearchivist    |     missing_videos = handler.scan()
tubearchivist    |                      ^^^^^^^^^^^^^^
tubearchivist    |   File "/app/home/src/download/subscriptions.py", line 340, in scan
tubearchivist    |     self.scan_playlists()
tubearchivist    |   File "/app/home/src/download/subscriptions.py", line 359, in scan_playlists
tubearchivist    |     missing = playlist_handler.find_missing()
tubearchivist    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tubearchivist    |   File "/app/home/src/download/subscriptions.py", line 291, in find_missing
tubearchivist    |     is_active = playlist.update_playlist()
tubearchivist    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
tubearchivist    |   File "/app/home/src/index/playlist.py", line 173, in update_playlist
tubearchivist    |     self.build_json(scrape=True)
tubearchivist    |   File "/app/home/src/index/playlist.py", line 50, in build_json
tubearchivist    |     self.get_entries(ids_found)
tubearchivist    |   File "/app/home/src/index/playlist.py", line 96, in get_entries
tubearchivist    |     if not entry["channel"]:
tubearchivist    |            ~~~~~^^^^^^^^^^^
tubearchivist    | KeyError: 'channel'
tubearchivist    | [2024-11-16 18:06:29,656: WARNING/ForkPoolWorker-16] e718e3c8-4add-4e2b-8d87-85cb556361b9 return callback

# ad 2 - playlist is deactivated when synced with `0.4.11`, but download of everything else still works

tubearchivist    | [2024-11-16 18:33:02,777: WARNING/ForkPoolWorker-16] PLbFspXL_XHc0Vk1MoObY9XLa1i6c-L1Pb: get metadata from es
tubearchivist    | [2024-11-16 18:33:02,781: WARNING/ForkPoolWorker-16] PLbFspXL_XHc0Vk1MoObY9XLa1i6c-L1Pb: get metadata from youtube
tubearchivist    | [2024-11-16 18:33:03,348: WARNING/ForkPoolWorker-16] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
tubearchivist    | [2024-11-16 18:33:03,349: WARNING/ForkPoolWorker-16] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (1/3)...
tubearchivist    | [2024-11-16 18:33:03,569: WARNING/ForkPoolWorker-16] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
tubearchivist    | [2024-11-16 18:33:03,569: WARNING/ForkPoolWorker-16] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (2/3)...
tubearchivist    | [2024-11-16 18:33:03,798: WARNING/ForkPoolWorker-16] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
tubearchivist    | [2024-11-16 18:33:03,798: WARNING/ForkPoolWorker-16] WARNING: [youtube:tab] Incomplete yt initial data received. Retrying (3/3)...
tubearchivist    | [2024-11-16 18:33:04,033: WARNING/ForkPoolWorker-16] WARNING: [youtube:tab] unable to extract yt initial data; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
tubearchivist    | [2024-11-16 18:33:04,033: WARNING/ForkPoolWorker-16] WARNING: [youtube:tab] Incomplete yt initial data received; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest
version using  yt-dlp -U. Giving up after 3 retries
tubearchivist    | [2024-11-16 18:33:04,035: WARNING/ForkPoolWorker-16] ERROR: [youtube:tab] PLbFspXL_XHc0Vk1MoObY9XLa1i6c-L1Pb: Playlists that require authentication may not extract correctly without a successful webpage download. If you are not downloading private content, or your cookies are only for the first account and channel, pass "--extractor-args youtubetab:skip=authcheck" to skip this check
tubearchivist    | [2024-11-16 18:33:04,035: WARNING/ForkPoolWorker-16] https://www.youtube.com/playlist?list=PLbFspXL_XHc0Vk1MoObY9XLa1i6c-L1Pb: failed to get info from youtube with message ERROR: [youtube:tab] PLbFspXL_XHc0Vk1MoObY9XLa1i6c-L1Pb: Playlists that require authentication may not extract correctly without a successful webpage download. If you are not downloading private content, or your cookies are only
for the first account and channel, pass "--extractor-args youtubetab:skip=authcheck" to skip this check
tubearchivist    | [2024-11-16 18:33:04,035: WARNING/ForkPoolWorker-16] PLbFspXL_XHc0Vk1MoObY9XLa1i6c-L1Pb: deactivate document

Anything else?

https://www.youtube.com/playlist?list=PLbFspXL_XHc0Vk1MoObY9XLa1i6c-L1Pb

if i'd had to guess, this video might be the problem https://www.youtube.com/watch?v=-mTmvZjNPdU
@bbilly1
Copy link
Member

bbilly1 commented Nov 18, 2024

You are getting blocked by YT, see our FAQ with some workarounds. https://docs.tubearchivist.com/faq/#am-i-getting-blockedthrottled

Once you unblock your IP/account this should work again.

@bbilly1 bbilly1 added the question Further information is requested label Nov 18, 2024
@gebi
Copy link
Author

gebi commented Nov 18, 2024

oh wow... i've no idea what happened.

playlist always shown as having zero videos, new ip and updated session did not help, BUT after adding one new video to the playlist and deleting the cookie in ta it synced the new video and now the playlist is not shown as empty anymore but contains all videos, no idea how because it definitely did not download the rest of the videos, but works again now.

thx for your patience, and sry for wasting your time :/

@ajw107
Copy link

ajw107 commented Nov 27, 2024

I was having the same problem with the same video -mTmvZjNPdU (not playlist), not even sure how the video got into my downloads, as I never requested it (not that it shows there, just see it mentioned at the end of a download subscription scan in the logs). But removing your cookie, and re-importing a new one solves it (no need to add a video to a playlist, although I couldn't as I didn't have that playlist).
However you may then run into this insane bug: @yt-dlp/yt-dlp#11640 where we now need to turn off cookies for ios api, and update yt-dlp really for mweb support by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gebi @ajw107 @bbilly1