-
-
Notifications
You must be signed in to change notification settings - Fork 230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: Manual update - raise ValueError("failed to add item to index") #803
Labels
bug
Something isn't working
Comments
Same issue. I can confirm same as above with specific channels on 'latest' branch docker build. Shell log posted below: 2024-09-18 21:28:49 [2024-09-18 21:28:49,947: INFO/MainProcess] Task check_reindex[fc51500e-ccc5-4ecf-aca6-30feb8edc68f] received
2024-09-18 21:28:49 [2024-09-18 21:28:49,949: WARNING/ForkPoolWorker-32] check_reindex create callback
2024-09-18 21:28:49 [2024-09-18 21:28:49,952: WARNING/ForkPoolWorker-32] [task][check_reindex] reindex {'channel': ['UCzGrGrvf9g8CVVzh_LvGf-g']}
2024-09-18 21:28:49 [2024-09-18 21:28:49,968: WARNING/ForkPoolWorker-32] UCzGrGrvf9g8CVVzh_LvGf-g: get metadata from es
2024-09-18 21:28:49 [2024-09-18 21:28:49,974: WARNING/ForkPoolWorker-32] UCzGrGrvf9g8CVVzh_LvGf-g: get metadata from youtube
2024-09-18 21:28:52 [2024-09-18 21:28:52,810: WARNING/ForkPoolWorker-32] WARNING: [youtube] tLVVhfJqohU: nsig extraction failed: Some formats may be missing
2024-09-18 21:28:52 Install PhantomJS to workaround the issue. Please download it from https://phantomjs.org/download.html
2024-09-18 21:28:52 n = Nbc6T-IasNB0Ydh87 ; player = https://www.youtube.com/s/player/a9d81eca/player_ias.vflset/en_US/base.js
2024-09-18 21:28:54 [2024-09-18 21:28:54,084: WARNING/ForkPoolWorker-32] UCzGrGrvf9g8CVVzh_LvGf-g: download channel thumbnail
2024-09-18 21:28:56 [2024-09-18 21:28:56,414: WARNING/ForkPoolWorker-32] {"error":{"root_cause":[{"type":"parse_exception","reason":"[value] required property is missing","property_name":"value","processor_type":"set"}],"type":"parse_exception","reason":"[value] required property is missing","property_name":"value","processor_type":"set"},"status":400}
2024-09-18 21:28:56 [2024-09-18 21:28:56,414: WARNING/ForkPoolWorker-32] {'description': 'UCzGrGrvf9g8CVVzh_LvGf-g', 'processors': [{'set': {'field': 'channel.channel_active', 'value': True}}, {'set': {'field': 'channel.channel_description', 'value': 'Nirvana on Vevo - Official Music Videos, Live Performances, Interviews and more...'}}, {'set': {'field': 'channel.channel_id', 'value': 'UCzGrGrvf9g8CVVzh_LvGf-g'}}, {'set': {'field': 'channel.channel_last_refresh', 'value': 1726720134}}, {'set': {'field': 'channel.channel_name', 'value': 'NirvanaVEVO'}}, {'set': {'field': 'channel.channel_subs', 'value': None}}, {'set': {'field': 'channel.channel_subscribed', 'value': False}}, {'set': {'field': 'channel.channel_tags', 'value': ['vevo music video official nirvana']}}, {'set': {'field': 'channel.channel_banner_url', 'value': 'https://yt3.googleusercontent.com/icXgmwCoMtg49hJBRlRnjmScm4-_dkDZAX6ghByMNU5XDfqluUx4ItdL_1vmRKARAi6fbB-kVWI=w2560-fcrop64=1,00005a57ffffa5a8-k-c0xffffffff-no-nd-rj'}}, {'set': {'field': 'channel.channel_thumb_url', 'value': 'https://yt3.googleusercontent.com/ytc/AIdro_nEzankJ2oekN31HOpZjk5u-eEyCyLcxEjvL1bWrYjyv5M=s900-c-k-c0x00ffffff-no-rj'}}, {'set': {'field': 'channel.channel_tvart_url', 'value': 'https://yt3.googleusercontent.com/icXgmwCoMtg49hJBRlRnjmScm4-_dkDZAX6ghByMNU5XDfqluUx4ItdL_1vmRKARAi6fbB-kVWI=s0'}}, {'set': {'field': 'channel.channel_views', 'value': 0}}]}
2024-09-18 21:28:56 [2024-09-18 21:28:56,418: WARNING/ForkPoolWorker-32] fc51500e-ccc5-4ecf-aca6-30feb8edc68f Failed callback
2024-09-18 21:28:56 [2024-09-18 21:28:56,420: ERROR/ForkPoolWorker-32] Task check_reindex[fc51500e-ccc5-4ecf-aca6-30feb8edc68f] raised unexpected: ValueError('failed to add item to index')
2024-09-18 21:28:56 Traceback (most recent call last):
2024-09-18 21:28:56 File "/root/.local/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
2024-09-18 21:28:56 R = retval = fun(*args, **kwargs)
2024-09-18 21:28:56 ^^^^^^^^^^^^^^^^^^^^
2024-09-18 21:28:56 File "/root/.local/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
2024-09-18 21:28:56 return self.run(*args, **kwargs)
2024-09-18 21:28:56 ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-18 21:28:56 File "/app/home/tasks.py", line 203, in check_reindex
2024-09-18 21:28:56 handler.reindex_all()
2024-09-18 21:28:56 File "/app/home/src/index/reindex.py", line 276, in reindex_all
2024-09-18 21:28:56 self.reindex_type(name, index_config)
2024-09-18 21:28:56 File "/app/home/src/index/reindex.py", line 291, in reindex_type
2024-09-18 21:28:56 reindex(youtube_id)
2024-09-18 21:28:56 File "/app/home/src/index/reindex.py", line 378, in _reindex_single_channel
2024-09-18 21:28:56 channel.sync_to_videos()
2024-09-18 21:28:56 File "/app/home/src/index/channel.py", line 167, in sync_to_videos
2024-09-18 21:28:56 _, _ = ElasticWrap(ingest_path).put(data)
2024-09-18 21:28:56 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-09-18 21:28:56 File "/app/home/src/es/connect.py", line 113, in put
2024-09-18 21:28:56 raise ValueError("failed to add item to index")
2024-09-18 21:28:56 ValueError: failed to add item to index
2024-09-18 21:28:56 [2024-09-18 21:28:56,421: WARNING/ForkPoolWorker-32] fc51500e-ccc5-4ecf-aca6-30feb8edc68f return callback |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've read the documentation
Operating System
Docker (TA 0.4.10 / ES8.14.3 /Redis-Stack latest) on Debian ARM64
Your Bug Report
Describe the bug
manage.py ta_fix_channels stopped with ValueError("failed to add item to index") with certain channels.
Steps To Reproduce
Expected behavior
Process finish with no errors.
Relevant log output
Anything else?
skip troubled channel instead of a stop (halt)?
Thanks for your time.
The text was updated successfully, but these errors were encountered: