Open
Description
- I am reporting a bug.
- I am running the latest version of BDfR
- I have read the Opening an issue
Description
I don't think that --exclude-id-file is working, files still get downloaded
Command
bdfr download H:\programs\bdfr -L 10 -s photographs --exclude-id-file successful.txt
Environment (please complete the following information)
- OS: W10
- Python version: 3.11.0
Logs
If the files already exist in the directory then the log looks like this
[2023-08-15 15:36:25,803 - bdfr.connector - DEBUG] - Disabling the following modules:
[2023-08-15 15:36:25,804 - bdfr.connector - Level 9] - Created download filter
[2023-08-15 15:36:25,804 - bdfr.connector - Level 9] - Created time filter
[2023-08-15 15:36:25,804 - bdfr.connector - Level 9] - Created sort filter
[2023-08-15 15:36:25,889 - bdfr.connector - Level 9] - Create file name formatter
[2023-08-15 15:36:25,889 - bdfr.connector - DEBUG] - Using unauthenticated Reddit instance
[2023-08-15 15:36:26,387 - bdfr.connector - Level 9] - Created site authenticator
[2023-08-15 15:36:27,660 - bdfr.connector - DEBUG] - Added submissions from subreddit photographs
[2023-08-15 15:36:27,661 - bdfr.connector - Level 9] - Retrieved subreddits
[2023-08-15 15:36:27,661 - bdfr.connector - Level 9] - Retrieved multireddits
[2023-08-15 15:36:27,661 - bdfr.connector - Level 9] - Retrieved user data
[2023-08-15 15:36:27,662 - bdfr.connector - Level 9] - Retrieved submissions for given links
[2023-08-15 15:36:28,121 - bdfr.downloader - DEBUG] - Attempting to download submission 15ohypx
[2023-08-15 15:36:28,973 - bdfr.site_downloaders.youtube - ERROR] - �[0;31mERROR:�[0m [generic] None: '/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/' is not a valid URL. Set --default-search "ytsearch" (or run yt-dlp "ytsearch:/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/" ) to search YouTube
Traceback (most recent call last):
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1518, in wrapper
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1594, in __extract_info
ie_result = ie.extract(url)
^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\extractor\common.py", line 694, in extract
ie_result = self._real_extract(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\extractor\generic.py", line 2344, in _real_extract
raise ExtractorError(
yt_dlp.utils.ExtractorError: [generic] None: '/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/' is not a valid URL. Set --default-search "ytsearch" (or run yt-dlp "ytsearch:/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/" ) to search YouTube
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python311\Lib\site-packages\bdfr\site_downloaders\youtube.py", line 73, in get_video_data
result = ydl.extract_info(url, download=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1507, in extract_info
return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1536, in wrapper
self.report_error(str(e), e.format_traceback())
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1015, in report_error
self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 955, in trouble
raise DownloadError(message, exc_info)
yt_dlp.utils.DownloadError: �[0;31mERROR:�[0m [generic] None: '/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/' is not a valid URL. Set --default-search "ytsearch" (or run yt-dlp "ytsearch:/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/" ) to search YouTube
[2023-08-15 15:36:28,976 - bdfr.downloader - ERROR] - Could not download submission 15ohypx: No downloader module exists for url /r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/
[2023-08-15 15:36:28,977 - bdfr.downloader - DEBUG] - Attempting to download submission 15rtjy0
[2023-08-15 15:36:28,978 - bdfr.downloader - DEBUG] - Using Direct with url https://live.staticflickr.com/65535/53118203559_1b93b02cd1_o.png
[2023-08-15 15:36:29,005 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\IncredulousDylan_Tiny Sunrise_15rtjy0.png from submission 15rtjy0 already exists, continuing
[2023-08-15 15:36:29,005 - bdfr.downloader - INFO] - Downloaded submission 15rtjy0 from photographs
[2023-08-15 15:36:29,005 - bdfr.downloader - DEBUG] - Attempting to download submission 15rwgs2
[2023-08-15 15:36:29,006 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/mdsivtcqpaib1.jpg
[2023-08-15 15:36:29,007 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\WinnerUnable1722_Northern lights_15rwgs2.jpg from submission 15rwgs2 already exists, continuing
[2023-08-15 15:36:29,007 - bdfr.downloader - INFO] - Downloaded submission 15rwgs2 from photographs
[2023-08-15 15:36:29,008 - bdfr.downloader - DEBUG] - Attempting to download submission 15rr9sf
[2023-08-15 15:36:29,008 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/36hhsg37p9ib1.jpg
[2023-08-15 15:36:29,009 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\Short_Desk_1273_Golitha Falls, Cornwall._15rr9sf.jpg from submission 15rr9sf already exists, continuing
[2023-08-15 15:36:29,009 - bdfr.downloader - INFO] - Downloaded submission 15rr9sf from photographs
[2023-08-15 15:36:29,010 - bdfr.downloader - DEBUG] - Attempting to download submission 15rxrid
[2023-08-15 15:36:29,010 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/5r21ojiqyaib1.jpg
[2023-08-15 15:36:29,011 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\CitiClix_Hong Kong tram street scene_15rxrid.jpg from submission 15rxrid already exists, continuing
[2023-08-15 15:36:29,012 - bdfr.downloader - INFO] - Downloaded submission 15rxrid from photographs
[2023-08-15 15:36:29,012 - bdfr.downloader - DEBUG] - Attempting to download submission 15rvgmi
[2023-08-15 15:36:29,012 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/ze9p0yqviaib1.jpg
[2023-08-15 15:36:29,013 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\KickinAP1985_Lake Michigan sunset_15rvgmi.jpg from submission 15rvgmi already exists, continuing
[2023-08-15 15:36:29,013 - bdfr.downloader - INFO] - Downloaded submission 15rvgmi from photographs
[2023-08-15 15:36:29,014 - bdfr.downloader - DEBUG] - Attempting to download submission 15rw5sp
[2023-08-15 15:36:29,014 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/wl6hbl0onaib1.jpg
[2023-08-15 15:36:29,016 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\elizabeth896_Orchid._15rw5sp.jpg from submission 15rw5sp already exists, continuing
[2023-08-15 15:36:29,016 - bdfr.downloader - INFO] - Downloaded submission 15rw5sp from photographs
[2023-08-15 15:36:29,016 - bdfr.downloader - DEBUG] - Attempting to download submission 15rv9cu
[2023-08-15 15:36:29,017 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/3aaun7vghaib1.jpg
[2023-08-15 15:36:29,018 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\BananaMilkshakes8_Beach sunset_15rv9cu.jpg from submission 15rv9cu already exists, continuing
[2023-08-15 15:36:29,018 - bdfr.downloader - INFO] - Downloaded submission 15rv9cu from photographs
[2023-08-15 15:36:29,018 - bdfr.downloader - DEBUG] - Attempting to download submission 15rsz6z
[2023-08-15 15:36:29,019 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/neohu38q1aib1.jpg
[2023-08-15 15:36:29,019 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\bajanbigrock_Long time no NYC_15rsz6z.jpg from submission 15rsz6z already exists, continuing
[2023-08-15 15:36:29,020 - bdfr.downloader - INFO] - Downloaded submission 15rsz6z from photographs
[2023-08-15 15:36:29,020 - bdfr.downloader - DEBUG] - Attempting to download submission 15s0kx1
[2023-08-15 15:36:29,021 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/n18v7ydehbib1.jpg
[2023-08-15 15:36:29,022 - bdfr.downloader - DEBUG] - File H:\programs\bdfr\photographs\0x3Alex_Sleepy Monkey_15s0kx1.jpg from submission 15s0kx1 already exists, continuing
[2023-08-15 15:36:29,022 - bdfr.downloader - INFO] - Downloaded submission 15s0kx1 from photographs
[2023-08-15 15:36:29,023 - root - INFO] - Program complete
If the file does not exist already the log looks like this
[2023-08-15 15:39:05,961 - bdfr.connector - DEBUG] - Disabling the following modules:
[2023-08-15 15:39:05,962 - bdfr.connector - Level 9] - Created download filter
[2023-08-15 15:39:05,962 - bdfr.connector - Level 9] - Created time filter
[2023-08-15 15:39:05,962 - bdfr.connector - Level 9] - Created sort filter
[2023-08-15 15:39:06,047 - bdfr.connector - Level 9] - Create file name formatter
[2023-08-15 15:39:06,048 - bdfr.connector - DEBUG] - Using unauthenticated Reddit instance
[2023-08-15 15:39:06,484 - bdfr.connector - Level 9] - Created site authenticator
[2023-08-15 15:39:07,868 - bdfr.connector - DEBUG] - Added submissions from subreddit photographs
[2023-08-15 15:39:07,869 - bdfr.connector - Level 9] - Retrieved subreddits
[2023-08-15 15:39:07,869 - bdfr.connector - Level 9] - Retrieved multireddits
[2023-08-15 15:39:07,869 - bdfr.connector - Level 9] - Retrieved user data
[2023-08-15 15:39:07,870 - bdfr.connector - Level 9] - Retrieved submissions for given links
[2023-08-15 15:39:08,837 - bdfr.downloader - DEBUG] - Attempting to download submission 15ohypx
[2023-08-15 15:39:09,628 - bdfr.site_downloaders.youtube - ERROR] - �[0;31mERROR:�[0m [generic] None: '/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/' is not a valid URL. Set --default-search "ytsearch" (or run yt-dlp "ytsearch:/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/" ) to search YouTube
Traceback (most recent call last):
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1518, in wrapper
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1594, in __extract_info
ie_result = ie.extract(url)
^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\extractor\common.py", line 694, in extract
ie_result = self._real_extract(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\extractor\generic.py", line 2344, in _real_extract
raise ExtractorError(
yt_dlp.utils.ExtractorError: [generic] None: '/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/' is not a valid URL. Set --default-search "ytsearch" (or run yt-dlp "ytsearch:/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/" ) to search YouTube
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python311\Lib\site-packages\bdfr\site_downloaders\youtube.py", line 73, in get_video_data
result = ydl.extract_info(url, download=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1507, in extract_info
return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1536, in wrapper
self.report_error(str(e), e.format_traceback())
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 1015, in report_error
self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
File "C:\Python311\Lib\site-packages\yt_dlp\YoutubeDL.py", line 955, in trouble
raise DownloadError(message, exc_info)
yt_dlp.utils.DownloadError: �[0;31mERROR:�[0m [generic] None: '/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/' is not a valid URL. Set --default-search "ytsearch" (or run yt-dlp "ytsearch:/r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/" ) to search YouTube
[2023-08-15 15:39:09,631 - bdfr.downloader - ERROR] - Could not download submission 15ohypx: No downloader module exists for url /r/photography/comments/15o93ii/focal_point_52_weeks_prompts_august_2023/
[2023-08-15 15:39:09,632 - bdfr.downloader - DEBUG] - Attempting to download submission 15rtjy0
[2023-08-15 15:39:09,633 - bdfr.downloader - DEBUG] - Using Direct with url https://live.staticflickr.com/65535/53118203559_1b93b02cd1_o.png
[2023-08-15 15:39:11,543 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\IncredulousDylan_Tiny Sunrise_15rtjy0.png
[2023-08-15 15:39:11,544 - bdfr.downloader - DEBUG] - Hash added to master list: 19cbb907064870505a70ec5e713f194c
[2023-08-15 15:39:11,544 - bdfr.downloader - INFO] - Downloaded submission 15rtjy0 from photographs
[2023-08-15 15:39:11,546 - bdfr.downloader - DEBUG] - Attempting to download submission 15rwgs2
[2023-08-15 15:39:11,546 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/mdsivtcqpaib1.jpg
[2023-08-15 15:39:12,197 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\WinnerUnable1722_Northern lights_15rwgs2.jpg
[2023-08-15 15:39:12,198 - bdfr.downloader - DEBUG] - Hash added to master list: a6e9cd27820466bc25e7ddb1cdace69f
[2023-08-15 15:39:12,198 - bdfr.downloader - INFO] - Downloaded submission 15rwgs2 from photographs
[2023-08-15 15:39:12,199 - bdfr.downloader - DEBUG] - Attempting to download submission 15rr9sf
[2023-08-15 15:39:12,199 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/36hhsg37p9ib1.jpg
[2023-08-15 15:39:13,282 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\Short_Desk_1273_Golitha Falls, Cornwall._15rr9sf.jpg
[2023-08-15 15:39:13,284 - bdfr.downloader - DEBUG] - Hash added to master list: 35b992aa74afdd0a9c47157d34c4f3d9
[2023-08-15 15:39:13,284 - bdfr.downloader - INFO] - Downloaded submission 15rr9sf from photographs
[2023-08-15 15:39:13,285 - bdfr.downloader - DEBUG] - Attempting to download submission 15rxrid
[2023-08-15 15:39:13,285 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/5r21ojiqyaib1.jpg
[2023-08-15 15:39:13,976 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\CitiClix_Hong Kong tram street scene_15rxrid.jpg
[2023-08-15 15:39:13,977 - bdfr.downloader - DEBUG] - Hash added to master list: be3677c6505ce07921f0a3af7f03240a
[2023-08-15 15:39:13,977 - bdfr.downloader - INFO] - Downloaded submission 15rxrid from photographs
[2023-08-15 15:39:13,978 - bdfr.downloader - DEBUG] - Attempting to download submission 15rvgmi
[2023-08-15 15:39:13,978 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/ze9p0yqviaib1.jpg
[2023-08-15 15:39:15,062 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\KickinAP1985_Lake Michigan sunset_15rvgmi.jpg
[2023-08-15 15:39:15,063 - bdfr.downloader - DEBUG] - Hash added to master list: 2218685edd2e528e8acbea52814300b1
[2023-08-15 15:39:15,063 - bdfr.downloader - INFO] - Downloaded submission 15rvgmi from photographs
[2023-08-15 15:39:15,064 - bdfr.downloader - DEBUG] - Attempting to download submission 15rw5sp
[2023-08-15 15:39:15,064 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/wl6hbl0onaib1.jpg
[2023-08-15 15:39:15,825 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\elizabeth896_Orchid._15rw5sp.jpg
[2023-08-15 15:39:15,826 - bdfr.downloader - DEBUG] - Hash added to master list: 34b115c180ead6695d8b6e516dea3fed
[2023-08-15 15:39:15,826 - bdfr.downloader - INFO] - Downloaded submission 15rw5sp from photographs
[2023-08-15 15:39:15,827 - bdfr.downloader - DEBUG] - Attempting to download submission 15rv9cu
[2023-08-15 15:39:15,827 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/3aaun7vghaib1.jpg
[2023-08-15 15:39:16,504 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\BananaMilkshakes8_Beach sunset_15rv9cu.jpg
[2023-08-15 15:39:16,505 - bdfr.downloader - DEBUG] - Hash added to master list: 24c9421903da2deb8f49680352153927
[2023-08-15 15:39:16,505 - bdfr.downloader - INFO] - Downloaded submission 15rv9cu from photographs
[2023-08-15 15:39:16,506 - bdfr.downloader - DEBUG] - Attempting to download submission 15rsz6z
[2023-08-15 15:39:16,506 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/neohu38q1aib1.jpg
[2023-08-15 15:39:17,732 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\bajanbigrock_Long time no NYC_15rsz6z.jpg
[2023-08-15 15:39:17,733 - bdfr.downloader - DEBUG] - Hash added to master list: d89986e62ea138cdc1e51f723de7e7a1
[2023-08-15 15:39:17,733 - bdfr.downloader - INFO] - Downloaded submission 15rsz6z from photographs
[2023-08-15 15:39:17,734 - bdfr.downloader - DEBUG] - Attempting to download submission 15s0kx1
[2023-08-15 15:39:17,734 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/n18v7ydehbib1.jpg
[2023-08-15 15:39:19,022 - bdfr.downloader - DEBUG] - Written file to H:\programs\bdfr\photographs\0x3Alex_Sleepy Monkey_15s0kx1.jpg
[2023-08-15 15:39:19,023 - bdfr.downloader - DEBUG] - Hash added to master list: d8f7a229006994afcdd37fcaaf4847f5
[2023-08-15 15:39:19,023 - bdfr.downloader - INFO] - Downloaded submission 15s0kx1 from photographs
[2023-08-15 15:39:19,024 - root - INFO] - Program complete
Extra
This is the list of files in the directory showing that the successful.txt file does exist
and this is a snip of successful.txt showing the IDs
Activity