Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[deviantart][warning] API responded with 429 Too Many Requests #2143

Open
rufi-san opened this issue Dec 26, 2021 · 40 comments
Open

[deviantart][warning] API responded with 429 Too Many Requests #2143

rufi-san opened this issue Dec 26, 2021 · 40 comments

Comments

@rufi-san
Copy link

rufi-san commented Dec 26, 2021

Sin títuloo

so i tried to download in deviantart but this happen, i tried to clear the cache and puts some comand (for window) i now like

gallery-dl.exe url

gallery-dl -u "username" -p "pasword" url

gallery-dl -v -o "extractor.oauth.browser=false" oauth:deviantart

in other sites download perfectly but i dont now why this happens now in deviantart

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

You already asked it #1930.
Do it again.

UPD:
See the guide here: #2143 (comment)

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

Nice title bar text.

@Twi-Hard
Copy link

You already asked it #1930. Do it again.

This is a different problem.

You need to use your own API credentials to not get 429 errors.
https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractordeviantartclient-id--client-secret

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

Based on his response #1930 (comment) it looks that he has resolved both problems (throttling and private works access) then.

@rufi-san
Copy link
Author

You already asked it #1930. Do it again.

yeah i do that method but still have the same problem, like i said i have put it like this

gallery-dl -v -o "extractor.oauth.browser=false" oauth:deviantart

and still the same

and not a acount i submit thinks so dont worry for the cath xD

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

You already asked it #1930. Do it again.

This is a different problem.

You need to use your own API credentials to not get 429 errors. https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractordeviantartclient-id--client-secret

edited:

I have the client id and cliente secret but i dont now how to put it in the configuration (i work in window)

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#configuration


  1. Create gallery-dl.conf in C:%HOMEPATH%\ (your home dir).

  2. Fill it with the content:

{
    "extractor":
    {
        "base-directory": "./",
        "deviantart":
        {
            "quality": 100,
            "directory": ["[gallery-dl]", "[{category}] {author[username]}"],
            "filename": "[{category}] {author[username]}—{index}—{date:%Y.%m.%d}—{title}.{extension}",
            "client-id": "12345",
            "client-secret": "0123456789abcdef0123456789abcdef",
            "refresh-token": "0123456789abcdef0123456789abcdef01234567"
        }
    }
}
  1. Replace
 "client-id": "12345",
 "client-secret": "0123456789abcdef0123456789abcdef",
 "refresh-token": "0123456789abcdef0123456789abcdef01234567"

with yours.

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

I highly recommend to use "filename" from my example above instead of the default filename pattern.

With it your files will look so:

  • [deviantart] Hosha-Usagi—798062379—2019.05.18—_CM_ Mimi 1_2_ ViPOP.png
  • [deviantart] Hosha-Usagi—801389877—2019.06.12—CM_ Sakura_ babypippo.png
  • [deviantart] Hosha-Usagi—877532727—2021.04.25—CM_Myriel_Ruzakali.png

instead of

  • deviantart_798062379__CM_ Mimi 1_2_ ViPOP.png
  • deviantart_801389877_CM_ Sakura_ babypippo.png
  • deviantart_877532727_CM_Myriel_Ruzakali.png

Also, to save the text descriptions, you can add this:

"#": "put description texts into a separate directory",
"metadata": true,
"postprocessors": [
{
"name": "metadata",
"mode": "custom",
"directory" : "Descriptions",
"content-format" : "{description}\n",
"extension-format": "descr.txt"
}
]

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

thanks for all the examples and the effort i very apreciated but still dont now how to put it, i copy and do like you said to me but the cmd dont recognize the comand

like this

1

and when i have the refresh token and the put this

G: ( i try to copy the comand like you told me and repplace de id, secret and token but dont recognize any of the comand, or very much i put it wrong) {
"extractor":
{
"base-directory": "./",
"deviantart":
{
"quality": 100,
"directory": ["[gallery-dl]", "[{category}] {author[username]}"],
"filename": "[{category}] {author[username]}—{index}—{date:%Y.%m.%d}—{title}.{extension}",
"client-id": "12345",
"client-secret": "0123456789abcdef0123456789abcdef",
"refresh-token": "0123456789abcdef0123456789abcdef01234567"
}
}
}

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

A command? It's the file's content.

gallery-dl.conf is a text file in your home dir.

  • Create it.
  • Put the content to the file.
  • Edit it by replacing the example credentials with yours.

gallery-dl.conf, not gallery-dl.conf.txt.


https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractordeviantartclient-id--client-secret

@rufi-san
Copy link
Author

A command? It's the file's content.

gallery-dl.conf is a text file in your home dir.

  • Create it.
  • Put the content to the file.
  • Edit it by replacing the example credentials with yours.

gallery-dl.conf, not gallery-dl.conf.txt.

https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractordeviantartclient-id--client-secret

i dont have a text file, i only have the gallery.exe , i downloated for window

i have to download a text file? where?

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

Say the third time: in home directory aka C:%HOMEPATH%.

In C:\Users\Usuario\ create: gallery-dl.conf file:

  • C:\Users\Usuario\gallery-dl.conf

Manually, or with CMD command: copy NUL "C:%HOMEPATH%\gallery-dl.conf"
(it creates an empty file)

Then edit it with a text editor.

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

Say the third time: in home directory aka C:%HOMEPATH%.

In C:\Users\Usuario\ create: gallery-dl.conf file:

  • C:\Users\Usuario\gallery-dl.conf

Manually, or with CMD command: copy NUL "C:%HOMEPATH%\gallery-dl.conf" (it creates an empty file)

Then edit it with a text editor.

thanks for your patiente and I´m very very sorry for not understand but i really dumb in this. i do this (put i text in the homepath i copy past the comands) but i really stuck and i dont now what to do
3

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

You have missed } at the last line in the screenshot. It's required.

The command "copy NUL "C:%HOMEPATH%\gallery-dl.conf"" is no need anymore.

@rufi-san
Copy link
Author

i put the } in the text and then refresh the token and replaced in the text and then still the same issue or i dont now what to do next

4

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

Technically it should work.

Do the new downloaded files have the new filename format from the config?
If yes, gallery-dl correctly sees the config file.

By the way, it looks that "refresh-token" is not required to specify manually.

Try this config (don't forget to replace "client-id", "client-secret" again).

{
    "extractor":
    {
        "base-directory": "./",
        "deviantart":
        {
            "quality": 100,
            "directory": ["[gallery-dl]", "[{category}] {author[username]}"],
            "filename": "[{category}] {author[username]}—{index}—{date:%Y.%m.%d}—{title}.{extension}",
            "client-id": "123",
            "client-secret": "123123123"
        },
        "twitter":
        {
            "retweets": "original",
            "videos": true,
            "directory": ["[gallery-dl]", "[{category}] {author[name]}"],
            "filename": "[{category}] {author[name]}—{date:%Y.%m.%d}—{retweet_id|tweet_id}—{filename}.{extension}"
        }
    }
}

with https://twitter.com/SpaceX/status/1473671809870725121 (download this)

The result filename should be:

  • [twitter] SpaceX—2021.12.22—1473671809870725121—FHOHgQGUYAIM3r9.jpg

So, the config is used well (if the filename is so).

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

Technically it should work.

Do the new downloaded files have the new filename format from the config? If yes, gallery-dl correctly sees the config file.

By the way, it looks that "refresh-token" is not required to specify manually.

Try this config (don't forget to replace "client-id", "client-secret" again).

{
    "extractor":
    {
        "base-directory": "./",
        "deviantart":
        {
            "quality": 100,
            "directory": ["[gallery-dl]", "[{category}] {author[username]}"],
            "filename": "[{category}] {author[username]}—{index}—{date:%Y.%m.%d}—{title}.{extension}",
            "client-id": "12345",
            "client-secret": "0123456789abcdef0123456789abcdef"
        },
        "twitter":
        {
            "retweets": "original",
            "videos": true,
            "directory": ["[gallery-dl]", "[{category}] {author[name]}"],
            "filename": "[{category}] {author[name]}—{date:%Y.%m.%d}—{retweet_id|tweet_id}—{filename}.{extension}"
        }
    }
}

with https://twitter.com/SpaceX/status/1473671809870725121 (download this)

The result filename should be:

  • [twitter] SpaceX—2021.12.22—1473671809870725121—FHOHgQGUYAIM3r9.jpg

Do the new downloaded files have the new filename format from the config? what? i dont understand this, sorry

and i tried but dosent download like yourse with the [twitter] SpaceX—2021.12.22—1473671809870725121—FHOHgQGUYAIM3r9.jpg , for sure i did something wrong or put the text some place wrong (i put the text in the %HOMEPATH user like you told me)
5

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

It only means that gallery-dl does not see the gallery-dl.conf file.

It's strange.
While I use gallery-dl installed with "pip", but I think it should work the same way for the standalone executable.

For example, this command

type C:%HOMEPATH%\gallery-dl.conf

should list the content of the gallery-dl.conf.

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

It only means that gallery-dl does not see the gallery-dl.conf file.

It's strange. While I use gallery-dl installed with "pip", but I think it should work the same way for the standalone executable.

For example, this command

type C:%HOMEPATH%\gallery-dl.conf

should list the content of the gallery-dl.conf.

i think the same, like they dont detect the gallery-dl (i have the gallery-dl.exe in the %HOMEPATH% too)

and dont recognize the comand C:%HOMEPATH%\gallery-dl.conf

7

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

type is a part of the command, but even without it: C:%HOMEPATH%\gallery-dl.conf should just open the text file.

Okay, what is the result of these commands:

echo %HOMEPATH%
echo %USERPROFILE%

Move gallery-dl.conf to the result of echo %USERPROFILE%.


https://github.com/mikf/gallery-dl#configuration

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

type is a part of the command, but even without it: C:%HOMEPATH%\gallery-dl.conf should just open the text file.

Okay, what is the result of these commands:

echo %HOMEPATH%
echo %USERPROFILE%

Move gallery-dl.conf to the result of echo %USERPROFILE%.

https://github.com/mikf/gallery-dl#configuration

i typed the comand echo %USERPROFILE% and i alredy have de gallery-dl.conf in the userprofile
8

edited

should i put the gallery-dl.configuration in another format?

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

Possibly (for unknown for me reason) you already have the config file in
%APPDATA%\gallery-dl\config.json
or in
%USERPROFILE%\gallery-dl\config.json.

Remove them if they are.

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

Possibly (for unknown for me reason) you already have the config file in %APPDATA%\gallery-dl\config.json or in %USERPROFILE%\gallery-dl\config.json.

Remove them if they are.

in the app data i think i have only the cache, the text for the gallery-dl.configuration is the first time i made it and i have put it in the user file like you tolde me

and the user profile i have the gallery-dl.configuration and the gallery-dl
9

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

gallery-dl.conf, not gallery-dl.conf.txt.

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

Enable showing of the file extensions.

Remove "gallery-dl.conf" which is "gallery-dl.conf.txt" actually.

Then create it with the correct name:

copy NUL "C:%HOMEPATH%\gallery-dl.conf"

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

@mikf
Add in --version showing if there is any config file in use.
Add a command to create a dummy config file.
Read also gallery-dl.conf.txt?

To resolve such problems in future.

Or add a command to open (and create an empty file if it does not exist) the config file.
--open-config

@rufi-san
Copy link
Author

10

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

I have clearly said that to delete.

Or enable showing of the file extensions in File Explorer settings and rename gallery-dl.conf.txt to gallery-dl.conf.

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

I have clearly said that to delete.

Or enable showing of the file extensions in File Explorer settings and rename gallery-dl.conf.txt to gallery-dl.conf.

okey, i changed the in gallery-dl.conf.txt (sorry i have a very bad english so sometimes i dont understand cleary )

add --version in what comand?

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

add --version in what comand?

It was a message not for you.


i changed the in gallery-dl.conf.txt

If you just have changed the name of the config file from gallery-dl.conf.txt to gallery-dl.conf.
Now it should work.

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

add --version in what comand?

It was a message not for you.

ah sorry, but something happend with changing the name of gallery-dl.txt

11

they copy a archive text
12

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

Just enable the showing of the file extensions in File Explorer settings.
https://www.thewindowsclub.com/show-file-extensions-in-windows

@rufi-san
Copy link
Author

rufi-san commented Dec 27, 2021

Just enable the showing of the file extensions in File Explorer settings. https://www.thewindowsclub.com/show-file-extensions-in-windows

my menu is in spanish and i dont now how to search very well, but the issue still the same and the delay in deviantart still the same too

edited: I have the option for showing the hiden files, folder and drivers activated

13

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

Because of gallery-dl.conf is empty.

(gallery-dl sees it now finally)

@rufi-san
Copy link
Author

so i need to copy the text of the gallery-dl.conf.txt in the gallery-dl? right?

@AlttiRi
Copy link

AlttiRi commented Dec 27, 2021

to copy the text of the gallery-dl.conf.txt in the gallery-dl

Just do it.

I have the option for showing the hiden files, folder and drivers activated

I asked for the other thing.

In fact your gallery-dl.conf.txt is gallery-dl.conf.txt.txt and gallery-dl is gallery-dl.conf.

@rufi-san
Copy link
Author

Just do it.

they work!! aleluya!! thank you very much!! realy! i dont now how you have so many patient but thankyou!!! ;_;!

@ghost
Copy link

ghost commented Dec 28, 2021

I'm having the same issue, and nothing works. I followed all the instructions here, and it's still stuck giving me errors. I didn't have to deal with this when I started using gallery-dl a few weeks ago, so I don't understand why it's happening now.

@AlttiRi
Copy link

AlttiRi commented Dec 28, 2021

But there are only two things to do:

1. Create custom gallery-dl.conf file

  1. Create empty gallery-dl.conf file by running in CMD.exe:
copy NUL "%USERPROFILE%\gallery-dl.conf"
  1. Open the file with a text editor. Just run in the console:
"%USERPROFILE%\gallery-dl.conf"
  1. Fill the file with:
{
    "extractor":
    {
        "base-directory": "./",
        "deviantart":
        {
            "directory": ["[gallery-dl]", "[{category}] {author[username]}"],
            "filename": "[{category}] {author[username]}—{index}—{date:%Y.%m.%d}—{title}.{extension}",
            "client-id": "12345",
            "client-secret": "0123456789abcdef0123456789abcdef"
        },
        "twitter":
        {
            "retweets": "original",
            "videos": true,
            "directory": ["[gallery-dl]", "[{category}] {author[name]}"],
            "filename": "[{category}] {author[name]}—{date:%Y.%m.%d}—{retweet_id|tweet_id}—{filename}.{extension}"
        }
    }
}
  1. Verify the config is working:
gallery-dl https://twitter.com/SpaceX/status/1473671809870725121

The result filename should be:

  • [twitter] SpaceX—2021.12.22—1473671809870725121—FHOHgQGUYAIM3r9.jpg

2. Then use your credentials

Then follow this guide:
https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#extractordeviantartclient-id--client-secret

To get "client-id" and "client-secret" in order to replace with them the dummy values in gallery-dl.conf to bypass 429 error and download sub-only works on Deviantart.

@ghost
Copy link

ghost commented Dec 28, 2021

The first method got it working. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants
@Twi-Hard @AlttiRi @rufi-san and others