Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keyring errors during non-publishing operations #1917

Closed
wahuneke opened this issue Jan 18, 2020 · 71 comments · Fixed by #8227
Closed

Keyring errors during non-publishing operations #1917

wahuneke opened this issue Jan 18, 2020 · 71 comments · Fixed by #8227
Labels
kind/bug Something isn't working as expected

Comments

@wahuneke
Copy link

I gather that keyring integration was added as a convenience feature so that you can publish packages using keys stored in your keyring. This ticket describes that feature:

#210

But it appears that poetry tries to access the keyring even for install operations. In one of my first times using poetry - I, as a security paranoid person, decided to abort installation of a py package I wanted because the installation process seemed to be wanting to access my keyring.

I think people should be careful about granting programs access to things they should not need to access. This behavior is an example of a problem that makes that hard to achieve.

For me, I worked around the problem by pip uninstalling the 'keyring' package from that virt env.

In the future, I think it would be beneficial to poetry's adoption and to its users, if users are not prompted for keyring unless keys should actually be needed (for a 'publish' operation).

Thanks! I'm happy to be trying out poetry.

@wahuneke wahuneke added the kind/bug Something isn't working as expected label Jan 18, 2020
@sdispater
Copy link
Member

Do you have private repositories present in your pyproject.toml file?

I think you are affected by the issue that is fixed by #1892. The fix will be available in the next bug fix release.

@colatkinson
Copy link

I was affected by something similar on the latest release (1.0.5), but I'm not sure if it's entirely the same.

The keyring will be hit for private repos if there is an auth.toml with only the username set, even if credentials are provided via environment variables.

I believe the relevant code is here. In this case, auth will be truthy, causing the keyring path to be taken.

In our case, the "fix" was to remove any leftover auth.toml files in the environment. Manually uninstalling keyring had the same effect.

I was wondering your thoughts on adding a config variable that would prevent any keyring hits altogether (by e.g. unconditionally marking the keyring as unavailable). This would have largely the same effect as @wahuneke's workaround, but might be a bit cleaner for CI pipelines and such.

@asandeep
Copy link

+1 to this. I am facing similar issue while integrating poetry with Jenkins. Even though private repo credentials were provided via environment variables, poetry still tries to look into keyring for credentials.

@dariobig
Copy link

dariobig commented Jun 4, 2020

+1 running into this issue while working over ssh

@bdowling
Copy link

fwiw, I came across this issue and agree with the concerns mentioned above.

But fwiw, I came here due to a issue with my distro that shows a dependency mismatch on keyring. keyring==18 is required (circa 2019), where as my distro has keyring-21.2 available, not sure without looking on the compatibility issues, but thought I'd mention if in case that can be relaxed....

$ poetry init
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (keyring 21.2.1 (/usr/lib/python3.8/site-packages), Requirement.parse('keyring==18.*,>=18.0.0'), {'poetry'})

@kakarukeys
Copy link

kakarukeys commented Jun 18, 2021

I'm seeing this issue in poetry 1.1.6

My setup:
MacOS Mojave 10.14.6
localhost private pypiserver with authentication required for upload only

[[tool.poetry.source]]
name = "foo"
url = "http://localhost/simple/"
secondary = true
$ poetry config --list
cache-dir = "..."
experimental.new-installer = true
installer.parallel = true
repositories.foo.url = "http://localhost"
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.path = "{cache-dir}/virtualenvs" 
$ cat auth.toml
[http-basic]
[http-basic.foo]
username = "kakarukeys"

The commands that poetry asked for keyring access:
poetry add lxml ??? why? it should get the package from the official PyPI
poetry run python --version this makes no sense either.

if you need the installation scripts for the local private pypi let me know

@alesdakshanin
Copy link

+1 this

@abn
Copy link
Member

abn commented Jun 1, 2022

Poetry searches for a package in all package sources by default.

Additionally you might want to add the publishing repository with a different name, if the source uses the same name authenticator will load the credentials.

@erooke
Copy link

erooke commented Sep 3, 2022

Poetry 1.2 now hits the keyring for most operations (add, update, install, self update, etc) rendering it unusable for me. I can "fix" the problem by exporting PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring but it would be much nicer if poetry just didn't hit the keyring at all if it wasn't needed.

@txtsd
Copy link

txtsd commented Sep 6, 2022

I tried poetry for the first time today.


This command will guide you through creating your pyproject.toml config.

Package name [herpderp]:  
Version [0.1.0]:  
Description []:  
Author [txtsd <[email protected]>, n to skip]:  
License []:  GPL-3.0-only
Compatible Python versions [^3.10]:  

Would you like to define your main dependencies interactively? (yes/no) [yes] yes
You can specify a package in the following forms:
  - A single name (requests): this will search for matches on PyPI
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Package to add or search for (leave blank to skip): requests
Found 20 packages matching requests
Showing the first 10 matches

Enter package # to add, or the complete package name if it is not listed []:
 [ 0] requests
 [ 1] requests5
 [ 2] requests3
 [ 3] requests2
 [ 4] pycopy-requests
 [ 5] requests-middleware
 [ 6] grasspy-requests
 [ 7] requests-kerberos
 [ 8] fed-requests
 [ 9] requests-httpsproxy
 [10] 
 > 1
Enter the version constraint to require (or leave blank to use the latest version): 

Failed to open keyring: org.freedesktop.DBus.Error.ServiceUnknown: The name :1.146 was not provided by any .service files.

What a horrible experience. If there isn't a keyring, assume it doesn't exist and continue. Why hard fail at this stage? It's not even like I don't have a keyring. I use KeePassXC's freedesktop.org Secrets integration. It works fine system-wide.

@neersighted
Copy link
Member

@txtsd Keyring support is a new feature, and this is a complex interaction across the matrix of possible platforms, configurations, versions, and backends that a user may invoke Poetry in. Many of these sharp edges simply haven't been filed off because of how large that matrix is and how dependent on external factors keychain interactions are.

MRs improving the authentication experience or docs are welcome -- we're going to have to gradually file down these rough edges as keychain support matures.

@willforde
Copy link

Had the same issue. Had to change the keyring package config to point to the keyring.backends.SecretService.Keyring backend. For me the keyring package was trying to use kwallet, but I'm using Gnome not KDE.

OS = Arch Linux
DE = Gnome

@txtsd
Copy link

txtsd commented Sep 7, 2022

Okay, I created this file ~/.config/python_keyring/keyringrc.cfg and populated it with

[backend]
default-keyring=keyring.backends.SecretService.Keyring

Then I unset PYTHON_KEYRING_BACKEND and ran poetry init:

λ poetry init

This command will guide you through creating your pyproject.toml config.

Package name [test]:  
Version [0.1.0]:  
Description []:  
Author [txtsd <[email protected]>, n to skip]:  
License []:  
Compatible Python versions [^3.10]:  

Would you like to define your main dependencies interactively? (yes/no) [yes] 
You can specify a package in the following forms:
  - A single name (requests): this will search for matches on PyPI
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Package to add or search for (leave blank to skip): lxml
Found 20 packages matching lxml
Showing the first 10 matches

Enter package # to add, or the complete package name if it is not listed []:
 [ 0] lxml
 [ 1] lxml-stubs
 [ 2] zsi-lxml
 [ 3] lxml-wrapper
 [ 4] readability-lxml
 [ 5] lxml2dict
 [ 6] suds-lxml
 [ 7] json-lxml
 [ 8] lxml-asserts
 [ 9] gocept.lxml
 [10] 
 > 0
Enter the version constraint to require (or leave blank to use the latest version): 

Empty module name

Different error now.

@neersighted
Copy link
Member

neersighted commented Sep 7, 2022

There's some busted keyring behavior with your particular backends that seem to

  1. invoke with poor error handling during init -- odd (though it's possible that the search code in poetry init was not made robust against keyring errors as an oversight)
  2. bypass most of our error handling and spit out unhelpful messages

Some more insight into your system (what backend should be available, OS version, package versions), etc would be helpful for those that attempt to address yours/similar issues @txtsd.

In the meantime, you can revert to 1.1-style writing to disk using the keyring.backends.null.Keyring backend, of course.

@txtsd
Copy link

txtsd commented Sep 7, 2022

@neersighted
I use Arch Linux

λ pip freeze | grep poetry
poetry==1.2.0
poetry-core==1.1.0
poetry-plugin-export==1.0.6

I set the config to use keyring.backends.null.Keyring for now. Thanks!

@croqaz
Copy link

croqaz commented Sep 19, 2022

Happened to me today for the first time. I used Poetry until last month without any issues, but today I saw same error.
export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring seems to fix the issue.

@lazka
Copy link
Contributor

lazka commented Sep 21, 2022

I'm seeing the same error on Windows with libsecret installed:

  • Installing urllib3 (1.26.12): Failed
  Error
  g-dbus-error-quark: The name org.freedesktop.secrets is unknown (2)

rnestler added a commit to rnestler/dotfiles that referenced this issue Sep 21, 2022
Poetry tries to use the keyring even on systems were no keyring agent is
running.

See python-poetry/poetry#1917
@viteski
Copy link

viteski commented Aug 15, 2023

you can use this in your dev environment to work around the issue on Debian/Ubuntu machines

function unlock_keyring() {
    export "$(echo -n "$(pass)" | gnome-keyring-daemon --replace --unlock)"
    unset "$(pass)"
}

you'll have to apt install the gnome-keyring package
gnome-keyring-pkcs11/jammy-updates,now 40.0-3ubuntu3 amd64 [installed,automatic]
gnome-keyring/jammy-updates,now 40.0-3ubuntu3 amd64 [installed,automatic]
libpam-gnome-keyring/jammy-updates,now 40.0-3ubuntu3 amd64 [installed,automatic]

But, it is troublesome.

@bgyarfas
Copy link

Another workaround in the case you don't need the keyring at all

python3 -m keyring --disable

If you are also running pyenv you'll need to run this with the system python

pyenv shell system
python3 -m keyring --disable

bryanvaz added a commit to bryanvaz/.dotfiles that referenced this issue Sep 7, 2023
alexanderkjackson-protonmail-com added a commit to alexanderkjackson-protonmail-com/etcWatch that referenced this issue Sep 14, 2023
See: python-poetry/poetry#1917
Prior duplicate with exact matching error message:
   python-poetry/poetry#7032

Encountered a poetry bug: "Failed to unlock the collection"
Unsure if this bug is per-project. If it recurs:
   Temporary fix:
   ```PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring```
      Or switch to a working backend.
   Permanent fix (disables Python keyring facilities):
   ```python3 -m keyring --disable```

I am uncertain if this bug is elicited by particular project settings,
but it is by particular environments.
@vipcxj
Copy link

vipcxj commented Sep 18, 2023

The same issue for the latest version. A big bong bug not fixed for 3 years?

@ericriff
Copy link

I'm just dealing with this myself, on the latest version (1.6.1).
This workaround worked for me

python3 -m keyring --disable

but it is not ideal. I handle the environment of many devs on a company, I can't ask them all to run this command.

@13steinj
Copy link

The same issue for the latest version. A big bong bug not fixed for 3 years?

Because

  • this is open source software-- unless someone picks it up it won't be priority. I think this was picked up at some point but then dropped.
  • I don't think the maintainers care about this particular issue as a "big deal"

In fairness to them, it doesn't happen in most desktop environments. However companies in various fashion have moved many devs to "use an ssh terminal and be stuck with it." or something of the like. Which is sometimes a necessity, but isn't what open source devs "target" in terms of usability testing (if they do usability testing at all).

Similarly in fairness, after doing some evil hack to avoid this issue once, it's not a recurring problem.

@remram44
Copy link
Contributor

"this is open source software" is a terrible excuse, the community has provided many patches that have all been ignored.

@13steinj
Copy link

is a terrible excuse, the community has provided many patches that have all been ignored.

I don't disagree, it's just the excuse always given to me by every project I've made a similar comment on 🙃

It may be possible that the maintainers don't have enough time to go through the issues / prs at this point (there are quite a few, after all). It may be possible that they've stopped caring. It may be possible that they don't accept the patches made in various fashion.

In fact, yours, explicitly was asked for tests and it doesn't appear as if you've written them; so, bit odd to complain about ignored patches. And in that there is a reference (without response from the author) about #6612 which has been superseded (and relatively recent) by #8078

I can understand the frustration, but it's not as if things aren't moving and it's not as if the fault is purely on maintainers for not merging any one person's PR.

@dimbleby
Copy link
Contributor

dimbleby commented Sep 18, 2023

for what it's worth there have been a couple of abandoned tries at this (I know of #6612, #7037 - neither of which were ignored) - but #8227 looks to me as though it's pretty close. So with any luck that'll go in soon

radoering pushed a commit to real-yfprojects/poetry that referenced this issue Oct 3, 2023
@moberst
Copy link

moberst commented Oct 3, 2023

Just came across this multi-year-old issue for the first time only to find that it was just merged into the master branch. Hurray!

For anyone else in a similar position who comes across this issue and is looking for the fix, you can install the current master branch via the following (taken from the installation docs here). Looking forward to this being in the stable version.

curl -sSL https://install.python-poetry.org | python3 - --git https://github.com/python-poetry/poetry.git@master

@simaotwx
Copy link

For now I'm using the workaround mentioned before

python3 -m keyring --disable

Bad timing that I just got to use poetry for the first time and got hit by a flood of error messages.

Looks like it has been fixed so I hope this will not be an issue in the future.
IMO it should not hard-fail since the keyring is not necessary to install packages.

@arturtoshev
Copy link

Installing the latest version from the master branch as suggested above did the job yesterday, but I now ran into poetry install getting stuck halfway through.

What I found to really solve the problem for me was clearing the cache:

curl -sSL https://install.python-poetry.org | python3 - --uninstall  # uninstall poetry
rm -r $HOME/.cache/pypoetry  # clear cache
curl -sSL https://install.python-poetry.org | python3 -  # install latest release

Could someone add clearing the cache as an option during uninstall or at least mention it in the documentation?

LuzianHahn added a commit to LuzianHahn/dotfiles that referenced this issue Oct 26, 2023
When using poetry I recently ran into an issue as described here:
 python-poetry/poetry#1917
It seems that the poetry authors enabled the automatic support of
 keyring. This does not check if keyring is actually used.

I decided to apply this fix here, since I think of poetry as a useful
 tool to steer my python projects. Maybe this is fixed within the next
 poetry release anyhow. Then I'll just remove it. Otherwise this fix is
 useful across this system, since I don't use keyring up to now that
 much. Worst case I can deactivate it on specific systems.
@ilyagr
Copy link
Contributor

ilyagr commented Nov 4, 2023

I installed poetry 1.7.0, and I now have poetry install and poetry lock hang forever.

Removing the cache as suggested by the previous comment did not help.

The workaround of export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring still works and solves the problem.

See some more details in https://gist.github.com/ilyagr/1bebabebc93e6662d139009d19ed8c1b; I'll probably file a proper bug later. Update: Filed #8623.

Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected
Projects
None yet