Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Uproot 5.2.0+ no longer respects URL parameters #1168

Open
chrisburr opened this issue Mar 12, 2024 · 2 comments
Open

Uproot 5.2.0+ no longer respects URL parameters #1168

chrisburr opened this issue Mar 12, 2024 · 2 comments
Labels
bug (unverified) The problem described would be a bug, but needs to be triaged

Comments

@chrisburr
Copy link
Member

Since moving to fsspec uproot now strips URL parameters from XRootD URLs. In this case I use them to add EOS tokens to authenticate, but I've also known them be required to even access files for quirks of the storage system.

Running this on lxplus at CERN:

export MYEOSTOKEN=$(eos token --path /eos/lhcb/user/c/cburr/test1/ --permission r)
# Destroy kerberos credentials so the token in the URL is the only available auth method
kdestroy
  • 5.2.2
In [1]: import os
   ...: import uproot
   ...: print(f"{uproot.__version__}")
   ...: f = uproot.open(f"root://eoslhcb.cern.ch//eos/lhcb/user/c/cburr/test.root?xrd.wantprot=unix&authz={os.environ['MYEOSTOKEN']}")
   ...: f.keys()
5.2.2

... truncated

OSError: File did not open properly: [ERROR] Server responded with an error: [3010] Unable to give access - user access restricted - unauthorized identity used ; Permission denied
  • 5.1.2
In [1]: import os
   ...: import uproot
   ...: print(f"{uproot.__version__}")
   ...: f = uproot.open(f"root://eoslhcb.cern.ch//eos/lhcb/user/c/cburr/test.root?xrd.wantprot=unix&authz={os.environ['MYEOSTOKEN']}")
   ...: f.keys()
5.1.2
Out[1]:
['HltRoutingBitsMonitor;1',
 'HltRoutingBitsMonitor/rb_frac;1',
 'HltRoutingBitsMonitor/rb_count;1',
 'HltDecReportsMonitor;1',
 'HltDecReportsMonitor/pass_frac_ul;1',
 'HltDecReportsMonitor/pass_frac;1',
 'HltDecReportsMonitor/pass_count_ul;1',
 'HltDecReportsMonitor/pass_count;1']
@chrisburr chrisburr added the bug (unverified) The problem described would be a bug, but needs to be triaged label Mar 12, 2024
@nsmith-
Copy link
Collaborator

nsmith- commented Aug 6, 2024

This is due to CoffeaTeam/fsspec-xrootd#64

@nsmith-
Copy link
Collaborator

nsmith- commented Aug 6, 2024 •
edited
Loading

As an aside, you can do this with http as well:

url = "https://eoslhcb.cern.ch//eos/lhcb/user/c/cburr/test.root?xrd.wantprot=unix&authz=" + token
fin = uproot.open(url, verify_ssl=False)
fin.keys()

or equivalently,

fin = uproot.open(url, verify_ssl=False, params={"xrd.wantprot": "unix", "authz": token})

Edit: to be secure on a machine without IGTF grid certificates trusted systemwide you can download the CERN CA, convert it to pem:

openssl x509 -in CERN\ Root\ Certification\ Authority\ 2.crt -out CERN\ Root\ Certification\ Authority\ 2.pem -outform PEM

and load it with

import ssl

sslctx = ssl.create_default_context()
sslctx.load_verify_locations("CERN Root Certification Authority 2.pem")
fin = uproot.open(
    url,
    ssl=sslctx,
    params={"xrd.wantprot": "unix", "authz": token},
)

though at this point one might as well use x509 user certs as in #1241

@github-project-automation github-project-automation bot moved this to Dask and high-level behavior in Finalization Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug (unverified) The problem described would be a bug, but needs to be triaged
Projects
Finalization
Status: Dask and high-level behavior
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chrisburr @nsmith-