Add ability to access github Release Asset API. #525

ankona · 2017-02-09T15:59:21Z

API Documentation found here:
https://developer.github.com/v3/repos/releases/#list-assets-for-a-release
https://developer.github.com/v3/repos/releases/#upload-a-release-asset
https://developer.github.com/v3/repos/releases/#get-a-single-release-asset
https://developer.github.com/v3/repos/releases/#edit-a-release-asset
https://developer.github.com/v3/repos/releases/#delete-a-release-asset

See: https://developer.github.com/v3/repos/releases

nhomar · 2017-02-16T04:20:36Z

github/Requester.py

+            if "Content-Type" in headers:
+                mime_type = headers["Content-Type"]
+            else:
+                import mimetypes


Why not import globally?

nhomar · 2017-02-16T04:21:01Z

github/tests/GitRelease.py

+                                 "application/zip")
+
+
+


Cleanup unused blanklines to avoid introduce new PEP8 failures.

ankona · 2017-02-21T19:29:18Z

@nhomar I had 3 places where I'd imported locally (with no good reason). I moved all of them and removed the extra whitespace as well.

kujenga · 2017-03-14T23:49:47Z

@nhomar Is there anything else left on this PR? I'd like to use this functionality for a project I'm working on. Happy to help out if necessary/possible.

Thanks for the great work @THEHighlander!

ankona

modified per review notes

NicoHood · 2017-04-02T14:52:16Z

Am I wrong or are the releases themselves also missing for this module?

I try to search for an existing release using the tag id:
https://api.github.com/repos/username/project/releases/tags/1.0.0

And then I want to upload release assets to this release or create it if it does not exist yet. But the docs have no result for "release", so is there any doc about this yet?

ankona · 2017-04-11T14:07:17Z

@nhomar is there anything else you'd like me to do here?

petelopez · 2017-04-20T14:26:42Z

@nhomar Can we get this PR accepted?

petelopez · 2017-04-20T14:32:34Z

@NicoHood Releases are present in some form but it's not documented or at least wasn't as of a couple of months ago. This PR includes additional functionality around the release API as it pertains to assets.

This file has been around forever but I agree, it wasn't in the docs: https://github.com/PyGithub/PyGithub/blob/master/github/GitRelease.py

NicoHood · 2017-05-12T15:10:04Z

Maybe this project can help for now:
https://github.com/seletz/ghrelease

NicoHood · 2017-05-12T17:06:00Z

@THEHighlander I have problems uploading release assets. Listing works fine, uploading does not.

This is my code (excerpt):

       # Note for github issue: yes, i am verifying the release exists.
       self.release = self.githubrepo.get_release(self.config['tag'])

        # Upload assets
        for asset in self.newassets:
            assetpath = os.path.join(self.config['output'], asset)
            print(':: Uploading', assetpath)
            self.release.upload_asset(assetpath, "Testlabel", "application/x-xz")

The error:

:: Uploading /home/arch/hackallthethings/attiny/archive/attiny-1.2.27.tar.xz
Traceback (most recent call last):
  File "../gpgit/gpgit.py", line 841, in <module>
    sys.exit(main(sys.argv[1:]))
  File "../gpgit/gpgit.py", line 833, in main
    if not gpgit.run():
  File "../gpgit/gpgit.py", line 791, in run
    if step.run():
  File "../gpgit/gpgit.py", line 110, in run
    if substep.run():
  File "../gpgit/gpgit.py", line 87, in run
    return self.funct()
  File "../gpgit/gpgit.py", line 772, in step_5_1
    self.release.upload_asset(assetpath, "unit test artifact", "application/x-xz")
  File "/usr/lib/python3.6/site-packages/github/GitRelease.py", line 145, in upload_asset
    input=path
  File "/usr/lib/python3.6/site-packages/github/Requester.py", line 243, in requestBlob
    status, responseHeaders, output = self.__requestEncode(None, verb, url, parameters, headers, input, encode)
  File "/usr/lib/python3.6/site-packages/github/Requester.py", line 268, in __requestEncode
    status, responseHeaders, output = self.__requestRaw(cnx, verb, url, requestHeaders, encoded_input)
  File "/usr/lib/python3.6/site-packages/github/Requester.py", line 293, in __requestRaw
    requestHeaders
  File "/usr/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1055, in _send_output
    for chunk in chunks:
  File "/usr/lib/python3.6/http/client.py", line 1010, in _read_readable
    datablock = readable.read(blocksize)
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 0: invalid start byte

ankona · 2017-05-12T18:23:28Z

@NicoHood I'll dig into it after work

NicoHood · 2017-05-14T17:43:54Z

@THEHighlander thanks! Any progress so far? I have no idea what causes this issue.

You can find the full sourcecode of my project here:
https://github.com/NicoHood/gpgit/tree/python

ankona · 2017-05-15T20:18:42Z

@NicoHood So far, I'm unable to reproduce your defect.

I made the mistake of first rebasing my local branch off of master and that did introduce a defect (the underlying request code kills off the correct host name for uploads). I'm going to stash these changes and try again with the code as it was during the PR.

Edit: I just realized you're using python 3.6 and I'm running locally using 2.7. I'll try that, too.

NicoHood · 2017-05-16T14:44:08Z

@THEHighlander Thanks for testing! I have merged your changes into master and created a new tag:
https://github.com/NicoHood/PyGithub/releases/tag/v1.34.1

Yes I am using python3 and that is possibly the reason why it crashes (UFT8 changes were made in python3)

ankona · 2017-05-16T21:23:22Z

@NicoHood I've found and fixed the encoding issue in python3. there were two issues:

the response from http.client is a byte array
opening/uploading the file as plain text
i've run out of time for today, but i'll have the fix committed tomorrow.

NicoHood · 2017-05-17T18:22:57Z

@THEHighlander thanks for taking the time. The test now fails:


======================================================================
ERROR: testGetAsset (github.tests.GitRelease.Release)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/tests/GitRelease.py", line 97, in testGetAsset
    the_asset = the_repo.get_release_asset(asset_id)
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/Repository.py", line 2305, in get_release_asset
    return github.GitReleaseAsset.GitReleaseAsset(self._requester, {}, data, completed=True)
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/GithubObject.py", line 240, in __init__
    GithubObject.__init__(self, requester, headers, attributes, completed)
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/GithubObject.py", line 78, in __init__
    self._storeAndUseAttributes(headers, attributes)
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/GithubObject.py", line 90, in _storeAndUseAttributes
    self._useAttributes(attributes)
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/GitReleaseAsset.py", line 200, in _useAttributes
    self._url = self._makeStringAttribute(attributes["url"])
TypeError: string indices must be integers

======================================================================
FAIL: testUploadAsset (github.tests.GitRelease.Release)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/tests/GitRelease.py", line 125, in testUploadAsset
    "application/zip")
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/GitRelease.py", line 145, in upload_asset
    input=path
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/Requester.py", line 181, in requestBlobAndCheck
    return self.__check(*self.requestBlob(verb, url, parameters, headers, input))
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/Requester.py", line 247, in requestBlob
    return self.__requestEncode(None, verb, url, parameters, headers, input, encode)
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/Requester.py", line 270, in __requestEncode
    status, responseHeaders, output = self.__requestRaw(cnx, verb, url, requestHeaders, encoded_input)
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/Requester.py", line 295, in __requestRaw
    requestHeaders
  File "/build/python-pygithub/src/PyGithub-1.34.2/build/lib/github/tests/Framework.py", line 148, in request
    self.__testCase.assertEqual(self.__host, readLine(self.__file))
AssertionError: 'uploads.github.com' != 'api.github.com'
- uploads.github.com
? ^ ^^^^^
+ api.github.com
? ^ ^


----------------------------------------------------------------------
Ran 424 tests in 2.951s

FAILED (failures=1, errors=1)
Test failed: <unittest.runner.TextTestResult run=424 errors=1 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=424 errors=1 failures=1>
==> ERROR: A failure occurred in check().
    Aborting...
==> ERROR: Build failed, check /var/lib/archbuild/extra-x86_64/arch/build

ankona · 2017-05-17T18:56:18Z

@NicoHood Yeah... I'm working it.

One down...

ankona · 2017-05-17T22:16:29Z

@NicoHood Ready

NicoHood · 2017-05-18T15:09:23Z

@THEHighlander Thanks for the help. Now I get a different error while using:

:: Uploading /data/gpgit/PyGithub-v1.34.3.tar.xz
Traceback (most recent call last):
  File "../gpgit/gpgit.py", line 916, in <module>
    sys.exit(main())
  File "../gpgit/gpgit.py", line 903, in main
    err_msg = gpgit.run()
  File "../gpgit/gpgit.py", line 837, in run
    err_msg = substep.funct()
  File "../gpgit/gpgit.py", line 676, in substep2
    self.release.upload_asset(assetpath, "Testlabel", "application/x-xz")
  File "/usr/lib/python3.6/site-packages/github/GitRelease.py", line 145, in upload_asset
    input=path
  File "/usr/lib/python3.6/site-packages/github/Requester.py", line 181, in requestBlobAndCheck
    return self.__check(*self.requestBlob(verb, url, parameters, headers, input))
  File "/usr/lib/python3.6/site-packages/github/Requester.py", line 186, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.GithubException: 400 {'message': 'Bad Content-Length: ', 'request_id': 'DEC0:6858:529B6F:55DEA0:591DB84B'}

I am using a hardcoded label and mimetype. I also have a question about that. Do I need to specify those and to which values does those default to? From what I can see in the source the mime type is auto guessed and the label is left out, am I right with this?

Checkout my tool to reproduce the error:
https://github.com/NicoHood/gpgit/tree/python

ankona · 2017-05-18T15:20:58Z

@NicoHood You're correct on both label & mime type.

Label is optional. If you provide one, it just appears as the text in the download link on the release page in github (e.g. <a href='filename.ext'>Label Here</a>, see screenshot above). If you don't provide it, the link just shows the raw file name.

The mime type is also technically optional, but it's better to provide it. You noticed that I'm using mimetypes.guess_type(input_file) to try to determine it if it's not provided, then just falling back to application/octet-stream if that fails.

I haven't seen the bad content length error in my tests. I'll get it figured out today.

NicoHood · 2017-05-19T20:37:38Z

@THEHighlander Thanks! Its finally working now. Thanks so much for your help and effort!

@nhomar Please merge this PR, its tested and really helpful.

NicoHood · 2017-08-06T20:14:33Z

@jayfk Any chance to get a review from your side?

xiapoy · 2017-08-27T13:36:00Z

+1

matejak · 2017-11-22T13:07:15Z

Please merge this if it is ready, it is a really useful stuff.

steven-hh-ding · 2017-12-14T18:35:57Z

github/Requester.py

+    def requestBlobAndCheck(self, verb, url, parameters=None, headers=None, input=None):
+        o = urlparse.urlparse(url)
+        self.__hostname = o.hostname
+        return self.__check(*self.requestBlob(verb, url, parameters, headers, input))


Shouldnt we change back the __hostname here? My subsequent api calls failed due to the host name assertion fails in requester..

See fix in #771

NicoHood · 2018-01-06T10:19:28Z

@jasonwhite @nhomar Thanks for merging. Can you please tag a new release, so its available on pypi and the linux distros?

You might also want to check the comment by @steven-hh-ding , maybe also @THEHighlander can do.

rpatterson · 2018-05-20T18:35:42Z

This functionality remains undocumented in the reference.

Add ability to access github Release Asset API.

65edbcf

See: https://developer.github.com/v3/repos/releases

ankona mentioned this pull request Feb 9, 2017

Missing support for release asset APIs #526

Closed

nhomar requested changes Feb 16, 2017

View reviewed changes

PEP8 compliance (move imports & remove extra spacing)

9b0eb60

ankona commented Mar 17, 2017

View reviewed changes

NicoHood mentioned this pull request Apr 25, 2017

Release 2.0.0 NicoHood/GPGit#5

Merged

15 tasks

ankona added 3 commits May 17, 2017 12:15

Add requestBlobAndCheck to fix python3 result bug reported by @NicoHood

6a6d5ca

Fix non-text files failing to upload in python3

d01da1d

Ensure upload url hostname is used during uploads

47723c9

ankona added 2 commits May 17, 2017 18:12

fix incorrect headers supplied to obj contructors

b57e1e0

fix replay data for upload_asset test case

77bb4d1

fix invalid-content-length for python > 3.5.1

a4f19c1

ankona added 2 commits May 19, 2017 16:17

add content length to replaydata for unittest

778b78e

fix unclosed file warning

85a167b

nhomar approved these changes Nov 27, 2017

View reviewed changes

nhomar merged commit 5244964 into PyGithub:master Nov 27, 2017

steven-hh-ding reviewed Dec 14, 2017

View reviewed changes

sfdye mentioned this pull request Mar 22, 2018

Add read-only Releases API with Assets #361

Closed

jesseli2002 mentioned this pull request May 27, 2020

Add ability to download binary content of release asset #1541

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to access github Release Asset API. #525

Add ability to access github Release Asset API. #525

ankona commented Feb 9, 2017

nhomar Feb 16, 2017

nhomar Feb 16, 2017

ankona commented Feb 21, 2017

kujenga commented Mar 14, 2017

ankona left a comment

NicoHood commented Apr 2, 2017

ankona commented Apr 11, 2017

petelopez commented Apr 20, 2017

petelopez commented Apr 20, 2017

NicoHood commented May 12, 2017

NicoHood commented May 12, 2017

ankona commented May 12, 2017

NicoHood commented May 14, 2017

ankona commented May 15, 2017 •

edited

Loading

NicoHood commented May 16, 2017

ankona commented May 16, 2017

NicoHood commented May 17, 2017

ankona commented May 17, 2017 •

edited

Loading

ankona commented May 17, 2017

NicoHood commented May 18, 2017

ankona commented May 18, 2017 •

edited

Loading

NicoHood commented May 19, 2017

NicoHood commented Aug 6, 2017

xiapoy commented Aug 27, 2017

matejak commented Nov 22, 2017

steven-hh-ding Dec 14, 2017

mfonville Apr 25, 2018

NicoHood commented Jan 6, 2018

rpatterson commented May 20, 2018

Add ability to access github Release Asset API. #525

Add ability to access github Release Asset API. #525

Conversation

ankona commented Feb 9, 2017

nhomar Feb 16, 2017

Choose a reason for hiding this comment

nhomar Feb 16, 2017

Choose a reason for hiding this comment

ankona commented Feb 21, 2017

kujenga commented Mar 14, 2017

ankona left a comment

Choose a reason for hiding this comment

NicoHood commented Apr 2, 2017

ankona commented Apr 11, 2017

petelopez commented Apr 20, 2017

petelopez commented Apr 20, 2017

NicoHood commented May 12, 2017

NicoHood commented May 12, 2017

ankona commented May 12, 2017

NicoHood commented May 14, 2017

ankona commented May 15, 2017 • edited Loading

NicoHood commented May 16, 2017

ankona commented May 16, 2017

NicoHood commented May 17, 2017

ankona commented May 17, 2017 • edited Loading

ankona commented May 17, 2017

NicoHood commented May 18, 2017

ankona commented May 18, 2017 • edited Loading

NicoHood commented May 19, 2017

NicoHood commented Aug 6, 2017

xiapoy commented Aug 27, 2017

matejak commented Nov 22, 2017

steven-hh-ding Dec 14, 2017

Choose a reason for hiding this comment

mfonville Apr 25, 2018

Choose a reason for hiding this comment

NicoHood commented Jan 6, 2018

rpatterson commented May 20, 2018

ankona commented May 15, 2017 •

edited

Loading

ankona commented May 17, 2017 •

edited

Loading

ankona commented May 18, 2017 •

edited

Loading