<newcontributor> I've just created my first PR, could someone please /allow me? https://github.com/gitgitgadget/git/pull/12345
<veteran> newcontributor: it is done
<newcontributor> thanks!

Changes since v1:
- Fixed a typo in the commit message (found by ...)
- Added a code comment to ... as suggested by ...
...

git fetch https://github.com/gitgitgadget/git pr-git-1146/mahozad/add-github-releases-workflow-v1

git fetch --no-tags https://github.com/gitgitgadget/git tag pr-git-1146/mahozad/add-github-releases-workflow-v1

Hi Mahdi,

I've already written up these concerns on GitHub [1] and you've replied 
there, but Johannes asked me to also send this to the mailing list, so 
please bear with me for mostly repeating the same points.

On Thu, 25 Nov 2021, Mahdi Hosseinzadeh via GitGitGadget wrote:

> From: Mahdi Hosseinzadeh <[email protected]>
>
> GitHub now allows users to subscribe only to
> "release" notifications of a repository.
> So, users can be notified of new releases and their
> changelog/release notes automatically.
>
> This workflow works whenever:
>    a new version tag
>    (with the format following the regex "v\d+\..*")
>    is pushed to the repository
> AND
>    the commit that the tag points to, created/modified
>    a release notes file from Doumentation/RelNotes/ directory.
>
> The script for generating the temporary changelog file is
> written in Kotlin language which can be a much better alternative
> to shell scripts in terms of features and readability
> (it is like a python script but with static typing support).
> The Kotlin runner is pre-installed in GitHub Actions environments;
> for more information see
>    https://github.com/actions/virtual-environments/
>    https://stackoverflow.com/a/69116750/8583692
>
> The "Release Notes (yyyy-MM-dd)" link in https://git-scm.com/
> website can also link to these GitHub release pages instead of
> to the raw .txt release note file in the repository.
>
> See the issue related to GitHub release notifications:
> https://github.com/isaacs/github/issues/410
>
> Also see GitHub announcement for this feature:
> https://github.blog/changelog/2018-11-27-watch-releases/

Nit: "Github now allows users" sounds like a new feature, not one that's 
three years old.

>
> Signed-off-by: Mahdi Hosseinzadeh <[email protected]>
> ---
>    Add a workflow for creating GitHub release notes
>
>    Because this is not directly the git code and is related to GitHub CI, I
>    post it here.
>
>    This pull request adds a new GitHub Actions workflow that automatically
>    creates releases on GitHub repository when pushing a new tag to the
>    repository.
>
>    GitHub now allows users to subscribe only to "release" notifications of
>    a repository. So, users can be notified of new releases and their
>    changelog/release notes automatically.
>
>    This workflow works whenever: a new version tag (with the format
>    following the regex v\d+\..*) is pushed to the repository AND the commit
>    that the tag points to, created/modified a release notes file from
>    Doumentation/RelNotes/ directory.
>
>    The script for generating the temporary changelog file is written in
>    Kotlin language [https://kotlinlang.org/] which can be a better
>    alternative to shell scripts in terms of features and readability (it is
>    like a python script but with static typing support). The Kotlin runner
>    is pre-installed in GitHub Actions environments; for more information
>    see https://github.com/actions/virtual-environments/
>    https://stackoverflow.com/a/69116750/8583692
>
>    The Release Notes (yyyy-MM-dd) link in https://git-scm.com/ website can
>    also link to these GitHub release pages instead of to the raw .txt
>    release note file in the repository.
>
>    See the issue related to GitHub release notifications:
>    https://github.com/isaacs/github/issues/410
>
>    Also see GitHub announcement for this feature:
>    https://github.blog/changelog/2018-11-27-watch-releases/
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1146%2Fmahozad%2Fadd-github-releases-workflow-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1146/mahozad/add-github-releases-workflow-v1
> Pull-Request: https://github.com/git/git/pull/1146
>
> .../generate-github-changelog.main.kts        | 21 ++++++++++
> .github/workflows/create-release.yml          | 40 +++++++++++++++++++
> 2 files changed, 61 insertions(+)
> create mode 100644 .github/scripts/generate-github-changelog.main.kts
> create mode 100644 .github/workflows/create-release.yml
>
> diff --git a/.github/scripts/generate-github-changelog.main.kts b/.github/scripts/generate-github-changelog.main.kts
> new file mode 100644
> index 00000000000..e57fd2a6ae5
> --- /dev/null
> +++ b/.github/scripts/generate-github-changelog.main.kts
> @@ -0,0 +1,21 @@
> +#!/usr/bin/env kotlin
> +
> +/**
> + * Copies contents of the release notes file created/modified
> + * in this commit to a new file to be used by the workflow.
> + */
> +
> +import java.io.File
> +
> +println("Files modified in this commit:")
> +args.forEachIndexed { index, name ->
> +    println("\t${index + 1}- $name")
> +}
> +
> +val notesFile = args
> +    .map(::File)
> +    .singleOrNull { "RelNotes" in it.parent }
> +
> +notesFile
> +    ?.copyTo(File("changelog.txt"))
> +    ?: println("No release notes file modified in this commit")

We need to spin up a JVM for 21 lines of code just to copy a single 
file. I think a single call of `cp` is faster and more readable than that.

> diff --git a/.github/workflows/create-release.yml b/.github/workflows/create-release.yml
> new file mode 100644
> index 00000000000..711ba105e42
> --- /dev/null
> +++ b/.github/workflows/create-release.yml
> @@ -0,0 +1,40 @@
> +name: Create GH release
> +
> +# Create a GitHub release for each new tag.
> +# The release notes are taken from the release notes file
> +# modified in that commit located in Documentation/RelNotes directory.
> +
> +on:
> +  push:
> +    tags:
> +      - v[0-9]+.*

I think we should probably exclude the release candidates from this. As 
Johhannes pointed out[2], marking them as full releases would 
periodically cause https://github.com/git/git/releases/latest to point
to a pre-release instead of the latest full release.

> +
> +permissions:
> +  contents: write
> +
> +jobs:
> +  create-gh-release:
> +    name: Create a new release or update an existing release in the GitHub repository
> +    runs-on: ubuntu-latest
> +    steps:
> +      - name: Checkout the repository
> +        uses: actions/checkout@v2
> +        with:
> +          fetch-depth: 2  # OR '0' To retrieve all preceding commit.

The value 2 seems pretty arbitrary and the comment adds nothing.

> +      - name: Get changed files
> +        uses: tj-actions/[email protected]
> +        id: changed-files

You've replied on Github that you need the last two commits for this 
action [3], but I don't think we care about wether or not the release 
notes where changed in the last commit. We only need the version number 
(from the pushed tag) to determine the correct release notes file.

> +        with:
> +          separator: ' '
> +      - name: Generate the changelog
> +        run: kotlin .github/scripts/generate-github-changelog.main.kts ${{ steps.changed-files.outputs.all_changed_and_modified_files }}
> +      - name: Create the release
> +        uses: actions/create-release@v1
> +        env:
> +          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
> +        with:
> +          tag_name: ${{ github.ref_name }}
> +          release_name: ${{ github.ref_name }}
> +          body_path: changelog.txt

If we just use the file directly we don't even need to copy the file to 
changelog.txt

> +          draft: false
> +          prerelease: false

If we don't exclude release candidates, we should set prelease to true for 
those tags.

>
> base-commit: 5f439a0ecfb4657100ec1e56ef9c6eca963b5a94
> -- 
> gitgitgadget
>

All in all I think this is too convoluted for what it's trying to 
achieve. I think we should be able to achieve the same result with 
something like this:

  .github/workflows/create-release.yml | 37 ++++++++++++++++++++++++++++
  1 file changed, 37 insertions(+)
  create mode 100644 .github/workflows/create-release.yml

diff --git a/.github/workflows/create-release.yml 
b/.github/workflows/create-release.yml
new file mode 100644
index 0000000000..5b9fdf0372
--- /dev/null
+++ b/.github/workflows/create-release.yml
@@ -0,0 +1,37 @@
+name: Create GH release
+
+# Create a GitHub release for each new tag.
+# The release notes are taken from the release notes file
+# modified in that commit located in Documentation/RelNotes directory.
+
+on:
+  push:
+    tags:
+      - v[0-9]+.[0-9]+.[0-9]+
+
+permissions:
+  contents: write
+
+jobs:
+  create-gh-release:
+    name: Create a new release or update an existing release in the 
GitHub repository
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout the repository
+        uses: actions/checkout@v2
+        with:
+          fetch-depth: 1
+      - name: Get version number
+        shell: bash
+        run: |
+          echo GIT_VERSION=${GITHUB_REF#refs/tags/v} >> $GITHUB_ENV
+      - name: Create the release
+        uses: actions/create-release@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          tag_name: ${{ github.ref_name }}
+          release_name: ${{ github.ref_name }}
+          body_path: Documentation/RelNotes/${{ env.GIT_VERSION }}.txt
+          draft: false
+          prerelease: false
-- 
2.25.1

An example of the result this reduced action produces can be found at [4] 
(release notes for v2.34.1, but the tagged commit isn't v2.34.1).

Best regards

Matthias

[1] https://github.com/git/git/pull/1146
[2] https://github.com/git/git/pull/1146#discussion_r756854259
[3] https://github.com/git/git/pull/1146#discussion_r756845042
[4] https://github.com/rimrul/git/releases/tag/v2.34.1

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323328-1858230353-1637935150=:63
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi,

On Thu, 25 Nov 2021, Matthias A=C3=9Fhauer wrote:

> On Thu, 25 Nov 2021, Mahdi Hosseinzadeh via GitGitGadget wrote:
>
> > diff --git a/.github/workflows/create-release.yml
> > b/.github/workflows/create-release.yml
> > new file mode 100644
> > index 00000000000..711ba105e42
> > --- /dev/null
> > +++ b/.github/workflows/create-release.yml
> > @@ -0,0 +1,40 @@
> > +name: Create GH release
> > +
> > +# Create a GitHub release for each new tag.
> > +# The release notes are taken from the release notes file
> > +# modified in that commit located in Documentation/RelNotes directory=
.
> > +
> > +on:
> > +  push:
> > +    tags:
> > +      - v[0-9]+.*
> > [...]
>
> All in all I think this is too convoluted for what it's trying to
> achieve. I think we should be able to achieve the same result with
> something like this:
>
>  .github/workflows/create-release.yml | 37 ++++++++++++++++++++++++++++
>  1 file changed, 37 insertions(+)
>  create mode 100644 .github/workflows/create-release.yml
>
> diff --git a/.github/workflows/create-release.yml
> b/.github/workflows/create-release.yml
> new file mode 100644
> index 0000000000..5b9fdf0372
> --- /dev/null
> +++ b/.github/workflows/create-release.yml
> @@ -0,0 +1,37 @@
> +name: Create GH release
> +
> +# Create a GitHub release for each new tag.
> +# The release notes are taken from the release notes file
> +# modified in that commit located in Documentation/RelNotes directory.
> +
> +on:
> +  push:
> +    tags:
> +      - v[0-9]+.[0-9]+.[0-9]+
> +
> +permissions:
> +  contents: write
> +
> +jobs:
> +  create-gh-release:
> +    name: Create a new release or update an existing release in the Git=
Hub
> repository
> +    runs-on: ubuntu-latest
> +    steps:
> +      - name: Checkout the repository
> +        uses: actions/checkout@v2
> +        with:
> +          fetch-depth: 1
> +      - name: Get version number
> +        shell: bash
> +        run: |
> +          echo GIT_VERSION=3D${GITHUB_REF#refs/tags/v} >> $GITHUB_ENV
> +      - name: Create the release
> +        uses: actions/create-release@v1
> +        env:
> +          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
> +        with:
> +          tag_name: ${{ github.ref_name }}
> +          release_name: ${{ github.ref_name }}
> +          body_path: Documentation/RelNotes/${{ env.GIT_VERSION }}.txt
> +          draft: false
> +          prerelease: false
> --
> 2.25.1
>
> An example of the result this reduced action produces can be found at [4=
]
> (release notes for v2.34.1, but the tagged commit isn't v2.34.1).
>
> Best regards
>
> Matthias
>
> [1] https://github.com/git/git/pull/1146
> [2] https://github.com/git/git/pull/1146#discussion_r756854259
> [3] https://github.com/git/git/pull/1146#discussion_r756845042
> [4] https://github.com/rimrul/git/releases/tag/v2.34.1

One thing I had not thought of earlier: do we really want to do this in
every fork of git/git? I know for a fact that microsoft/git has a very
different workflow upon pushing a tag.

So maybe we need something like this, too:

   create-gh-release:
+    if: github.repository =3D=3D 'git/git'
     name: Create a new release or update an existing release in the GitHu=
b repository

Ciao,
Dscho

--8323328-1858230353-1637935150=:63--

On Fri, 26 Nov 2021, Johannes Schindelin wrote:

> Hi,
>

[...]

> One thing I had not thought of earlier: do we really want to do this in
> every fork of git/git? I know for a fact that microsoft/git has a very
> different workflow upon pushing a tag.
>
> So maybe we need something like this, too:
>
>   create-gh-release:
> +    if: github.repository == 'git/git'
>     name: Create a new release or update an existing release in the GitHub repository

I think you're right. This would have unidesirable side effects if it ran 
in forks.

>
> Ciao,
> Dscho
>

Best regards

Matthias

On Fri, Nov 26 2021, Matthias Aßhauer wrote:

> On Fri, 26 Nov 2021, Johannes Schindelin wrote:
>
>> Hi,
>>
>
> [...]
>
>> One thing I had not thought of earlier: do we really want to do this in
>> every fork of git/git? I know for a fact that microsoft/git has a very
>> different workflow upon pushing a tag.
>>
>> So maybe we need something like this, too:
>>
>>   create-gh-release:
>> +    if: github.repository == 'git/git'
>>     name: Create a new release or update an existing release in the GitHub repository
>
> I think you're right. This would have unidesirable side effects if it
> ran in forks.

Rather than hardcode given repositories, which e.g. for testing the CI
itself can be bothersome, perhaps a better thing is to put this into the
existing ci-config? I.e. git/git.git could opt itself in to behavior
that would be off by default?

I don't know how much that matters in this case, but I don't see why
we'd hardcode repository paths in general since we've got the ci-config.

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323328-898605708-1638272764=:63
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi =C3=86var,

On Mon, 29 Nov 2021, =C3=86var Arnfj=C3=B6r=C3=B0 Bjarmason wrote:

> On Fri, Nov 26 2021, Matthias A=C3=9Fhauer wrote:
>
> > On Fri, 26 Nov 2021, Johannes Schindelin wrote:
> >
> >> Hi,
> >>
> >
> > [...]
> >
> >> One thing I had not thought of earlier: do we really want to do this =
in
> >> every fork of git/git? I know for a fact that microsoft/git has a ver=
y
> >> different workflow upon pushing a tag.
> >>
> >> So maybe we need something like this, too:
> >>
> >>   create-gh-release:
> >> +    if: github.repository =3D=3D 'git/git'
> >>     name: Create a new release or update an existing release in the G=
itHub repository
> >
> > I think you're right. This would have unidesirable side effects if it
> > ran in forks.
>
> Rather than hardcode given repositories, which e.g. for testing the CI
> itself can be bothersome, perhaps a better thing is to put this into the
> existing ci-config? I.e. git/git.git could opt itself in to behavior
> that would be off by default?

You probably missed that the purpose of this workflow is something that
does not make sense in the forks of git.git.

Its purpose is to create releases for all tags that are pushed to the
repository. Most forks of git.git have no business creating releases, and
those that do already have their own processes in place.

As such, the situation is very different from the CI/PR runs that some
contributors might want to restrict to only certain branches in their
forks.

> I don't know how much that matters in this case, but I don't see why
> we'd hardcode repository paths in general since we've got the ci-config.

Integrating it into `ci-config` is not even possible in this instance
because the newly-introduced workflows should only ever run on tags, while
the `ci-config` step is part of a workflow that needs to run on every new
push and PR.

Those are two very, very different things. So even if it would have made
sense to use `ci-config` in the workflow under discussion, it is not
applicable because that step lives in a different workflow that is
triggered for a different set of events.

Ciao,
Johannes

--8323328-898605708-1638272764=:63--

Johannes Schindelin <[email protected]> writes:

>> Rather than hardcode given repositories, which e.g. for testing the CI
>> itself can be bothersome, perhaps a better thing is to put this into the
>> existing ci-config? I.e. git/git.git could opt itself in to behavior
>> that would be off by default?
>
> You probably missed that the purpose of this workflow is something that
> does not make sense in the forks of git.git.
>
> Its purpose is to create releases for all tags that are pushed to the
> repository. Most forks of git.git have no business creating releases, and
> those that do already have their own processes in place.
>
> As such, the situation is very different from the CI/PR runs that some
> contributors might want to restrict to only certain branches in their
> forks.

All true.  But https://github.com/git/git/ itself has no business
creating releases, as we already have the release process that
pushes the release material to kernel.org to be distributed from
there.

So... do we still need to discuss this patch?  In other words,
what's the benefit of creating "releases for all tags that are
pushed to the repository" there?  Does it give us redundancy in case
kernel.org goes down?  Does it risk confusing users having to wonder
release materials from which source is more "genuine"?

On 02.12.2021 11:05, Junio C Hamano wrote:
>Johannes Schindelin <[email protected]> writes:
>
>>> Rather than hardcode given repositories, which e.g. for testing the CI
>>> itself can be bothersome, perhaps a better thing is to put this into the
>>> existing ci-config? I.e. git/git.git could opt itself in to behavior
>>> that would be off by default?
>>
>> You probably missed that the purpose of this workflow is something that
>> does not make sense in the forks of git.git.
>>
>> Its purpose is to create releases for all tags that are pushed to the
>> repository. Most forks of git.git have no business creating releases, and
>> those that do already have their own processes in place.
>>
>> As such, the situation is very different from the CI/PR runs that some
>> contributors might want to restrict to only certain branches in their
>> forks.
>
>All true.  But https://github.com/git/git/ itself has no business
>creating releases, as we already have the release process that
>pushes the release material to kernel.org to be distributed from
>there.
>
>So... do we still need to discuss this patch?  In other words,
>what's the benefit of creating "releases for all tags that are
>pushed to the repository" there?  Does it give us redundancy in case
>kernel.org goes down?  Does it risk confusing users having to wonder
>release materials from which source is more "genuine"?
>

One benefit that I see is that github offers APIs & Notifications around 
releases and lots of CI integration already exist for it. If my (non github) 
CI includes building the git source then i can easily trigger when upstream 
releases a new version. Just pulling the repo and watching for the tag works 
just as well of course.

However the "which source is genuine" is a valid objection. You certainly 
don't want the CI to accidentally create a new release that does not exist 
on kernel.org (yet).

Fabian Stelzer <[email protected]> writes:

> One benefit that I see is that github offers APIs & Notifications
> around releases and lots of CI integration already exist for it. If my
> (non github) CI includes building the git source then i can easily
> trigger when upstream releases a new version. Just pulling the repo
> and watching for the tag works just as well of course.

Ahh, thanks.

If some sort of "push" notification is available only for "there is
a new release" but not for "there is a new tag", then I can sort of
see why having a "release" would be nice.  Listening to notifications
and acting on them is more pleasant than having to poll.

Do I understand what you said correctly?

On 04.12.2021 17:25, Junio C Hamano wrote:
>Fabian Stelzer <[email protected]> writes:
>
>> One benefit that I see is that github offers APIs & Notifications
>> around releases and lots of CI integration already exist for it. If my
>> (non github) CI includes building the git source then i can easily
>> trigger when upstream releases a new version. Just pulling the repo
>> and watching for the tag works just as well of course.
>
>Ahh, thanks.
>
>If some sort of "push" notification is available only for "there is
>a new release" but not for "there is a new tag", then I can sort of
>see why having a "release" would be nice.  Listening to notifications
>and acting on them is more pleasant than having to poll.
>
>Do I understand what you said correctly?
>

Yes, thats correct. 

Github has a webhook for releases:
https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#release

Getting tags means listening to every push and filtering yourself:
https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#push

Still, if this gets added to git/git I think the risk of users considering 
the github release to be the primary source is quite high since lots of 
tools and CI integrations use them. I'm not a fan of depending on github for 
everything, but as long as the kernel.org releases don't go away I don't 
think this is a big deal.

Fabian Stelzer <[email protected]> writes:

> Still, if this gets added to git/git I think the risk of users
> considering the github release to be the primary source is quite high
> since lots of tools and CI integrations use them.

That's a bit of downside.  I guess not using this patch then would
make our life simpler, then.