Skip to content

Commit d1764fc

Browse files
mheverymhevery
committed
build: Add GitHub scripts for rebasing PRs (angular#18359)
PR Close angular#18359
1 parent 2e714f9 commit d1764fc

10 files changed

Lines changed: 706 additions & 2 deletions

File tree

‎docs/CARETAKER.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Caretaker
2+
3+
Caretaker is responsible for merging PRs into the individual branches and internally at Google.
4+
5+
## Responsibilities
6+
7+
- Draining the queue of PRs ready to be merged. (PRs with [`PR action: merge`](https://github.com/angular/angular/pulls?q=is%3Aopen+is%3Apr+label%3A%22PR+action%3A+merge%22) label)
8+
- Assigining [new issues](https://github.com/angular/angular/issues?q=is%3Aopen+is%3Aissue+no%3Alabel) to individual component authors.
9+
10+
## Setup
11+
12+
### Set `upstream` to fetch PRs into your local repo
13+
14+
Use this conmmands to configure your `git` to fetch PRs into your local repo.
15+
16+
```
17+
git remote add upstream [email protected]:angular/angular.git
18+
git config --add remote.upstream.fetch +refs/pull/*/head:refs/remotes/upstream/pr/*
19+
```
20+
21+
22+
## Merging the PR
23+
24+
A PR needs to have `PR action: merge` and `PR target: *` labels to be considered
25+
ready to merge. Merging is performed by running `merge-pr` with a PR number to merge.
26+
27+
NOTE: before running `merge-pr` ensure that you have synced all of the PRs
28+
locally by running:
29+
30+
```
31+
$ git fetch upstream
32+
```
33+
34+
To merge a PR run:
35+
36+
```
37+
$ ./scripts/github/merge-pr 1234
38+
```
39+
40+
The `merge-pr` script will:
41+
- Ensure that all approriate labels are on the PR.
42+
- That the current branch (`master` or `?.?.x` patch) mathches the `PR target: *` label.
43+
- It will `cherry-pick` all of the SHAs from the PR into the current branch.
44+
- It will rewrite commit history by automatically adding `Close #1234` and `(#1234)` into the commit message.
45+
46+
47+
### Recovering from failed `merge-pr` due to conflicts
48+
49+
When running `merge-pr` the script will output the commands which it is about to run.
50+
51+
```
52+
$ ./scripts/github/merge-pr 1234
53+
======================
54+
GitHub Merge PR Steps
55+
======================
56+
git cherry-pick upstream/pr/1234~1..upstream/pr/1234
57+
git filter-branch -f --msg-filter "/usr/local/google/home/misko/angular-pr/scripts/github/utils/github.closes 1234" HEAD~1..HEAD
58+
```
59+
60+
If the `cherry-pick` command fails than resolve conflicts and use `git cherry-pick --continue` once ready. After the `cherry-pick` is done cut&paste and run the `filter-branch` command to properly rewrite the messages
61+
62+
## Cherry-picking PRs into patch branch
63+
64+
In addition to merging PRs into the master branch, many PRs need to be also merged into a patch branch.
65+
Follow these steps to get path brach up to date.
66+
67+
1. Check out the most recent patch branch: `git checkout 4.3.x`
68+
2. Get a list of PRs merged into master: `git log master --oneline -n10`
69+
3. For each PR number in the commit message run: `././scripts/github/merge-pr 1234`
70+
- The PR will only merge if the `PR target:` matches the branch.
71+
72+
Once all of the PRs are in patch branch, push the all branches and tags to github using `push-upstream` script.
73+
74+
75+
## Pushing merged PRs into github
76+
77+
Use `push-upstream` script to push all of the branch and tags to github.
78+
79+
```
80+
$ ./scripts/github/push-upstream
81+
git push [email protected]:angular/angular.git master:master 4.3.x:4.3.x
82+
Counting objects: 25, done.
83+
Delta compression using up to 6 threads.
84+
Compressing objects: 100% (17/17), done.
85+
Writing objects: 100% (25/25), 2.22 KiB | 284.00 KiB/s, done.
86+
Total 25 (delta 22), reused 8 (delta 7)
87+
remote: Resolving deltas: 100% (22/22), completed with 18 local objects.
88+
To github.com:angular/angular.git
89+
079d884b6..d1c4a94bb master -> master
90+
git push --tags -f [email protected]:angular/angular.git patch_sync:patch_sync
91+
Everything up-to-date
92+
```

‎docs/COMMITTER.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,8 @@ Change approvals in our monorepo are managed via [pullapprove.com](https://about
1313

1414
# Merging
1515

16-
Once a change has all the approvals either the last approver or the PR author (if PR author has the project collaborator status) should mark the PR with "PR: merge" label.
17-
This signals to the caretaker that the PR should be merged.
16+
Once a change has all the approvals either the last approver or the PR author (if PR author has the project collaborator status) should mark the PR with `PR: merge` as well as `PR target: *` labels.
17+
This signals to the caretaker that the PR should be merged. See [merge instructions](../CARETAKER.md).
1818

1919
# Who is the Caretaker?
2020

‎scripts/github/merge-pr‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
#!/bin/sh
2+
3+
set -u -e -o pipefail
4+
5+
BASEDIR=$(dirname "$0")
6+
BASEDIR=`(cd $BASEDIR; pwd)`
7+
8+
if [ $# -eq 0 ]; then
9+
echo "Merge github PR into the current branch"
10+
echo
11+
echo "$0 PR_NUMBER"
12+
echo
13+
exit 0
14+
fi
15+
16+
PR_NUMBER="$1"
17+
CURRENT_BRANCH=`git rev-parse --abbrev-ref HEAD`
18+
PR_SHA_COUNT=`curl -s https://api.github.com/repos/angular/angular/pulls/$PR_NUMBER | node $BASEDIR/utils/json_extract.js commits`
19+
PR_LABELS=`curl -s https://api.github.com/repos/angular/angular/issues/$PR_NUMBER/labels`
20+
PR_ACTION=`echo "$PR_LABELS" | node $BASEDIR/utils/json_extract.js "name=^PR action:"`
21+
PR_TARGET=`echo "$PR_LABELS" | node $BASEDIR/utils/json_extract.js "name=^PR target:"`
22+
PR_CLA=`echo "$PR_LABELS" | node $BASEDIR/utils/json_extract.js "name=^cla"`
23+
MASTER_BRANCH='master'
24+
SHA=`git rev-parse HEAD`
25+
PATCH_BRANCH=`git branch --list '*.x' | cut -d ' ' -f2- | sort -r | head -n1`
26+
# Trim whitespace
27+
PATCH_BRANCH=`echo $PATCH_BRANCH`
28+
29+
if [[ "$PR_ACTION" != "PR action: merge" ]]; then
30+
echo The PR is missing 'PR action: merge' label, found: $PR_ACTION
31+
exit 1
32+
fi
33+
34+
if [[ "$PR_CLA" != "cla: yes" ]]; then
35+
echo The PR is missing 'cla: Yes' label, found: $PR_CLA
36+
exit 1
37+
fi
38+
39+
40+
if [[ $PR_TARGET == "PR target: master & patch" ]]; then
41+
MERGE_MASTER=1
42+
MERGE_PATCH=1
43+
elif [[ $PR_TARGET == "PR target: master-only" ]]; then
44+
MERGE_MASTER=1
45+
MERGE_PATCH=0
46+
elif [[ $PR_TARGET == "PR target: patch-only" ]]; then
47+
MERGE_MASTER=0
48+
MERGE_PATCH=1
49+
else
50+
echo "Unknown PR target format: $PR_TARGET"
51+
exit 1;
52+
fi
53+
54+
if [[ $CURRENT_BRANCH == $MASTER_BRANCH ]]; then
55+
if [[ $MERGE_MASTER == 0 ]]; then
56+
echo "This PR is not intended for master branch: $PR_TARGET"
57+
exit 1
58+
fi
59+
elif [[ $CURRENT_BRANCH == $PATCH_BRANCH ]]; then
60+
if [[ $MERGE_PATCH == 0 ]]; then
61+
echo "This PR is not intended for patch branch: $PR_TARGET"
62+
exit 1
63+
fi
64+
else
65+
echo "Current branch $CURRENT_BRANCH does not match $MASTER_BRANCH or $PATCH_BRANCH."
66+
exit 1
67+
fi
68+
69+
70+
CHERRY_PICK_PR="git cherry-pick upstream/pr/$PR_NUMBER~$PR_SHA_COUNT..upstream/pr/$PR_NUMBER"
71+
REWRITE_MESSAGE="git filter-branch -f --msg-filter \"$BASEDIR/utils/github_closes.js $PR_NUMBER\" HEAD~$PR_SHA_COUNT..HEAD"
72+
73+
echo "======================"
74+
echo "GitHub Merge PR Steps"
75+
echo "======================"
76+
echo " $CHERRY_PICK_PR"
77+
echo " $REWRITE_MESSAGE"
78+
echo "----------------------"
79+
80+
echo ">>> Cherry Pick: $CHERRY_PICK_PR"
81+
$CHERRY_PICK_PR
82+
83+
echo
84+
echo ">>> Rewrite Messages: $REWRITE_MESSAGE"
85+
# Next line should work, but it errors, hence copy paste the command.
86+
# $REWRITE_MESSAGE
87+
git filter-branch -f --msg-filter "$BASEDIR/utils/github_closes.js $PR_NUMBER" HEAD~$PR_SHA_COUNT..HEAD
88+
89+
echo
90+
echo ">>>>>> SUCCESS <<<<<<"

‎scripts/github/push-upstream‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/bin/sh
2+
3+
set -u -e -o pipefail
4+
5+
PATCH_BRANCH=`git branch --list '*.x' | cut -d ' ' -f2- | sort -r | head -n1`
6+
# Trim whitespace
7+
PATCH_BRANCH=`echo $PATCH_BRANCH`
8+
9+
PUSH_BRANCHES="git push [email protected]:angular/angular.git master:master $PATCH_BRANCH:$PATCH_BRANCH"
10+
11+
echo $PUSH_BRANCHES
12+
$PUSH_BRANCHES

‎scripts/github/rebase-pr‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
#!/bin/sh
2+
3+
set -u -e -o pipefail
4+
5+
BASEDIR=$(dirname "$0")
6+
7+
if [ $# -eq 0 ]; then
8+
echo "Rebase github PR onto a branch"
9+
echo
10+
echo "$0 branch_to_rebase_on PR_NUMBER"
11+
echo
12+
exit 0
13+
fi
14+
15+
REBASE_ON="$1"
16+
PR_NO="$2"
17+
18+
19+
HEAD_LABEL=`curl -s https://api.github.com/repos/angular/angular/pulls/$PR_NO | node $BASEDIR/utils/json_extract.js head.label`
20+
echo $HEAD_LABEL
21+
IFS=':' read -r -a array <<< "$HEAD_LABEL"
22+
USER="${array[0]}"
23+
USER_GIT_URL="[email protected]:$USER/angular.git"
24+
BRANCH="${array[1]}"
25+
OLD_BRANCH=`git branch | grep \* | cut -d ' ' -f2`
26+
27+
echo =====================================================
28+
echo Rebasing $USER_GIT_URL branch $BRANCH onto $REBASE_ON
29+
echo =====================================================
30+
31+
git fetch $USER_GIT_URL $BRANCH
32+
git co FETCH_HEAD
33+
PUSH_CMD="git push $USER_GIT_URL HEAD:$BRANCH -f";
34+
RESTORE_CMD="git co $OLD_BRANCH"
35+
git rebase upstream/master
36+
if [ $? -eq 0 ]; then
37+
$PUSH_CMD
38+
$RESTORE_CMD
39+
else
40+
echo ===========================
41+
echo Git rebase failed. RECOVER WITH:
42+
echo " $PUSH_CMD"
43+
echo " $RESTORE_CMD"
44+
echo ===========================
45+
fi

‎scripts/github/utils/github_closes.js‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/usr/bin/env node
2+
3+
var msg = '';
4+
5+
if (require.main === module) {
6+
process.stdin.setEncoding('utf8');
7+
8+
process.stdin.on('readable', () => {
9+
const chunk = process.stdin.read();
10+
if (chunk !== null) {
11+
msg += chunk;
12+
}
13+
});
14+
15+
process.stdin.on('end', () => {
16+
var argv = process.argv.slice(2);
17+
console.log(rewriteMsg(msg, argv[0]));
18+
});
19+
}
20+
21+
function rewriteMsg(msg, prNo) {
22+
var lines = msg.split(/\n/);
23+
lines[0] += ' (#' + prNo +')';
24+
lines.push('PR Close #' + prNo);
25+
return lines.join('\n');
26+
}
27+
28+
exports.rewriteMsg = rewriteMsg;

‎scripts/github/utils/json_extract.js‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
#!/usr/bin/env node
2+
3+
var json = '';
4+
5+
if (require.main === module) {
6+
process.stdin.setEncoding('utf8');
7+
8+
process.stdin.on('readable', () => {
9+
const chunk = process.stdin.read();
10+
if (chunk !== null) {
11+
json += chunk;
12+
}
13+
});
14+
15+
process.stdin.on('end', () => {
16+
var obj = JSON.parse(json);
17+
var argv = process.argv.slice(2);
18+
extractPaths(obj, argv).forEach(function(line) {
19+
console.log(line);
20+
})
21+
});
22+
}
23+
24+
function extractPaths(obj, paths) {
25+
var lines = [];
26+
paths.forEach(function(exp) {
27+
var objs = obj instanceof Array ? [].concat(obj) : [obj];
28+
exp.split('.').forEach(function(name) {
29+
for(var i = 0; i < objs.length; i++) {
30+
var o = objs[i];
31+
if (o instanceof Array) {
32+
// Expand and do over
33+
objs = objs.slice(0, i).concat(o).concat(objs.slice(i+1, objs.length));
34+
i--;
35+
} else {
36+
name.split("=").forEach(function(name, index) {
37+
if (index == 0) {
38+
objs[i] = o = o[name];
39+
} else if (name.charAt(0) == '^') {
40+
if (o.indexOf(name.substr(1)) !== 0) {
41+
objs.splice(i, 1);
42+
i--;
43+
}
44+
}
45+
});
46+
}
47+
}
48+
});
49+
lines.push(objs.join("|"));
50+
});
51+
return lines;
52+
}
53+
54+
exports.extractPaths = extractPaths;

‎scripts/github/utils/json_extract_test.js‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
#!/usr/bin/env node
2+
3+
var assert = require("assert");
4+
var extractPaths = require('./json_extract').extractPaths;
5+
6+
var SAMPLE_LABELS = [
7+
{
8+
"id": 149476251,
9+
"url": "https://api.github.com/repos/angular/angular/labels/cla:%20yes",
10+
"name": "cla: yes",
11+
"color": "009800",
12+
"default": false
13+
},
14+
{
15+
"id": 533874619,
16+
"url": "https://api.github.com/repos/angular/angular/labels/comp:%20aio",
17+
"name": "comp: aio",
18+
"color": "c7def8",
19+
"default": false
20+
},
21+
{
22+
"id": 133556520,
23+
"url": "https://api.github.com/repos/angular/angular/labels/PR%20action:%20merge",
24+
"name": "PR action: merge",
25+
"color": "99ff66",
26+
"default": false
27+
},
28+
{
29+
"id": 655699838,
30+
"url": "https://api.github.com/repos/angular/angular/labels/PR%20target:%20master%20&%20patch",
31+
"name": "PR target: master & patch",
32+
"color": "5319e7",
33+
"default": false
34+
}
35+
];
36+
37+
assert.deepEqual(extractPaths({head: {label: 'value1'}}, ['head.label']), ['value1']);
38+
assert.deepEqual(extractPaths(SAMPLE_LABELS, ['name']), ['cla: yes|comp: aio|PR action: merge|PR target: master & patch']);
39+
assert.deepEqual(extractPaths(SAMPLE_LABELS, ['name=^PR target:']), ['PR target: master & patch']);

0 commit comments

Comments
 (0)