Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add function to query git for all generated files
  • Loading branch information
mbg committed Nov 19, 2025
commit 1512f400b38466450275d117a9ba7e3ef72b298f
39 changes: 39 additions & 0 deletions src/git-utils.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import * as fs from "fs";
import * as os from "os";
import * as path from "path";

import * as core from "@actions/core";
Expand Down Expand Up @@ -392,3 +393,41 @@ test("getFileOidsUnderPath throws on unexpected output format", async (t) => {
runGitCommandStub.restore();
}
});

test("listFiles returns array of file paths", async (t) => {
sinon
.stub(gitUtils, "runGitCommand")
.resolves(["dir/file.txt", "README.txt"].join(os.EOL));

await t.notThrowsAsync(async () => {
const result = await gitUtils.listFiles("/some/path");
t.is(result.length, 2);
t.is(result[0], "dir/file.txt");
});
});
Comment on lines +397 to +407
Copy link

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stub created in this test is not restored after the test completes. This could cause test pollution if other tests rely on runGitCommand behavior. Consider wrapping the test in a try-finally block or using t.teardown() to restore the stub, similar to how it's done in other tests in this file (e.g., lines 393-394).

Copilot uses AI. Check for mistakes.

test("getGeneratedFiles returns generated files only", async (t) => {
const runGitCommandStub = sinon.stub(gitUtils, "runGitCommand");

runGitCommandStub
.onFirstCall()
.resolves(["dir/file.txt", "test.json", "README.txt"].join(os.EOL));
runGitCommandStub
.onSecondCall()
.resolves(
[
"dir/file.txt: linguist-generated: unspecified",
"test.json: linguist-generated: true",
"README.txt: linguist-generated: false",
].join(os.EOL),
);

await t.notThrowsAsync(async () => {
const result = await gitUtils.getGeneratedFiles("/some/path");

t.assert(runGitCommandStub.calledTwice);

t.is(result.length, 1);
t.is(result[0], "test.json");
});
Comment on lines +425 to +432
Copy link

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stub runGitCommandStub is not restored after the test completes. Add a try-finally block or use t.teardown() to ensure the stub is restored, similar to how it's done in other tests in this file (e.g., lines 393-394).

Suggested change
await t.notThrowsAsync(async () => {
const result = await gitUtils.getGeneratedFiles("/some/path");
t.assert(runGitCommandStub.calledTwice);
t.is(result.length, 1);
t.is(result[0], "test.json");
});
try {
await t.notThrowsAsync(async () => {
const result = await gitUtils.getGeneratedFiles("/some/path");
t.assert(runGitCommandStub.calledTwice);
t.is(result.length, 1);
t.is(result[0], "test.json");
});
} finally {
runGitCommandStub.restore();
}

Copilot uses AI. Check for mistakes.
});
33 changes: 33 additions & 0 deletions src/git-utils.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import * as os from "os";

import * as core from "@actions/core";
import * as toolrunner from "@actions/exec/lib/toolrunner";
import * as io from "@actions/io";
Expand Down Expand Up @@ -408,3 +410,34 @@ export async function isAnalyzingDefaultBranch(): Promise<boolean> {

return currentRef === defaultBranch;
}

export async function listFiles(workingDirectory: string): Promise<string[]> {
const stdout = await runGitCommand(
workingDirectory,
["ls-files"],
"Unable to list tracked files.",
);
return stdout.split(os.EOL);
Copy link

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Splitting on os.EOL will include an empty string at the end of the array if the git output ends with a newline (which is typical). This empty string will then be passed to git check-attr in getGeneratedFiles. Consider filtering out empty strings: return stdout.split(os.EOL).filter((line) => line.length > 0);

Suggested change
return stdout.split(os.EOL);
return stdout.split(os.EOL).filter((line) => line.length > 0);

Copilot uses AI. Check for mistakes.
}
Comment on lines +414 to +421
Copy link

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing JSDoc documentation. Consider adding a comment explaining what this function does, its parameters, and return value. For example: /**\n * Lists all files tracked by git in the given working directory.\n * @param workingDirectory The directory to list files in.\n * @returns An array of file paths relative to the working directory.\n */

Copilot uses AI. Check for mistakes.

export async function getGeneratedFiles(
workingDirectory: string,
): Promise<string[]> {
Comment on lines +423 to +425
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you measured how long this operation takes overall on a large mono-repo?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it should be relatively quick, since Git caches this information IIRC. That said, I haven't explicitly tested it on a large repo, but I can do that.

const files = await listFiles(workingDirectory);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could potentially be a very large number of files, too many to pass on the command line.

If we're mainly interested in CCR, could we filter down to just the diff here?

Alternatively, we could parse globs from the .gitattributes file rather than finding all files that match. That would be more likely to contain one entry for a large directory rather than potentially hundreds.

Or we could add a limit on the number of files on which we'll run check-attr.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could potentially be a very large number of files, too many to pass on the command line.

True. The files could be passed in via stdin as well I think, which might be a more robust solution.

If we're mainly interested in CCR, could we filter down to just the diff here?

Potentially. We should first look at whether there is actually enough of a performance hit for large repos for this to make sense though.

Alternatively, we could parse globs from the .gitattributes file rather than finding all files that match. That would be more likely to contain one entry for a large directory rather than potentially hundreds.

I considered this, but I'd like to avoid it if we can get the information from git itself so we don't have to try and duplicate what it does.

const stdout = await runGitCommand(
workingDirectory,
["check-attr", "linguist-generated", "--", ...files],
"Unable to check attributes of files.",
Comment on lines +426 to +430
Copy link

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing all files as command-line arguments via the spread operator could hit command-line length limits in repositories with very large numbers of files (e.g., tens of thousands). Consider using git check-attr --stdin instead, which reads file paths from stdin and avoids this limitation. This would involve piping the file list to git rather than passing it as arguments.

Copilot uses AI. Check for mistakes.
);

const generatedFiles: string[] = [];
const regex = /^([^:]+): linguist-generated: true$/;
for (const result of stdout.split(os.EOL)) {
Copy link

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Splitting on os.EOL will include an empty string at the end of the array if the git output ends with a newline (which is typical). The regex won't match empty strings, but it's cleaner to filter them out explicitly: for (const result of stdout.split(os.EOL).filter((line) => line.length > 0)). This pattern is used elsewhere in the codebase (e.g., line 284 checks if (line) before processing).

Suggested change
for (const result of stdout.split(os.EOL)) {
for (const result of stdout.split(os.EOL).filter((line) => line.length > 0)) {

Copilot uses AI. Check for mistakes.
const match = result.match(regex);
if (match) {
generatedFiles.push(match[1]);
}
}

return generatedFiles;
}
Comment on lines +423 to +443
Copy link

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing JSDoc documentation. Consider adding a comment explaining what this function does, its parameters, and return value. For example: /**\n * Returns a list of files marked as generated via the linguist-generated attribute in .gitattributes.\n * @param workingDirectory The directory to check for generated files.\n * @returns An array of file paths (relative to the working directory) that are marked as generated.\n */

Copilot uses AI. Check for mistakes.