Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve Reproducibility (Reproducible Builds) #54

Merged
merged 2 commits into from
Aug 3, 2022
Merged

Improve Reproducibility (Reproducible Builds) #54

merged 2 commits into from
Aug 3, 2022

Conversation

pacordova
Copy link
Contributor

@pacordova pacordova commented Aug 2, 2022 •
edited
Loading

On POSIX systems, find may have indeterministic output (I believe based on the inode on the filesystem).
sort is needed to ensure the output is deterministic.

In my builds, this resulted in the build order in the generated Makefile changing around.
Thus building on a fresh partition made with mkfs.ext4 would result in the binary being different each time.
The compiled binary code in thebc binary shuffled around depending on which files were built first.
Possibly the build order in the Makefile was affecting the order in which files were linked?

I added LC_ALL=C recommended by reproducible-builds.org.
See below for reference:

@pacordova
improve reproducibility
8c7d5fa 
find has indeterministic output based on the inode on the filesystem,
so sort is needed to ensure the output is deterministic
@pacordova
remove -z
6ea4d12
@gavinhoward gavinhoward merged commit 71215b0 into gavinhoward:master Aug 3, 2022
@gavinhoward
Copy link
Owner

The only hangup I would have had is if the way you did it was not portable to POSIX sh. I consulted the standard, and your solution is portable!

I agree that reproducible builds are very important. So thank you for your contribution! I have accepted it, and it will be in the release out in a day or two. I just have to make sure everything builds correctly.

@pacordova
Copy link
Contributor Author

pacordova commented Aug 3, 2022 •
edited
Loading

No problem! Thank you for accepting my pull request!
One additional comment:
With this change I do get reproducible builds, although I do not use -flto
Your recommended optimization of -O3 I think should be ok, but I do know that Link Time Optimization (LTO) and Profile Guided Optimization (PGO) can both potentially break build determinism.
Either way the end user should be able get reproducible builds if they want it with this change.
The CFLAGS I was testing with were CFLAGS='-march=x86-64 -pipe -Os -fstack-protector-strong -fstack-clash-protection' with GCC 11.2.0.

@gavinhoward
Copy link
Owner

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pacordova @gavinhoward