Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for building sanitizer-enabled jlls #255

Merged
merged 2 commits into from
Aug 15, 2022
Merged

Conversation

Keno
Copy link
Contributor

@Keno Keno commented Jul 31, 2022

This adds support for automatically adding the appropriate -fsanitize=
flags when the platform includes a sanitizer tag. This is particularly
intended for msan, which requires that all .so's in the system are built
using it, but could be useful for other sanitizers also.

There's a couple annoyances. One is that we don't currently actually ship
the sanitizer runtime libraries in our rootfs. Thus, if we want to start
using this, we'd have to add a BuildDependency on LLVMCompilerRT_jll and
manually copy the runtime libraries into place. Not the end of the world,
but certainly a wart.

The other issue is that platforms_match (which is defined in Base) of
course currently ignores the sanitizer tag. In theory it is possible
to add a custom compare strategy, but since we're specifying the target
by triplet, we can't really add that. Easy enough to add manually here,
but does make me wonder whether the custom compare strategies in Base
actually fit the use case.

@Keno Keno requested a review from staticfloat July 31, 2022 11:06
Copy link
Member

@staticfloat staticfloat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, this looks great to me. What would be a good test, so we can verify that this is working?

@Keno
Copy link
Contributor Author

Keno commented Aug 2, 2022

You mean for CI? Maybe just check that something like:

int main() {
   return *(int*)1;
}

triggers an asan/msan report?

@Keno
Copy link
Contributor Author

Keno commented Aug 2, 2022

We should probably also teach the auditor to check for the presence of the asan/msan symbols in the executable to make sure that the thing that got compiled actually had the options set.

src/Platforms.jl Outdated
@@ -146,3 +146,4 @@ const ARCHITECTURE_FLAGS = Dict(
),
)
march(p::AbstractPlatform; default=nothing) = get(tags(p), "march", default)
sanitizer(p::AbstractPlatform; default=nothing) = get(tags(p), "sanitizer", default)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to call this sanitizer or sanitize? The latter would match better with -fsanitize.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like sanitize

This adds support for automatically adding the appropriate `-fsanitize=`
flags when the platform includes a `sanitizer` tag. This is particularly
intended for msan, which requires that all .so's in the system are built
using it, but could be useful for other sanitizers also.

There's a couple annoyances. One is that we don't currently actually ship
the sanitizer runtime libraries in our rootfs. Thus, if we want to start
using this, we'd have to add a BuildDependency on LLVMCompilerRT_jll and
manually copy the runtime libraries into place. Not the end of the world,
but certainly a wart.

The other issue is that `platform_matches` (which is defined in Base) of
course currently ignores the `sanitizer` tag. In theory it is possible
to add a custom compare strategy, but since we're specifying the target
by triplet, we can't really add that. Easy enough to add manually here,
but does make me wonder whether the custom compare strategies in Base
actually fit the use case.
@Keno
Copy link
Contributor Author

Keno commented Aug 12, 2022

Renamed this to sanitize and tested by building SuiteSparse and running the SparseArray tests on JuliaLang/julia#46336, so I think this is ok for an initial cut.

@giordano
Copy link
Member

What's the benefit of having this as a platform tag? Is a library built with sanitizer incompatible with another library built without it? This is probably what you meant with

This is particularly
intended for msan, which requires that all .so's in the system are built
using it

?

we'd have to add a BuildDependency on LLVMCompilerRT_jll and
manually copy the runtime libraries into place.

Would JuliaPackaging/Yggdrasil#1681 be a better long-term solution?

@Keno
Copy link
Contributor Author

Keno commented Aug 14, 2022

Is a library built with sanitizer incompatible with another library built without it?

Yes

@Keno
Copy link
Contributor Author

Keno commented Aug 14, 2022

Would JuliaPackaging/Yggdrasil#1681 be a better long-term solution?

Yes - I haven't looked at that PR in detail, but presumably.

@Keno
Copy link
Contributor Author

Keno commented Aug 14, 2022

Is a library built with sanitizer incompatible with another library built without it?

Yes

In particular, julia would detect that it's built with sanitizers and append the platform tag itself.

@Keno
Copy link
Contributor Author

Keno commented Aug 15, 2022

We should probably split out memory into memory and memory_origins, since the origin tracking is another 2-3 perf overhead and incompatible. If you're using rr anyway, the origin tracking doesn't really buy you anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants