HWAddressSanitizer support #81506

vo4 · 2021-01-29T08:47:16Z

Motivation

Compared to regular ASan, HWASan has a smaller overhead. The difference in practice is that HWASan'ed code is more usable, e.g. Android device compiled with HWASan can be used as a daily driver.

Example

fn main() {
    let xs = vec![0, 1, 2, 3];
    let _y = unsafe { *xs.as_ptr().offset(4) };
}

==223==ERROR: HWAddressSanitizer: tag-mismatch on address 0xefdeffff0050 at pc 0xaaaad00b3468
READ of size 4 at 0xefdeffff0050 tags: e5/00 (ptr/mem) in thread T0
    #0 0xaaaad00b3464  (/root/main+0x53464)
    #1 0xaaaad00b39b4  (/root/main+0x539b4)
    #2 0xaaaad00b3dd0  (/root/main+0x53dd0)
    #3 0xaaaad00b61dc  (/root/main+0x561dc)
    #4 0xaaaad00c0574  (/root/main+0x60574)
    #5 0xaaaad00b6290  (/root/main+0x56290)
    #6 0xaaaad00b6170  (/root/main+0x56170)
    #7 0xaaaad00b3578  (/root/main+0x53578)
    #8 0xffff81345e70  (/lib64/libc.so.6+0x20e70)
    #9 0xaaaad0096310  (/root/main+0x36310)

[0xefdeffff0040,0xefdeffff0060) is a small allocated heap chunk; size: 32 offset: 16
0xefdeffff0050 is located 0 bytes to the right of 16-byte region [0xefdeffff0040,0xefdeffff0050)
allocated here:
    #0 0xaaaad009bcdc  (/root/main+0x3bcdc)
    #1 0xaaaad00b1eb0  (/root/main+0x51eb0)
    #2 0xaaaad00b20d4  (/root/main+0x520d4)
    #3 0xaaaad00b2800  (/root/main+0x52800)
    #4 0xaaaad00b1cf4  (/root/main+0x51cf4)
    #5 0xaaaad00b33d4  (/root/main+0x533d4)
    #6 0xaaaad00b39b4  (/root/main+0x539b4)
    #7 0xaaaad00b61dc  (/root/main+0x561dc)
    #8 0xaaaad00b3578  (/root/main+0x53578)
    #9 0xaaaad0096310  (/root/main+0x36310)

Thread: T0 0xeffe00002000 stack: [0xffffc0590000,0xffffc0d90000) sz: 8388608 tls: [0xffff81521020,0xffff815217d0)
Memory tags around the buggy address (one tag corresponds to 16 bytes):
  0xfefcefffef80: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffef90: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefa0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefb0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefc0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefd0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffefe0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefcefffeff0: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
=>0xfefceffff000: a2  a2  05  00  e5 [00] 00  00  00  00  00  00  00  00  00  00 
  0xfefceffff010: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff020: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff030: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff040: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff050: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff060: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff070: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
  0xfefceffff080: 00  00  00  00  00  00  00  00  00  00  00  00  00  00  00  00 
Tags for short granules around the buggy address (one tag corresponds to 16 bytes):
  0xfefcefffeff0: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  .. 
=>0xfefceffff000: ..  ..  c5  ..  .. [..] ..  ..  ..  ..  ..  ..  ..  ..  ..  .. 
  0xfefceffff010: ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  .. 
See https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#short-granules for a description of short granule tags
Registers where the failure occurred (pc 0xaaaad00b3468):
    x0  e500efdeffff0050  x1  0000000000000004  x2  0000ffffc0d8f5a0  x3  0200efff00000000
    x4  0000ffffc0d8f4c0  x5  000000000000004f  x6  00000ffffc0d8f36  x7  0000efff00000000
    x8  e500efdeffff0050  x9  0200efff00000000  x10 0000000000000000  x11 0200efff00000000
    x12 0200effe000006b0  x13 0200effe000006b0  x14 0000000000000008  x15 00000000c00000cf
    x16 0000aaaad00a0afc  x17 0000000000000003  x18 0000000000000001  x19 0000ffffc0d8f718
    x20 ba00ffffc0d8f7a0  x21 0000aaaad00962e0  x22 0000000000000000  x23 0000000000000000
    x24 0000000000000000  x25 0000000000000000  x26 0000000000000000  x27 0000000000000000
    x28 0000000000000000  x29 0000ffffc0d8f650  x30 0000aaaad00b3468

Comments/Caveats

HWASan is only supported on arm64.
I'm not sure if I should add a feature gate or piggyback on the existing one for sanitizers.
HWASan requires -C target-feature=+tagged-globals. That flag should probably be set transparently to the user. Not sure how to go about that.

TODO

Need more tests.
Update documentation.
Fix symbolization.
Integrate with CI

rust-highfive · 2021-01-29T08:47:18Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @oli-obk (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

src/bootstrap/test.rs

src/test/ui/sanitize/hwaddress.rs

tmiasko · 2021-01-29T12:19:50Z

Clang also enables emission of lifetime markers by default for HWASAN, should we do we same? I couldn't immediately identify how those are used, but if they are useful the code is in

rust/compiler/rustc_session/src/session.rs

Lines 1125 to 1130 in 099f27b

    
           pub fn emit_lifetime_markers(&self) -> bool { 
        
               self.opts.optimize != config::OptLevel::No 
        
               // AddressSanitizer uses lifetimes to detect use after scope bugs. 
        
               // MemorySanitizer uses lifetimes to detect use of uninitialized stack variables. 
        
               || self.opts.debugging_opts.sanitizer.intersects(SanitizerSet::ADDRESS | SanitizerSet::MEMORY) 
        
           }

vo4 · 2021-02-01T19:32:47Z

Clang also enables emission of lifetime markers by default for HWASAN, should we do we same? I couldn't immediately identify how those are used, but if they are useful the code is in

rust/compiler/rustc_session/src/session.rs

Lines 1125 to 1130 in 099f27b

pub fn emit_lifetime_markers(&self) -> bool {

self.opts.optimize != config::OptLevel::No

// AddressSanitizer uses lifetimes to detect use after scope bugs.

// MemorySanitizer uses lifetimes to detect use of uninitialized stack variables.

|| self.opts.debugging_opts.sanitizer.intersects(SanitizerSet::ADDRESS | SanitizerSet::MEMORY)

}

Good spot! I didn't see the instrumentation pass or the runtime use lifetime markers, so I left it out. But if clang emits them, maybe they are actually used. I couldn't find out how/if HWASan uses them though. @pcc @eugenis do you folks know?

pcc · 2021-02-01T19:39:41Z

We don't currently use them, but we do plan to use them to implement use-after-scope checks in HWASan.

oli-obk · 2021-02-02T10:01:39Z

I have honestly no clue about the changes in this PR.

r? @tmiasko you seem to be on top of this

compiler/rustc_session/src/options.rs

compiler/rustc_typeck/src/collect.rs

tmiasko · 2021-02-05T18:46:32Z

The documentation for sanitizers is currently in src/doc/unstable-book/src/compiler-flags/sanitizer.md. It can be built with ./x.py doc src/doc/unstable-book/, and later found in ./build/$target/doc/unstable-book/index.html.
The aarch64-gnu CI job should run the tests during merge, if you would like you can also enable this runner temporarily for pull request CI by including the job in pr section of src/ci/github-actions/ci.yml.

r? @nagisa

nagisa

Overall LGTM, r=me once nits are fixed.

compiler/rustc_session/src/session.rs

vo4 · 2021-02-09T07:16:34Z

r= @nagisa

nagisa · 2021-02-11T20:01:51Z

@bors r+

bors · 2021-02-11T20:01:53Z

📋 Looks like this PR is still in progress, ignoring approval.

Hint: Remove [WIP] from this PR's title when it is ready for review.

nagisa · 2021-02-11T20:02:10Z

@bors r+

bors · 2021-02-11T20:02:11Z

📌 Commit 9c34c14 has been approved by nagisa

HWAddressSanitizer support # Motivation Compared to regular ASan, HWASan has a [smaller overhead](https://source.android.com/devices/tech/debug/hwasan). The difference in practice is that HWASan'ed code is more usable, e.g. Android device compiled with HWASan can be used as a daily driver. # Example ``` fn main() { let xs = vec![0, 1, 2, 3]; let _y = unsafe { *xs.as_ptr().offset(4) }; } ``` ``` ==223==ERROR: HWAddressSanitizer: tag-mismatch on address 0xefdeffff0050 at pc 0xaaaad00b3468 READ of size 4 at 0xefdeffff0050 tags: e5/00 (ptr/mem) in thread T0 #0 0xaaaad00b3464 (/root/main+0x53464) rust-lang#1 0xaaaad00b39b4 (/root/main+0x539b4) rust-lang#2 0xaaaad00b3dd0 (/root/main+0x53dd0) rust-lang#3 0xaaaad00b61dc (/root/main+0x561dc) rust-lang#4 0xaaaad00c0574 (/root/main+0x60574) rust-lang#5 0xaaaad00b6290 (/root/main+0x56290) rust-lang#6 0xaaaad00b6170 (/root/main+0x56170) rust-lang#7 0xaaaad00b3578 (/root/main+0x53578) rust-lang#8 0xffff81345e70 (/lib64/libc.so.6+0x20e70) rust-lang#9 0xaaaad0096310 (/root/main+0x36310) [0xefdeffff0040,0xefdeffff0060) is a small allocated heap chunk; size: 32 offset: 16 0xefdeffff0050 is located 0 bytes to the right of 16-byte region [0xefdeffff0040,0xefdeffff0050) allocated here: #0 0xaaaad009bcdc (/root/main+0x3bcdc) rust-lang#1 0xaaaad00b1eb0 (/root/main+0x51eb0) rust-lang#2 0xaaaad00b20d4 (/root/main+0x520d4) rust-lang#3 0xaaaad00b2800 (/root/main+0x52800) rust-lang#4 0xaaaad00b1cf4 (/root/main+0x51cf4) rust-lang#5 0xaaaad00b33d4 (/root/main+0x533d4) rust-lang#6 0xaaaad00b39b4 (/root/main+0x539b4) rust-lang#7 0xaaaad00b61dc (/root/main+0x561dc) rust-lang#8 0xaaaad00b3578 (/root/main+0x53578) rust-lang#9 0xaaaad0096310 (/root/main+0x36310) Thread: T0 0xeffe00002000 stack: [0xffffc0590000,0xffffc0d90000) sz: 8388608 tls: [0xffff81521020,0xffff815217d0) Memory tags around the buggy address (one tag corresponds to 16 bytes): 0xfefcefffef80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefcefffef90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefcefffefa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefcefffefb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefcefffefc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefcefffefd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefcefffefe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefcefffeff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0xfefceffff000: a2 a2 05 00 e5 [00] 00 00 00 00 00 00 00 00 00 00 0xfefceffff010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefceffff020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefceffff030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefceffff040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefceffff050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefceffff060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefceffff070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0xfefceffff080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Tags for short granules around the buggy address (one tag corresponds to 16 bytes): 0xfefcefffeff0: .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. =>0xfefceffff000: .. .. c5 .. .. [..] .. .. .. .. .. .. .. .. .. .. 0xfefceffff010: .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. See https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#short-granules for a description of short granule tags Registers where the failure occurred (pc 0xaaaad00b3468): x0 e500efdeffff0050 x1 0000000000000004 x2 0000ffffc0d8f5a0 x3 0200efff00000000 x4 0000ffffc0d8f4c0 x5 000000000000004f x6 00000ffffc0d8f36 x7 0000efff00000000 x8 e500efdeffff0050 x9 0200efff00000000 x10 0000000000000000 x11 0200efff00000000 x12 0200effe000006b0 x13 0200effe000006b0 x14 0000000000000008 x15 00000000c00000cf x16 0000aaaad00a0afc x17 0000000000000003 x18 0000000000000001 x19 0000ffffc0d8f718 x20 ba00ffffc0d8f7a0 x21 0000aaaad00962e0 x22 0000000000000000 x23 0000000000000000 x24 0000000000000000 x25 0000000000000000 x26 0000000000000000 x27 0000000000000000 x28 0000000000000000 x29 0000ffffc0d8f650 x30 0000aaaad00b3468 ``` # Comments/Caveats * HWASan is only supported on arm64. * I'm not sure if I should add a feature gate or piggyback on the existing one for sanitizers. * HWASan requires `-C target-feature=+tagged-globals`. That flag should probably be set transparently to the user. Not sure how to go about that. # TODO * Need more tests. * Update documentation. * Fix symbolization. * Integrate with CI

Rollup of 10 pull requests Successful merges: - rust-lang#79775 (Fix injected errors when running doctests on a crate named after a keyword) - rust-lang#81012 (Stabilize the partition_point feature) - rust-lang#81479 (Allow casting mut array ref to mut ptr) - rust-lang#81506 (HWAddressSanitizer support) - rust-lang#81741 (Increment `self.index` before calling `Iterator::self.a.__iterator_ge…) - rust-lang#81850 (use RWlock when accessing os::env) - rust-lang#81911 (GAT/const_generics: Allow with_opt_const_param to return GAT param def_id) - rust-lang#82022 (Push a `char` instead of a `str` with len one into a String) - rust-lang#82023 (Remove unnecessary lint allow attrs on example) - rust-lang#82030 (Use `Iterator::all` instead of open-coding it) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

This CL removes the patch that enables HWASan support for Rust. This functionality was upstreamed here: rust-lang/rust#81506 Test: Build 1.52.1 Change-Id: I194f8eaf95000cb272b7dad695aa21218347717f

rust-highfive assigned oli-obk Jan 29, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 29, 2021

tmiasko reviewed Jan 29, 2021

View reviewed changes

src/bootstrap/test.rs Outdated Show resolved Hide resolved

src/test/ui/sanitize/hwaddress.rs Show resolved Hide resolved

rust-highfive assigned tmiasko and unassigned oli-obk Feb 2, 2021

tmiasko reviewed Feb 5, 2021

View reviewed changes

compiler/rustc_session/src/options.rs Outdated Show resolved Hide resolved

compiler/rustc_typeck/src/collect.rs Show resolved Hide resolved

rust-highfive assigned nagisa and unassigned tmiasko Feb 5, 2021

nagisa approved these changes Feb 5, 2021

View reviewed changes

compiler/rustc_session/src/session.rs Outdated Show resolved Hide resolved

Tri Vo added 2 commits February 7, 2021 23:48

HWASan support

c7d9bff

HWASan documentation

9c34c14

nagisa changed the title ~~[WIP] HWAddressSanitizer support~~ HWAddressSanitizer support Feb 11, 2021

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 11, 2021

Dylan-DPC-zz mentioned this pull request Feb 11, 2021

Rollup of 12 pull requests #82010

Closed

Dylan-DPC-zz mentioned this pull request Feb 12, 2021

Rollup of 11 pull requests #82036

Closed

Dylan-DPC-zz mentioned this pull request Feb 12, 2021

Rollup of 10 pull requests #82045

Merged

bors merged commit 58d72ae into rust-lang:master Feb 13, 2021

rustbot added this to the 1.52.0 milestone Feb 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HWAddressSanitizer support #81506

HWAddressSanitizer support #81506

vo4 commented Jan 29, 2021

rust-highfive commented Jan 29, 2021

tmiasko commented Jan 29, 2021

vo4 commented Feb 1, 2021

pcc commented Feb 1, 2021

oli-obk commented Feb 2, 2021

tmiasko commented Feb 5, 2021

nagisa left a comment

vo4 commented Feb 9, 2021

nagisa commented Feb 11, 2021

bors commented Feb 11, 2021

nagisa commented Feb 11, 2021

bors commented Feb 11, 2021

HWAddressSanitizer support #81506

HWAddressSanitizer support #81506

Conversation

vo4 commented Jan 29, 2021

Motivation

Example

Comments/Caveats

TODO

rust-highfive commented Jan 29, 2021

tmiasko commented Jan 29, 2021

vo4 commented Feb 1, 2021

pcc commented Feb 1, 2021

oli-obk commented Feb 2, 2021

tmiasko commented Feb 5, 2021

nagisa left a comment

Choose a reason for hiding this comment

vo4 commented Feb 9, 2021

nagisa commented Feb 11, 2021

bors commented Feb 11, 2021

nagisa commented Feb 11, 2021

bors commented Feb 11, 2021