Commit Graph

11288 Commits

Author SHA1 Message Date
Kostya Kortchinsky
7547170ca4 [scudo][standalone] Do not error out on spurious C(XX) flags
Summary:
The standalone Scudo version is being built with `-Werror` which can be
tripped by extraneous command line arguments. We have little control over
those as they can be passed down to us by `CMAKE_C(XX)_FLAGS`, the reported
scenario involving `-stdlib=libc++` (see https://reviews.llvm.org/D57412#1384504).

To work around this, disable `-Wunused-command-line-argument`.

Reviewers: eugenis, vitalybuka, Eugene.Zelenko

Reviewed By: eugenis

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D57757

llvm-svn: 353418
2019-02-07 15:44:36 +00:00
Dmitry Vyukov
bdfba86047 tsan: add more benchmarks
func_entry_exit.cc is for __tsan_func_entry/exit (spends ~75% there),
we don't yet have any.
mop.cc is for memory access functions, as compared to mini_bench_local/shared.cc
this benchmark passes through deduplication logic (ContainsSameAccess).

llvm-svn: 353407
2019-02-07 14:04:04 +00:00
Dmitry Vyukov
fddaf1f369 tsan: Optimize performance of Thread sanitizer memory access functions
- Manually unwind code in MemoryAccessImpl1() because clang do not optimize it
- Check for .rodata section only in read operations
- Place LIKELY/UNLIKELY on fast paths

This speeds up synthetic memory access benchmarks by 10-20%.

[dvyukov: fixed up consts in check_analyze.sh]

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57882
Context: https://reviews.llvm.org/D54889

llvm-svn: 353401
2019-02-07 12:43:09 +00:00
Dmitry Vyukov
baf2f35ec4 sanitizers: Introduce ThreadType enum
Replace bool workerthread flag with ThreadType enum.
This change is preparation for fiber support.

[dvyukov: fixed build of sanitizer_thread_registry_test.cc]

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57839
Context: https://reviews.llvm.org/D54889

llvm-svn: 353390
2019-02-07 11:01:22 +00:00
Dmitry Vyukov
17132b62e0 tsan: Implement pthread_exit() interceptor for Thread sanitizer
This change is preparation for fiber support.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57876
Context: https://reviews.llvm.org/D54889

llvm-svn: 353385
2019-02-07 10:46:02 +00:00
Vitaly Buka
c41835230a [sanitizer] Fix Android tests
On Android some fields can be null

llvm-svn: 353377
2019-02-07 07:40:33 +00:00
Petr Hosek
55a2fbe234 [CMake] Mark runtime library link libraries as private
There's no need to expose these dependencies to consumers. This
matches the change made to other runtimes in D57456.

Differential Revision: https://reviews.llvm.org/D57873

llvm-svn: 353376
2019-02-07 06:32:09 +00:00
Vitaly Buka
2b93ef7ef6 [sanitizer] Re-enabled getpw_getgr.cc on Android
Reviewers: eugenis

Subscribers: srhines, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57785

llvm-svn: 353366
2019-02-07 01:55:07 +00:00
Vitaly Buka
1665ea19e9 [sanitizer] Remove unneeded pointer check
Summary: unpoison_passwd and unpoison_group support nullptrs

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57784

llvm-svn: 353365
2019-02-07 01:54:59 +00:00
Vitaly Buka
eb7031ecf6 [sanitizer] Don't unpoison buffer in getpw/getgr functions
Summary:
Buffer should be referenced by results so used parts will be unpoisoned with unpoison_group and unpoison_passwd.

This fixes TSAN performance issue made us to disable this interceptors.

Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, krytarowski, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57731

llvm-svn: 353351
2019-02-07 00:08:14 +00:00
Evgeniy Stepanov
54c82886cb [sanitizer] Fix fuchsia and windows build.
llvm-svn: 353261
2019-02-06 01:58:23 +00:00
Evgeniy Stepanov
443c034391 [sanitizer] Decorate /proc/self/maps better.
Summary:
Refactor the way /proc/self/maps entries are annotated to support most
(all?) posix platforms, with a special implementation for Android.
Extend the set of decorated Mmap* calls.

Replace shm_open with internal_open("/dev/shm/%s"). Shm_open is
problematic because it calls libc open() which may be intercepted.

Generic implementation has limits (max number of files under /dev/shm is
64K on my machine), which can be conceivably reached when sanitizing
multiple programs at once. Android implemenation is essentially free, and
enabled by default.

The test in sanitizer_common is copied to hwasan and not reused directly
because hwasan fails way too many common tests at the moment.

Reviewers: pcc, vitalybuka

Subscribers: srhines, kubamracek, jfb, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D57720

llvm-svn: 353255
2019-02-06 01:14:50 +00:00
Kamil Rytarowski
3349bd662a Update the ioctl(2) list in sanitizers with NetBSD 8.99.34
llvm-svn: 353224
2019-02-05 22:20:25 +00:00
Reid Kleckner
0a142346b3 [InstrProf] Merge COFF .lprof* and .lcovmap sections into .data/.rdata
There is no reason for these sections to remain separate in the final
DLL or EXE.

I have not yet added a InstrProfilingPlatformWindows.c for these, since
avoiding dynamic profile data registration is a larger project for
later.

llvm-svn: 353221
2019-02-05 21:26:31 +00:00
Vitaly Buka
0ca702f8d5 [sanitizer] Fix unused function 'unpoison_passwd'
llvm-svn: 353205
2019-02-05 19:46:44 +00:00
Peter Collingbourne
6141b037a9 gn build: Upgrade to NDK r19.
NDK r19 includes a sysroot that can be used directly by the compiler
without creating a standalone toolchain, so we just need a handful
of flags to point Clang there.

Differential Revision: https://reviews.llvm.org/D57733

llvm-svn: 353139
2019-02-05 05:10:19 +00:00
Vitaly Buka
d2a42c3815 [tsan] Disable fgetpwent_r to work around performance issues
This was missed from D54041 when SANITIZER_INTERCEPT_FGETPWENT_R was branched from SANITIZER_INTERCEPT_GETPWENT_R

llvm-svn: 353110
2019-02-04 23:25:55 +00:00
Jonathan Metzman
7c907bac6f [libFuzzer][Windows] Clean up RawPrint
Summary:
Use `_write` instead of the deprecated alias `write` on Windows.
Also, remove comment saying RawPrint is untested on Windows.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: vitalybuka

Differential Revision: https://reviews.llvm.org/D57589

llvm-svn: 353108
2019-02-04 23:01:06 +00:00
Kostya Kortchinsky
47f0d136f1 [scudo] Initial standalone skeleton check-in
Summary:
This is the initial check-in for the Standalone version of Scudo.

The project is initially going to live in scudo/standalone then will
replace scudo. See http://lists.llvm.org/pipermail/llvm-dev/2019-January/129113.html
for details.

This initial CL is meant to lay out the project structure, of both
code & tests, providing a minimal amount of functionalities, namely
various definitions, some atomic helpers and an intrusive list.
(empty.cc is just here to have a compilation unit, but will go away
in the upcoming CLs).

Initial support is restricted to Linux i386 & x86_64 in make files
and will be extended once things land & work.

We will grow organically from here, adding functionalities in limited
amounts.

Reviewers: morehouse, eugenis, vitalybuka, kcc, mcgrathr, flowerhack

Reviewed By: morehouse, vitalybuka

Subscribers: srhines, mgorny, krytarowski, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D57412

llvm-svn: 353055
2019-02-04 16:25:40 +00:00
Jonathan Metzman
e2469b11a5 [libFuzzer][Windows] Reenable passing tests
Summary:
Enable tests that were previously disabled because they didn't work on
Windows.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: morehouse

Differential Revision: https://reviews.llvm.org/D57563

llvm-svn: 353000
2019-02-03 16:53:32 +00:00
Reid Kleckner
33706e3cbf [ubsan] Make suppressions.cpp test pass for me on Windows
The test seems to be failing because the module suppression file
contains a colon. I found that it was sufficient to just use the
basename of the suppression file.

While I was here, I noticed that we don't implement IsAbsolutePath for
Windows, so I added it.

llvm-svn: 352921
2019-02-01 21:35:17 +00:00
Peter Collingbourne
6fa43f8b9c Mark __rela_iplt_{start,end} as weak.
Should fix non-lld links.

llvm-svn: 352823
2019-02-01 00:43:31 +00:00
Kostya Serebryany
ae667c49d2 [libFuzzer] make a test for exploding dfsan labels more agressive
llvm-svn: 352819
2019-01-31 23:43:54 +00:00
Peter Collingbourne
886b7cc107 hwasan: Add __hwasan_init_static() function.
This function initializes enough of the runtime to be able to run
instrumented code in a statically linked executable. It replaces
__hwasan_shadow_init() which wasn't doing enough initialization for
instrumented code that uses either TLS or IFUNC to work.

Differential Revision: https://reviews.llvm.org/D57490

llvm-svn: 352816
2019-01-31 23:37:12 +00:00
Jonathan Metzman
6f94a033a7 [libFuzzer][Windows] Temporarily disable value-profile-cmp2.test on Win
Summary:
Temporarily disable value-profile-cmp2.test on Win.
https://reviews.llvm.org/D57465 causes the test to fail on Win.
However, it seems that the behavior of libFuzzer on Win was broken
before that patch. It crashes in the exit handler when not used with
ASAN. Prior to the patch, the crash handler would run, tricking the
test into thinking libFuzzer on Win had exited properly.

Reviewers: morehouse, vitalybuka

Reviewed By: morehouse

Subscribers: yln

Differential Revision: https://reviews.llvm.org/D57551

llvm-svn: 352815
2019-01-31 23:34:56 +00:00
Jonathan Metzman
fc7faecb40 [fuzzer] Use RawPrint instead of Printf for instrumentation warning
Summary:
Use RawPrint instead of Printf for instrumentation warning because
Printf doesn't work on Win when instrumentation is being
initialized (since OutputFile is not yet initialized).

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D57531

llvm-svn: 352789
2019-01-31 20:32:20 +00:00
Kostya Serebryany
02a86e6c7e [libFuzzer] relax a flaky test
llvm-svn: 352778
2019-01-31 18:40:12 +00:00
Jeremy Morse
d778d4fded Revert r352732: [libFuzzer] replace slow std::mt19937 with a much faster std::minstd_rand
This causes a failure on the following bot as well as our internal ones:

http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/23103

llvm-svn: 352747
2019-01-31 12:44:14 +00:00
Kostya Serebryany
046cac61c1 [libFuzzer] replace slow std::mt19937 with a much faster std::minstd_rand
llvm-svn: 352732
2019-01-31 06:52:55 +00:00
Petr Hosek
e2c021d8b7 [CMake][compiler-rt] Enable statically linking unwinder and c++abi
Rather than guessing whether to use static or shared version of
unwinder and c++abi when using linking against the in-tree versions,
provide a CMake option to control this.

Differential Revision: https://reviews.llvm.org/D57492

llvm-svn: 352723
2019-01-31 03:38:43 +00:00
Julian Lettner
7e0219d979 [libFuzzer] Update Darwin test
Support for -fsanitize-coverage=trace-pc[-guard] was removed from
libFuzzer, which makes this currently fail.

This commit aligns this Darwin-specific test with its Linux counterpart
which changed in this commit:
3a94519a77

llvm-svn: 352721
2019-01-31 03:03:06 +00:00
Kostya Serebryany
23482e1d07 [libFuzzer] set libFuzzer's own SEGV handler even one is already present, but call that handler from ours (unless we are unprotecting lazy counters). Call ProtectLazyCounters later, so that it runs after the initialization code in the target.
llvm-svn: 352713
2019-01-31 01:40:14 +00:00
Julian Lettner
15df273eb4 [libFuzzer] Set default sanitizer options in fuzzer tests
Summary:
Set default `ASAN_OPTIONS` when running libFuzzer tests. This allows us
to remove special casing in code for Darwin where we usually pass
`abort_on_error=0` to override platform defaults for tests.

A previous commit changed the code to make the tests pass:
7764a04af0

Adapted a few tests to use `%env_asan_opts=` instead of directly setting
the environment variable.

rdar://problem/47515276

Reviewers: kcc, george.karpenkov

Differential Revision: https://reviews.llvm.org/D57465

llvm-svn: 352711
2019-01-31 01:24:01 +00:00
Kostya Serebryany
8da9479e40 [libFuzzer] experimental performance optimization -lazy_counters, off by default. Posix-only for now, tested on Linux
llvm-svn: 352700
2019-01-31 00:09:43 +00:00
Kostya Serebryany
5c0751ec3e [libFuzzer] remove stale code, NFC
llvm-svn: 352604
2019-01-30 06:21:20 +00:00
Kostya Serebryany
6fd4d8ab9c [libFuzzer] refactor the handling of instrumentation counters so that they are grouped in regions one full page each. Needed for future optimization. NFC
llvm-svn: 352603
2019-01-30 06:15:52 +00:00
Kostya Serebryany
3a94519a77 [libFuzzer] update a test
llvm-svn: 352577
2019-01-30 00:36:28 +00:00
Kostya Serebryany
2891b257c2 [libFuzzer] remove stale code
llvm-svn: 352571
2019-01-29 23:53:28 +00:00
Kostya Serebryany
035c08f73d [libFuzzer] revert an accidental commit
llvm-svn: 352567
2019-01-29 23:41:50 +00:00
Kostya Serebryany
62d7270610 [libFuzzer] remove deprecated support for -fsanitize-coverage=trace-pc[-guard]
llvm-svn: 352566
2019-01-29 23:40:05 +00:00
Kostya Serebryany
50a1c69712 [libFuzzer] remove deprecated support for -fsanitize-coverage=trace-pc[-guard]
llvm-svn: 352564
2019-01-29 23:37:20 +00:00
James Y Knight
5d71fc5d7b Adjust documentation for git migration.
This fixes most references to the paths:
 llvm.org/svn/
 llvm.org/git/
 llvm.org/viewvc/
 github.com/llvm-mirror/
 github.com/llvm-project/
 reviews.llvm.org/diffusion/

to instead point to https://github.com/llvm/llvm-project.

This is *not* a trivial substitution, because additionally, all the
checkout instructions had to be migrated to instruct users on how to
use the monorepo layout, setting LLVM_ENABLE_PROJECTS instead of
checking out various projects into various subdirectories.

I've attempted to not change any scripts here, only documentation. The
scripts will have to be addressed separately.

Additionally, I've deleted one document which appeared to be outdated
and unneeded:
  lldb/docs/building-with-debug-llvm.txt

Differential Revision: https://reviews.llvm.org/D57330

llvm-svn: 352514
2019-01-29 16:37:27 +00:00
Jeremy Morse
b0d7cf5df4 Add 'REQUIRES: zlib' to a fuzzer test
Fixes tests on build environments that don't have zlib. See also r352483.

llvm-svn: 352487
2019-01-29 13:06:17 +00:00
Jeremy Morse
2601aa5406 Add zlib feature test to lit configuration
Some new tests in libfuzzer have dependencies on zlib: add a feature test
for zlib so that we can add a REQUIRES field to the relevant tests.

Patch by Matthew Voss.

Differential Revision: https://reviews.llvm.org/D57366

llvm-svn: 352483
2019-01-29 12:00:01 +00:00
Igor Kudrin
0f13000958 [compiler-rt] Fix reporting unrecognized flags in unit tests.
Previously, the warning messages might be issued within a wrong test case.

Differential Revision: https://reviews.llvm.org/D57318

llvm-svn: 352447
2019-01-29 02:31:57 +00:00
Petr Hosek
12f4b86808 Revert "[CMake] Use __libc_start_main rather than fopen when checking for C library"
This reverts commit r352341: it broke the build on macOS which doesn't
seem to provide __libc_start_main in its C library.

llvm-svn: 352411
2019-01-28 19:26:41 +00:00
Jonathan Metzman
8f7fc95ab7 [libFuzzer][Windows] Use dllexport for all declarations in FuzzerInterface.h
Summary:
Use dllexport for all declarations in FuzzerInterface.h Use it for clang
even though clang supports default visibility attribute to prevent a
warning from being thrown when LLVMFuzzerMutate is defined with dllexport.
This makes `FUZZER_INTERFACE_VISIBILITY` (FuzzerInterface.h) consistent with
`ATTRIBUTE_INTERFACE` (FuzzerDefs.h) when using clang on Windows.

Reviewers: vitalybuka, morehouse

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D57305

llvm-svn: 352395
2019-01-28 17:51:13 +00:00
Jonathan Metzman
3720e2b39e [fuzzer][afl] Remove AFL_DRIVER_EXTRA_STATS_FILENAME
Summary:
Remove this feature as it is unused, buggy, and not worth correcting
since the forkserver makes it difficult.

Reviewers: morehouse, jfb

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D57308

llvm-svn: 352392
2019-01-28 17:15:49 +00:00
Stefan Granitz
6163039ce0 [CMake] Quick-Fix FileCheck target does not exist when building against LLVM install-tree with COMPILER_RT_INCLUDE_TESTS=ON
The issue came up during release testing for LLVM 8: https://bugs.llvm.org/show_bug.cgi?id=40443

Differential Revision: https://reviews.llvm.org/D57224

llvm-svn: 352381
2019-01-28 16:14:57 +00:00
Petr Hosek
b667153cf6 [CMake] Use __libc_start_main rather than fopen when checking for C library
The check_library_exists CMake uses a custom symbol definition. This
is a problem when checking for C library symbols because Clang
recognizes many of them as builtins, and returns the
-Wbuiltin-requires-header (or -Wincompatible-library-redeclaration)
error. When building with -Werror which is the default, this causes
the check_library_exists check fail making the build think that C
library isn't available.

To avoid this issue, we should use a symbol that isn't recognized by
Clang and wouldn't cause the same issue. __libc_start_main seems like
reasonable choice that fits the bill.

Differential Revision: https://reviews.llvm.org/D57142

llvm-svn: 352341
2019-01-28 04:12:54 +00:00