Note that the Fuzzer contains the main() function.
Preferably do *not* use sanitizers while building the Fuzzer.
* Build the library you are going to test with
- `-fsanitize-coverage={bb,edge}[,indirect-calls]`
+ `-fsanitize-coverage={bb,edge}[,indirect-calls,8bit-counters]`
and one of the sanitizers. We recommend to build the library in several
different modes (e.g. asan, msan, lsan, ubsan, etc) and even using different
optimizations options (e.g. -O0, -O1, -O2) to diversify testing.
fuzzer (a directory with test inputs, one file per input).
The better your inputs are the faster you will find something interesting.
Also try to keep your inputs small, otherwise the Fuzzer will run too slow.
+ By default, the Fuzzer limits the size of every input to 64 bytes
+ (use ``-max_len=N`` to override).
* Run the fuzzer with the test corpus. As new interesting test cases are
discovered they will be added to the corpus. If a bug is discovered by
the sanitizer (asan, etc) it will be reported as usual and the reproducer
will be written to disk.
Each Fuzzer process is single-threaded (unless the library starts its own
threads). You can run the Fuzzer on the same corpus in multiple processes
- in parallel. For run-time options run the Fuzzer binary with '-help=1'.
+ in parallel.
The Fuzzer is similar in concept to AFL_,
but the Fuzzer itself does not (and should not) depend on any
part of LLVM and can be used for other projects w/o requiring the rest of LLVM.
+Flags
+=====
+The most important flags are::
+
+ seed 0 Random seed. If 0, seed is generated.
+ runs -1 Number of individual test runs (-1 for infinite runs).
+ max_len 64 Maximum length of the test input.
+ cross_over 1 If 1, cross over inputs.
+ mutate_depth 5 Apply this number of consecutive mutations to each input.
+ timeout 1200 Timeout in seconds (if positive). If one unit runs more than this number of seconds the process will abort.
+ help 0 Print help.
+ save_minimized_corpus 0 If 1, the minimized corpus is saved into the first input directory
+ jobs 0 Number of jobs to run. If jobs >= 1 we spawn this number of jobs in separate worker processes with stdout/stderr redirected to fuzz-JOB.log.
+ workers 0 Number of simultaneous worker processes to run the jobs. If zero, "min(jobs,NumberOfCpuCores()/2)" is used.
+ sync_command 0 Execute an external command "<sync_command> <test_corpus>" to synchronize the test corpus.
+ sync_timeout 600 Minimum timeout between syncs.
+ use_traces 0 Experimental: use instruction traces
+ only_ascii 0 If 1, generate only ASCII (isprint+isspace) inputs.
+
+
+For the full list of flags run the fuzzer binary with ``-help=1``.
+
Usage examples
==============
(cd pcre; ./autogen.sh; CC="clang -fsanitize=address $COV_FLAGS" ./configure --prefix=`pwd`/../inst && make -j && make install)
# Build lib/Fuzzer files.
clang -c -g -O2 -std=c++11 Fuzzer/*.cpp -IFuzzer
- # Build the the actual function that does something interesting with PCRE2.
+ # Build the actual function that does something interesting with PCRE2.
cat << EOF > pcre_fuzzer.cc
#include <string.h>
#include "pcre2posix.h"
Advanced features
=================
-Tokens
-------
+Data-flow-guided fuzzing
+------------------------
-By default, the fuzzer is not aware of complexities of the input language
-and when fuzzing e.g. a C++ parser it will mostly stress the lexer.
-It is very hard for the fuzzer to come up with something like ``reinterpret_cast<int>``
-from a test corpus that doesn't have it.
-See a detailed discussion of this topic at
-http://lcamtuf.blogspot.com/2015/01/afl-fuzz-making-up-grammar-with.html.
+*EXPERIMENTAL*.
+With an additional compiler flag ``-fsanitize-coverage=trace-cmp`` (see SanitizerCoverageTraceDataFlow_)
+and extra run-time flag ``-use_traces=1`` the fuzzer will try to apply *data-flow-guided fuzzing*.
+That is, the fuzzer will record the inputs to comparison instructions, switch statements,
+and several libc functions (``memcmp``, ``strcmp``, ``strncmp``, etc).
+It will later use those recorded inputs during mutations.
-lib/Fuzzer implements a simple technique that allows to fuzz input languages with
-long tokens. All you need is to prepare a text file containing up to 253 tokens, one token per line,
-and pass it to the fuzzer as ``-tokens=TOKENS_FILE.txt``.
-Three implicit tokens are added: ``" "``, ``"\t"``, and ``"\n"``.
-The fuzzer itself will still be mutating a string of bytes
-but before passing this input to the target library it will replace every byte ``b`` with the ``b``-th token.
-If there are less than ``b`` tokens, a space will be added instead.
+This mode can be combined with DataFlowSanitizer_ to achieve better sensitivity.
AFL compatibility
-----------------
Then you can subtract the set of covered PCs from the set of all instrumented PCs in the binary,
see SanitizerCoverage_ for details.
+User-supplied mutators
+----------------------
+
+LibFuzzer allows to use custom (user-supplied) mutators,
+see FuzzerInterface.h_
+
Fuzzing components of LLVM
==========================
Optionally build other kinds of binaries (asan+Debug, msan, ubsan, etc).
-TODO: commit the pre-fuzzed corpus to svn (?).
-
Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=23052
clang-fuzzer
------------
-The default behavior is very similar to ``clang-format-fuzzer``.
-Clang can also be fuzzed with Tokens_ using ``-tokens=$LLVM/lib/Fuzzer/cxx_fuzzer_tokens.txt`` option.
+The behavior is very similar to ``clang-format-fuzzer``.
Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=23057
+llvm-as-fuzzer
+--------------
+
+Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=24639
+
Buildbot
--------
git clone https://github.com/kcc/fuzzing-with-sanitizers.git
bin/clang-format-fuzzer fuzzing-with-sanitizers/llvm/clang-format/C1
bin/clang-fuzzer fuzzing-with-sanitizers/llvm/clang/C1/
- bin/clang-fuzzer fuzzing-with-sanitizers/llvm/clang/TOK1 -tokens=$LLVM/llvm/lib/Fuzzer/cxx_fuzzer_tokens.txt
+ bin/llvm-as-fuzzer fuzzing-with-sanitizers/llvm/llvm-as/C1 -only_ascii=1
FAQ
to crash on invalid inputs.
Examples: regular expression matchers, text or binary format parsers.
+Trophies
+========
+* GLIBC: https://sourceware.org/glibc/wiki/FuzzingLibc
+
+* MUSL LIBC:
+
+ * http://git.musl-libc.org/cgit/musl/commit/?id=39dfd58417ef642307d90306e1c7e50aaec5a35c
+ * http://www.openwall.com/lists/oss-security/2015/03/30/3
+
+* pugixml: https://github.com/zeux/pugixml/issues/39
+
+* PCRE: Search for "LLVM fuzzer" in http://vcs.pcre.org/pcre2/code/trunk/ChangeLog?view=markup
+
+* ICU: http://bugs.icu-project.org/trac/ticket/11838
+
+* LLVM:
+
+ * Clang: https://llvm.org/bugs/show_bug.cgi?id=23057
+
+ * Clang-format: https://llvm.org/bugs/show_bug.cgi?id=23052
+
+ * libc++: https://llvm.org/bugs/show_bug.cgi?id=24411
+
+ * llvm-as: https://llvm.org/bugs/show_bug.cgi?id=24639
+
+
+
.. _pcre2: http://www.pcre.org/
.. _AFL: http://lcamtuf.coredump.cx/afl/
.. _SanitizerCoverage: http://clang.llvm.org/docs/SanitizerCoverage.html
+.. _SanitizerCoverageTraceDataFlow: http://clang.llvm.org/docs/SanitizerCoverage.html#tracing-data-flow
+.. _DataFlowSanitizer: http://clang.llvm.org/docs/DataFlowSanitizer.html
.. _Heartbleed: http://en.wikipedia.org/wiki/Heartbleed
+
+.. _FuzzerInterface.h: https://github.com/llvm-mirror/llvm/blob/master/lib/Fuzzer/FuzzerInterface.h
projects / oota-llvm.git / blobdiff