diff options
Diffstat (limited to 'tools/dexfuzz/README')
-rw-r--r-- | tools/dexfuzz/README | 130 |
1 files changed, 130 insertions, 0 deletions
diff --git a/tools/dexfuzz/README b/tools/dexfuzz/README new file mode 100644 index 0000000..c4795f2 --- /dev/null +++ b/tools/dexfuzz/README @@ -0,0 +1,130 @@ +DexFuzz +======= + +DexFuzz is primarily a tool for fuzzing DEX files. Fuzzing is the introduction of +subtle changes ("mutations") to a file to produce a new test case. These test cases +can be used to test the various modes of execution available to ART (Interpreter, +Quick compiler, Optimizing compiler) to check for bugs in these modes of execution. +This is done by differential testing - each test file is executed with each mode of +execution, and any differences between the resulting outputs may be an indication of +a bug in one of the modes. + +For a wider overview of DexFuzz, see: + +http://community.arm.com/groups/android-community/blog/2014/11/26/the-art-of-fuzz-testing + +In typical operation, you provide DexFuzz with a set of DEX files that are the "seeds" +for mutation - e.g. some tests taken from the ART test suite - and point it at an +ADB-connected Android device, and it will fuzz these seed files, and execute the +resulting new tests on the Android device. + +How to run DexFuzz +================== + +1. Build dexfuzz with mmm tools/dexfuzz from within art/. +2. Make sure you have an Android device connected via ADB, that is capable of + having DEX files pushed to it and executed with the dalvikvm command. +3. Make sure you're in the Android build environment! + (That is, . build/envsetup.sh && lunch) +4. Create a new directory, and place some DEX files in here. These are the seed files + that are mutated to form new tests. +5. Create a directory on your device that mutated test files can be pushed to and + executed in, using dalvikvm. For example, /data/art-test/ +6. If you currently have multiple devices connected via ADB, find out the name of + your device using "adb devices -l". +7. Run this command: + +dexfuzz --inputs=<seeds dir> --execute --repeat=<attempts> \ + --dump-output <combination of ISA(s) and and backend(s)> + +You MUST specify one of the following ISAs: + --arm + --arm64 + --x86 + --x86_64 + --mips + --mips64 + +And also at least two of the following backends: + --interpreter + --quick + --optimizing + +Note that if you wanted to test both ARM and ARM64 on an ARM64 device, you can use +--allarm. Also in this case only one backend is needed, if i.e., you wanted to test +ARM Quick Backend vs. ARM64 Quick Backend. + +Some legal examples: + --arm --quick --optimizing + --x86 --quick --optimizing --interpreter + --allarm --quick + +Add in --device=<device name, e.g. device:generic> if you want to specify a device. +Add in --execute-dir=<dir on device> if you want to specify an execution directory. + (The default is /data/art-test/) + +As the fuzzer works, you'll see output like: + +|-----------------------------------------------------------------| +|Iterations|VerifyFail|MutateFail|Timed Out |Successful|Divergence| +|-----------------------------------------------------------------| +| 48 | 37 | 4 | 0 | 6 | 1 | + +Iterations - number of attempts we've made to mutate DEX files. +VerifyFail - the number of mutated files that ended up failing to verify, either + on the host, or the target. +MutateFail - because mutation is a random process, and has attempt thresholds to + avoid attempting to mutate a file indefinitely, it is possible that + an attempt to mutate a file doesn't actually mutate it. This counts + those occurrences. +Timed Out - mutated files that timed out for one or more backends. + Current timeouts are: + Quick - 5 seconds + Optimizing - 5 seconds + Intepreter - 30 seconds + (use --short-timeouts to set all backends to 2 seconds.) +Successful - mutated files that executed and all backends agreed on the resulting + output. NB: if all backends crashed with the same output, this would + be considered a success - proper detection of crashes is still to come. +Divergence - mutated files that executed and some backend disagreed about the + resulting output. Divergent programs are run multiple times with a + single backend, to check if they diverge from themselves, and these are + not included in the count. If multiple architectures are being used + (ARM/ARM64), and the divergences align with different architectures, + these are also not included in the count. + +8. Check report.log for the full report, including input file and RNG seed for each + test program. This allows you to recreate a bad program with, e.g.: + +dexfuzz --input=<input file> --seed=<seed value> + +Check dexfuzz --help for the full list of options. + +NOTE: DEX files with unicode strings are not fully supported yet, and DEX files with +JNI elements are not supported at all currently. + +Mutation Likelihoods +==================== + +Each bytecode mutation has a chance out of 100% of firing. Following is the listing +of each mutation's probability. If you wish to easily adjust these values, copy +these values into a file called likelihoods.txt, and run dexfuzz with +--likelihoods=likelihoods.txt. + +ArithOpChanger 75 +BranchShifter 30 +CmpBiasChanger 30 +ConstantValueChanger 70 +ConversionRepeater 50 +FieldFlagChanger 40 +InstructionDeleter 40 +InstructionDuplicator 80 +InstructionSwapper 80 +NewMethodCaller 10 +NonsenseStringPrinter 10 +PoolIndexChanger 30 +RandomInstructionGenerator 30 +SwitchBranchShifter 30 +TryBlockShifter 40 +ValuePrinter 40 +VRegChanger 60 |