Developer’s Guide to Fuzz Testing
Fuzz testing continues to grow in popularity among developers in the open source community. Google’s Open Source Security (OSS) Team recently reported finding over 40,000 bugs in 650 open source projects through the use of fuzz testing. But if you have no idea what fuzz testing is (still) about, don’t worry. You have come to the right place.
What is the fuzz test?
Fuzz testing is a dynamic testing method for finding bugs and security issues in software. In a fuzz test, a program or function under test is executed with invalid, unexpected, or random inputs to discover unlikely or unexpected edge cases.
Value of fuzzing for DevSecOps
As we know, DevSecOps extends the approaches of DevOps and Agile by adding appropriate security testing methods to each phase of the software development lifecycle (SDLC), creating a dynamic and continuous testing process.
Part of this testing process can and should involve fuzzing to detect security and stability issues in all phases of the SDLC and enable developers to deliver software quickly and securely. For example, to prevent regressions from going live by running automated fuzz tests on every code change.
Especially in the automotive industry and complex embedded projects, where a lot of unreliable and complicated data is processed, fuzzing brings great benefits to DevSecOps. Indeed, developers can apply fuzzing similar to unit tests to automatically test their security-critical modules for possible vulnerabilities. But with fuzzing at the integration level, they can also cover all the complicated and dangerous edge cases that may arise in the interaction between different modules.
Benefits of fuzz testing
Fuzzing code generates thousands of automated test cases in a second, with the fuzzer receiving detailed information about code coverage and program states reached to simulate the interaction of different modules while executing source code. The fuzzer can adapt and disable its inputs based on runtime feedback in subsequent iterations to maximize code coverage. Some of the benefits are:
- (Almost) no false positives: If you find a crash, fuzzing provides you with the source of the error and the malicious entry causing the problem. So you can be sure that every discovery is a real vulnerability.
- Debug reproducible results: Modern fuzz testing tools will give you the input that caused the problem, including the source of the error. With this, you already have a lot of useful information to reproduce the bug and load it directly into the debugger.
- Find problems in forgotten edge cases: Since fuzzing runs a program with invalid or random input, it also uncovers unlikely and unexpected edge-case errors that otherwise would not have been revealed.
Who uses fuzz testing?
As cybersecurity regulations and standards continue to expand globally, more organizations are expanding their cybersecurity programs and running automated security tests before shipping software. Entire industries, especially those involving advanced quality and safety regulations, are setting standards for safety testing that recommend fuzz testing, including automotive, aviation, finance, healthcare, telecommunications and energy.
Digging deeper into the automotive industry, for example, various standards from the International Organization for Standardization (ISO) and other organizations now recommend fuzzing:
The most effective way to do security testing, including fuzzing, is to do continuously. This means integrating fuzzing directly into your CI/CD pipeline. This creates fast feedback cycles so developers can fix security vulnerabilities before code is released. Integrating fuzzing tools into code hosting systems like GitHub is also valuable. This allows communication and alerts to be shared across your DevOps team as soon as an issue is discovered, making it easier to fix vulnerabilities.
What bugs can I find?
Although the practice is still relatively new, fuzzing has already discovered thousands of bugs in different use cases. Types of bugs and vulnerabilities discovered include:
- Remote Code Executions
- Injections, which can be triggered by untrusted inputs
- Memory leaks: incorrect memory allocation
- Exposure of sensitive data: Accidental exposure of personal data
- Functional bugs: entries not answered
and many other bugs, such as undefined behaviors, uncaught exceptions, and buffer overflow. Learn more.
You can try an open source fuzzer, like Jazzer (for Java tests) or the CI Fuzz CLI (for C/C++ testing). When you are comfortable with fuzzing and want to use this approach in more complex environments, enterprise solutions offer additional features such as API fuzzing, reporting, and CI/CD integration.
So take your chance and join the community of open source developers using fuzzing to quickly and securely ship their software.