Practical Jazzer for the Snazzy Fuzzer | Security Lab
Author: Sam Shahsavar
Description: We will review some lessons learned and tips for effectively fuzzing your applications with Jazzer.
Date: 2024-07-08T00:00:00-07:00

# Practical Jazzer for the Snazzy Fuzzer

Fuzzing is an automated methodology that attempts to identify bugs within software, especially security vulnerabilities. We on the Security Research Team at ServiceNow utilize the fuzzer Jazzer internally for specific use cases.

> Jazzer is a coverage-guided, in-process fuzzer for the JVM platform developed by Code Intelligence. It is based on libFuzzer and brings many of its instrumentation-powered mutation features to the JVM.1

While Code Intelligence no longer maintains Jazzer, the fuzzer remains functional and available for use from GitHub.

When it comes to large-scale enterprise Java applications, throwing a fuzzer at some code is unlikely to identify interesting vulnerabilities. Instead, you need to think about application state and what sort of state indicates a potential security issue. After that, it’s a matter of wrangling a fuzzer to effectively identify that state.

Unfortunately, like many technical tools, the documentation for Jazzer is a little lighter than we’d like, and examples across the Internet leave a bit to be desired complexity-wise. In this post, we will share and compile some lessons learned and advanced tips for effectively making use of Jazzer with an enterprise Java application.

## Extra Fuzz Target Methods

If you stick to the documentation on GitHub or official blogs from Code Insight, you might easily overlook some helpful methods available to you when creating fuzz targets.

The optional `fuzzerInitialize` method runs before iterating through calls of `fuzzerTestOneInput`. This initialization is especially helpful when fuzzing larger applications. You might need to prepare some of the normal application, but you don’t need to actually run the entire thing. Early initialization steps might also avoid unnecessary overhead on each call to `fuzzerTestOneInput`. A basic example is included below.

    public class ExampleApp{

      public static void fuzzerInitialize() {
        System.setProperty("test.property.1", "false");
        System.setProperty("test.db.name", "db_master");
        TestProperties.loadSystemProperties();
        TestProperties.reload();

        new TestEnvironment();
      }

      public static void fuzzerTestOneInput(FuzzedDataProvider data) {
        String c = data.consumeString(100);

        //Do something....
      }
    }

In this example, we set some system properties, load those properties within the application context, and then prepare a mock application state with `new TestEnvironment()`. All of this will wildly differ depending on the minimum components your application require to run for fuzzing.

In a similar vein, optional method `fuzzerTearDown` can be used to clean up resources after the completion of a fuzzing run.

## Method Hooks & Classpath

With a memory safe language like Java, you’re unlikely to be interested in fuzzing for memory corruption vulnerabilities. Additionally, an enterprise Java application is less likely to be as affected by denial of service conditions. So you’re generally going to be most interested in fuzzing for particular application states that indicate specific types of vulnerabilities only relevant in the context of your application.

Code Intelligence provides a decent resource on using method hooks to detect these sort of application states. Still, the devil is in the details.

Method hooks themselves must be built and provided in a separate JAR file from the fuzzing target. It’s an easy to miss detail, but it prevents all sorts of access and classpath issues down the road. So a basic run example might look like this:

    java --cp=jazzer_standalone.jar:target.jar:hook.jar \
    com.code_intelligence.jazzer.Jazzer \
    --target_class=sn.securityresearch.fuzzing.target.ExampleApp \
    --custom_hooks=sn.securityresearch.fuzzing.hooks.ExampleHook

Second, during runtime, method hooks themselves appear to be limited in scope to system level JARs. This occurs whether or not the method hook itself was compiled with other JARs in its classpath and those same JARs are available on the classpath provided to Jazzer.

So, when you’re attempting to peek at your application state from a method hook, you might find yourself limited in the classes you would normally use to do so within the application itself. What’s the alternative then? Reflection!

In a slightly contrived example, imagine we have an application that allows users to run certain types of script. Low privilege users get access to a restricted script environment, normally the `UserContext` class, which contains a limited set of APIs. Admin level users get access to the entire script environment, normally the `AdminContext` class, which contains the full set of APIs. We’ve created a before-type method hook at the actual lower level script API invoker,`APIRunner.handleInvocation` to attempt to detect scenarios where certain script has escaped the low privilege context `UserContext` and entered the privileged `AdminContext`.

The normal method for `APIRunner.handleInvocation` looks like the following.

    public class APIRunner {

        protected static Object handleInvocation(ScriptAPI api, Runtime rt, Context cx) {
            // Invoke API...
        }
    }

Within the application itself, it would be trivial to poke at the classes `Runtime` and `Context` and their respective fields to identify an escalation from a restricted to privileged environment. We simply could grab the type of `cx` and then compare with the `callContext` field from `rt`. Instead, since our method hook is unaware of `Runtime` and `Context`, we must use reflection to tease out the same information, resulting in a slightly less readable method hook like below.

    public class ExampleHook {

        @MethodHook(
                type = HookType.BEFORE,
                targetClassName = "com.securityresearch.script.APIRunner",
                targetMethod = "handleInvocation"
        )
        public static void checkContextEscape(MethodHandle method, Object thisObject, Object[] arguments, int hookId) {
            detectContextEscape(arguments);
        }

        private static void detectContextEscape(Object[] arguments) {

            String currentContext = null;
            String callContext = null;

            try {
                // Current Context
                currentContext = arguments[2].getClass().getSimpleName();

                // Calling Context
                Class c = arguments[1].getClass();
                Field f = c.getDeclaredField("callContext");
                f.setAccessible(true);
                callContext = f.get(arguments[1]).getClass().getSimpleName();

            } catch (Exception ignore) {}

            if (currentContext.equals("AdminContext") && callContext.equals("UserContext")) {
                Jazzer.reportFindingFromHook(new FuzzerSecurityIssueCritical("Possible Context Escape"));
            }
        }
    }

The `@MethodHook` annotation tells Jazzer the class and method to target and first execute the custom method `checkContextEscape`. Since we do not directly have access to the `Context` object, we use reflection, `arguments[2].getClass().getSimpleName()`, to identify the class of the `cx` argument. We similarly parse out the call context by inspecting the `callContext` field of the `Runtime` argument `rt`. We then compare the current and call context values we identified through reflection to see if a privilege escalation occurs. `Jazzer.reportFindingFromHook` is used to report the state as a finding.

## Timeouts

LibFuzzer itself reports timeouts, stored on disk alongside findings as “timeout-<sha1>”. Timeouts are defined with a default value of 1200 seconds (20 minutes), though this can be tweaked when calling Jazzer and passing `-timeout=120` to libFuzzer. Unfortunately, timeouts are terminal and may kill a fuzzing run if encountered. This may be desirable for some folks, but we’ve often found the need to gracefully handle timeouts ourselves in the context of a larger applications and extended fuzzing runs.

There are a couple options in this scenario. LibFuzzer has a few experimental settings you may configure when calling Jazzer. In particular, fork mode allows you to spin up parallel processes for fuzzing, with the side effect of crash resistance. By default, timeouts are ignored, but this can be configured with `-ignore_timeouts` flag. So a run might look like this:

    java --cp=jazzer_standalone.jar:target.jar:hook.jar \
    com.code_intelligence.jazzer.Jazzer \
    -fork=4 \
    --target_class=sn.securityresearch.fuzzing.target.ExampleApp \
    --custom_hooks=sn.securityresearch.fuzzing.hooks.ExampleHook

Alternatively, it’s possible to detect and handle timeouts yourself within the context of the fuzz target. This is tricky and not typically going to be necessary outside niche use cases. We can utilize the standard Java `Future` API to detect and throw our own Jazzer security findings for timeouts. In the example below, our `fuzzerTestOneInput` method merely serves as a wrapper to pass off our fuzzer input to a separate worker, `ExampleApplicationWorker`.

    public class ExampleApp{

        public static void fuzzerTestOneInput(FuzzedDataProvider data) {
            String c = data.consumeString(100);

            ExecutorService executor = Executors.newSingleThreadExecutor();
            Future future = executor.submit(new ExampleApplicationWorker(c));

            try {
                future.get(30, TimeUnit.SECONDS);
            } catch (TimeoutException e) {
                future.cancel(true);
                throw new FuzzerSecurityIssueCritical("Potential Timeout");
            } catch (Exception ignore) {
            } finally {
                executor.shutdownNow();
            }
        }
    }

The standard `fuzzerTestOneInput` method takes in a string from the fuzzer itself, and passes it to a separate thread, `ExampleApplicationWorker`. The worker thread is monitored to ensure it finishes in 30 seconds, or the thread is terminated and a finding thrown.

Within `ExampleApplicationWorker`, we place the functionality that resides in a typical fuzz target.

    public class ExampleApplicationWorker implements Runnable{

        private String input;

        public ExampleApplicationWorker(String input) {
            this.input = input;
        }

        @Override
        public void run() {
            // Do Something....
        }
    }

Jazzer continues to functional normally, method hooks included, and we can handle timeouts how we wish, though that might mean more logging or cleaning up application resources in true timeout scenarios.

## Advanced Run Options

Both Jazzer itself and libFuzzer contain even more options useful for customizing your fuzzing runs. We’ve included a few we found particularly useful.

### Disable Existing Hooks

As we mentioned earlier, fuzzing becomes particularly useful in the context of your own application and detecting vulnerable states, which means working with your own custom method hooks. Jazzer, by default, comes with several existing method hooks it will run with by default. You may find these useful; you may not. If you’re looking to optimize your fuzzing run and hone in on your own findings, you can disable these existing method hooks with the `--disabled_hooks` flag. In particular, we found the `RegexInjection` and `SqlInjection` method hooks to be troublesome for our use cases and application, resulting in false positives or unnecessarily consuming fuzzing resources.

    java --cp=jazzer_standalone.jar:target.jar:hook.jar \
    com.code_intelligence.jazzer.Jazzer \
    --target_class=sn.securityresearch.fuzzing.target.ExampleApp \
    --custom_hooks=sn.securityresearch.fuzzing.hooks.ExampleHook \
    --disabled_hooks=com.code_intelligence.jazzer.sanitizers.RegexInjection:com.code_intelligence.jazzer.sanitizers.SqlInjection

You can see some of the other method hooks here.

### Instrumentation Includes

To hone in on your own application, or even better, specific focus areas of your own application, you may find it useful to limit instrumentation using the `--instrumentation_includes` flag. If you have a large enterprise Java application, you really don’t want or need Jazzer instrumenting your entire codebase, including potentially hundreds of dependencies.

    java --cp=jazzer_standalone.jar:target.jar:hook.jar \
    com.code_intelligence.jazzer.Jazzer \
    --target_class=sn.securityresearch.fuzzing.target.ExampleApp \
    --custom_hooks=sn.securityresearch.fuzzing.hooks.ExampleHook \
    --disabled_hooks=com.code_intelligence.jazzer.sanitizers.RegexInjection:com.code_intelligence.jazzer.sanitizers.SqlInjection \
    --instrumentation_includes=sn.securityresearch.**:com.snc.**

### Keep Going

If you’re going through the work of creating a fairly complicated fuzzing target and method hook, you’re probably not going to want to stop fuzzing runs after a single crash/finding. We’d recommend running Jazzer with the `--keep_going` flag. Jazzer is more than capable enough to not report duplicate crashes, assuming the stack traces match.

    java --cp=jazzer_standalone.jar:target.jar:hook.jar \
    com.code_intelligence.jazzer.Jazzer \
    --target_class=sn.securityresearch.fuzzing.target.ExampleApp \
    --custom_hooks=sn.securityresearch.fuzzing.hooks.ExampleHook \
    --disabled_hooks=com.code_intelligence.jazzer.sanitizers.RegexInjection:com.code_intelligence.jazzer.sanitizers.SqlInjection \
    --instrumentation_includes=sn.securityresearch.**:com.snc.** \
    --keep_going=10

## Final Thoughts

We hope this post will save at least a few folks some time and improve their fuzzing performance with Jazzer going forward. If you take anything from this post, it’s that running out of the box Jazzer with the default configurations may not be the best to fuzz sizable Java applications for meaningful vulnerabilities. However, by considering the specific application state that indicates a potential vulnerability and using the techniques outlined here, you can use Jazzer to more effectively identify actionable security issues.

1. https://github.com/CodeIntelligenceTesting/jazzer ↩

Enter your search term...