Log4J Vulnerability: The Latest Information Your Software Developers Need To Know About Log4Shell

Happy Holidays!

If you’re responsible for application security (AppSec) in your organization—and especially if your applications are written in Java—you received an early Christmas present this past weekend in the form of a vulnerability for Log4j. Also known as Log4Shell, this exploit is being described as possibly the worst software vulnerability since Heartbleed.

Ok, so maybe that’s not really a present – maybe we’re actually getting coal in our Christmas stockings this year.

Log4Shell Vulnerability Overview

Over the past few days, the industry continues to learn more about the severity and potential blast radius of the Log4Shell vulnerability. According to SecurityBoulevard, log4j ranks as one of the most popular components out there, with 28.6 million downloads to date. Needless to say, the potential impact of this vulnerability is massive!

Given our experience developing runtime alerts for our cloud native application security platform, Deepfactor, we’ve spent the last few days carefully examining PoC exploits and related research, hoping to better understand this exploit and what we can do to help our customers triage and mitigate vulnerable systems.

Once a bad actor identifies a vulnerable system (check out this honeypot, for example), Log4Shell is exploited by tricking the JVM running your Java app to download, and subsequently execute a class file (or serialized class bytecode) from malicious LDAP and other JNDI related endpoints. In security parlance, this security flaw is known as Remote Code Execution (RCE).

Of course, that code could be anything the attacker wants to run. For example, the first payloads that we saw in the wild were coin miners, followed by several examples of payloads that were trying to ascertain secret keys, such as AWS Secret Keys stored in the JVM’s environment. Now we’re seeing more complex payloads, and it’s our theory that this might become a ransomware and/or botnet hook in short order.

The Swiss government (via GovCERT.ch) created the following diagram to explain how easily things can go sideways behind the scenes:

In addition to illustrating the exploit, the Swiss government provided several recommendations for triaging and managing particular areas of the vulnerability chain. From disabling remote codebases to upgrading, patching or even disabling log4j, no solution is [currently] perfect. And while some solutions shown above may be “doable” in your environment, they may require replacing hundreds or thousands of lines of code (like disabling log4j entirely or replacing it with another logging framework).

What About WAFs as a solution?

Over the past few days, many companies have recommended the use of a Web Application Firewall (WAF) to scan for offending log strings at the application’s perimeter … However, many of these suggestions are missing an important question – what exactly should the WAF prevent?

Well, you might be thinking, “Why not block ${jndi:ldap?” Unfortunately, this was worked around pretty quickly by PoC exploit kits. As a matter of fact, tools were quickly developed specifically to get around the WAF at the perimeter by hiding or obfuscating the poison log string. For example, consider the log4j-payload generator from woodpecker-appstore/log4j-payload-generator:

The “raw payload” (at the top of the image above) is the key string an attacker wants to actually send to log4j, but “jndi:ldap” might be blocked by a WAF. But can your WAF decode the obfuscated versions shown later in the image? Maybe, but it’s just going to need a minor permutation by an attacker to change those logged strings to something else your WAF might not see.

Could Deepfactor Have Detected That I Was Vulnerable Before This Was Announced?

One question that we’ve been asked over the past 48 hours or so: “Could Deepfactor have detected that I was vulnerable before this was announced?” If you aren’t familiar with Deepfactor, we’re a cloud native application security platform that enables developers to quickly discover and resolve security vulnerabilities, supply chain risks, and compliance issues during development.

The short answer is “yes, but not in all cases.” The longer answer is a bit more interesting. While really no AppSec vendor knew about this before about December 1 (that’s when CDNs started seeing fingerprints of this vulnerability being probed), what Deepfactor could tell you, that relates to this issue, are the following behavioral fingerprints:

• Your application is suddenly making outbound DNS queries it didn’t do before (this is done to find the IP of the attacker’s LDAP server, provided as part of the poisoned log string)
• Your application is suddenly making LDAP connections (which, for most internet-facing applications, should be a giant red flag (since when does your Tomcat server need to make LDAP queries?)

As a software developer if I said to you that your application is suddenly doing these two things it didn’t do before, I would hope that you would go investigate why. These are the types of insights that Deepfactor provides.

Another interesting insight we’d show would help out in that payload scenario I talked about above—storing secrets in environment variables. There’s almost always a better way to store sensitive information (vaults, etc) than in environment variables, and this is why we tell you about it. Even if you *weren’t* using a vulnerable version of log4j, you still shouldn’t rely on environment variables being secret.

Why does that even matter though? Why would an attacker be interested in AWS secret keys they found in a Java VM they hijacked?

If I have your AWS secret key/access token, I can launch VMs, access your S3 buckets, glacier storage, RDS databases, etc. Think of the wealth of information I can exfiltrate (or the number of coin miner VMs I can start that you’ll have to pay for).

We’ve established that Deepfactor can tell you about the behavioral changes exhibited by your application if someone were to try to exploit this vulnerability while your Java app was being monitored by our runtime observability platform. The next question that you might logically ask is “Can Deepfactor tell me if I’m using a vulnerable version of log4j in the first place?”

This might seem like a simple answer since Deepfactor incorporates a software composition (SCA) module for this very purpose. Our SCA module (with input from the runtime observability part of the product) produces an SBOM which should be able to tell you if you’re using a vulnerable version of log4j. And, in the majority of cases, that’s true; we can tell you if you imported (directly or indirectly) a version that has this vulnerability. But I’d like to point out that since log4j is just an open source package that could be (in theory) self-built by some developer, it’s not a 100% “slam-dunk” guarantee that any SCA tool can give you a definite yes or no answer to that question. The good news here is that it’s pretty unlikely someone is making their own small batch artisanal log4j builds and including those in a dependency that you imported, but you should be aware that this is a possibility (and this really applies to all SCA tools, not just Deepfactor).

Trust, but Verify Your Information on Log4Shell—There’s Some Misinformation Out There

This is a vulnerability that is going to evolve substantially over the next days/weeks/months. There have already been several misleading pieces of advice given by various people/companies:

• If you use a “new” version of Java, you’re immune. Nope: As per Márcio Almeida on Twitter:

“Just added support to LDAP Serialized Payloads in the JNDI-Exploit-Kit. This attack path works in *ANY* java version as long the classes used in the Serialized payload are in the application classpath. Do not rely on your java version being up-to-date and update your log4j ASAP!”

This was proven false about 3 days after the vulnerability was announced. But there are still blog posts, Twitter posts, and ambulance-chasing emails that make this claim.

• If you use v2.15 of log4j, you’re immune. Nope: Check out CVE-2021-45046. The statement about v2.15 being sufficient to avoid the vulnerability was reported in one of the initial CVEs, but it didn’t consider all scenarios that could ultimately lead to the same exploit.
  (Note: v2.16 looks good so far, but stay tuned for Part 2 of this blog where we tell you to install v2.17 </sarcasm>)
• If you use an ancient version of log4j, you’re immune (like, a version from years and years ago). Ok, while this one might be technically true, many of these old versions have so many other vulnerabilities this really isn’t good advice at all.

The takeaway here is “trust, but verify” what you see out there. This is going to be an extended, rough ride for everyone impacted, and there’s still much to learn about this vulnerability.

What’s Next?

While writing this blog post, I set up a sample PoC environment for us to use at Deepfactor. I used this PoC environment to understand how Deepfactor would have performed had one of our customers been susceptible to one of these attacks. If you’re interested to see exactly what Deepfactor reported, follow Deepfactor on Twitter or LinkedIn for updates.

I hope everyone has a fun and safe holiday season … those of us in the AppSec community will be over here shoveling coal until next Christmas!