Elasticsearch Log4j Vulnerability and Mitigation

Recently a dangerous zero-day exploit in the popular Java Apache Log4j library was disclosed. Hot on the heels of the first exploit, CVE-2021-44228, a second vulnerability was announced, CVE-2021-45046. Yet a third vulnerability was found, CVE-2021-45105, which allows DoS attacks even with Log4j 2.16.0. The exploits potentially enable Remote Code Execution (RCE) in software using the library and information leak is also possible. Unfortunately, Elasticsearch does use Log4j for logging. While information leak cannot expose Elasticsearch data directly, it does allow access to information readable through Log4j lookups, such as environment variables. Depending on your Elasticsearch and JDK version, you could also be susceptible to RCE.

The initial exploit could be mediated simply setting a JVM option,  log4j2.formatMsgNoLookups=true and restarting Elasticsearch, but this is not a valid mitigation for the second RCE exploit. Elastic have released patched versions of Elasticsearch 6 and 7 that remove the vulnerable class, JndiLookup altogether and adds the JVM flag.

Versions Affected

Recent Elasticsearch versions (6.8.9+, 7.8+) running JDK 9+ are not susceptible to either RCE or information leakage, due to Elasticsearch's usage of Java Security Manager, introduced in Elasticsearch 2.0. This is achieved via the Java Security Manager's strict SocketPermission limits in the security.policy file. To be on the safe side, you should still upgrade to Elasticsearch ≥ 7.16.2 or ≥ 6.8.22.  This would protect you against any future or currently unknown exploits that circumvent the protection from the Java Security Manager.

Recent Elasticsearch versions (6.8.9+, 7.8+) with older JDKs are only vulnerable to information leak, but not RCE. Consider upgrading your JDK to 9+ and upgrading to Elasticsearch ≥ 7.16.2 or ≥ 6.8.22.

Older Elasticsearch versions (5.6.11+, 6.4.0+ and 7.0.0+) are vulnerable to both RCE and information leak.

Early ES 5 (before 5.6.11) is susceptible to both information leak and RCE. Upgrade to Elasticsearch 5.6.16, add log4j2.formatMsgNoLookups=true JVM option, and potentially remove JndiLookup.class from log4j.

Elasticsearch 2.x does not seem to be affected. It uses an earlier 1.x version of Log4j that is not vulnerable.

All versions of Logstash before the latest patch (Dec 15, 2021) are affected.

See this official Elastic post for more information and updates.

Mitigations

Upgrade

The best course of action is upgrade to Elasticsearch ≥ 7.16.2 or ≥ 6.8.22 as soon as possible. Elastic has released 6.8.22 and 7.16.2 which removes the vulnerable JndiLookup class from Log4j and sets log4j2.formatMsgNoLookups=true JVM option. It also upgrades Log4j to 2.17.0 which addresses the third vulnerability found.

Firewall Rules

If you have a WAF in place, that may be configured to filter Log4j information out of HTTP requests. The HTTP body, headers, and url are all attack vectors. Major WAF vendors now support rules to mitigate Log4Shell attacks. See:

• Google Cloud Armor WAF
• Cloudflare WAF
• AWS WAF

If you're on AWS Elasticsearch, make sure that you're running it behind a VPC. If you have publicly accessible servers, consider blocking egress to all but whitelisted IPs and ports.

Scrub Environment Variables

If you're susceptible to information leak, then you should ensure that you don't store any sensitive data, like AWS keys, in environment variables. This is a best practice outside of the Log4Shell exploit, too.

Remove JndiLookup class

For those that can't upgrade their Elasticsearch clusters, there is the option of removing the JndiLookup class altogether.

You can remove the class from the log4j-core JAR file in-place with:

zip -q -d <ES_HOME>/lib/log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

And validate it was removed with:

zipinfo <ES_HOME>/lib/log4j-core-*.jar | grep -c JndiLookup.class

You should also add the log4j2.formatMsgNoLookups JVM flag. You can confirm that Elasticsearch is running with that flag:

ps -ef | grep elasticsearch | grep formatMsgNoLookups

Elasticsearch must still be restarted for this to take effect.

"Vaccine"

If Elasticsearch absolutely cannot be restarted there are so called "vaccines" like Logout4Shell which use the vulnerability itself to "patch" Java process in place. These are produced by third parties and are not guaranteed to be safe, so we would heavily discourage using those and try to adhere to official remedies whenever possible.

Please enjoy these memes as you upgrade and restart your Elasticsearch clusters with the patches: https://log4jmemes.com/.