Analysis and Solutions for Invalid Request Target Issues with '|' Character in Query Parameters in Tomcat 8

Problem Background and Technical Analysis

When using Apache Tomcat 8 as a web server, developers may encounter the following exception: when HTTP GET requests contain the pipe character | in query parameters, Tomcat throws java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986. The root cause of this issue lies in Tomcat's enhanced validation of characters in HTTP request lines starting from specific versions.

Version Changes and Security Enhancements

According to Apache Tomcat's official changelog, this behavioral change primarily appeared in the following versions:

• Tomcat 7.0.73
• Tomcat 8.0.39
• Tomcat 8.5.7

These versions introduced stricter request target validation mechanisms to ensure that characters in HTTP request lines comply with RFC 7231 and RFC 3986 standards. Specifically, Tomcat 8.5.3 began rejecting HTTP method names containing non-token characters, while version 8.5.7 added additional checks for valid characters in request lines, enabling earlier rejection of invalid requests.

Solution 1: Configuring the relaxedQueryChars Attribute

This is currently the recommended primary solution. By adding the relaxedQueryChars attribute to Tomcat's Connector configuration, specific special characters can be allowed in query strings.

<Connector port="8080" protocol="HTTP/1.1"
           connectionTimeout="20000"
           redirectPort="8443"
           relaxedQueryChars="|{}[]" />

This configuration example allows pipe, curly brace, and square bracket characters in query parameters. If other characters need to be allowed, simply add them to the attribute value, separated by commas.

Solution 2: Using the requestTargetAllow System Property

In versions prior to Tomcat 8.5, specific characters could be allowed by setting the system property tomcat.util.http.parser.HttpParser.requestTargetAllow. Add the following to the catalina.properties file:

tomcat.util.http.parser.HttpParser.requestTargetAllow=|{}

Note that this option has been marked as deprecated in Tomcat 8.5, and it is recommended to use relaxedQueryChars instead for new projects.

Solution 3: Client-Side URL Encoding

From a standards compliance perspective, the most ideal solution is to properly encode URLs on the client side. This ensures that all special characters are appropriately escaped, avoiding server-side parsing issues.

In JavaScript, the encodeURI() or encodeURIComponent() functions can be used:

// Encode the entire URL
encodeURI("http://localhost:8080/app/handleResponse?msg=name|id|")
// Output: http://localhost:8080/app/handleResponse?msg=name%7Cid%7C

// Encode only the query string
encodeURIComponent("msg=name|id|")
// Output: msg%3Dname%7Cid%7C

This approach not only solves the pipe character issue but also handles other potentially problematic special characters such as square brackets [], curly braces {}, etc.

Solution Comparison and Best Practices

The following table compares the characteristics of the three main solutions:

Based on the above analysis, we recommend the following best practices:

1. For new projects or situations where client code can be controlled, prioritize the client-side URL encoding solution, as it complies with HTTP standards and offers the highest security.
2. For legacy systems or situations where the client cannot be modified, use the relaxedQueryChars configuration approach to appropriately relax character restrictions in Tomcat's server.xml.
3. Avoid using the deprecated requestTargetAllow property unless compatibility with older Tomcat versions is necessary.
4. Regularly check Tomcat security updates to ensure server configurations do not introduce vulnerabilities.

Deep Understanding of RFC Standards

To fully understand this issue, knowledge of relevant RFC standards is essential:

• RFC 3986: Defines the generic syntax for URIs, categorizing characters into reserved, unreserved, and percent-encoded characters.
• RFC 7230: Defines HTTP/1.1 message syntax and routing, specifically specifying format requirements for request lines.

In these standards, the pipe character | belongs to the "sub-delimiter" category and may require encoding in certain parts of a URI. Tomcat's strict validation aims to ensure full compliance with these standards.

Practical Application Example

Consider a Java web application that needs to handle requests from external systems containing pipe-separated parameters:

// Original request
GET /api/data?params=value1|value2|value3 HTTP/1.1

// Server-side processing code
@WebServlet("/api/data")
public class DataServlet extends HttpServlet {
    protected void doGet(HttpServletRequest request, HttpServletResponse response) {
        String params = request.getParameter("params");
        // params value is "value1|value2|value3"
        String[] values = params.split("\|");
        // Further processing...
    }
}

By correctly configuring Tomcat or ensuring client-side encoding, this application can properly handle request parameters containing pipe characters.

Security Considerations

When relaxing character restrictions, security implications must be considered:

1. Only allow special characters that are genuinely needed, avoiding excessive relaxation of restrictions.
2. Regularly review the list of allowed characters and remove any that are no longer necessary.
3. Consider combining with other security measures, such as input validation and output encoding.
4. Monitor abnormal request patterns to promptly identify potential attack behaviors.

By understanding Tomcat's character validation mechanisms, mastering multiple solutions, and following best practices, developers can effectively handle HTTP requests containing special characters while ensuring the security and stability of web applications.