You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm trying to run a unit test on my spring boot project using the HtmlUnit Webclient.
The test does a POST Request submitting form data to the Controller which will create a database entry and send a redirect to the client. The response will contain a URL in the Location header which will be encoded if necessary.
Everything works fine until the HtmlUnit Webclient tries to follow the redirect and the org.springframework.security.web.firewall.StrictHttpFirewall.getFirewalledRequest(HttpServletRequest) method is called which prevents the request from being processed. It complains about the % in the URL (see stack trace).
But when running the application and using an actual browser the problem does not occur. Searching the web I found #16067 which first looked familiar.
So I wrote a small project to reproduce the issue, and I think that the problem with HtmlUnit WebClient is a little bit different.
When running the webClientTestStringWithEncoding test method and debugging into org.springframework.security.web.firewall.StrictHttpFirewall#rejectedBlocklistedUrls(HttpServletRequest), one can see that request.getServletPath() still contains the encoded path while the called method decodedUrlContains seems to expect that it has been decoded.
If you run the mockMvcTestURI test method and debug the same line you can see that request.getServletPath() is empty but request.getPathInfo() contains an decoded path. request.getRequestURI() contains the encoded path in both cases.
In case you're wondering why the first test uses the webclient and the other one the mockMvc: I didn't find a way to pass either the encoded URL or the unencoded version (with spaces and an Umlaut) to the HtmlUnit Webclient (as you can see by the other test methods)
Perhaps the problem lies in the org.springframework.test.web.servlet.htmlunit.HtmlUnitRequestBuilder.buildRequest(ServletContext) where the servletPath is set on the request and should be decoded.
I think I can work around that issue in my test but would be happy if you could have a look at this.
Thanks in advance,
Michael
java.io.IOException: org.springframework.security.web.firewall.RequestRejectedException: The request was rejected because the URL contained a potentially malicious String "%"
at org.springframework.test.web.servlet.htmlunit.MockMvcWebConnection.getResponse(MockMvcWebConnection.java:152)
at org.springframework.test.web.servlet.htmlunit.MockMvcWebConnection.getResponse(MockMvcWebConnection.java:134)
at org.springframework.test.web.servlet.htmlunit.DelegatingWebConnection.getResponse(DelegatingWebConnection.java:79)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1596)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1518)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:493)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:413)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:548)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:529)
at io.gitlab.thuri.spring.htmlunit.TestWithEncodedUriIssue.webClientTestStringWithEncoding(TestWithEncodedUriIssue.java:63)
// cut junit and eclipse stacktrace entries
Caused by: org.springframework.security.web.firewall.RequestRejectedException: The request was rejected because the URL contained a potentially malicious String "%"
at org.springframework.security.web.firewall.StrictHttpFirewall.rejectedBlocklistedUrls(StrictHttpFirewall.java:463)
at org.springframework.security.web.firewall.StrictHttpFirewall.getFirewalledRequest(StrictHttpFirewall.java:429)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:196)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:134)
at org.springframework.test.web.servlet.MockMvc.perform(MockMvc.java:199)
at org.springframework.test.web.servlet.htmlunit.MockMvcWebConnection.getResponse(MockMvcWebConnection.java:149)
... 78 more
The text was updated successfully, but these errors were encountered:
HtmlUnitRequestBuilder does not seem to support well URLs with encoded paths, including paths that are not encoded but become encoded when HtmlUnit's WebClient prepares a java.net.URL. The problem, as you pointed out, is that the servletPath is supposed to be decoded, but HtmlUnitRequestBuilder essentially uses the full path minus any contextPath, and without further decoding.
One reason this might not have failed as much is that Boot apps have alwaysUseFullPath set by default so effectively the servletPath is not used for mapping purposes, so only Spring Security's firewall is affected.
I think this can be corrected. I don't see another alternative, and hopefully it shouldn't cause issues since the servletPath is not often used for mapping purposes and in any case it should be decoded.
sbrannen
changed the title
HTMLUnit WebClient can't call encoded URL if Spring Security is activated
HtmlUnit WebClient can't call encoded URL if Spring Security is activated
Jan 7, 2022
rstoyanchev
changed the title
HtmlUnit WebClient can't call encoded URL if Spring Security is activated
StrictHttpFirewall rejects request when HtmlUnit WebClient is called with encoded URL
Jan 11, 2022
Affects: 5.3.13
I'm trying to run a unit test on my spring boot project using the HtmlUnit Webclient.
The test does a POST Request submitting form data to the Controller which will create a database entry and send a redirect to the client. The response will contain a URL in the Location header which will be encoded if necessary.
Everything works fine until the HtmlUnit Webclient tries to follow the redirect and the
org.springframework.security.web.firewall.StrictHttpFirewall.getFirewalledRequest(HttpServletRequest)
method is called which prevents the request from being processed. It complains about the % in the URL (see stack trace).But when running the application and using an actual browser the problem does not occur. Searching the web I found #16067 which first looked familiar.
So I wrote a small project to reproduce the issue, and I think that the problem with HtmlUnit WebClient is a little bit different.
When running the
webClientTestStringWithEncoding
test method and debugging intoorg.springframework.security.web.firewall.StrictHttpFirewall#rejectedBlocklistedUrls(HttpServletRequest)
, one can see thatrequest.getServletPath()
still contains the encoded path while the called methoddecodedUrlContains
seems to expect that it has been decoded.If you run the
mockMvcTestURI
test method and debug the same line you can see thatrequest.getServletPath()
is empty butrequest.getPathInfo()
contains an decoded path.request.getRequestURI()
contains the encoded path in both cases.In case you're wondering why the first test uses the webclient and the other one the mockMvc: I didn't find a way to pass either the encoded URL or the unencoded version (with spaces and an Umlaut) to the HtmlUnit Webclient (as you can see by the other test methods)
Perhaps the problem lies in the
org.springframework.test.web.servlet.htmlunit.HtmlUnitRequestBuilder.buildRequest(ServletContext)
where theservletPath
is set on the request and should be decoded.I think I can work around that issue in my test but would be happy if you could have a look at this.
Thanks in advance,
Michael
The text was updated successfully, but these errors were encountered: