Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hotfix rest Chinese encoding #13617

Merged
merged 11 commits into from
Jan 19, 2024
Merged

Conversation

MajaChen
Copy link
Contributor

@MajaChen MajaChen commented Jan 6, 2024

What is the purpose of the change

When utilizing Dubbo REST with the @QueryParam annotation on parameters, issues with Chinese character decoding may arise. Refer to the 13349 issue for a comprehensive and detailed description of the problem, including its cause, solution, and an explanation of why it works.

Brief changelog

RequestFacade.initParameters will decode the request uri with UTF-8 before parsing uri to parameters.

Verifying this change

  • I add more unit tests , then can pass, and I recommend you refer the 13349 issue

Checklist

  • Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
  • Each commit in the pull request should have a meaningful subject line and body.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
  • Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
  • Add some description to dubbo-website project if you are requesting to add a feature.
  • GitHub Actions works fine on your own branch.
  • If this contribution is large, please follow the Software Donation Guide.

@codecov-commenter
Copy link

codecov-commenter commented Jan 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (e583187) 70.38% compared to head (4b17bc9) 70.37%.
Report is 17 commits behind head on 3.2.

Additional details and impacted files
@@            Coverage Diff             @@
##              3.2   #13617      +/-   ##
==========================================
- Coverage   70.38%   70.37%   -0.02%     
==========================================
  Files        1606     1606              
  Lines       69999    70036      +37     
  Branches    10098    10105       +7     
==========================================
+ Hits        49272    49290      +18     
- Misses      16103    16114      +11     
- Partials     4624     4632       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AlbumenJ
Copy link
Member

AlbumenJ commented Jan 7, 2024

@suncairong163 PTAL

@MajaChen
Copy link
Contributor Author

I'm wondering if anybody can review my code

Comment on lines 61 to 66
String decodedRequestURI = requestURI;
try {
decodedRequestURI = URLDecoder.decode(decodedRequestURI, "UTF-8");
} catch (UnsupportedEncodingException e) {
// do nothing
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is hardcoded, we can set utf-8 by default, but it must come from Accept-Charset first.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I'll make a adjustment

Copy link
Contributor Author

@MajaChen MajaChen Jan 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made an adjustment according to your opinon, please take a look again, take your time

@kimmking
Copy link
Member

As json standard format, non-ansi chars are not allowed, so is nessary to do this?

@MajaChen
Copy link
Contributor Author

Take an example of SpringMVC framework, Chinese character encoding problems are very common among Chinese developers, and it offers solution. I think dubbo should also offer such solution to avoid such problems, even though it's not common to pass Chinese character as parameter. But in my situation,I does need to do so.

@MajaChen
Copy link
Contributor Author

And many users including me, are not familiar with json standard format(sorry for that).So this bug can be a trap

@kimmking
Copy link
Member

I know your case now.
BTW: It need/maybe process q param to parse Accept-Charset field like:

Accept-Charset: <charset>

// Multiple types, weighted with the [quality value](https://developer.mozilla.org/zh-CN/docs/Glossary/Quality_values) syntax:
Accept-Charset: utf-8, iso-8859-1;q=0.5

refer:
https://developer.mozilla.org/zh-CN/docs/Web/HTTP/Headers/Accept-Charset

@MajaChen
Copy link
Contributor Author

PATL, thanks

Copy link

sonarcloud bot commented Jan 18, 2024

Quality Gate Passed Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

1 New issue
0 Security Hotspots
93.2% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

@CrazyHZM CrazyHZM added this to the 3.2.11 milestone Jan 19, 2024
@CrazyHZM CrazyHZM merged commit 7a3ad4a into apache:3.2 Jan 19, 2024
19 checks passed
Will-well pushed a commit to Will-well/dubbo that referenced this pull request Jan 31, 2024
* decode uri in RequestFacade

* supplement the unit test

* fix code style violations

* remove duplicate codes

* consider Accept-Charset as enc

* run mvn spotless apply

* add comment

* take weight into consideration

* fix code style violations according sonar

* add DEFAULT_CHARSET

---------

Co-authored-by: chenxinyuan1 <chenxinyuan1@xiaomi.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants