Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport HTTPRequest gzip decompression #48581

Closed
wants to merge 84 commits into from
Closed

Conversation

tavurth
Copy link
Contributor

@tavurth tavurth commented May 9, 2021

Sorry if creating incomplete PRs is not allowed! I can remove this if needed.

I was trying to handle the backport of this PR.

Most of the work seems to be done, however I've run into a few issues, and I'm not familiar enough with the Godot source to understand what's happening here.

I'm currently running into the following errors:

[ 78%] scene/main/http_request.cpp:352:70: error: no viable conversion from 'Vector<uint8_t>' (aka 'Vector<unsigned char>') to 'const PoolVector<uint8_t>' (aka 'const PoolVector<unsigned char>')
                                Error err = client->request_raw(method, request_string, headers, request_data);
                                                                                                 ^~~~~~~~~~~~
./core/pool_vector.h:466:2: note: candidate constructor not viable: no known conversion from 'Vector<uint8_t>' (aka 'Vector<unsigned char>') to 'const PoolVector<unsigned char> &' for 1st argument
        PoolVector(const PoolVector &p_pool_vector) {
        ^
./core/io/http_client.h:205:118: note: passing argument to parameter 'p_body' here
        Error request_raw(Method p_method, const String &p_url, const Vector<String> &p_headers, const PoolVector<uint8_t> &p_body);
                                                                                                                            ^
[ 78%] scene/main/http_request.cpp:476:86: error: no member named 'ptr' in 'PoolVector<unsigned char>'
                int result = Compression::decompress_dynamic(decompressed, body_size_limit, p_data.ptr(), p_data.size(), mode);
                                                                                            ~~~~~~ ^
[ 79%] 2 errors generated.
[ 80%] scons: *** [scene/main/http_request.osx.tools.x86_64.o] Error 1
core/io/compression.cpp:222:7: error: member reference base type 'uint8_t' (aka 'unsigned char') is not a structure or union
        p_dst->resize(0);
        ~~~~~^ ~~~~~~
core/io/compression.cpp:228:8: error: member reference base type 'uint8_t' (aka 'unsigned char') is not a structure or union
                p_dst->resize(p_dst->size() + gzip_chunk);
                ~~~~~^ ~~~~~~
core/io/compression.cpp:228:22: error: member reference base type 'uint8_t' (aka 'unsigned char') is not a structure or union
                p_dst->resize(p_dst->size() + gzip_chunk);
                              ~~~~~^ ~~~~
core/io/compression.cpp:230:14: error: member reference base type 'uint8_t' (aka 'unsigned char') is not a structure or union
                dst = p_dst->ptrw();
                      ~~~~~^ ~~~~
core/io/compression.cpp:251:11: error: member reference base type 'uint8_t' (aka 'unsigned char') is not a structure or union
                                        p_dst->resize(0);
                                        ~~~~~^ ~~~~~~
core/io/compression.cpp:261:9: error: member reference base type 'uint8_t' (aka 'unsigned char') is not a structure or union
                        p_dst->resize(0);
                        ~~~~~^ ~~~~~~
core/io/compression.cpp:267:49: error: member reference base type 'uint8_t' (aka 'unsigned char') is not a structure or union
        if (ret == Z_STREAM_END && (unsigned long)p_dst->size() > strm.total_out) {
                                                  ~~~~~^ ~~~~
core/io/compression.cpp:268:8: error: member reference base type 'uint8_t' (aka 'unsigned char') is not a structure or union
                p_dst->resize(strm.total_out);
                ~~~~~^ ~~~~~~
8 errors generated.
scons: *** [core/io/compression.osx.tools.x86_64.o] Error 1
scons: building terminated because of errors.

Does anyone have any idea how to bypass this issue?

[ 78%] scene/main/http_request.cpp:352:70: error: no viable conversion from 'Vector<uint8_t>' (aka 'Vector<unsigned char>') to 'const PoolVector<uint8_t>' (aka 'const PoolVector<unsigned char>')
                                Error err = client->request_raw(method, request_string, headers, request_data);

akien-mga and others added 30 commits April 22, 2021 15:30
Directly RC as we'll keep changes conservative to keep this branch usable
in production at any time.
It's possible to link against system zlib on Linux, so we should use system paths.

(cherry picked from commit 93b7406)
* Better handling of the scene's environment energy in the lightmapper
  bakes.
* Fixed a bug where ProceduralSky::get_panorama() returned a reference
  instead of a copy.
* Removed includes to Embree's internal header files.

(cherry picked from commit 2db2d11)
We need to propagate the hacky checks from the raycast config to the
lightmapper config, as the failure of a `can_build()` check is not notified to
other modules (which might even be checked further depending on the processing
order in SConstruct).

A more thorough fix would be to change SConstruct to do two loops on modules:
one to check `can_build()` and disable modules which can't build, then another
one to rechecked `can_build()` with the new lineup and do further config.
But there would be more risk for regressions than with this ad hoc hack.

Similar story for the `platform/x11/detect.py` change... oh my eyes :(

(cherry picked from commit a2c68d9)
…ss too

`tech_debt++`, that's what we get for not taking the time to cleanup all this
and do it right...

Follow-up to godotengine#48073 and godotengine#48102.

(cherry picked from commit a14b51d)
Follow-up to godotengine#46810, this was missed in godotengine#47079 when fixing the issue
for other platforms.

Fixes godotengine#48135.

(cherry picked from commit a09f383)
- Modified classes: RigidBody, PhysicalBone, VehicleBody, RigidBody2D, KinematicBody2D
- The input argument is untrusted even in release mode

(cherry picked from commit e075b6b)
This updates global classes and exposes base member variables.
Fixes godotengine#39713

(cherry picked from commit b16bb33)
The final_modulate was incorrectly being set in the uniform on light passes in GLES3 in situations where color was baked in the vertices. This was already correct in GLES2. This PR makes prevents setting final_modulate in this situation.

(cherry picked from commit 35c5ccc)
The translation to larger vertex formats was assuming that batches were rects, and not accounting that the num_commands had a different meaning for lines and polys, so the calculation for number of vertices to translate was incorrect in these cases.

Also prevents infinite loop if a single polygon has too many vertices to fit in the batch buffer.

(cherry picked from commit d08cf5f)
(cherry picked from commit decdf4f)
The min requirement was upped by godotengine#45618 to have proper support for C++14.

Related to godotengine#48222.

(cherry picked from commit 8851fa7)
After further testing it seems to work fine now when building binaries with GCC 5
on Ubuntu 16.04 (previously we were using GCC 9 on Ubuntu 14.04).

Follow-up to godotengine#45629.

(cherry picked from commit aa15ad7)
We still use Emscripten 1.39.9 for official Mono builds so ideally we want to test
against an old Emscripten version to ensure we don't break compatibility.

But then google-closure-compiler-linux broke compatibility for us and is not properly
pinned, so we need to use a more recent version for now to fix CI.

Cf. emscripten-core/emsdk#802

(cherry picked from commit 9571ae3)
(cherry picked from commit 7501c7f)
Now correctly computes the timeout value in milliseconds.

(cherry picked from commit 46f7b0f)
@aaronfranke aaronfranke changed the base branch from 3.3 to 3.x May 11, 2021 15:33
@aaronfranke aaronfranke removed request for a team May 11, 2021 15:36
@tavurth
Copy link
Contributor Author

tavurth commented May 11, 2021

Ok thank you @aaronfranke!

I've opened a new PR with the 3.x branch.

#48651

@tavurth tavurth closed this May 11, 2021
@aaronfranke aaronfranke removed request for a team May 11, 2021 19:19
@tavurth tavurth deleted the 3.3 branch May 11, 2021 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.