chore: Upgrade to later version of gcp-uploader and protobuf

[lqiu96]

Update to use the latest version of gcp-uploader package and unpin the dependencies.

[mpeddada1]

@lqiu96 After this change, we will be able to re-enable renovate-bot for these files:

synthtool/renovate.json

Line 5 in 91c4e39

"ignorePaths": ["*/**", "synthtool/gcp/templates/java_library/.kokoro/requirements.txt", "synthtool/docker/owlbot/java/src/requirements.txt"],

[lqiu96]

Seeing this error:

Step #9: ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
Step #9:     SecretStorage>=3.2 from https://files.pythonhosted.org/packages/54/24/b4293291fa1dd830f353d2cb163295742fa87f179fcc8a20a306a81978b7/SecretStorage-3.3.3-py3-none-any.whl (from keyring==23.4.1->-r requirements.txt (line 223))

[lqiu96]

@lqiu96 After this change, we will be able to re-enable renovate-bot for these files:

synthtool/renovate.json

Line 5 in 91c4e39

"ignorePaths": ["*/**", "synthtool/gcp/templates/java_library/.kokoro/requirements.txt", "synthtool/docker/owlbot/java/src/requirements.txt"],

@mpeddada1 Any objections if I include that change in this PR?

I think the change would be

"ignorePaths": ["*/**", "synthtool/gcp/templates/java_library/.kokoro/requirements.txt", "synthtool/docker/owlbot/java/src/requirements.txt"]

becomes

"ignorePaths": ["*/**"],

Is that right?

[mpeddada1]

@mpeddada1 Any objections if I include that change in this PR?

I think the change would be

"ignorePaths": ["*/**", "synthtool/gcp/templates/java_library/.kokoro/requirements.txt", "synthtool/docker/owlbot/java/src/requirements.txt"]

becomes

"ignorePaths": ["*/**"],

Is that right?

Making the change in this PR sounds good! And yup, it would become ["*/**"]. Oh one note, could you also update https://github.com/googleapis/synthtool/blob/master/docker/owlbot/java/src/requirements.in before removing it from the ignorePaths option?

[mpeddada1]

Thanks @lqiu96!

[lqiu96]

Oh one note, could you also update https://github.com/googleapis/synthtool/blob/master/docker/owlbot/java/src/requirements.in before removing it from the ignorePaths option?

@mpeddada1 What did you mean by update? Should I just unpin the dependencies or pin the current latest versions or something else?

[mpeddada1]

@mpeddada1 What did you mean by update? Should I just unpin the dependencies or pin the current latest versions or something else?

I think we can now update the dependencies in the associated requirement.txt to the latest version now that we're using Python 3.9. It'll help us resolve these open PRs: #1480, #1731.

And for requirements.in, mostly a clean up but afaik, pinning is only required to freeze the version/prevent a dep from being upgraded to the latest. This may not be needed anymore with the Python upgrade so we should be fine with unpinning the versions there. wdyt?

[lqiu96]

@mpeddada1 What did you mean by update? Should I just unpin the dependencies or pin the current latest versions or something else?

I think we can now update the dependencies in the associated requirement.txt to the latest version now that we're using Python 3.9. It'll help us resolve these open PRs: #1480, #1731.

And for requirements.in, mostly a clean up but afaik, pinning is only required to freeze the version/prevent a dep from being upgraded to the latest. This may not be needed anymore with the Python upgrade so we should be fine with unpinning the versions there. wdyt?

Sounds good. I'll unpin the versions

[lqiu96]

Adding Do Not Merge to this PR. Plan is to merge after the next release of the libraries-bom (~approximately 2 weeks from now).

[blakeli0]

And for requirements.in, mostly a clean up but afaik, pinning is only required to freeze the version/prevent a dep from being upgraded to the latest

@mpeddada1 Did you validate this approach with other languages lead or rennie, based on his "Pip install vulnerability" doc, I think we may still need it.

[suztomo]

@lqiu96 Would you add an assertion that this python module synthtool.languages.java is available? I think it would look like this:

python3 -c 'import synthtool.languages.java; print("import successs")'

An example failure would look like this:

suztomo@suztomo:~/java-storage$ docker run --user "$(id -u):$(id -g)" --entrypoint /bin/bash --rm -it  a8e0ab6765a4
I have no name!@752c3d65c2dd:/workspace$ python3 -c 'import synthtool.languages.java; print("import successs")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'synthtool.languages'
I have no name!@752c3d65c2dd:/workspace$ 

It gives more confidence that a change for OwlBot Postprocessor works fine in owlbot.py (example location)

[lqiu96]

Sure, I can add that validation in

[lqiu96]

Added the new validation in as java synthtool validation which runs the commands above.

FYI, couldn't find the docs for schemaVersion 1.0.0 and the docs on https://github.com/GoogleContainerTools/container-structure-test were showing docs for 2.0.0, so I used that for local testing

[suztomo]

Thank you.

[mpeddada1]

And for requirements.in, mostly a clean up but afaik, pinning is only required to freeze the version/prevent a dep from being upgraded to the latest

@mpeddada1 Did you validate this approach with other languages lead or rennie, based on his "Pip install vulnerability" doc, I think we may still need it.

@blakeli0 sorry I missed your comment earlier! Yup, according to the document pip install --require-hashes -r /synthtool/requirements.txt line ensures that we mitigate the vulnerability. The pinning of versions was a java-specific issue and we needed it because we were still using an older version of Python that wasn't always compatible with the latest versions of the dependencies. Languages like Python don't pin the versions (for example: https://github.com/googleapis/synthtool/blob/master/synthtool/gcp/templates/python_library/.kokoro/requirements.in).

[suztomo]

Thank you.

[lqiu96]

Removing the Do Not Merge label... Planning on releasing python v3.9.