Use Application Default Credentials to get App/Compute Engine credentials #337

ajkannan · 2015-11-07T02:48:55Z

Fixes #334

2 notes:
-- This PR should be merged in only once #336 is merged in
-- I left the order for project ID determination the same for now. There was a suggestion to make it the same order as the authentication credentials lookup. However, since we check for App Engine ID before Google Cloud SDK ID, we don't run into security exceptions with using the files API if we run in App Engine. However, if we change the order to move Google Cloud SDK first, we have to try and catch that exception, and the effective order would be slightly different than what we might report. Somebody might change their JRE locally to allow read access to the Google Cloud SDK config file, but then it would crash in production.

…re no default auth credentials on Travis

aozarov · 2015-11-07T04:06:15Z

I am not sure I understand your comment about the order.
What we want is to switch the order of project default to look up at "Google Cloud SDK"
before "Compute Engine" which is similar to the lookup order for auth. In both cases (auth or projectId) AE is picked before and therefore we should not have the file access issue.

aozarov · 2015-11-07T04:15:47Z

Ah, I get you now. This PR actually changed the lookup order of auth as we now rely on google-auth-library-java for AE auth. I think we could still swith the projectid lookup order of CE with SDK, no?

gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/SerializationTest.java

-        .authCredentials(AuthCredentials.noCredentials())
-        .force(true)
-        .build();
+    options =


ajkannan · 2015-11-09T16:27:00Z

@aozarov Added a commit to switch the project ID lookup order of CE and SDK.

ajkannan · 2015-11-11T01:24:00Z

Waiting on this bug to be fixed, will test and update PR as necessary following that fix.

aozarov · 2015-11-17T20:58:56Z

I assume this PR also depends on googleapis/google-auth-library-java#45

ajkannan · 2015-11-17T21:03:31Z

Yup, that's correct.

ajkannan · 2015-11-18T23:40:17Z

This PR now is only blocked by this feature request from the auth library.

aozarov · 2015-11-20T18:54:26Z

Once we resolve this issue we should also fix #334

ajkannan · 2015-11-24T02:19:37Z

Added back in the app engine credentials since it looks like reflection won't be added to the auth library.

gcloud-java-core/src/main/java/com/google/gcloud/ServiceOptions.java

@@ -111,12 +115,22 @@ public HttpTransport create() {
      }
      // Consider Compute
      try {
-        return AuthCredentials.getComputeCredential().getTransport();
+        return getComputeHttpTransport();


aozarov · 2015-11-24T03:32:39Z

Looks good. One cleanup suggestion that needs to be validated.

ajkannan · 2015-11-25T03:01:55Z

The compute credentials cleanup suggestion worked, so I added a commit for it. I also noticed that service credentials didn't work with Datastore (I had been testing with storage earlier) because the v1beta2 dependency pulls in an outdated version of com.google.api-client:google-api-client. I added a commit that fixes this as well.

Also, two cleanup questions:

There's GoogleCredential from com.google.api-client:google-api-client and there's GoogleCredentials (with an 's') from google-auth-library-java. Could we use the google-auth-library-java's GoogleCredentials here? It looks like both have the fromStream. Then we can get rid of the com.google.api-client dependency in the PR that fixes Remove the HttpIntializer dependency from AuthCredentials #403 (once the HttpInitializer code that uses GoogleCredential is gone).
We have a ServiceAccountCredentials class, but google-auth-library-java does as well. Do you think it'd be worthwhile to use google-auth-library-java's instead? It seems a bit more heavyweight, but it is a subclass of GoogleCredentials, which could be helpful for more auth restructuring later.

aozarov · 2015-11-25T18:20:24Z

Yes, we should use google-auth-library-java Credentialsinstead of the GoogleCredential from google-api-client(however we do need that artifcat).

We can't use ServiceAccountCredentialsfrom google-auth-library-java directly (as it is not restorable) however we can delegate to it.

Use Application Default Credentials to get App/Compute Engine credentials

This PR was generated using Autosynth. 🌈 <details><summary>Log from Synthtool</summary> ``` 2020-12-15 22:41:00,355 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-resourcemanager/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2020-12-15 22:41:01,259 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ``` </details> Full log will be available here: https://source.cloud.google.com/results/invocations/6a383c9a-220f-416e-a702-a377af0ea86a/targets - [ ] To automatically regenerate this PR, check this box.

…-plugin to v3.4.1 (#337) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [org.apache.maven.plugins:maven-javadoc-plugin](https://maven.apache.org/plugins/) | `3.4.0` -> `3.4.1` | [![age](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-javadoc-plugin/3.4.1/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-javadoc-plugin/3.4.1/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-javadoc-plugin/3.4.1/compatibility-slim/3.4.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-javadoc-plugin/3.4.1/confidence-slim/3.4.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Renovate will not automatically rebase this PR, because other commits have been found. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. ⚠ **Warning**: custom changes will be lost. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-channel).

…337) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.protobuf:protobuf-java](https://developers.google.com/protocol-buffers/) ([source](https://togithub.com/protocolbuffers/protobuf)) | `3.19.4` -> `3.20.1` | [![age](https://badges.renovateapi.com/packages/maven/com.google.protobuf:protobuf-java/3.20.1/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.protobuf:protobuf-java/3.20.1/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.protobuf:protobuf-java/3.20.1/compatibility-slim/3.19.4)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.protobuf:protobuf-java/3.20.1/confidence-slim/3.19.4)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>protocolbuffers/protobuf</summary> ### [`v3.20.1`](https://togithub.com/protocolbuffers/protobuf/releases/v3.20.1) ##### PHP - Fix building packaged PHP extension ([#9727](https://togithub.com/protocolbuffers/protobuf/issues/9727)) - Fixed composer.json to only advertise compatibility with PHP 7.0+. ([#9819](https://togithub.com/protocolbuffers/protobuf/issues/9819)) ##### Ruby - Disable the aarch64 build on macOS until it can be fixed. ([#9816](https://togithub.com/protocolbuffers/protobuf/issues/9816)) ##### Other - Fix versioning issues in 3.20.0 ### [`v3.20.0`](https://togithub.com/protocolbuffers/protobuf/releases/v3.20.0) 2022-03-25 version 3.20.0 (C++/Java/Python/PHP/Objective-C/C#/Ruby/JavaScript) ##### Ruby - Dropped Ruby 2.3 and 2.4 support for CI and releases. ([#9311](https://togithub.com/protocolbuffers/protobuf/issues/9311)) - Added Ruby 3.1 support for CI and releases ([#9566](https://togithub.com/protocolbuffers/protobuf/issues/9566)). - Message.decode/encode: Add recursion_limit option ([#9218](https://togithub.com/protocolbuffers/protobuf/issues/9218)/[#9486](https://togithub.com/protocolbuffers/protobuf/issues/9486)) - Allocate with xrealloc()/xfree() so message allocation is visible to the Ruby GC. In certain tests this leads to much lower memory usage due to more frequent GC runs ([#9586](https://togithub.com/protocolbuffers/protobuf/issues/9586)). - Fix conversion of singleton classes in Ruby ([#9342](https://togithub.com/protocolbuffers/protobuf/issues/9342)) - Suppress warning for intentional circular require ([#9556](https://togithub.com/protocolbuffers/protobuf/issues/9556)) - JSON will now output shorter strings for double and float fields when possible without losing precision. - Encoding and decoding of binary format will now work properly on big-endian systems. - UTF-8 verification was fixed to properly reject surrogate code points. - Unknown enums for proto2 protos now properly implement proto2's behavior of putting such values in unknown fields. ##### Java - Revert "Standardize on Array copyOf" ([#9400](https://togithub.com/protocolbuffers/protobuf/issues/9400)) - Resolve more java field accessor name conflicts ([#8198](https://togithub.com/protocolbuffers/protobuf/issues/8198)) - Don't support map fields in DynamicMessage.Builder.{getFieldBuilder,getRepeatedFieldBuilder} - Fix parseFrom to only throw InvalidProtocolBufferException - InvalidProtocolBufferException now allows arbitrary wrapped Exception types. - Fix bug in `FieldSet.Builder.mergeFrom` - Flush CodedOutputStream also flushes underlying OutputStream - When oneof case is the same and the field type is Message, merge the subfield. (previously it was replaced.)’ - Add [@CheckReturnValue](https://togithub.com/CheckReturnValue) to some protobuf types - Report original exceptions when parsing JSON - Add more info to [@deprecated](https://togithub.com/deprecated) javadoc for set/get/has methods - Fix initialization bug in doc comment line numbers - Fix comments for message set wire format. ##### Kotlin - Add test scope to kotlin-test for protobuf-kotlin-lite ([#9518](https://togithub.com/protocolbuffers/protobuf/issues/9518)) - Add orNull extensions for optional message fields. - Add orNull extensions to all proto3 message fields. ##### Python - Dropped support for Python < 3.7 ([#9480](https://togithub.com/protocolbuffers/protobuf/issues/9480)) - Protoc is now able to generate python stubs (.pyi) with --pyi_out - Pin multibuild scripts to get manylinux1 wheels back ([#9216](https://togithub.com/protocolbuffers/protobuf/issues/9216)) - Fix type annotations of some Duration and Timestamp methods. - Repeated field containers are now generic in field types and could be used in type annotations. - Protobuf python generated codes are simplified. Descriptors and message classes' definitions are now dynamic created in internal/builder.py. Insertion Points for messages classes are discarded. - has_presence is added for FieldDescriptor in python - Loosen indexing type requirements to allow valid **index**() implementations rather than only PyLongObjects. - Fix the deepcopy bug caused by not copying message_listener. - Added python JSON parse recursion limit (default 100) - Path info is added for python JSON parse errors - Pure python repeated scalar fields will not able to pickle. Convert to list first. - Timestamp.ToDatetime() now accepts an optional tzinfo parameter. If specified, the function returns a timezone-aware datetime in the given time zone. If omitted or None, the function returns a timezone-naive UTC datetime (as previously). - Adds client_streaming and server_streaming fields to MethodDescriptor. - Add "ensure_ascii" parameter to json_format.MessageToJson. This allows smaller JSON serializations with UTF-8 or other non-ASCII encodings. - Added experimental support for directly assigning numpy scalars and array. - Improve the calculation of public_dependencies in DescriptorPool. - \[Breaking Change] Disallow setting fields to numpy singleton arrays or repeated fields to numpy multi-dimensional arrays. Numpy arrays should be indexed or flattened explicitly before assignment. ##### Compiler - Migrate IsDefault(const std::string\*) and UnsafeSetDefault(const std::string\*) - Implement strong qualified tags for TaggedPtr - Rework allocations to power-of-two byte sizes. - Migrate IsDefault(const std::string\*) and UnsafeSetDefault(const std::string\*) - Implement strong qualified tags for TaggedPtr - Make TaggedPtr Set...() calls explicitly spell out the content type. - Check for parsing error before verifying UTF8. - Enforce a maximum message nesting limit of 32 in the descriptor builder to guard against stack overflows - Fixed bugs in operators for RepeatedPtrIterator - Assert a maximum map alignment for allocated values - Fix proto1 group extension protodb parsing error - Do not log/report the same descriptor symbol multiple times if it contains more than one invalid character. - Add UnknownFieldSet::SerializeToString and SerializeToCodedStream. - Remove explicit default pointers and deprecated API from protocol compiler ##### Arenas - Change Repeated\*Field to reuse memory when using arenas. - Implements pbarenaz for profiling proto arenas - Introduce CreateString() and CreateArenaString() for cleaner semantics - Fix unreferenced parameter for MSVC builds - Add UnsafeSetAllocated to be used for one-of string fields. - Make Arena::AllocateAligned() a public function. - Determine if ArenaDtor related code generation is necessary in one place. - Implement on demand register ArenaDtor for InlinedStringField ##### C++ - Enable testing via CTest ([#8737](https://togithub.com/protocolbuffers/protobuf/issues/8737)) - Add option to use external GTest in CMake ([#8736](https://togithub.com/protocolbuffers/protobuf/issues/8736)) - CMake: Set correct sonames for libprotobuf-lite.so and libprotoc.so ([#8635](https://togithub.com/protocolbuffers/protobuf/issues/8635)) ([#9529](https://togithub.com/protocolbuffers/protobuf/issues/9529)) - Add cmake option `protobuf_INSTALL` to not install files ([#7123](https://togithub.com/protocolbuffers/protobuf/issues/7123)) - CMake: Allow custom plugin options e.g. to generate mocks ([#9105](https://togithub.com/protocolbuffers/protobuf/issues/9105)) - CMake: Use linker version scripts ([#9545](https://togithub.com/protocolbuffers/protobuf/issues/9545)) - Manually \*struct Cord fields to work better with arenas. - Manually destruct map fields. - Generate narrower code - Fix [https://github.com/protocolbuffers/protobuf/issues/9378](https://togithub.com/protocolbuffers/protobuf/issues/9378) by removing shadowed *cached_size* field - Remove GetPointer() and explicit nullptr defaults. - Add proto_h flag for speeding up large builds - Add missing overload for reference wrapped fields. - Add MergedDescriptorDatabase::FindAllFileNames() - RepeatedField now defines an iterator type instead of using a pointer. - Remove obsolete macros GOOGLE_PROTOBUF_HAS_ONEOF and GOOGLE_PROTOBUF_HAS_ARENAS. ##### PHP - Fix: add missing reserved classnames ([#9458](https://togithub.com/protocolbuffers/protobuf/issues/9458)) - PHP 8.1 compatibility ([#9370](https://togithub.com/protocolbuffers/protobuf/issues/9370)) ##### C\# - Fix trim warnings ([#9182](https://togithub.com/protocolbuffers/protobuf/issues/9182)) - Fixes NullReferenceException when accessing FieldDescriptor.IsPacked ([#9430](https://togithub.com/protocolbuffers/protobuf/issues/9430)) - Add ToProto() method to all descriptor classes ([#9426](https://togithub.com/protocolbuffers/protobuf/issues/9426)) - Add an option to preserve proto names in JsonFormatter ([#6307](https://togithub.com/protocolbuffers/protobuf/issues/6307)) ##### Objective-C - Add prefix_to_proto_package_mappings_path option. ([#9498](https://togithub.com/protocolbuffers/protobuf/issues/9498)) - Rename `proto_package_to_prefix_mappings_path` to `package_to_prefix_mappings_path`. ([#9552](https://togithub.com/protocolbuffers/protobuf/issues/9552)) - Add a generation option to control use of forward declarations in headers. ([#9568](https://togithub.com/protocolbuffers/protobuf/issues/9568)) </details> --- ### Configuration 📅 **Schedule**: At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-iam).

🤖 I have created a release *beep* *boop* --- ### [1.3.3](googleapis/java-iam@v1.3.2...v1.3.3) (2022-04-22) ### Dependencies * update dependency com.google.protobuf:protobuf-java to v3.20.1 ([#337](googleapis/java-iam#337)) ([ff257ff](googleapis/java-iam@ff257ff)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

ajkannan added auth type: cleanup An internal cleanup or hygiene concern. labels Nov 7, 2015

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Nov 7, 2015

Auth cleanup

e063bc5

ajkannan force-pushed the auth-cleanup branch from 5d53adb to e063bc5 Compare November 7, 2015 02:51

Remove DefaultAuthCredentials from serialization tests, since there a…

9394d28

…re no default auth credentials on Travis

aozarov reviewed Nov 7, 2015
View reviewed changes

gcloud-java-datastore/src/test/java/com/google/gcloud/datastore/SerializationTest.java

.authCredentials(AuthCredentials.noCredentials())

.force(true)

.build();

options =

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

ajkannan force-pushed the auth-cleanup branch from 3fb055a to b6e5f70 Compare November 9, 2015 16:22

Modify project id lookup order and revert style changes

6074605

ajkannan force-pushed the auth-cleanup branch from b6e5f70 to 6074605 Compare November 9, 2015 16:58

ajkannan mentioned this pull request Nov 10, 2015

Scopes not configured for service account error #341

Closed

ajkannan added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Nov 11, 2015

Ajay Kannan added 2 commits November 18, 2015 15:28

Use version auth library verison 0.3.1 to avoid NPE

a560777

Merge to get AuthCredentials updates

f995b12

Add back AppEngineCredentials

539ac61

ajkannan force-pushed the auth-cleanup branch from 1322fc9 to 539ac61 Compare November 24, 2015 02:15

ajkannan removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Nov 24, 2015

aozarov reviewed Nov 24, 2015
View reviewed changes

Ajay Kannan added 2 commits November 24, 2015 13:20

Use NetHttpTransport when running on Compute

2ef47e8

Fix dependency on two different versions of the GoogleCredential class

f87a394

aozarov added a commit that referenced this pull request Nov 25, 2015

Merge pull request #337 from ajkannan/auth-cleanup

2c90245

Use Application Default Credentials to get App/Compute Engine credentials

aozarov merged commit 2c90245 into googleapis:master Nov 25, 2015

ajkannan deleted the auth-cleanup branch November 30, 2015 20:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Application Default Credentials to get App/Compute Engine credentials #337

Use Application Default Credentials to get App/Compute Engine credentials #337

ajkannan commented Nov 7, 2015

aozarov commented Nov 7, 2015

aozarov commented Nov 7, 2015

This comment was marked as spam.

This comment was marked as spam.

ajkannan commented Nov 9, 2015

ajkannan commented Nov 11, 2015

aozarov commented Nov 17, 2015

ajkannan commented Nov 17, 2015

ajkannan commented Nov 18, 2015

aozarov commented Nov 20, 2015

ajkannan commented Nov 24, 2015

This comment was marked as spam.

aozarov commented Nov 24, 2015

ajkannan commented Nov 25, 2015

aozarov commented Nov 25, 2015

Use Application Default Credentials to get App/Compute Engine credentials #337

Use Application Default Credentials to get App/Compute Engine credentials #337

Conversation

ajkannan commented Nov 7, 2015

aozarov commented Nov 7, 2015

aozarov commented Nov 7, 2015

This comment was marked as spam.

This comment was marked as spam.

ajkannan commented Nov 9, 2015

ajkannan commented Nov 11, 2015

aozarov commented Nov 17, 2015

ajkannan commented Nov 17, 2015

ajkannan commented Nov 18, 2015

aozarov commented Nov 20, 2015

ajkannan commented Nov 24, 2015

This comment was marked as spam.

aozarov commented Nov 24, 2015

ajkannan commented Nov 25, 2015

aozarov commented Nov 25, 2015