Why does AmazonWebServiceClient.setRegion computes Signer with isRegionIdAsSignerParam=false?

[zz22394]

Dear team:

I have a question,
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/AmazonWebServiceClient.java#L342
Why does AmazonWebServiceClient.setRegion computes Signer with isRegionIdAsSignerParam=false?

I am trying to use this library to connect an S3-Compatible storage service.
I use the override region file and set region to the internal one.

System.setProperty(SDKGlobalConfiguration.REGIONS_FILE_OVERRIDE_SYSTEM_PROPERTY, "/tmp/sso_regions.xml");
Region ssoBeta = RegionUtils.getRegion("beta");
client.setRegion(ssoBeta);

However, It fails when authration and got 400 Bad request.
In the request header, SDK library sent

Authorization: AWS4-HMAC-SHA256 Credential=AK....MyKey...A/20160913/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;user-agent;x-amz-content-sha256;x-amz-date, Signature=bbd27c....MyKey...9b5b3

I traced the source code,
AmazonS3Client.setRegion(Region region)
-->super.setRegion(region)
-->'computeSignerByServiceRegion(serviceNameForSigner, region.getName(), signerRegionOverride, false)'
Because AWSS3V4Signer use Region in header, regionId should be used.
So the isRegionIdAsSignerParam parameter here should be true, am I right?

[shorea]

Wow this code is confusing. We'll actually try and set the region in the signer provider which might recompute the region if there's a redirect (https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/internal/auth/S3SignerProvider.java#L53).

The region on the signer itself is apparently more of an override, similar to the signerRegionOverride on the client (https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/auth/AWS4Signer.java#L79). When not set it too will attempt to parse the region from the hostname (https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/auth/internal/AWS4SignerRequestParams.java#L88)

What's your custom endpoints file look like? Would you mind sharing or is it sensitive?

[zz22394]

@shorea Thanks for your reply.
Our custom endpoints file(/tmp/sso_regions.xml):

<XML>
  <Regions>
    <Region>
      <Name>beta</Name>
      <Domain>beta.storage.testdomain.com</Domain>
      <Endpoint>
        <ServiceName>s3</ServiceName>
        <Http>true</Http>
        <Https>true</Https>
        <Hostname>beta.storage.testdomain.com</Hostname>
      </Endpoint>
    </Region>
    <Region>
      <Name>region01</Name>
      <Domain>region01.storage.storage-testdomain.com</Domain>
      <Endpoint>
        <ServiceName>s3</ServiceName>
        <Http>true</Http>
        <Https>true</Https>
        <Hostname>region01.storage.storage-testdomain.com</Hostname>
      </Endpoint>
    </Region>
    <Region>
      <Name>region02</Name>
      <Domain>region02.storage.storage-testdomain.com</Domain>
      <Endpoint>
        <ServiceName>s3</ServiceName>
        <Http>true</Http>
        <Https>true</Https>
        <Hostname>region02.storage.storage-testdomain.com</Hostname>
      </Endpoint>
    </Region>
  </Regions>
  <Services>
    <Service>
      <Name>s3</Name>
      <FullName>testdomain Simple Storage Service</FullName>
      <RegionName>beta</RegionName>
      <RegionName>region01</RegionName>
      <RegionName>region02</RegionName>
    </Service>
  </Services>
</XML>

Because there is no "s3" hint in the Hostname (eg. beta.storage.testdomain.com), maybe we will have to make some change to the AwsHostNameUtils to compute our internal region name.
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/util/AwsHostNameUtils.java#L120

[shorea]

I believe you can add a mapping in an overridden config file like so.
https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/resources/com/amazonaws/internal/config/awssdk_config_default.json#L145

The override file is expected to be at the root of the classpath with the name 'awssdk_config_override.json'

aws-sdk-java/aws-java-sdk-core/src/main/java/com/amazonaws/internal/config/InternalConfig.java

Line 55 in a51792e

static final String CONFIG_OVERRIDE_RESOURCE = "awssdk_config_override.json";

[zz22394]

Thanks a lot, I will have a try

[zz22394]

Thanks!
it worked.
I put a customized awssdk_config_override.json in my proj/src/main/resources/ folder,
then the region could be read correctly.

With my edited sso_regions.xml file, the AWS SDK got our internal S3-compatible correctly.

Thanks a lot, it helped me.

[shorea]

Good to hear!