Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Weldr/Cloud API: simplify GCP upload options #3023

Merged
merged 10 commits into from
Oct 11, 2022
Merged

Weldr/Cloud API: simplify GCP upload options #3023

merged 10 commits into from
Oct 11, 2022

Commits on Oct 11, 2022

  1. worker/aws: prefer bucket from TargetOptions if provided 

    Flip the logic when deciding if to use the Bucket from the job or worker
configuration. Previously, the Bucket from the worker configuration was
always preferred if it was set, even if it was provided in the job
itself. This made it impossible to override the configuration.

Change the logic to use the Bucket from the worker configuration only if
it was not set in the job.

Report an error if no bucket name was provided with the job and there is
also none specified in the configuration.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    7c6de92 View commit details
    Browse the repository at this point in the history

  2. worker/aws: don't generate object key in worker 

    There is a desire to make the worker as "dumb" as possible. Therefore it
is not desired to generate the AWS object key names in the worker if it
was not provided in the job.

Modify the worker code to not generate the AWS object key in any case
and instead set an error in case the object key was not provided.

Modify Weldr API implementation to generate the object key, if it was
not provided by the user. This is consistent with Cloud API
implementation.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    2d5fbf0 View commit details
    Browse the repository at this point in the history

  3. worker/osbuild: use dedicated struct for GCP config internally 

    Previously, the internal `OSBuildJobImpl` structure defined only
`GCPCreds` member. This is not practical, once there will be more
than one GCP-related variable.

Define a new `GCPConfiguration` structure, move the credentials variable
into it and use it in `OSBuildJobImpl` instead.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    267ea12 View commit details
    Browse the repository at this point in the history

  4. worker/gcp: allow setting Bucket in worker configuration 

    Extend the worker's configuration to allow setting GCP Bucket to use
when uploading images to GCP. The value from the configuration is used
only if not provided in the TargetOptions of the job.

In GCP, the region of the bucket does not limit importing of the image
to a particular region. So it is completely possible to use a single
Bucket to import images to any and all regions.

Return an error in case no bucket name was set in the job nor in the
worker configuration.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    6706a70 View commit details
    Browse the repository at this point in the history

  5. internal/target/gcp: make Bucket optional 

    The Bucket can now be set also in the worker configuration.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    10d6072 View commit details
    Browse the repository at this point in the history

  6. cloudapi: make Bucket optional in GCPUploadOptions 

    GCP Bucket to use can be now configured in the worker configuration.
Make the `Bucket` optional in the Cloud API when uploading image to GCP.

Adjust the Cloud API test case to configure GCP Bucket on the worker and
not provide it in the API request.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    6acb77e View commit details
    Browse the repository at this point in the history

  7. worker/gcp: return error if job doesn't specify object key 

    The object key is required in order to upload the image to GCP. Return
an error if it is not set.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    d616f6b View commit details
    Browse the repository at this point in the history

  8. weldr/upload/gcp: make Object optional 

    Previously, it was expected from the user to provide the Object name
when uploading image to GCP. The object name does not matter much,
because the object is deleted once image import finishes. Make
the specification of the object name optional and generate it if not
provided.

Adjust the GCP Weldr test case to not provide the Object name when
uploading the image.

The user can still provide the Object name if needed.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    e000bf1 View commit details
    Browse the repository at this point in the history

  9. templates/packer: set the GCP bucket in the worker configuration 

    Similar to AWS, set the GCP bucket in the worker configuration.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    a7ab90c View commit details
    Browse the repository at this point in the history

  10. templates/packer: add comment to get_aws_creds.sh 

    Add a comment explaining why it is important to set the AWS bucket in
the worker configuration, even if the `AWS_ACCOUNT_IMAGE_BUILDER_ARN` is
empty.
    @thozza
    thozza committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    a80273c View commit details
    Browse the repository at this point in the history