Update stack deployment docs and other small fixes

.github/workflows/publish_stack_templates.yml

@@ -23,14 +23,3 @@ jobs:
       - name: Push CloudFormation template to S3
         run: |-
           aws s3 cp infra/aws/aws-ecr-s3-sagemaker.yaml s3://zenml-cf-templates/aws-ecr-s3-sagemaker.yaml
-      # Setup gcloud CLI
-      - uses: google-github-actions/setup-gcloud@v0
-        with:
-          service_account_email: ${{ secrets.GCP_CLOUDBUILD_EMAIL }}
-          service_account_key: ${{ secrets.GCP_CLOUDBUILD_KEY }}
-          project_id: ${{ secrets.GCP_CLOUDBUILD_PROJECT }}
-      # Publish the Deployment Manager function code to GCS
-      - name: Push Deployment Manager function to GCS
-        run: |-
-          zip -j gcp-dm-stack-register.zip infra/gcp/main.py infra/gcp/requirements.txt
-          gsutil cp gcp-dm-stack-register.zip gs://zenml-public-bucket/zenml-gcp-dm-templates/

docs/book/how-to/stack-deployment/deploy-a-cloud-stack.md

@@ -62,7 +62,7 @@ Once the configuration is finished, you will see a deployment page:
 
 ![Deploying the new stack](../../.gitbook/assets/deploy_stack_aws_2.png)
 
-During this process, you will be redirected to a Cloud Formation page on AWS Console. 
+Clicking on the "Deploy in AWS" button will redirect you to a Cloud Formation page on AWS Console. 
 
 ![Cloudformation page](../../.gitbook/assets/deploy_stack_aws_cloudformation_intro.png)
 
@@ -76,13 +76,16 @@ pre-filled configuration and create the stack.
 If you choose `gcp` as your provider, you will see a page where you will have 
 to select a region and a name for your new stack:
 
-![Configuring the new stack](../../.gitbook/assets/deploy_stack_gcp.png)
+![Deploy GCP Stack - Step 1](../../.gitbook/assets/deploy_stack_gcp.png)
+![Deploy GCP Stack - Step 1 Continued](../../.gitbook/assets/deploy_stack_gcp_2.png)
 
 Once the configuration is finished, you will see a deployment page:
 
-![Deploying the new stack](../../.gitbook/assets/deploy_stack_gcp_2.png)
+![Deploy GCP Stack - Step 2](../../.gitbook/assets/deploy_stack_gcp_3.png)
 
-During this process, you will be redirected to a Cloud Shell session on GCP.
+Make note of the configuration values provided to you in the ZenML dashboard. You will need these in the next step.
+
+Clicking on the "Deploy in GCP" button will redirect you to a Cloud Shell session on GCP.
 
 ![GCP Cloud Shell start page](../../.gitbook/assets/deploy_stack_gcp_cloudshell_start.png)
 
@@ -96,6 +99,8 @@ GCP projects. You will also get a chance to review the scripts that will be
 executed in the Cloud Shell session before proceeding.
 {% endhint %}
 
+![GCP Cloud Shell intro](../../.gitbook/assets/deploy_stack_gcp_cloudshell_intro.png)
+
 After the Cloud Shell session starts, you will be guided through the process of
 authenticating with GCP, configuring your deployment, and finally provisioning
 the resources for your new GCP stack using Deployment Manager.
@@ -106,10 +111,12 @@ enabled and to configure your terminal with the selected project:
 ![GCP Cloud Shell tutorial step 1](../../.gitbook/assets/deploy_stack_gcp_cloudshell_step_1.png)
 
 Next, you will be asked to configure your deployment by pasting the configuration
-values that were provided to you in the ZenML dashboard:
+values that were provided to you earlier in the ZenML dashboard. You may need to switch back to the ZenML dashboard to copy these values if you did not do so earlier:
 
 ![GCP Cloud Shell tutorial step 2](../../.gitbook/assets/deploy_stack_gcp_cloudshell_step_2.png)
 
+![Deploy GCP Stack pending](../../.gitbook/assets/deploy_stack_pending.png)
+
 You can take this opportunity to review the script that will be executed at the
 next step. You will notice that this script starts by enabling some necessary
 GCP service APIs and configuring some basic permissions for the service accounts
@@ -129,6 +136,8 @@ GCP console:
 Once the deployment is complete, you may close the Cloud Shell session and return
 to the ZenML dashboard to view the newly created stack:
 
+![GCP Cloud Shell tutorial step 4](../../.gitbook/assets/deploy_stack_gcp_cloudshell_step_4.png)
+
 ![GCP Stack dashboard output](../../.gitbook/assets/deploy_stack_gcp_dashboard_output.png)
 
 {% endtab %}
@@ -153,20 +162,44 @@ guide](../../component-guide/component-guide.md) for further details.
 
 ### AWS
 
-If you choose `aws` as your provider, this command will redirect you to a 
-Cloud Formation page on AWS: 
+If you choose `aws` as your provider, the command will walk you through
+deploying a Cloud Formation stack on AWS. It will start by showing some
+information about the stack that will be created:
+
+![CLI AWS stack deploy](../../.gitbook/assets/deploy_stack_aws_cli.png)
+
+Upon confirmation, the command will redirect you to a Cloud Formation page on
+AWS Console where you will have to deploy the stack:
 
 ![Cloudformation page](../../.gitbook/assets/deploy_stack_aws_cloudformation_intro.png)
 
 You will have to log in to your AWS account, have permission to deploy an AWS 
-CloudFormation script, review and confirm the pre-filled configuration and 
+Cloud Formation stack, review and confirm the pre-filled configuration and 
 create the stack.
 
 ![Finalizing the new stack](../../.gitbook/assets/deploy_stack_aws_cloudformation.png)
 
+The Cloud Formation stack will provision the necessary resources for your new
+AWS stack and automatically register the stack with your ZenML server. You can
+monitor the progress of the stack in your AWS console:
+
+![AWS Cloud Formation progress](../../.gitbook/assets/deploy_stack_aws_cf_progress.png)
+
+Once the provisioning is complete, you may close the AWS Cloud Formation page
+and return to the ZenML CLI to view the newly created stack:
+
+![AWS Stack CLI output](../../.gitbook/assets/deploy_stack_aws_cli_output.png)
+
+
 ### GCP
 
-If you choose `gcp` as your provider, this command will redirect you to a Cloud Shell session on GCP.
+If you choose `gcp` as your provider, the command will walk you through
+deploying a Deployment Manager template on GCP. It will start by showing some
+information about the stack that will be created:
+
+![CLI GCP stack deploy](../../.gitbook/assets/deploy_stack_gcp_cli.png)
+
+Upon confirmation, the command will redirect you to a Cloud Shell session on GCP.
 
 ![GCP Cloud Shell start page](../../.gitbook/assets/deploy_stack_gcp_cloudshell_start.png)
 
@@ -180,6 +213,8 @@ GCP projects. You will also get a chance to review the scripts that will be
 executed in the Cloud Shell session before proceeding.
 {% endhint %}
 
+![GCP Cloud Shell intro](../../.gitbook/assets/deploy_stack_gcp_cloudshell_intro.png)
+
 After the Cloud Shell session starts, you will be guided through the process of
 authenticating with GCP, configuring your deployment, and finally provisioning
 the resources for your new GCP stack using Deployment Manager.
@@ -190,7 +225,8 @@ enabled and to configure your terminal with the selected project:
 ![GCP Cloud Shell tutorial step 1](../../.gitbook/assets/deploy_stack_gcp_cloudshell_step_1.png)
 
 Next, you will be asked to configure your deployment by pasting the configuration
-values that were provided to you in the ZenML CLI:
+values that were provided to you in the ZenML CLI. You may need to switch back
+to the ZenML CLI to copy these values if you did not do so earlier:
 
 ![GCP Cloud Shell tutorial step 2](../../.gitbook/assets/deploy_stack_gcp_cloudshell_step_2.png)
 
@@ -213,6 +249,8 @@ GCP console:
 Once the deployment is complete, you may close the Cloud Shell session and return
 to the ZenML CLI to view the newly created stack:
 
+![GCP Cloud Shell tutorial step 4](../../.gitbook/assets/deploy_stack_gcp_cloudshell_step_4.png)
+
 ![GCP Stack CLI output](../../.gitbook/assets/deploy_stack_gcp_cli_output.png)
 
 {% endtab %}

docs/book/how-to/stack-deployment/register-a-cloud-stack.md

@@ -52,18 +52,33 @@ support to Azure as well. Stay in touch for further updates.
 
 ### Define Service Connector
 
-Below you will find cloud-specific selection options. Based on your selection, you will have to provide the required parameters listed 
-below. This will allow ZenML to create a Service Connector and 
-authenticate you to use your cloud resources.
+As the very first step the configuration wizard will check if the selected 
+cloud provider credentials can be acquired automatically from the local environment.
+If the credentials are found, you will be offered to use them or proceed to 
+manual configuration.
 
-#### AWS
+{% code title="Example prompt for AWS auto-configuration" %}
+```
+AWS cloud service connector has detected connection 
+credentials in your environment.
+Would you like to use these credentials or create a new 
+configuration by providing connection details? [y/n] (y):
+```
+{% endcode %}
+
+If you decline auto-configuration next you might be offered the list of already 
+created service connectors available on the server: pick one of them and proceed or pick 
+`0` to create a new one.
+
+#### AWS connection options
 
 If you select `aws` as your cloud provider, and you haven't selected a connector
-yet, you will be prompted to select an authentication method for your stack.
+or declined auto-configuration, you will be prompted to select an authentication 
+method for your cloud connector.
 
-{% code title="Example Command Output" %}
+{% code title="Available authentication methods for AWS" %}
 ```
-                  Available authentication methods for aws                   
+                  Available authentication methods for AWS                   
 ┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
 ┃ Choice  ┃ Name                           ┃ Required                       ┃
 ┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
@@ -108,14 +123,19 @@ yet, you will be prompted to select an authentication method for your stack.
 ```
 {% endcode %}
 
+Based on your selection, you will have to provide the required parameters 
+listed below. This will allow ZenML to create a Service Connector and authenticate 
+you to use your cloud resources.
+
 #### GCP
 
 If you select `gcp` as your cloud provider, and you haven't selected a connector
-yet, you will be prompted to select an authentication method for your stack.
+or declined auto-configuration, you will be prompted to select an authentication 
+method for your cloud connector.
 
-{% code title="Example Command Output" %}
+{% code title="Available authentication methods for GCP" %}
 ```
-                  Available authentication methods for gcp                   
+                  Available authentication methods for GCP                   
 ┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
 ┃ Choice  ┃ Name                           ┃ Required                       ┃
 ┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
@@ -156,24 +176,57 @@ yet, you will be prompted to select an authentication method for your stack.
 ```
 {% endcode %}
 
+Based on your selection, you will have to provide the required parameters 
+listed below. This will allow ZenML to create a Service Connector and authenticate 
+you to use your cloud resources.
+
 ### Defining cloud components
 
-Next, for each missing component, the available resources will be listed to 
-you as follows:
+Next, you will define three major components of your target stack:
+- artifact store
+- orchestrator
+- container registry
+
+All three are crucial for a basic cloud stack. Extra components can be added later 
+if they are needed.
+
+For each component, you will be asked:
+- if you would like to reuse one of the existing components connected via a defined 
+service connector (if any)
+
+{% code title="Example Command Output for available orchestrator" %}
+```
+                    Available orchestrator
+┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
+┃ Choice           ┃ Name                                               ┃
+┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
+│ [0]              │ Create a new orchestrator                          │
+├──────────────────┼────────────────────────────────────────────────────┤
+│ [1]              │ existing_orchestrator_1                            │
+├──────────────────┼────────────────────────────────────────────────────┤
+│ [2]              │ existing_orchestrator_2                            │
+└──────────────────┴────────────────────────────────────────────────────┘
+```
+{% endcode %}
+
+- to create a new one from available to the service connector resources 
+(if the existing not picked)
 
 {% code title="Example Command Output for Artifact Stores" %}
 ```
-                           Available GCP storages                            
-┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
-┃ Choice        ┃ Storage                                                   ┃
-┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
-│ [0]           │ gs://***************************                          │
-├───────────────┼───────────────────────────────────────────────────────────┤
-│ [1]           │ gs://***************************                          │
-└───────────────┴───────────────────────────────────────────────────────────┘
+                        Available GCP storages                            
+┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
+┃ Choice        ┃ Storage                                               ┃
+┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
+│ [0]           │ gs://***************************                      │
+├───────────────┼───────────────────────────────────────────────────────┤
+│ [1]           │ gs://***************************                      │
+└───────────────┴───────────────────────────────────────────────────────┘
 ```
 {% endcode %}
 
+
+
 ### Final steps
 
 Based on your selection, ZenML will create the stack component and ultimately 

infra/README.md

@@ -19,8 +19,5 @@ A Deployment Manager template is used to provision the infrastructure in GCP. Th
 Files:
 
 * [gcp/gcp-gar-gcs-vertex.jinja](gcp/gcp-gar-gcs-vertex.jinja): Deployment Manager template for provisioning GCS and GCR resources along with a GCP service account and credentials. The template also uses a Cloud Function instance to register the ZenML stack with the ZenML server.
-* [gcp/main.py](gcp/main.py): The Python script that is used by the Cloud Function instance to register the ZenML stack with the ZenML server. The script is triggered by a Cloud Function call that is sent by the Deployment Manager template after the stack resources have been provisioned.
 * [gcp/gcp-gar-gcs-vertex-deploy.sh](gcp/gcp-gar-gcs-vertex-deploy.sh): Deployment script that the user must run in the Cloud Shell to deploy the stack. In addition to deploying the Deployment Manager template, the script also takes care of enabling the necessary GCP APIs and configuring the necessary permissions for the various service accounts involved.
 * [gcp/gcp-gar-gcs-vertex.md](gcp/gcp-gar-gcs-vertex.md): A markdown file that provides the user with instructions on how to deploy the stack using the deployment script. This is powered by the tutorial walkthrough feature in the Google Cloud Shell.
-
-The Cloud Function script is uploaded to GCP GCS using a GitHub action during the release process at the following location: gs://zenml-public-bucket/zenml-gcp-dm-templates/gcp-dm-stack-register.zip