Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Sagemaker pipeline sample #272

Merged
merged 2 commits into from
Apr 19, 2024
Merged

Conversation

prayeole
Copy link
Contributor

@prayeole prayeole commented Apr 9, 2024

Issue #, if available:

Description of changes: Adding a Sagemaker pipeline example for creating and executing a pipeline run.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ack-prow ack-prow bot requested a review from jsitu777 April 9, 2024 19:05
@ack-prow ack-prow bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Apr 9, 2024
Copy link

ack-prow bot commented Apr 9, 2024

Hi @prayeole. Thanks for your PR.

I'm waiting for a aws-controllers-k8s member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Copy link
Member

@a-hilaly a-hilaly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thank you @prayeole ! I left two tiny comments in-line


### Modify/Create a JSON pipeline definition

Create the JSON pipeline definition using the JSON schema documented at https://aws-sagemaker-mlops.github.io/sagemaker-model-building-pipeline-definition-JSON-schema/. You will need to convert your JSON pipeline definition into String format to pass to the *.spec.pipelineDefinition* key in the Kubernetes YAML spec. You may use online third-party tools to convert from JSON to String format. In this sample, you are provided a sample pipeline definition with one Training step.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible to pass a json object as is using the yaml-multi-line directive. Something similar to https://github.com/aws-controllers-k8s/iam-controller/blob/main/test/e2e/resources/role_simple.yaml#L9-L21 .

However not sure if the sagemaker API accepts indented json strings

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @a-hilaly . I updated the pipeline spec with inline JSON object, however, still called out passing as string as another option in the README, because it may be easier especially, when the pipeline definition is large with multiple steps.

Comment on lines 33 to 41

To list all pipelines created using the ACK controller use the following command:
```
$ kubectl get pipeline.sagemaker.services.k8s.aws
```
If it is a pipeline executions it is endpointsconfigs.sagemaker.services.k8s.aws
```
$ kubectl get pipelineexecution.sagemaker.services.k8s.aws
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you could also use short names such as pipeline and pipelineexecution

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this as suggested.

Copy link
Contributor Author

@prayeole prayeole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updates made. Comments in-line.

Copy link
Member

@ryansteakley ryansteakley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks for the contribution! Lets see if @suryans-commit has any comments

@ack-prow ack-prow bot added the approved label Apr 10, 2024
@a-hilaly
Copy link
Member

/ok-to-test

@ack-prow ack-prow bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 15, 2024
@ryansteakley
Copy link
Member

/lgtm

@ack-prow ack-prow bot added the lgtm Indicates that a PR is ready to be merged. label Apr 19, 2024
Copy link

ack-prow bot commented Apr 19, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: prayeole, ryansteakley

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ack-prow ack-prow bot merged commit b285666 into aws-controllers-k8s:main Apr 19, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm Indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants