This demonstration will be presented during Jenkins World 2017 in San Francisco, CA on August 29th-31st at CyberArk Booth # 504.
- Machine Identity
- By granting a machine identity to the Jenkins Master, we can trust any communication authenticated with it's API Key going forward. This allows Summon to use the Jenkins Master identity when reaching out to CyberArk Conjur for the secrets within secrets.yml.
- Role-Based Access Control (RBAC)
- Jenkins Master received it's identity, was added as a Host in Conjur and granted an API Key, and was added to the jenkins/masters Layer (or group of Hosts) to receive the associated Policy (policy.yml).
- On-Demand Secrets Allowing Rotation
- By using Summon rather than hardcoding the credentials, this allows us to retrieve the secrets on-demand allowing CyberArk Conjur to manage and rotate the AWS access keys while still serving out the secrets programatically, as needed.
Our JenkinsWorld2017
job in CloudBees Jenkins is tied to this repository. When the job's build is run, the sqsPost.py script will be run in a Shell Command build step within Jenkins. Rather than just calling python sqsPost.py
to test it in the workspace, we are executing summon python sqsPost.py
instead.
By having summon
run the python
provider, we can inject environment variables into python
that the sqsPost.py script can reference when it runs. summon
will read secrets.yml file and fetch the secret ID referenced within and place it in the given environment variable name in temporary memory. For example: ENV_VAR_NAME: !var /id/of/secret
Our sqsPost.py script is grabbing the environment variables AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
, logging into AWS API and sending a message to a Simple Queue Service (SQS) queue called JenkinsWorld2017 with a 2 minute TTL. The contents of the message are the values of the AWS secrets fetched from within CyberArk Conjur.
The secrets received in the message in AWS SQS can be checked against the Console Output of the Jenkins job build for confirmation of accuracy.
- CyberArk Conjur v4.x
- CyberArk Conjur CLI
- Jenkins v2
- Jenkins OSS or CloudBees Jenkins
sudo apt-get install jq
is needed for parsing JSON response.sudo apt-get install python27
is needed for testing sqsPost.py.
- Amazon Web Services (AWS) Free Tier Account
- Sending messages to Simple Queue Service (SQS) is considered free tier up to 1 million messages.
- Use caution if planning to exceed 1 million messages within a one-month billing period.
- Load policy.yml into CyberArk Conjur via CLI.
$ conjur authn login
$ conjur policy load --as-group security_admin policy.yml
- Install CyberArk Conjur CLI on Jenkins Master. Copy latest .deb release from cyberark/conjur-cli.
$ sudo curl -L -O https://github.com/cyberark/conjur-cli/releases/download/v5.4.0/conjur_5.4.0-1_amd64.deb
$ sudo dpkg -i ./conjur_5.4.0-1_amd64.deb
$ sudo apt-get install -f
- Do the following on the Jenkins Master:
$ mkdir ~/src
$ cd ~
- Copy identify.sh to the user running Jenkins' home directory on the Jenkins Master and change the commented variables for your environment.
- Login to the CyberArk Conjur UI, click on "Layers" in the left sidebar navigation and select the Layer created by our policy.yml.
- Scroll down to the "Host Factory" section and click "Add" to add one.
- Do the following on the Jenkins Master:
$ vi ~/src/hftoken.txt
Paste host factory token and save.
- Retrieve Machine Identity for Jenkins Master:
$ chmod +x identify.sh
$ ./identify.sh
- Login to Jenkins Web Interface
- Select
New Item
from left sidebar navigation. - Create a new Freestyle Project named
JenkinsWorldDemo
. - Under
Source Code Management
, selectGit
and use this repository forked under your GitHub user account. - Under
Build
,Add build step
and chooseExecute shell
with the following command:summon python sqsPost.py
- Save the job.
- Do the following from the Docker host:
docker exec -u 0 jenkins pip install boto3
- Navigate to Summon Releases and copy the URL to download the latest
summon-linux-amd64.tar.gz
. - Do the following from the Jenkins Master:
$ curl -L -O https://github.com/cyberark/summon/releases/download/v0.6.5/summon-linux-amd64.tar.gz
$ tar -xvzf summon-linux-amd64.tar.gz
$ mv summon /usr/local/bin
- Navigate to Summon-Conjur Releases and copy the URL to download VERSION 0.2.0
summon-conjur-linux-amd64.tar.gz
. - Do the following from the Docker host:
docker exec -u 0 -it jenkins bash
- Do the following from the Jenkins Master as root now:
$ curl -L -O https://github.com/cyberark/summon-conjur/releases/download/v0.2.0/summon-conjur-linux-amd64.tar.gz
$ tar -xvzf summon-conjur-linux-amd64.tar.gz
$ mkdir -p /use/local/lib/summon
$ mv summon /usr/local/lib/summon
- Login to Amazon Web Services (AWS) as your Free Tier account.
- You should land on the
AWS services
page. SelectSimple Queue Service
underMessaging
. - Click the blue
Create New Queue
button. - Create a queue named
JenkinsWorldDemo
, selectFIFO Queue
, and clickConfigure Queue
. - Change the
Message Retention Period
to2 minutes
and click the blueCreate Queue
button. - You should now see
JenkinsWorldDemo.fifo
as an available queue. - Update the
queue_url=
value in postSQS.py to reflect your proper region and AWS Account Number.
- Login to Jenkins Web Interface.
- Select
JenkinsWorldDemo
from the dashboard. - Click
Build Now
in the left sidebar navigation. - After the Build Passes (or Fails), click the Job Number in the
Build History
pane. - Click
Console Output
in the left sidebar navigation.
Started by user CyberArk Demo
Building in workspace /root/operations-center/workspace/JenkinsWorldDemo
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/infamousjoeg/jenkinsworld-e2e # timeout=10
Fetching upstream changes from https://github.com/infamousjoeg/jenkinsworld-e2e
> git --version # timeout=10
using GIT_ASKPASS to set credentials GitHub Creds
> git fetch --tags --progress https://github.com/infamousjoeg/jenkinsworld-e2e +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 6f3f0dea7738d3e6fd6b11eded7f876159affb1e (refs/remotes/origin/master)
Commit message: "Delete host.json"
> git config core.sparsecheckout # timeout=10
> git checkout -f 6f3f0dea7738d3e6fd6b11eded7f876159affb1e
> git rev-list 6f3f0dea7738d3e6fd6b11eded7f876159affb1e # timeout=10
[JenkinsWorldDemo] $ /bin/sh -xe /tmp/jenkins6768624294872102283.sh
+ summon python sqsPost.py
Warning: this build has no associated authentication, so build permissions may be lacking, and downstream projects which cannot even be seen by an anonymous user will be silently skipped
Finished: SUCCESS
- Login to Amazon Web Services (AWS) Management Console.
- Navigate to the Simple Queue Service (SQS) where we configured the
JenkinsWorldDemo.fifo
queue earlier. - Select the
JenkinsWorldDemo.fifo
queue and selectQueue Actions
>View/Delete Messages
. - Click the blue
Start Polling for Messages
and all messages received in the past 2 minutes will begin to appear. The contents are a JSON blob of the Access Key ID and Secret Access Key retrieved from CyberArk Conjur.
For more information on Summon, please visit Summon on GitHub.