Python wrapper for cloud service provider APIs/SDKs.
- Microsoft Azure
./cloud-harness.py azure
, incorporating:- Azure Service Management APIs via Microsoft Azure Python SDK/API
- Azure Resource Management APIs
- from a working Python environment, run
pip install -r requirements.txt && git clone https://github.com/ab77/cloud-harness.git /opt/cloud-harness && pushd /opt/cloud-harness
- copy
cloud-harness.sample.conf
tocloud-harness.conf
- download and save your Azure PublishSettings file with
.publishsettings
extension to the same directory - run
./cloud-harness.py azure
for the first time to extract your management certificate and update the config file automatically (or manually if you wish) - set default
location_name
incloud-harness.conf
config file (e.g. East US), runpython ./cloud-harness.py azure
for the default actionlist_locations
- set other configuration properties as required in
cloud-harness.conf
config file (e.g.storage_account
andservice_certificate
) - to use the new Azure Resource Management (ARM) APIs, you'll need to follow these steps.
- run
python ./cloud-harness.py azure --help
to see all available command line options - to get get a list of required parameters for a particular action (e.g.
add_role
), runpython ./cloud-harness.py azure --action add_role
- specify
--verbose
flag to see various run-time properties - specify
--readonly
flag to limit operations, which would otherwise perform changes (usually together with--verbose
) - specify
--async
flag to return from calls immediately, without waiting for operation completion
The following resource extensions are supported (use --extension <extension> <extension> ... <ext>
):
- ChefClient (Windows|Linux)
- CustomScript (Windows|Linux)
- VMAccessAgent (Windows|Linux)
- OSPatching (Linux)
- DockerExtension (Linux)
- DSC (Windows)
- PuppetEnterpriseAgent (Windows|Linux)
- BGInfo (Windows|Linux)
- OctopusDeploy (Windows)
Some useful examples to deploy virtual machines and various resource extensions.
Create storage account (name must be unique as it forms part of the storage URL, check with --action check_storage_account_name_availability
):
./cloud-harness.py azure --action create_storage_account \
--account myuniquestorageaccountname01 \
--verbose
Remember to update storage_account
in cloud-harness.py
with the name of the storage account you've just created.
Create a new hosted service (name must be unique within cloudapp.net
domain, check with --action check_storage_account_name_availability
):
./cloud-harness.py azure --action create_hosted_service \
--service my-hosted-service \
--label 'my hosted service' \
--verbose
./cloud-harness.py azure --action add_service_certificate \
--service my-hosted-service \
--certificate service_certificate.cer \
--verbose
./cloud-harness.py azure --action create_reserved_ip_address \
--ipaddr my-reserved-ip-address \
--verbose
./cloud-harness.py azure --action create_virtual_network_site \
--network VNet1 \
--subnet Subnet-1 \
--subnetaddr 10.0.0.0/11 \
--vnetaddr 10.0.0.0/8 \
--verbose
./cloud-harness.py azure --action list_os_images
Create a new Linux virtual machine deployment and role with reserved IP, SSH authentication and CustomScript
resource extension[n3]:
./cloud-harness.py azure --action create_virtual_machine_deployment \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-ubuntu-virtual-machine \
--label 'my deployment' \
--account my-storage-account \
--blob b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-wily-15_10-amd64-server-20150722-en-us-30GB \
--os Linux \
--network my-virtual-network-name \
--subnet my-subnet-name \
--ipaddr my-reserved-ip-address \
--size Medium \
--extension CustomScript \
--ssh_auth \
--disable_pwd_auth \
--verbose
./cloud-harness.py azure --action add_resource_extension \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-ubuntu-virtual-machine \
--extension DockerExtension \
--docker_compose compose.yaml \
--verbose
Note, the Docker extension relies on certain pre-requisites, incl. certificates. See this article on how to generate these and don't forget to update the [DockerExtension]
section in cloud-harness.conf
afterwards.
Create a Linux virtual machine (role) with a random alpha-numeric password[n2], add CustomScript
and ChefClient
extensions:
./cloud-harness.py azure --action add_role \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-second-ubuntu-virtual-machine \
--label 'my Linux (Ubuntu) virtual machine' \
--account my-storage-account \
--blob b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-wily-15_10-amd64-server-20150722-en-us-30GB \
--os Linux \
--network my-virtual-network-name \
--subnet my-subnet-name \
--size Medium \
--extension CustomScript ChefClient \
--verbose
./cloud-harness.py azure --action add_data_disk \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-second-ubuntu-virtual-machine \
--account my-storage-account \
--verbose
Create a Windows virtual machine (role) with random alpha-numeric password, and CustomScript
extension[n4]:
./cloud-harness.py azure --action add_role \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-windows-virtual-machine \
--label 'my Windows 2K8R2 virtual machine' \
--account my-storage-account \
--os Windows \
--blob a699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-201505.01-en.us-127GB.vhd \
--network my-virtual-network-name \
--subnet my-subnet-name \
--size Medium \
--extension CustomScript \
--verbose
./cloud-harness.py azure --action add_resource_extension \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-windows-virtual-machine \
--extension ChefClient DSC \
--dsc_module IISInstall.ps1.zip \
--verbose
./cloud-harness.py azure --action add_resource_extension \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-second-ubuntu-virtual-machine \
--extension VMAccessAgent \
--password new-s3cure-passw0rd \
--verbose
./cloud-harness.py azure --action add_resource_extension \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-second-ubuntu-virtual-machine \
--extension OSPatching \
--patching_oneoff \
--verbose
./cloud-harness.py azure --action set_epacls \
--service my-hosted-service \
--deployment my-virtual-machine-deployment \
--name my-ubuntu-virtual-machine \
--subnet my-subnet-name \
--verbose
./cloud-harness.py azure --action delete_hosted_service \
--service my-hosted-service \
--delete_disks \
--delete_vhds \
--verbose
./cloud-harness.py azure --action delete_reserved_ip_address \
--ipaddr my-reserved-ip-address
To use Fiddler2
to capture HTTPS traffic to the API
- export your Azure Management Certificate as base64 encoded x.509 as ClientCertificate.cer
- place it into your
Fiddler2
directory (e.g.C:\Users\<user>\Documents\Fiddler2
) - set
proxy = True
incloud-harness.conf
and re-launchFiddler2
[n1]
Lots, including:
- implement new Azure Resource Management APIs
- implement (at least) one additional cloud service provider (e.g. DigitalOcean)
- add a suitable unittest framework
- implement additional VM extensions (OctopusDeploy, etc.)
- where it makes sense, move user-configurable defaults to config file
- automate Chef Server deployment from Azure image
-- ab1
[n1] For more information, see Using Fiddler to decipher Windows Azure PowerShell or REST API HTTPS traffic.
[n2] SSH authentication is not compatible with ChefClient
extension due to the way it currently handles certificates PR45.
[n3] CustomScript
extension on Linux by default, will run bootstrap.sh
to upgrade WAAgent
as well as un-pack/execute linux_custom_data.dat
where you can put additional bootstrap commands.
[n4] CustomScript
extension on Windows by default, will run bootstrap.ps1
to un-pack/execute windows_custom_data.dat
where you can put additional bootstrap commands.
[n5] update_role()
currently resets ACLs, use --action set_epacls
to set them again if you get a warning. Also, this operation will cause a reboot and currently generates new public facing port numbers.
[n6] Docker
is secured by default with SSL, using a server certificate signed by a private CA.
[n7] DSC
configuration archive can be compiled using PowerShell, run Publish-AzureVMDscConfiguration .\MyConfiguration.ps1 -ConfigurationArchivePath .\MyConfiguration.ps1.zip