Python-Terrascript¶
Introduction¶
About¶
Python-Terrascript is a Python package for generating Terraform configurations in JSON format.
Creating Terraform through a Python script offers a degree of flexibility superior to writing Terraform configurations by hand.
- Control structures like
if
/else
,for
/continue
/break
ortry
/except
/finally
. - More string methods.
- Python functions may be used as an alternative to Terraform Modules.
- Access to the Python Standard Library and third-party packages.
The file PROVIDERS.md lists all currently supported Terraform providers.
Installing Terrascript¶
Terrascript is available from the Python Package Repository PyPi or alternatively from its Github repository.
Installing Terrascript from PyPi¶
It is easiest to install Terrascript directly from the Python Package Index.
$ python3 -m pip install terrascript
Installing Terrascript from Github¶
Terrascript can also be installed from its Github repository.
$ git clone https://github.com/mjuenema/python-terrascript.git
$ cd python-terrascript/
$ git fetch
$ git fetch --tags
The master
branch should be identical to the version on PyPi.
$ git checkout master
$ python3 setup.py install
The develop
branch includes the latest changes but may not always
be in a stable state. Do not use the develop
branch unless you want
to submit a merge request on github.
$ git checkout develop
$ python3 setup.py install
Compatibility¶
Terraform releases¶
`Terraform 0.13`_ added the `required providers`_ block together with tighter integration with the Terraform Provider Registry. Terrascript 0.10.x reflects this change and is not backwards compatible with earlier Terraform releases.
Terraform 0.12 introduced some changes to how it deals with configuration files in JSON format. This is reflected in Terrascript by currently having separate releases for Terraform 0.12 and Terraform 0.11. Earlier releases of Terraform are not supported.
Terraform | Terrascript | Notes |
---|---|---|
0.13.x | 0.10.x | Introduced namespaces, many more providers, suppports Terraform 0.13 only |
0.13.x | 0.9.x | Cleanup efforts and bug fixes, dropping support for Python <3.6, supporting Terraform 0.13.x |
0.12.x | 0.8.x | Terrascript 0.8 are a (almost) complete rewrite |
0.12.x | 0.7.x | Never released |
0.11.x | 0.6.x | Last releases to support Terraform 0.11 and earlier |
Terrascript supports Python 3.6 and later.
Module layout¶
Python-Terrascript’s top-level directory layout is structured into providers, resources and data sources.
import terrascript
import terrascript.provider
import terrascript.resource
import terrascript.data
This is then further structured into namespaces to reflect the changes in Terraform 0.13. Below are examples for importing the modules for the
Amazon Web Services Provider which is maintained by Hashicorp.
import terrascript.provider.hashicorp.aws
import terrascript.resource.hashicorp.aws
import terrascript.data.hashicorp.aws
A first example¶
The following example has been taken from the official Terraform documentation for the AWS Provider and then converted into a Python script that generates the equivalent configuration in JSON syntax.
First the original Terraform HCL format, which since Terraform 0.13 must include
a required_providers
block inside the terraform
block.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "3.36.0"
}
}
}
provider "aws" {
region = "us-east-1"
}
resource "aws_vpc" "example" {
cidr_block = "10.0.0.0/16"
}
The Terrascript code would look like this. The required_providers
argument
is supplied as a nested dictionary. In this example only the
Amazon Web Services Provider is used, which is maintained by Hashicorp.
import terrascript
import terrascript.provider.hashicorp.aws
import terrascript.resource.hashicorp.aws
config = terrascript.Terrascript()
# The ``required_providers`` argument is supplied as a nested dictionary.
config += terrascript.Terraform(required_providers={'aws': {'source': 'hashicorp/aws',
'version': '3.36.0' }
})
# The provider is a module and a class: ***.***
config += terrascript.provider.hashicorp.aws.aws(region='us-east-1')
config += terrascript.resource.hashicorp.aws.aws_vpc('example', cidr_block='10.0.0.0/16')
with open('config.tf.json', 'wt') as fp:
fp.write(str(config))
The content of config.tf.json
is shown below. It is equivalent to the
original HCL format.
{
"terraform": {
"required_providers": {
"aws": {
"source": "hashicorp/aws",
"version": "3.36.0"
}
}
},
"provider": {
"aws": [
{
"region": "us-east-1"
}
]
},
"resource": {
"aws_vpc": {
"example": {
"cidr_block": "10.0.0.0/16"
}
}
}
}
Terrascript does not verify that the generated JSON code is a valid Terraform configuration. This is a deliberate design decision and is explained in the Frequently Asked Questions (FAQ)
Links¶
- Terraform.
- Terraform Provider Registry.
- Documentation for Python-Terrascript.
- Github page of Python-Terrascript.
- Terraform JSON syntax.
Tutorial¶
This section explains the different Terrascript Python classes that can be used to generate a Terraform configuration.
Since release 0.8.0 all Terrascript classes are available by importing just four modules.
import terrascript
import terrascript.provider
import terrascript.resource
import terrascript.data
Note
The old layout, e.g. import terarscript.aws.r
is still available
for backward compatibility but its use is discouraged.
Provider¶
Providers can be found in the terrascript.provider
module, with one class
for each provider. Terrascript supports most Terraform providers. The full list
can be found in List of Providers.
HCL
provider "aws" {
alias = "east"
region = "us-east-1"
}
provider "aws" {
alias = "west"
region = "us-west-1"
}
Python
import terrascript
import terrascript.provider
config = terrascript.Terrascript()
# Amazon Web Service with aliases
config += terrascript.provider.aws(alias="east", region="us-east-1")
config += terrascript.provider.aws(alias="west", region="us-west-1")
JSON
{
"provider": {
"aws": [
{
"alias": "east",
"region": "us-east-1"
},
{
"alias": "west",
"region": "us-west-1"
}
]
}
}
Resource¶
Resources can be found in the terrascript.resource
module. The example below shows
the original HCL syntax for creating an AWS S3 bucket and the equivalent Python code.
HCL
provider "aws" {
region = "us-east-1"
}
resource "aws_s3_bucket" "mybucket" {
bucket = "mybucket"
acl = "private"
tags = {
Name = "My bucket"
Environment = "Dev"
}
}
Python
And here is the same as a Python script. The first argument to terrascript.resource.aws_s3_bucket()
is the Terraform label under which it can be referenced later. Note how the tags
is a dictionary
as in the HCL syntax.
import terrascript
import terrascript.provider.hashicorp.aws
import terrascript.resource.hashicorp.aws
config = terrascript.Terrascript()
# AWS provider
config += terrascript.provider.hashicorp.aws.aws(region="us-east-1")
# Add an AWS S3 bucket resource
config += terrascript.resource.hashicorp.aws.aws_s3_bucket(
"mybucket",
bucket="mybucket",
acl="private",
tags={"Name": "My bucket", "Environment": "Dev"},
)
JSON
{
"provider": {
"aws": [
{
"region": "us-east-1",
"version": "3.60.0",
"source": "https://github.com/hashicorp/terraform-provider-aws"
}
]
},
"resource": {
"aws_s3_bucket": {
"mybucket": {
"bucket": "mybucket",
"acl": "private",
"tags": {
"Name": "My bucket",
"Environment": "Dev"
}
}
}
}
}
Data Source¶
Data Sources are located in the terrascript.data
module. The example creates a Google
Compute Instance based on the Debian-9 image. First the Terrascript HCL syntax.
provider "google" {
credentials = "${file("account.json")}"
project = "myproject"
region = "us-central1"
}
data "google_compute_image" "debian9" {
family = "debian-9"
project = "debian-cloud"
}
resource "google_compute_instance" "myinstance" {
name = "test"
machine_type = "n1-standard-1"
zone = "us-central1-a"
boot_disk {
initialize_params {
image = data.google_compute_image.debian9.self_link
}
}
network_interface {
network = "default"
access_config {
// Ephemeral IP
}
}
}
And the same as Python code.
import terrascript
import terrascript.provider
import terrascript.resource
import terrascript.data
config = terrascript.Terrascript()
# Google Cloud Compute provider
config += terrascript.provider.google(
credentials='${file("account.json")}', project="myproject", region="us-central1"
)
# Google Compute Image (Debian 9) data source
config += terrascript.data.google_compute_image("image", family="debian-9")
# Add Google Compute Instance resource
config += terrascript.resource.google_compute_instance(
"myinstance",
name="myinstance",
machine_type="n1-standard-1",
zone="us-central1-a",
boot_disk={
"initialize_params": {"image": "data.google_compute_image.image.self_link"}
},
network_interface={"network": "default", "access_config": {}},
)
The example above is mostly a one-to-one adaptation of the HCL syntax. Let’s make some changes to show how generating Terraform configurations through Python-Terrascript may help.
- Define the Google Compute Image family and Google Compute Instance machine type at the beginning of the script so they are easier to change.
- Reference an instance of the Python-Terrascript class
terrascript.data.google_compute_image()
as the boot disk image.
IMAGE_FAMILY = "debian-9"
MACHINE_TYPE = "n1-standard-1"
import terrascript
import terrascript.provider
import terrascript.resource
import terrascript.data
config = terrascript.Terrascript()
# Google Cloud Compute provider
config += terrascript.provider.google(
credentials='${file("account.json")}', project="myproject", region="us-central1"
)
# Google Compute Image (Debian 9) data source
image = terrascript.data.google_compute_image("image", family=IMAGE_FAMILY)
config += image
# Add Google Compute Instance resource
config += terrascript.resource.google_compute_instance(
"myinstance",
name="myinstance",
machine_type=MACHINE_TYPE,
zone="us-central1-a",
boot_disk={"initialize_params": {"image": image.self_link}},
network_interface={"network": "default", "access_config": {}},
)
Variable¶
The terrascript.Variable
class can be used to define variables that can be referenced
later. Python-Terrascript automatically takes care of converting a reference to a Python
variable into the correct Terraform JSON syntax.
HCL
provider "aws" {
region = "us-east-1"
}
variable "image_id" {
type = string
}
resource "aws_instance" "example" {
instance_type = "t2.micro"
ami = var.image_id
}
Python
import terrascript
import terrascript.provider
import terrascript.resource
config = terrascript.Terrascript()
# AWS provider
config += terrascript.provider.aws(region="us-east-1")
# Define Variable and add to config
v = terrascript.Variable("image_id", type="string")
config += v
# AWS EC2 instance referencing the variable.
config += terrascript.resource.aws_instance(
"example",
instance_type="t2.micro",
ami=v,
)
JSON
In the output the reference to the image_id
has been converted from a reference to a
Python variable ami=v
to the correct Terraform JSON syntax of ${var.image_id}
.
{
"provider": {
"aws": [
{
"region": "us-east-1"
}
]
},
"variable": {
"image_id": {
"type": "string"
}
},
"resource": {
"aws_instance": {
"example": {
"instance_type": "t2.micro",
"ami": "${var.image_id}"
}
}
}
}
Output¶
Output is implemented as the terrascript.Output
class.
HCL
provider "aws" {
region = "us-east-1"
}
variable "image_id" {
type = string
}
resource "aws_instance" "example" {
instance_type = "t2.micro"
ami = var.image_id
}
output "instance_ip_addr" {
value = aws_instance.server.private_ip
description = "The private IP address of the instance."
}
Python
import terrascript
import terrascript.provider
import terrascript.resource
config = terrascript.Terrascript()
# AWS provider
config += terrascript.provider.aws(region="us-east-1")
# Define Variable and add to config
v = terrascript.Variable("image_id", type="string")
config += v
# Define AWS EC2 instance and add to config
i = terrascript.resource.aws_instance("example", instance_type="t2.micro", ami=v)
config += i
# Output the instance's private IP
config += terrascript.Output(
"instance_ip_addr",
value=i.private_ip,
description="The private IP address of the instance.",
)
JSON
{
"provider": {
"aws": [
{
"region": "us-east-1"
}
]
},
"variable": {
"image_id": {
"type": "string"
}
},
"resource": {
"aws_instance": {
"example": {
"instance_type": "t2.micro",
"ami": "${var.image_id}"
}
}
},
"output": {
"instance_ip_addr": {
"value": "aws_instance.example.private_ip",
"description": "The private IP address of the instance."
}
}
}
Module¶
Calls to other Terraform modules are implemented through the terrascript.Module
class.
HCL
module "ec2_cluster" {
source = "terraform-aws-modules/ec2-instance/aws"
version = "~> 2.0"
name = "my-cluster"
instance_count = 5
ami = "ami-ebd02392"
instance_type = "t2.micro"
key_name = "user1"
monitoring = true
vpc_security_group_ids = ["sg-12345678"]
subnet_id = "subnet-eddcdzz4"
}
Python
"""Terrascript module example based on https://registry.terraform.io/modules/terraform-aws-modules/ec2-instance/aws/"""
import terrascript
import terrascript.provider
config = terrascript.Terrascript()
# AWS provider
config += terrascript.provider.aws(region="us-east-1")
# AWS EC2 module
config += terrascript.Module(
"ec2_cluster",
source="terraform-aws-modules/ec2-instance/aws",
version="~> 2.0",
name="my-cluster",
instance_count=5,
ami="ami-ebd02392",
instance_type="t2.micro",
key_name="user1",
monitoring=True,
vpc_security_group_ids=["sg-12345678"],
subnet_id="subnet-eddcdzz4",
)
JSON
{
"provider": {
"aws": [
{
"region": "us-east-1"
}
]
},
"module": {
"ec2_cluster": {
"source": "terraform-aws-modules/ec2-instance/aws",
"version": "~> 2.0",
"name": "my-cluster",
"instance_count": 5,
"ami": "ami-ebd02392",
"instance_type": "t2.micro",
"key_name": "user1",
"monitoring": true,
"vpc_security_group_ids": [
"sg-12345678"
],
"subnet_id": "subnet-eddcdzz4"
}
}
}
Backend¶
HCL
terraform {
backend "local" {
path = "example_app/terraform_state"
}
}
Python
import terrascript
config = terrascript.Terrascript()
backend = terrascript.Backend(
"local",
path="example_app/terraform_state",
)
config += terrascript.Terraform(backend=backend)
JSON
{
"terraform": {
"backend": {
"local": {
"path": "example_app/terraform_state"
}
}
}
}
Connection¶
Locals¶
Terraform local values are not supported. Use Python variables instead.
Python
import terrascript
import terrascript.provider
import terrascript.resource
config = terrascript.Terrascript()
# AWS provider
config += terrascript.provider.aws(region="us-east-1")
# Local values as Python variables
tags = {"service_name": "forum", "owner": "Community Team"}
# Resource with two provisioners
config += terrascript.resource.aws_instance(
"instance1",
instance_type="t2.micro",
ami="ami-4bf3d731",
tags=tags,
)
JSON
{
"provider": {
"aws": [
{
"region": "us-east-1"
}
]
},
"resource": {
"aws_instance": {
"instance1": {
"instance_type": "t2.micro",
"ami": "ami-4bf3d731",
"tags": {
"service_name": "forum",
"owner": "Community Team"
}
}
}
}
}
Provisioner¶
One or multiple provisioners can be added to a Terraform resource. Multiple provisioners must be added as a Python list, a single provisioner can be either on its own or inside a list.
The example adds one “create” and one “destroy” provisioner.
Python
import terrascript
import terrascript.provider
import terrascript.resource
config = terrascript.Terrascript()
# AWS provider
config += terrascript.provider.aws(region="us-east-1")
# Provisioners
create = terrascript.Provisioner("local-exec", command="echo 'Create'")
destroy = terrascript.Provisioner(
"local-exec",
when="destroy",
command="echo 'Destroy'",
)
# Resource with two provisioners
config += terrascript.resource.aws_instance(
"instance1",
instance_type="t2.micro",
ami="ami-4bf3d731",
provisioner=[
create,
destroy,
],
)
JSON
{
"provider": {
"aws": [
{
"region": "us-east-1"
}
]
},
"resource": {
"aws_instance": {
"instance1": {
"instance_type": "t2.micro",
"ami": "ami-4bf3d731",
"provisioner": [
{
"local-exec": {
"command": "echo 'Create'"
}
},
{
"local-exec": {
"when": "destroy",
"command": "echo 'Destroy'"
}
}
]
}
}
}
}
List of providers¶
This document lists the Terraform providers that are currently supported by Terrascript.
Official providers¶
Official providers can be imported with or without namespace.
# With namespace
import provider.hashicorp.aws
import resource.hashicorp.aws
import data.hashicorp.aws
# Without namespace
import provider.aws
import resource.aws
import data.aws
Terrascript currently supports the following official Terraform providers.
- ad (hashicorp/ad/0.4.3)
- alicloud (hashicorp/alicloud/1.136.0)
- archive (hashicorp/archive/2.2.0)
- aws (hashicorp/aws/3.60.0)
- azuread (hashicorp/azuread/2.4.0)
- azurerm (hashicorp/azurerm/2.78.0)
- azurestack (hashicorp/azurestack/0.10.0)
- boundary (hashicorp/boundary/1.0.5)
- ciscoasa (hashicorp/ciscoasa/1.2.0)
- cloudinit (hashicorp/cloudinit/2.2.0)
- consul (hashicorp/consul/2.13.0)
- dns (hashicorp/dns/3.2.1)
- external (hashicorp/external/2.1.0)
- fakewebservices (hashicorp/fakewebservices/0.2.1)
- github (hashicorp/github/4.15.1)
- google (hashicorp/google/3.85.0)
- google-beta (hashicorp/google-beta/3.85.0)
- googleworkspace (hashicorp/googleworkspace/0.4.1)
- hcp (hashicorp/hcp/0.17.0)
- hcs (hashicorp/hcs/0.5.0)
- helm (hashicorp/helm/2.3.0)
- http (hashicorp/http/2.1.0)
- kubernetes (hashicorp/kubernetes/2.5.0)
- kubernetes-alpha (hashicorp/kubernetes-alpha/0.6.0)
- local (hashicorp/local/2.1.0)
- nomad (hashicorp/nomad/1.4.15)
- null (hashicorp/null/3.1.0)
- oci (hashicorp/oci/4.45.0)
- opc (hashicorp/opc/1.4.1)
- oraclepaas (hashicorp/oraclepaas/1.5.3)
- random (hashicorp/random/3.1.0)
- template (hashicorp/template/2.2.0)
- terraform (hashicorp/terraform/1.0.2)
- tfe (hashicorp/tfe/0.26.1)
- time (hashicorp/time/0.7.2)
- tls (hashicorp/tls/3.1.0)
- vault (hashicorp/vault/2.24.0)
- vsphere (hashicorp/vsphere/2.0.2)
Partner providers¶
Partner providers can be imported with or without namespace.
# With namespace
import provider.akamai.akamai
import resource.akamai.akamai
import data.akamai.akamai
# Without namespace
import provider.akamai
import resource.akamai
import data.akamai
Terrascript currently supports the following partner Terraform providers.
- akamai (akamai/akamai/1.7.0)
- alicloud (aliyun/alicloud/1.136.0)
- amixr (alertmixer/amixr/0.2.3)
- avi (vmware/avi/21.1.1)
- aviatrix (AviatrixSystems/aviatrix/2.20.0)
- azurecaf (aztfmod/azurecaf/1.2.6)
- bigip (F5Networks/bigip/1.11.1)
- brightbox (brightbox/brightbox/2.0.6)
- circonus (circonus-labs/circonus/0.12.2)
- cloudflare (cloudflare/cloudflare/3.1.0)
- cloudsmith (cloudsmith-io/cloudsmith/0.0.6)
- databricks (databrickslabs/databricks/0.3.7)
- datadog (DataDog/datadog/3.4.0)
- digitalocean (digitalocean/digitalocean/2.12.0)
- exoscale (exoscale/exoscale/0.29.0)
- fastly (fastly/fastly/0.35.0)
- fortimanager (fortinetdev/fortimanager/1.3.4)
- fortios (fortinetdev/fortios/1.13.1)
- gridscale (gridscale/gridscale/1.13.0)
- hcloud (hetznercloud/hcloud/1.31.1)
- heroku (heroku/heroku/4.6.0)
- launchdarkly (launchdarkly/launchdarkly/2.0.1)
- linode (linode/linode/1.21.0)
- mongodbatlas (mongodb/mongodbatlas/1.0.1)
- ncloud (NaverCloudPlatform/ncloud/2.1.2)
- netapp-cloudmanager (NetApp/netapp-cloudmanager/21.9.2)
- netapp-elementsw (NetApp/netapp-elementsw/20.11.0)
- netapp-gcp (NetApp/netapp-gcp/21.9.0)
- newrelic (newrelic/newrelic/2.25.0)
- ns1 (ns1-terraform/ns1/1.12.1)
- nsxt (vmware/nsxt/3.2.4)
- oktaasa (oktadeveloper/oktaasa/1.0.1)
- onelogin (onelogin/onelogin/0.1.23)
- pagerduty (PagerDuty/pagerduty/1.11.0)
- pnap (phoenixnap/pnap/0.8.0)
- rancher2 (rancher/rancher2/1.20.0)
- rke (rancher/rke/1.2.3)
- scaleway (scaleway/scaleway/2.1.0)
- sdm (strongdm/sdm/1.0.28)
- sematext (sematext/sematext/0.4.0)
- signalfx (splunk-terraform/signalfx/6.7.7)
- stackpath (stackpath/stackpath/1.3.3)
- sumologic (SumoLogic/sumologic/2.10.0)
- tencentcloud (tencentcloudstack/tencentcloud/1.59.4)
- transloadit (transloadit/transloadit/0.4.0)
- triton (joyent/triton/0.8.2)
- turbot (turbot/turbot/1.8.2)
- vcd (vmware/vcd/3.3.1)
- vmc (vmware/vmc/1.7.0)
- vra (vmware/vra/0.3.11)
- vra7 (vmware/vra7/3.0.2)
- vultr (vultr/vultr/2.4.2)
- wavefront (vmware/wavefront/3.0.0)
Community providers¶
Community providers must be imported with namespace.
# With namespace
import provider.innovationnorway.git
Terrascript currently supports the following community Terraform providers.
- activedirectory (PortOfPortland/activedirectory/0.5.1-pre)
- ad (techBeck03/ad/0.4.3-patch)
- airtable (paultyng/airtable/0.1.0)
- ansiblevault (MeilleursAgents/ansiblevault/2.2.0)
- appdynamics (HarryEMartland/appdynamics/0.1.0)
- artifactory (cappyzawa/artifactory/2.2.15)
- asana (davidji99/asana/0.1.2)
- awslambda (dedunumax/awslambda/1.0.6)
- awsx (phzietsman/awsx/1.0.0)
- awx (mrcrilly/awx/0.1.2)
- azure-preview (innovationnorway/azure-preview/0.1.0-alpha.3)
- azuredevops (ellisdon-oss/azuredevops/0.0.2)
- berglas (sethvargo/berglas/0.2.0)
- bitbucketserver (gavinbunney/bitbucketserver/1.5.0)
- bless (chanzuckerberg/bless/0.5.0)
- cheesecake (joerx/cheesecake/0.2.3)
- christmas-tree (cappyzawa/christmas-tree/0.5.2)
- circleci (TomTucka/circleci/0.4.0)
- cloudknox (cloudknox/cloudknox/0.6.0)
- concourse (cappyzawa/concourse/0.1.2)
- confluentcloud (Mongey/confluentcloud/0.0.12)
- cosmic (MissionCriticalCloud/cosmic/0.5.0)
- cronitor (nauxliu/cronitor/1.0.8)
- ct (poseidon/ct/0.9.0)
- dbussecretservice (abergmeier/dbussecretservice/0.0.6)
- dmsnitch (plukevdh/dmsnitch/0.1.2)
- dns (someara/dns/2.3.0-pre)
- dnsimple (bgpat/dnsimple/0.5.1)
- domeneshop (innovationnorway/domeneshop/0.1.0)
- ecloud (ukfast/ecloud/2.0.0)
- eksctl (mumoshu/eksctl/0.16.2)
- elasticsearch (phillbaker/elasticsearch/2.0.0-beta.1)
- errorcheck (jb-abbadie/errorcheck/2.0.4)
- esxi (josenk/esxi/1.8.3)
- exasol (abergmeier/exasol/0.0.23)
- filesystem (sethvargo/filesystem/0.2.0)
- fortiadc (Ouest-France/fortiadc/0.3.2)
- freeipa (camptocamp/freeipa/0.7.0)
- geoserver (camptocamp/geoserver/0.0.3)
- git (innovationnorway/git/0.1.3)
- git (paultyng/git/0.1.0)
- gpg (invidian/gpg/0.3.0)
- graylog (terraform-provider-graylog/graylog/1.0.4)
- grid5000 (pmorillon/grid5000/0.0.7)
- gsuite (DeviaVir/gsuite/0.1.62)
- gsuite (paultyng/gsuite/0.2.2)
- guacamole (techBeck03/guacamole/1.2.7)
- harbor (Ouest-France/harbor/0.2.0)
- hashicups (hashicorp/hashicups/0.3.1)
- hdns (alxrem/hdns/0.2.0)
- hellosign (Mongey/hellosign/0.0.2)
- helmfile (mumoshu/helmfile/0.14.1)
- herokux (davidji99/herokux/0.30.3)
- hetznerdns (timohirt/hetznerdns/1.1.1)
- honeycombio (kvrhdn/honeycombio/0.1.4)
- hsdp (philips-software/hsdp/0.20.5)
- idm (DTherHtun/idm/0.0.2)
- infoblox (techBeck03/infoblox/2.0.7)
- iptables (jeremmfr/iptables/1.2.0)
- itop (Ouest-France/itop/0.6.1)
- javascript (apparentlymart/javascript/0.0.1)
- jenkins (taiidani/jenkins/0.9.0)
- jetstream (nats-io/jetstream/0.0.26)
- jsonnet (alxrem/jsonnet/1.0.3)
- junos (jeremmfr/junos/1.20.0)
- jwt (camptocamp/jwt/0.0.3)
- k8s (banzaicloud/k8s/0.9.1)
- kafka (Mongey/kafka/0.4.1)
- kafka-connect (Mongey/kafka-connect/0.2.3)
- kibana (mayjak/kibana/0.7.1)
- kind (unicell/kind/0.0.2-u2)
- kubectl (gavinbunney/kubectl/1.11.3)
- kubectl (mumoshu/kubectl/0.2.0)
- kubeflowpipelines (datarootsio/kubeflowpipelines/0.0.10)
- lastpass (nrkno/lastpass/0.5.3)
- ldap (Ouest-France/ldap/0.7.2)
- libvirt (invidian/libvirt/0.6.10-rc1)
- loadbalancer (ukfast/loadbalancer/1.0.0-alpha1)
- lvslb (jeremmfr/lvslb/1.1.0)
- lvsnetwork (jeremmfr/lvsnetwork/1.2.0)
- matchbox (poseidon/matchbox/0.4.1)
- middesk (Mongey/middesk/0.0.2)
- mikrotik (ddelnano/mikrotik/0.8.0)
- msgraph (yaegashi/msgraph/0.0.5)
- mssql (drarko/mssql/0.0.4)
- netbox (e-breuninger/netbox/0.2.4)
- netbox (innovationnorway/netbox/0.1.0-alpha.2)
- njalla (Sighery/njalla/0.9.1)
- nomadutility (AdrienneCohea/nomadutility/0.0.14)
- null (mildred/null/1.1.0)
- okta (chanzuckerberg/okta/3.10.3)
- oktafork (gavinbunney/oktafork/3.12.9)
- openshift (llomgui/openshift/1.1.0)
- opnsense (kradalby/opnsense/0.0.2-pre)
- opsgenie (opsgenie/opsgenie/0.6.5)
- orion (stobias123/orion/0.3.2)
- outlook (magodo/outlook/0.0.4)
- ovh (invidian/ovh/0.9.3)
- petstore (DTherHtun/petstore/1.0.1)
- phpipam (Ouest-France/phpipam/0.6.0)
- pingaccess (iwarapter/pingaccess/0.8.0)
- pingdom (nauxliu/pingdom/1.1.2)
- pingfederate (iwarapter/pingfederate/0.0.21)
- po (greg-gajda/po/1.0.0)
- postgresql (cyrilgdn/postgresql/1.14.0)
- puppetca (camptocamp/puppetca/1.3.0)
- puppetdb (camptocamp/puppetdb/1.2.0)
- pypi (jeffwecan/pypi/0.0.9)
- qingcloud (shaowenchen/qingcloud/1.2.6)
- rabbitmq (cyrilgdn/rabbitmq/1.6.0)
- rancher (eLobeto/rancher/1.5.1)
- restapi (gavinbunney/restapi/1.15.4)
- rollbar (davidji99/rollbar/1.5.1)
- sakuracloud (sacloud/sakuracloud/2.12.0)
- scaffolding (iwarapter/scaffolding/0.0.1)
- scaffolding (unicell/scaffolding/0.0.2)
- selectel (selectel/selectel/3.6.2)
- sendgrid (davidji99/sendgrid/0.1.1)
- sendgrid (Trois-Six/sendgrid/0.1.6)
- sentry (jianyuan/sentry/0.6.0)
- seq (innovationnorway/seq/0.1.0-alpha.5)
- shell (scottwinkler/shell/1.7.7)
- shellescape (bgpat/shellescape/0.0.2)
- snowflake (chanzuckerberg/snowflake/0.25.19)
- sops (carlpett/sops/0.6.3)
- spinnaker (mercari/spinnaker/0.3.0)
- split (davidji99/split/0.2.0)
- sql (paultyng/sql/0.4.0)
- sshcommand (invidian/sshcommand/0.2.2)
- statuspage (yannh/statuspage/0.1.7)
- stdlib (invidian/stdlib/0.1.1)
- sys (mildred/sys/1.3.25)
- systemd (mildred/systemd/0.0.1)
- teamcity (jeffwecan/teamcity/1.0.1-jeffwecan-fork)
- testing (apparentlymart/testing/0.0.2)
- tfvars (innovationnorway/tfvars/0.0.1)
- tls (invidian/tls/2.2.1)
- tls (someara/tls/2.3.0-pre)
- tozny (tozny/tozny/0.14.0)
- transip (aequitas/transip/0.1.11)
- travis (bgpat/travis/0.1.6)
- twitter (paultyng/twitter/0.1.0)
- ucloud (ucloud/ucloud/1.29.0)
- ucodecov (at-wat/ucodecov/0.1.2)
- ultradns (davidji99/ultradns/2.1.0)
- unifi (paultyng/unifi/0.34.0)
- uptimerobot (invidian/uptimerobot/0.5.1-gb83a310)
- utravis (kamatama41/utravis/0.5.0)
- vault (cyrilgdn/vault/2.23.1)
- vault (jeffwecan/vault/2.11.0-withsleep)
- vaultutility (AdrienneCohea/vaultutility/0.0.3)
- vinyldns (vinyldns/vinyldns/0.9.5)
- virtualbox (terra-farm/virtualbox/0.2.2-alpha.1)
- vsphere (techBeck03/vsphere/1.24.3-patch)
- windns (PortOfPortland/windns/0.5.3)
- xenorchestra (terra-farm/xenorchestra/0.21.0)
- zerotier (bltavares/zerotier/0.3.0)
Unsupported providers¶
The following providers are not supported.
- aiven (juniorz/aiven/0.1.0) - Failed to initialise provider
- appd (maskerade/appd/0.0.5) - Failed to initialise provider
- flexkube (invidian/flexkube/0.3.3) - Failed to initialise provider
- googlecalendar (sethvargo/googlecalendar/0.3.1) - Failed to initialise provider
- grafana (58231/grafana/0.0.2) - 58231 is not a valid Python identifier
- ipam (beauknowssoftware/ipam/0.2.6) - Failed to initialise provider
- k8s (mingfang/k8s/1.0.2) - Failed to process provider
- kind1 (unicell/kind1/0.0.2-u2) - Failed to initialise provider
- lxd (arren-ru/lxd/1.4.0) - Failed to initialise provider
- null (paultyng/null/0.1.1) - Failed to initialise provider
- null (ptyng/null/0.1.1) - Failed to initialise provider
- okta (gavinbunney/okta/3.12.7) - Failed to initialise provider
- okta (oktadeveloper/okta/3.13.13) - Failed to initialise provider
- pass (camptocamp/pass/2.0.0) - pass is a Python keyword
- safedns (ukfast/safedns/1.1.2) - Failed to process provider
- shakenfist (shakenfist/shakenfist/0.2.5) - Failed to initialise provider
- unofficial-travis (kamatama41/unofficial-travis/0.5.0) - Failed to initialise provider
- vmworkstation (elsudano/vmworkstation/0.2.3) - Failed to process provider
- zerotier (someara/zerotier/0.1.47) - Failed to initialise provider
Terraform JSON format¶
The following sections show the JSON output Python-Terrascript generates for the different Terraform elements.
Resource¶
Resources are coded as nested dictionaries. The top-level key is the type of the resource. The second-level key is the name of a resource instance.
Example:
- Two
aws_instance
resources namedinstance1
andinstance2
- One
aws_subnet
resource namedsubnet1
.
{
"resource": {
"aws_instance": {
"instance1": {
"instance_type": "t2.micro"
},
"instance2": {
"instance_type": "t2.micro"
}
},
"aws_subnet": {
"subnet1" {
"availability_zone": "usa-west-2a"
}
}
}
}
Provider¶
A Terraform configuration contains one or more provider sections. Unlike
resources and data sources providers don’t have a name and are distinguised
by their alias
attribute instead. Therefore multiple instances of the same
provider type are encoded as a list.
Example:
- Two
aws
providers with aliasesaws1
andaws2
. - One
google
provider with aliasgoogle1
.
{
"provider": {
"aws": [
{
"region": "us-east-1",
"alias": "aws1"
},
{
"region": "us-east-2",
"alias": "aws2"
}
],
"google": [
{
"region": "us-central-1",
"alias": "google1"
}
]
}
}
Variable¶
Variables are encoded as a dictionary whose keys are the names of the variables.
Example:
- Two variables names
image_id
andavailability_zone_names
.
{
"variable": {
"image_id": {
"type": "string"
},
"availability_zone_names": {
"type": "list(string)",
"default": [
"us-west-1a",
"us-west-1b"
]
}
}
}
Variable references¶
When using a variable as a attribute value for a object, a reference are used.
Example:
* Variable image_id
are used as name for instance.
{
"variable": {
"image_id": {
"type": "string"
}
},
"resource": {
"aws_instance": {
"instance1": {
"instance_type": "${var.image_id}"
}
}
}
}
Output Values¶
Output values are encoded as a dictionary whose keys are the names of the value.
Example:
- Output values for two resources.
{
"output": {
"instance1_ip_addr": {
"value": "instance1.server.private_ip"
},
"instance2_ip_addr": {
"value": "instance2.server.private_ip"
},
}
}
Local Values¶
Local values are encoded as a dictionary whose keys are the names of the value.
Example:
{
"locals": {
"service_name": "forum",
"owner": "Community Team",
"Service": "local.service_name",
"Owner": "local.owner"
}
}
Modules¶
Module calls are dictionaries keyed by the name of the module and module arguments as values.
Note
In contrast to calling existing modules, creating modules is not supported by Python-Terrascript as Python functions could be used as an alternative.
Example:
- Calling module
vpc
.
{
"module": {
"vpc": {
"source": "terraform-aws-modules/vpc/aws",
"version": "2.9.0"
}
}
}
Data Sources¶
Data sources are coded as nested dictionaries. The top-level key is the type of the resource. The second-level key is the name of the data source.
Example:
- Two
aws_ami
data sources namedami1
andami2
.
{
"data": {
"aws_ami": {
"ami1": {
"most_recent": true
},
"ami2": {
"most_recent": true
},
}
}
}
Expressions¶
Functions¶
Functions are encoded as text. Example: "content": "file('hello_world.txt')"
.
Terraform Settings¶
Terraform settings are a simple dictionary although the values of settings may contain nested data structures.
Example:
- Terraform backend configuration.
{
"terraform": {
"backend": {
"s3": {
"bucket": "mybucket"
}
}
}
}
Frequently Asked Questions¶
Questions I frequently asked myself ;-)
Why no error checking?¶
Python-Terrascript does not perform any error checking whatsoever! This was a deliberate design decision to keep the code simple. Therefore it is perfectly possible to generate JSON output that Terraform will later reject.
import terrascript
import terrascript.provider
import terrascript.resource
config = terrascript.Terrascript()
config += terrascript.provider.aws(region='us-east-1')
i = terrascript.resource.aws_instance('myinstance', foo='bar')
config += i
# AWS Instance resource does not have a `foo` argument but accepts it anyway.
assert i.foo == 'bar'
{
"provider": {
"aws": [
{
"region": "us-east-1"
}
]
},
"resource": {
"aws_instance": {
"myinstance": {
"foo": "bar"
}
}
}
Terraform will reject the generated JSON as the aws_instance resource does not accept the foo argument.
At an early stage I contemplated parsing the Terraform Go source code and auto-create Python code that does indeed verify whether the generated JSON configuration is valid Terraform input. This attempt proved much too difficult so I abandonded that approach.
Why is provider XYZ not supported?¶
All provider specific code is auto-generated through the tools/makecode.py
script
based on the list of providers in tools/providers.yml
. So if a provider is missing
it simply has not yet been added to tools/providers.yml
.
If you believe a provider is missing simply open a new issue or submit a pull
request with the missing provider added to tools/providers.yml
.
Why is there sometimes so little progress in this project?¶
Python-Terrascript is a hobby project which I can only work on in my very limited spare time.
Professionally I am a Systems and Network Engineer working on an Intelligent Transport Systems project. None of its infrastructure is in the Cloud, it’s all very physcially located on the freeways around Melbourne, Australia. Until someone writes Terraform modules for traffic cameras or vehicle sensors my manager simply wouldn’t appreciate if I spent time on Python-Terrascript during work hours.
Is Python-Terrascript better than writing configurations by hand?¶
I regard Python-Terrascript as an alternative to writing Terraform configurations by hand. Whether it is better or not is for everyone to decide for themselves.
How can I contribute to the project?¶
There a various ways you can contribute to the development of Python-Terrascript.
- Issues: Do not hestiate to raise an issue if you believe you found a bug or simply have a question. As the maintainer of a small Github project it is amazingly satisfying to see that there are other people out there who find Python-Terrascript useful. Do not get discouraged if I don’t respond immediately. As mentioned earlier I can only spend limited time on Python-Terrascript. You will hear from me eventually.
- Pull Requests: Yes, Pull Requests are welcome but please bear in my that this is a personal project and not a community project. I promise to provide an explanation if should I reject a Pull Request.
- Documentation: Documentation can always be improved so any support (Issues, Pull Requests) is very welcome. The biggest problem with documentation is always what may be obvious to me may not be obvious to the reader at all. I also tend to be rather terse when writing documentation which is not a good thing.
- Examples: I would like to include a collection of real-world examples of how Python-Terrascript is used. So if you are willing to share your code please come forward.
- Drinks: Anyone willing to catch-up for a chat over coffee (hot chocolate in my case) or beer when they are in Melbourne?
Are there any alternatives to Python-Terrascript?¶
I know that there are comparable projects to Python-Terrascript. I just haven’t managed to compile a list yet. Please stand-by for updates…