Full Module Docs

kingpin.actors.aws.base

The AWS Actors allow you to interact with the resources (such as SQS and ELB) inside your Amazon AWS account. These actors all support dry runs properly, but each actor has its own caveats with dry=True. Please read the instructions below for using each actor.

Required Environment Variables

_Note, these can be skipped only if you have a .aws/credentials file in place._

AWS_ACCESS_KEY_ID:  Your AWS access key AWS_SECRET_ACCESS_KEY:  Your AWS secret

exception kingpin.actors.aws.base.ELBNotFound

Raised when an ELB is not found

exception kingpin.actors.aws.base.InvalidMetaData

Raised when fetching AWS metadata.

exception kingpin.actors.aws.base.InvalidPolicy

Raised when Amazon indicates that policy JSON is invalid.

class kingpin.actors.aws.base.EnsurableAWSBaseActor(*args, **kwargs)

Ensurable version of the AWS Base Actor

kingpin.actors.aws.cloudformation

exception kingpin.actors.aws.cloudformation.CloudFormationError

Raised on any generic CloudFormation error.

exception kingpin.actors.aws.cloudformation.StackFailed

Raised any time a Stack fails to be created or updated.

exception kingpin.actors.aws.cloudformation.InvalidTemplate

An invalid CloudFormation template was supplied.

exception kingpin.actors.aws.cloudformation.StackAlreadyExists

The requested CloudFormation stack already exists.

exception kingpin.actors.aws.cloudformation.StackNotFound

The requested CloudFormation stack does not exist.

class kingpin.actors.aws.cloudformation.ParametersConfig

Validates the Parameters option.

A valid parameters option is a dictionary with simple Key/Value pairs of strings. No nested dicts, arrays or other objects.

class kingpin.actors.aws.cloudformation.CapabilitiesConfig

Validates the Capabilities option.

The capability options currently available are CAPABILITY_IAM and CAPABILITY_NAMED_IAM, either of which can be used to grant a Stack the capability to create IAM resources. You must use CAPABILITY_NAMED_IAM to create IAM resources with custom names.

class kingpin.actors.aws.cloudformation.OnFailureConfig

Validates the On Failure option.

The on_failure option can take one of the following settings: DO_NOTHING, ROLLBACK, DELETE

This option is applied at stack _creation_ time!

class kingpin.actors.aws.cloudformation.TerminationProtectionConfig

Validates the TerminationProtectionConfig option.

The enable_termination_protection option can take one of the following settings: 'UNCHANGED', False, True

UNCHANGED means on Create Stack it will default to False, however on

Ensure Stack no changes will be applied.

class kingpin.actors.aws.cloudformation.CloudFormationBaseActor(*args, **kwargs)

Base Actor for CloudFormation tasks

class kingpin.actors.aws.cloudformation.Create(*args, **kwargs)

Creates a CloudFormation stack.

Creates a CloudFormation stack from scratch and waits until the stack is fully built before exiting the actor.

Options

Name:

The name of the queue to create

Capabilities:

A list of CF capabilities to add to the stack.

On_failure:

(OnFailureConfig)

One of the following strings: DO_NOTHING, ROLLBACK, DELETE

Default: DELETE

Parameters:

A dictionary of key/value pairs used to fill in the parameters for the CloudFormation template.

Region:

AWS region (or zone) string, like ‘us-west-2’.

Role_arn:

The Amazon IAM Role to use when executing the stack.

Template:

String of path to CloudFormation template. Can either be in the form of a local file path (ie, /my_template.json) or a URI (ie https://my_site.com/cf.json).

Timeout_in_minutes:  

The amount of time that can pass before the stack status becomes CREATE_FAILED.

Enable_termination_protection:  

Whether termination protection is enabled for the stack.

Examples

{ "desc": "Create production backend stack",
  "actor": "aws.cloudformation.Create",
  "options": {
    "capabilities": [ "CAPABILITY_IAM" ],
    "name": "%CF_NAME%",
    "parameters": {
      "test_param": "%TEST_PARAM_NAME%",
    },
    "region": "us-west-1",
    "role_arn": "arn:aws:iam::123456789012:role/DeployRole",
    "template": "/examples/cloudformation_test.json",
    "timeout_in_minutes": 45,
    "enable_termination_protection": true,
  }
}

Dry Mode

Validates the template, verifies that an existing stack with that name does not exist. Does not create the stack.

class kingpin.actors.aws.cloudformation.Delete(*args, **kwargs)

Deletes a CloudFormation stack

Options

Name:The name of the queue to create Region:AWS region (or zone) string, like ‘us-west-2’

Examples

{ "desc": "Delete production backend stack",
  "actor": "aws.cloudformation.Create",
  "options" {
    "region": "us-west-1",
    "name": "%CF_NAME%",
  }
}

Dry Mode

Validates that the CF stack exists, but does not delete it.

class kingpin.actors.aws.cloudformation.Stack(*args, **kwargs)

Manages the state of a CloudFormation stack.

This actor can manage the following aspects of a CloudFormation stack in Amazon:

• Ensure that the Stack is present or absent.
• Monitor and update the stack Template and Parameters as necessary.

Default Parameters

If your CF stack defines parameters with defaults, Kingpin will use the defaults unless the parameters are explicitly specified.

NoEcho Parameters

If your CF stack takes a Password as a parameter or any other value thats secret and you set NoEcho: True on that parameter, Kingpin will be unable to diff it and compare whether or not the desired setting matches whats in Amazon. A warning will be thrown, and the rest of the actor will continue to operate as normal.

If any other difference triggers a Stack Update, the desired value for the parameter with NoEcho: True will be pushed in addition to all of the other stack parameters.

Options

Name:

The name of the queue to create

State:

(str) Present or Absent. Default: “present”

Capabilities:

(CapabilitiesConfig, None)

A list of CF capabilities to add to the stack.

Disable_rollback:  

Set to True to disable rollback of the stack if creation failed.

On_failure:

(OnFailureConfig, None)

One of the following strings: DO_NOTHING, ROLLBACK, DELETE

Default: DELETE

Parameters:

(ParametersConfig, None)

A dictionary of key/value pairs used to fill in the parameters for the CloudFormation template.

Region:

AWS region (or zone) string, like ‘us-west-2’.

Role_arn:

The Amazon IAM Role to use when executing the stack.

Template:

String of path to CloudFormation template. Can either be in the form of a local file path (ie, /my_template.json) or a URI (ie https://my_site.com/cf.json).

Timeout_in_minutes:  

The amount of time that can pass before the stack status becomes CREATE_FAILED.

Enable_termination_protection:  

Whether termination protection is enabled for the stack.

Examples

{ "actor": "aws.cloudformation.Create",
  "state": "present",
  "options": {
    "capabilities": [ "CAPABILITY_IAM" ],
    "on_failure": "DELETE",
    "name": "%CF_NAME%",
    "parameters": {
      "test_param": "%TEST_PARAM_NAME%",
    },
    "region": "us-west-1",
    "role_arn": "arn:aws:iam::123456789012:role/DeployRole",
    "template": "/examples/cloudformation_test.json",
    "timeout_in_minutes": 45,
    "enable_termination_protection": true,
  }
}

Dry Mode

Validates the template, verifies that an existing stack with that name does not exist. Does not create the stack.

kingpin.actors.aws.elb

exception kingpin.actors.aws.elb.CertNotFound

Raised when an ELB is not found

kingpin.actors.aws.elb.p2f(string)

Convert percentage string into float.

Converts string like ‘78.9%’ into 0.789

class kingpin.actors.aws.elb.ELBBaseActor(*args, **kwargs)

Base class for ELB actors.

class kingpin.actors.aws.elb.WaitUntilHealthy(*args, **kwargs)

Wait indefinitely until a specified ELB is considered “healthy”.

This actor will loop infinitely until a healthy threshold of the ELB is met. The threshold can be reached when the count as specified in the options is less than or equal to the number of InService instances in the ELB.

Another situation is for count to be a string specifying a percentage (see examples). In this case the percent of InService instances has to be greater than the count percentage.

Options

Name:The name of the ELB to operate on Count:Number, or percentage of InService instance to consider this ELB healthy Region:AWS region (or zone) name, such as us-east-1 or us-west-2

Examples

{ "actor": "aws.elb.WaitUntilHealthy",
  "desc": "Wait until production-frontend has 16 hosts",
  "options": {
    "name": "production-frontend",
    "count": 16,
    "region": "us-west-2"
  }
}

{ "actor": "aws.elb.WaitUntilHealthy",
  "desc": "Wait until production-frontend has 85% of hosts in-service",
  "options": {
    "name": "production-frontend",
    "count": "85%",
    "region": "us-west-2"
  }
}

Dry Mode

This actor performs the finding of the ELB as well as calculating its health at all times. The only difference in dry mode is that it will not re-count the instances if the ELB is not healthy. A log message will be printed indicating that the run is dry, and the actor will exit with success.

class kingpin.actors.aws.elb.SetCert(*args, **kwargs)

Find a server cert in IAM and use it for a specified ELB.

Options

Region:(str) AWS region (or zone) name, like us-west-2 Name:(str) Name of the ELB Cert_name:(str) Unique IAM certificate name, or ARN Port:(int) Port associated with the cert. (default: 443)

Example

{ "actor": "aws.elb.SetCert",
  "desc": "Run SetCert",
  "options": {
    "cert_name": "new-cert",
    "name": "some-elb",
    "region": "us-west-2"
  }
}

Dry run

Will check that ELB and Cert names are existent, and will also check that the credentials provided for AWS have access to the new cert for ssl.

class kingpin.actors.aws.elb.RegisterInstance(*args, **kwargs)

Add an EC2 instance to a load balancer or target group.

Options

Elb:(str) Name of the ELB or the Target Group ARN Instances:(str, list) Instance id, or list of ids. Default “self” id. Region:(str) AWS region (or zone) name, like us-west-2 Enable_zones:(bool) add all available AZ to the elb. Default: True

Example

{ "actor": "aws.elb.RegisterInstance",
  "desc": "Run RegisterInstance",
  "options": {
    "elb": "prod-loadbalancer",
    "instances": "i-123456",
    "region": "us-east-1",
  }
}

---
actor: aws.elb.RegisterInstance
desc: Run RegisterInstance
options:
  elb: prod-loadbalancer
  instances: i-123456
  region: us-east-1

Dry run

Will find the specified ELB, but not take any actions regarding instances.

class kingpin.actors.aws.elb.DeregisterInstance(*args, **kwargs)

Remove EC2 instance(s) from an ELB.

Options

Elb:(str) Name of the ELB. Optionally this may also be a *. Instances:(str, list) Instance id, or list of ids Region:(str) AWS region (or zone) name, like us-west-2 Wait_on_draining:  (bool) Whether or not to wait for connection draining

Example

{ "actor": "aws.elb.DeregisterInstance",
  "desc": "Run DeregisterInstance",
  "options": {
    "elb": "my-webserver-elb",
    "instances": "i-abcdeft",
    "region": "us-west-2"
  }
}

---
actor: aws.elb.DeregisterInstance
desc: Run DeregisterInstance
options:
  elb: prod-loadbalancer
  instances: i-123456
  region: us-east-1

Extremely simple way to remove the local instance running this code from all ELBs its been joined to:

{ "actor": "aws.elb.DeregisterInstance",
  "desc": "Run DeregisterInstance",
  "options": {
    "elb": "*",
    "region": "us-west-2"
  }
}

Dry run

Will find the ELB but not take any actions regarding the instances.

kingpin.actors.aws.iam

class kingpin.actors.aws.iam.User(*args, **kwargs)

Manages an IAM User.

This actor manages the state of an Amazon IAM User.

Currently we can:

• Ensure is present or absent
• Manage the inline policies for the user
• Manage the groups the user is in

Options

Name:(str) Name of the User profile to manage State:(str) Present or Absent. Default: “present” Groups:(str,array) A list of groups for the user to be a member of. Default: None Inline_policies:  (str,array) A list of strings that point to JSON files to use as inline policies. Default: None

Example

{ "actor": "aws.iam.User",
  "desc": "Ensure that Bob exists",
  "options": {
    "name": "bob",
    "state": "present",
    "groups": "my-test-group",
    "inline_policies": [
      "read-all-s3.json",
      "create-other-stuff.json"
    ]
  }
}

Dry run

Will let you know if the user exists or not, and what changes it would make to the users policy and settings. Will also parse the inline policies supplied, make sure any tokens in the files are replaced, and that the files are valid JSON.

class kingpin.actors.aws.iam.Group(*args, **kwargs)

Manages an IAM Group.

This actor manages the state of an Amazon IAM Group.

Currently we can:

• Ensure is present or absent
• Manage the inline policies for the group
• Purge (or not) all group members and delete the group

Options

Name:(str) Name of the Group profile to manage Force:(bool) Forcefully delete the group (explicitly purging all group memberships). Default: false State:(str) Present or Absent. Default: “present” Inline_policies:  (str,array) A list of strings that point to JSON files to use as inline policies. You can also pass in a single inline policy as a string. Default: None

Example

{ "actor": "aws.iam.Group",
  "desc": "Ensure that devtools exists",
  "options": {
    "name": "devtools",
    "state": "present",
    "inline_policies": [
      "read-all-s3.json",
      "create-other-stuff.json"
    ]
  }
}

Dry run

Will let you know if the group exists or not, and what changes it would make to the groups policy and settings. Will also parse the inline policies supplied, make sure any tokens in the files are replaced, and that the files are valid JSON.

class kingpin.actors.aws.iam.Role(*args, **kwargs)

Manages an IAM Role.

This actor manages the state of an Amazon IAM Role.

Currently we can:

• Ensure is present or absent
• Manage the inline policies for the role
• Manage the Assume Role Policy Document

Options

Name:(str) Name of the Role to manage State:(str) Present or Absent. Default: “present” Inline_policies:  (str,array) A list of strings that point to JSON files to use as inline policies. You can also pass in a single inline policy as a string. Default: None Assume_role_policy_document:  (str) A string with an Amazon IAM Assume Role policy. Not providing this causes Kingpin to ignore the value, and Amazon defaults the role to an ‘EC2’ style rule. Supplying the document will cause Kingpin to ensure the assume role policy is correct. Default: None

Example

{ "actor": "aws.iam.Role",
  "desc": "Ensure that myapp exists",
  "options": {
    "name": "myapp",
    "state": "present",
    "inline_policies": [
      "read-all-s3.json",
      "create-other-stuff.json"
    ]
  }
}

Dry run

Will let you know if the group exists or not, and what changes it would make to the groups policy and settings. Will also parse the inline policies supplied, make sure any tokens in the files are replaced, and that the files are valid JSON.

class kingpin.actors.aws.iam.InstanceProfile(*args, **kwargs)

Manages an IAM Instance Profile.

This actor manages the state of an Amazon IAM Instance Profile.

Currently we can:

• Ensure is present or absent
• Assign an IAM Role to the Instance Profile

Options

Name:(str) Name of the Role to manage State:(str) Present or Absent. Default: “present” Role:(str) Name of an IAM Role to assign to the Instance Profile. Default: None

Example

{ "actor": "aws.iam.InstanceProfile",
  "desc": "Ensure that my-ecs-servers exists",
  "options": {
    "name": "my-ecs-servers",
    "state": "present",
    "role": "some-iam-role",
  }
}

Dry run

Will let you know if the profile exists or not, and what changes it would make to the profile.

class kingpin.actors.aws.iam.UploadCert(*args, **kwargs)

Uploads a new SSL Cert to AWS IAM.

Options

Private_key_path:  (str) Path to the private key. Path:(str) The AWS “path” for the server certificate. Default: “/” Public_key_path:  (str) Path to the public key certificate. Name:(str) The name for the server certificate. Cert_chain_path:  (str) Path to the certificate chain. Optional.

Example

{ "actor": "aws.iam.UploadCert",
  "desc": "Upload a new cert",
  "options": {
    "name": "new-cert",
    "private_key_path": "/cert.key",
    "public_key_path": "/cert.pem",
    "cert_chain_path": "/cert-chain.pem"
  }
}

Dry run

Checks that the passed file paths are valid. In the future will also validate that the files are of correct format and content.

class kingpin.actors.aws.iam.DeleteCert(*args, **kwargs)

Delete an existing SSL Cert in AWS IAM.

Options

Name:(str) The name for the server certificate.

Example

{ "actor": "aws.iam.DeleteCert",
  "desc": "Run DeleteCert",
  "options": {
    "name": "fill-in"
  }
}

Dry run

Will find the cert by name or raise an exception if it’s not found.

kingpin.actors.aws.settings

Common settings used by many of the kingpin.actors.aws modules.

kingpin.actors.aws.settings.is_retriable_exception(exception)

Return true if this AWS exception is transient and should be retried.

Example:

>>> @retry(retry_on_exception=is_retriable_exception)

kingpin.actors.aws.sqs

exception kingpin.actors.aws.sqs.QueueNotFound

Raised by SQS Actor when a needed queue is not found.

exception kingpin.actors.aws.sqs.QueueDeletionFailed

Raised if Boto fails to delete an SQS queue.

http://boto.readthedocs.org/en/latest/ref/

sqs.html#boto.sqs.connection.SQSConnection.delete_queue

class kingpin.actors.aws.sqs.Create(*args, **kwargs)

Creates a new SQS queue with the specified name

Options

Name:(str) The name of the queue to create Region:(str) AWS region (or zone) string, like ‘us-west-2’

Examples

{ "actor": "aws.sqs.Create",
  "desc": "Create queue named async-tasks",
  "options": {
    "name": "async-tasks",
    "region": "us-east-1",
  }
}

Dry Mode

Will not create any queue, or even contact SQS. Will create a mock.Mock object and exit with success.

class kingpin.actors.aws.sqs.Delete(*args, **kwargs)

Deletes the SQS queues

Note: even if it`s not empty

Options

Name:(str) The name of the queue to destroy Region:(str) AWS region (or zone) string, like ‘us-west-2’ Idempotent:(bool) Will not raise errors if no matching queues are found. (default: False)

Examples

{ "actor": "aws.sqs.Delete",
  "desc": "Delete queue async-tasks",
  "options": {
    "name": "async-tasks",
    "region": "us-east-1"
  }
}

{ "actor": "aws.sqs.Delete",
  "desc": "Delete queues with 1234 in the name",
  "options": {
    "name": "1234",
    "region": "us-east-1"
  }
}

Dry Mode

Will find the specified queue, but will have a noop regarding its deletion. Dry mode will fail if no queues are found, and idempotent flag is set to False.

class kingpin.actors.aws.sqs.WaitUntilEmpty(*args, **kwargs)

Wait indefinitely until for SQS queues to become empty

This actor will loop infinitely as long as the count of messages in at least one queue is greater than zero. SQS does not guarantee exact count, so this can return a stale value if the number of messages in the queue changes rapidly.

Options

Name:(str) The name or regex pattern of the queues to operate on Region:(str) AWS region (or zone) string, like ‘us-west-2’ Required:(bool) Fail if no matching queues are found. (default: False)

Examples

{ "actor": "aws.sqs.WaitUntilEmpty",
  "desc": "Wait until release-0025a* queues are empty",
  "options": {
    "name": "release-0025a",
    "region": "us-east-1",
    "required": true
  }
}

Dry Mode

This actor performs the finding of the queue, but will pretend that the count is 0 and return success. Will fail even in dry mode if required option is set to True and no queues with the name pattern are found.

kingpin.actors.aws.s3

exception kingpin.actors.aws.s3.InvalidBucketConfig

Raised whenever an invalid option is passed to a Bucket

class kingpin.actors.aws.s3.PublicAccessBlockConfig

Provides JSON-Schema based validation of the supplied Public Access Block Configuration..

The S3 PublicAccessBlockConfiguration should look like this:

{ "block_public_acls": true,
  "ignore_public_acls": true,
  "block_public_policy": true,
  "restrict_public_buckets": true }

If you supply an empty dict, then we will explicitly remove the Public Access Block Configuration.

class kingpin.actors.aws.s3.LoggingConfig

Provides JSON-Schema based validation of the supplied logging config.

The S3 LoggingConfig format should look like this:

{ "target": "s3_bucket_name_here",
  "prefix": "an_optional_prefix_here" }

If you supply an empty target, then we will explicitly remove the logging configuration from the bucket. Example:

{ "target": "" }

class kingpin.actors.aws.s3.LifecycleConfig

Provides JSON-Schema based validation of the supplied Lifecycle config.

The S3 Lifecycle system allows for many unique configurations. Each configuration object defined in this schema will be turned into a boto.s3.lifecycle.Rule object. All of the rules together will be turned into a boto.s3.lifecycle.Lifecycle object.

[
  { "id": "unique_rule_identifier",
    "prefix": "/some_path",
    "status": "Enabled",
    "expiration": {
      "days": 365,
    },
    "noncurrent_version_expiration": {
      "noncurrent_days": 365,
    },
    "transition": {
      "days": 90,
      "date": "2016-05-19T20:04:17+00:00",
      "storage_class": "GLACIER",
    },
    "noncurrent_version_transition": {
      "noncurrent_days": 90,
      "storage_class": "GLACIER",
    }
  }
]

class kingpin.actors.aws.s3.TaggingConfig

Provides JSON-Schema based validation of the supplied tagging config.

The S3 TaggingConfig format should look like this:

[ { "key": "my_key", "value": "some_value" } ]

class kingpin.actors.aws.s3.Bucket(*args, **kwargs)

Manage the state of a single S3 Bucket.

The actor has the following functionality:

• Ensure that an S3 bucket is present or absent.
• Manage the bucket policy.
• Manage the bucket Lifecycle configurations.
• Enable or Suspend Bucket Versioning. Note: It is impossible to actually _disable_ bucket versioning – once it is enabled, you can only suspend it, or re-enable it.

Note about Buckets with Files

Amazon requires that an S3 bucket be empty in order to delete it. Although we could recursively search for all files in the bucket and then delete them, this is a wildly dangerous thing to do inside the confines of this actor. Instead, we raise an exception and alert the you to the fact that they need to delete the files themselves.

Options

Name:

The name of the bucket to operate on

State:

(str) Present or Absent. Default: “present”

Lifecycle:

(LifecycleConfig, None)

A list of individual Lifecycle configurations. Each dictionary includes keys for the id, prefix and status as required parameters. Optionally you can supply an expiration and/or transition dictionary.

If an empty list is supplied, or the list in any way does not match what is currently configured in Amazon, the appropriate changes will be made.

Logging:

(LoggingConfig, None)

If a dictionary is supplied ({'target': 'logging_bucket', 'prefix': '/mylogs'}), then we will configure bucket logging to the supplied bucket and prefix. If prefix is missing then no prefix will be used.

If target is supplied as an empty string (''), then we will disable logging on the bucket. If None is supplied, we will not manage logging either way.

Tags:

(TaggingConfig, None)

A list of dictionaries with a key and value key. Defaults to an empty list, which means that if you manually add tags, they will be removed.

Policy:

(str, None) A JSON file with the bucket policy. Passing in a blank string will cause any policy to be deleted. Passing in None (or not passing it in at all) will cause Kingpin to ignore the policy for the bucket entirely. Default: None

Public_access_block_configuration:  

(PublicAccessBlockConfig, None)

If a dictionary is supplied, then it must conform to the PublicAccessBlockConfig type and include all of the Public Access Block Configuration parameters.

If an empty dictionary is supplied, then Kingpin will explicitly remove any Public Access Block Configurations from the bucket.

Finally, if None is supplied, Kingpin will ignore the checks entirely on this portion of the bucket configuration.

Default: None

Region:

AWS region (or zone) name, such as us-east-1 or us-west-2

Versioning:

(bool, None): Whether or not to enable Versioning on the bucket. If “None”, then we don’t manage versioning either way. Default: None

Examples

{ "actor": "aws.s3.Bucket",
  "options": {
    "name": "kingpin-integration-testing",
    "region": "us-west-2",
    "policy": "./examples/aws.s3/amazon_put.json",
    "lifecycle": [
       { "id": "main",
         "prefix": "/",
         "status": "Enabled",
         "expiration": 30,
       }
    ],
    "logging": {
      "target": "logs.myco.com",
      "prefix": "/kingpin-integratin-testing"
    },
    "tags": [
      {"key": "my_key", "value": "billing-grp-1"},
    ],
    "versioning": true,
  }
}

Dry Mode

Finds the bucket if it exists (or tells you it would create it). Describes each potential change it would make to the bucket depending on the configuration of the live bucket, and the options that were passed into the actor.

Will gracefully fail and alert you if there are files in the bucket and you are trying to delete it.

kingpin.actors.base

Base Actor object class

An Actor object is a class that executes a single logical action on a resource as part of your deployment structure. For example, you may have an Actor that launches a server array in RightScale, or you may have one that sends an email.

Each Actor object should do one thing, and one thing only. Its responsible for being able to execute the operation in both ‘dry’ and ‘non-dry’ modes.

The behavior for ‘dry’ mode can contain real API calls, but should not make any live changes. It is up to the developer of the Actor to define what ‘dry’ mode looks like for that particular action.

class kingpin.actors.base.LogAdapter(logger, extra)

Simple Actor Logging Adapter.

Provides a common logging format for actors that uses the actors description and dry parameter as a prefix to the supplied log message.

class kingpin.actors.base.BaseActor(desc=None, options={}, dry=False, warn_on_failure=False, condition=True, init_context={}, init_tokens={}, timeout=None)

Abstract base class for Actor objects.

option(name)

Return the value for a given Actor option.

readfile(path)

Return file contents as a string.

Raises:

InvalidOptions if file is not found, or readable.

timeout(*args, **kwargs)

Wraps a Coroutine method in a timeout.

Used to wrap the self.execute() method in a timeout that will raise an ActorTimedOut exception if an actor takes too long to execute.

Note, Tornado 4+ does not allow you to actually kill a task on the IOLoop. This means that all we are doing here is notifying the caller (through the raised exception) that a problem has happened.

Fairly simple Actors should actually ‘stop executing’ when this exception is raised. Complex actors with very unique behaviors though (like the rightsacle.server_array.Execute actor) have the ability to continue to execute in the background until the Kingpin application quits. It is not the job of this method to try to kill these actors, but just to let the user know that a failure has happened.

str2bool(v, strict=False)

Returns a Boolean from a variety of inputs.

args:

value: String/Bool strict: Whether or not to _only_ convert the known words into booleans, or whether to allow “any” word to be considered True other than the known False words.

returns:

A boolean

get_orgchart(parent='')

Construct organizational chart describing this actor.

Return a list of actors handled by this actor. Most actors will return a list of just one object. Grouping actors will return a list of all actors that are called.

orgchart object:

id: unique string identifying this actor’s instance. class: kingpin class name desc: actor description parent_id: organizational relationship. Same as id above.

class kingpin.actors.base.EnsurableBaseActor(*args, **kwargs)

Base Class for Actors that “ensure” the state of a resource.

Many of our actors have a goal of ensuring that a particular resource is in a given state. This leads to a ton of boiler plate code to “get” the state of something, “compare” that to the desired state, and then maybe “set” the state.

This actor provides a framework allowing the user to simply write the getters and setters (and optionally compare), and lets the rest of the actor handle the order of operations.

Required Methods:

_set_state:

Creates or destroys the resource depending on the ‘state’ parameter that was passed in.

Note: The ‘state’ parameter is automatically added to the options. You do not need to define it.

_get_state:

Gets the current state of the resource.

_set_[option]:

A ‘setter’ for each option name passed in.

_get_[option]:

A ‘getter’ for each option name passed in.

Optional Methods:

_precache:Called before any setters/getters are triggered. Used to optionally populate a cache of data to make the getters faster. For example, if you can make one API call to get all of the data about a resource, then store that data locally for fast access. _compare_[option]:  Optionally you can write your own comparison method if you’re not doing a pure string comparison between the source and destination.

Examples

class MyClass(base.EnsurableBaseActor):

    all_options = {
        'name': (str, REQUIRED, 'Name of thing'),
        'description': (str, None, 'Description of thing')
    }

    unmanaged_options = ['name']

    @gen.coroutine
    def _set_state(self):
        if self.option('state') == 'absent':
            yield self.conn.delete_resource(
                name=self.option('name'))
        else:
            yield self.conn.create_resource(
                name=self.option('name'),
                desc=self.option('description'))

    @gen.coroutine
    def _set_description(self):
        yield self.conn.set_desc_of_resource(
            name=self.option('name'),
            desc=self.option('description'))

    @gen.coroutine
    def _get_description(self):
        yield self.conn.get_desc_of_resource(
            name=self.option('name'))

class kingpin.actors.base.HTTPBaseActor(desc=None, options={}, dry=False, warn_on_failure=False, condition=True, init_context={}, init_tokens={}, timeout=None)

Abstract base class for an HTTP-client based Actor object.

This class provides common methods for getting access to asynchronous HTTP clients, wrapping the executions in appropriate try/except blocks, timeouts, etc.

If you’re writing an Actor that uses a remote REST API, this is the base class you should subclass from.

kingpin.actors.exceptions

All common Actor exceptions

exception kingpin.actors.exceptions.ActorException

Base Kingpin Actor Exception

exception kingpin.actors.exceptions.RecoverableActorFailure

Base exception that allows script executions to continue on failure.

This exception class is used to throw an error when an Actor fails, but it was an expected and/or acceptable failure.

This should be used for exceptions that are somewhat normal … for example, trying to delete a ServerArray thats already gone.

exception kingpin.actors.exceptions.UnrecoverableActorFailure

Base exception for unrecoverable failures.

This exception class should be used for critical failures that should always stop a set of Kingpin actors in-place, regardless of the actors warn_on_failure setting.

Examples would be when credentials are incorrect, or an unexpected exception is caught and there is no known recovery point.

exception kingpin.actors.exceptions.ActorTimedOut

Raised when an Actor takes too long to execute

exception kingpin.actors.exceptions.InvalidActor

Raised when an invalid Actor name was supplied

exception kingpin.actors.exceptions.InvalidOptions

Invalid option arguments passed into the Actor object.

This can be used both for the actual options dict passed into the actor, as well as if a the wrong options were used when connecting to a remote API.

exception kingpin.actors.exceptions.InvalidCredentials

Invalid or missing credentials required for Actor object.

exception kingpin.actors.exceptions.UnparseableResponseFromEndpoint

Invalid response returned from a remote REST endpoint.

exception kingpin.actors.exceptions.BadRequest

An action failed due to a HTTP 400 error likely due to bad input.

kingpin.actors.group

Group a series of other BaseActor into either synchronous or asynchronous stages.

class kingpin.actors.group.BaseGroupActor(*args, **kwargs)

Group together a series of other kingpin.actors.base.BaseActor objects

Acts:[ <list of kingpin.actors.base.BaseActor objects to execute> ]

get_orgchart(parent='')

Generate an orgchart for all the acts specified.

class kingpin.actors.group.Sync(*args, **kwargs)

Execute a series of kingpin.actors.base.BaseActor synchronously.

Groups together a series of Actors and executes them synchronously in the order that they were defined.

Options

Acts:

An array of individual Actor definitions.

Contexts:

This variable can be one of two formats:

• A list of dictionaries with contextual tokens to pass into the actors at instantiation time. If the list has more than one element, then every actor defined in acts will be instantiated once for each item in the contexts list.
• A string that points to a file with a list of contexts, just like the above dictionary.
• (_Deprecation warning, this is going away in v0.4.0. Use the ‘str’ method above!_) A dictionary of file and tokens. The file should be a relative path with data formatted same as stated above. The tokens need to be the same format as a Macro actor: a dictionary passing token data to be used.

Timeouts

Timeouts are disabled specifically in this actor. The sub-actors can still raise their own kingpin.actors.exceptions.ActorTimedOut exceptions, but since the group actors run an arbitrary number of sub actors, we have chosen to not have this actor specifically raise its own kingpin.actors.exceptions.ActorTimedOut exception unless the user sets the timeout setting.

Examples

Creates two arrays … but sleeps 60 seconds between the two, then does not sleep at all after the last one:

{ "desc": "Clone, then sleep ... then clone, then sleep shorter...",
  "actor": "group.Sync",
  "options": {
    "contexts": [
      { "ARRAY": "First", "SLEEP": "60", },
      { "ARRAY": "Second", "SLEEP": "0", }
    ],
    "acts": [
      { "desc": "do something",
        "actor": "server_array.Clone",
        "options": {
          "source": "template",
          "dest": "{ARRAY}"
        }
      },
      { "desc": "sleep",
        "actor": "misc.Sleep",
        "options": {
          "sleep": "{SLEEP}",
        }
      }
    ]
  }
}

Alternatively if no contexts are needed you can use the array syntax.

[
  {
    "actor": "server_array.Clone",
    "options": {
      "source": "template",
      "dest": "%ARRAY%"
    }
  },
  {
    "actor": "misc.Sleep",
    "options": { "sleep": 30 }
  }
]

Dry Mode

Passes on the Dry mode setting to the acts that are called. Does not stop execution when one of the acts fails. Instead Group actor will finish all acts with warnings, and raise an error at the end of execution.

This provides the user with an insight to all the errors that are possible to encounter, rather than abort and quit on the first one.

Failure

In the event that an act fails, this actor will return the failure immediately. Because the acts are executed in-order of definition, the failure will prevent any further acts from executing.

The behavior is different in the dry run (read above.)

class kingpin.actors.group.Async(*args, **kwargs)

Execute several kingpin.actors.base.BaseActor objects asynchronously.

Groups together a series of Actors and executes them asynchronously - waiting until all of them finish before returning.

Options

Concurrency:

Max number of concurrent executions. This will fire off N executions in parallel, and continue with the remained as soon as the first execution is done. This is faster than creating N Sync executions.

Acts:

An array of individual Actor definitions.

Contexts:

This variable can be one of two formats:

• A list of dictionaries with contextual tokens to pass into the actors at instantiation time. If the list has more than one element, then every actor defined in acts will be instantiated once for each item in the contexts list.
• A dictionary of file and tokens. The file should be a relative path with data formatted same as stated above. The tokens need to be the same format as a Macro actor: a dictionary passing token data to be used.

Timeouts

Timeouts are disabled specifically in this actor. The sub-actors can still raise their own kingpin.actors.exceptions.ActorTimedOut exceptions, but since the group actors run an arbitrary number of sub actors, we have chosen to not have this actor specifically raise its own kingpin.actors.exceptions.ActorTimedOut exception unless the user sets the timeout setting.

Examples

Clone two arrays quickly.

{ "desc": "Clone two arrays",
  "actor": "group.Async",
  "options": {
    "contexts": [
      { "ARRAY": "NewArray1" },
      { "ARRAY": "NewArray2" }
    ],
    "acts": [
      { "desc": "do something",
        "actor": "server_array.Clone",
        "options": {
          "source": "template",
          "dest": "{ARRAY}",
        }
      }
    ]
  }
}

Dry Mode

Passes on the Dry mode setting to the sub-actors that are called.

Failure

In the event that one or more acts fail in this group, the entire group acts will return a failure to Kingpin. Because multiple actors are executing all at the same time, the all of these actors will be allowed to finish before the failure is returned.

kingpin.actors.hipchat

The Hipchat Actors allow you to send messages to a HipChat room at stages during your job execution. The actor supports dry mode by validating that the configured API Token has access to execute the methods, without actually sending the messages.

Required Environment Variables

HIPCHAT_TOKEN:HipChat API Token HIPCHAT_NAME:HipChat message from name (defaults to Kingpin)

class kingpin.actors.hipchat.HipchatBase(*args, **kwargs)

Simple Hipchat Abstract Base Object

class kingpin.actors.hipchat.Message(*args, **kwargs)

Sends a message to a room in HipChat.

Options

Room:(str) The string-name (or ID) of the room to send a message to Message:(str) Message to send

Examples

{ "actor": "hipchat.Message",
  "desc": "Send a message!",
  "options": {
    "room": "Operations",
    "message": "Beginning Deploy: v1.2"
  }
}

Dry Mode

Fully supported – does not actually send messages to a room, but validates that the API credentials would have access to send the message using the HipChat auth_test optional API argument.

class kingpin.actors.hipchat.Topic(*args, **kwargs)

Sets a HipChat room topic.

Options

• room - The string-name (or ID) of the room to set the topic of
• topic - String of the topic to send

Examples

{ "actor": "hipchat.Topic",
  "desc": "set the room topic",
  "options": {
    "room": "Operations",
    "topic": "Latest Deployment: v1.2"
  }
}

Dry Mode

Fully supported – does not actually set a room topic, but validates that the API credentials would have access to set the topic of the room requested.

kingpin.actors.librato

The Librato Actor allows you to post an Annotation to Librato. This is specifically useful for marking when deployments occur on your graphs for cause/effect analysis.

Required Environment Variables

LIBRATO_TOKEN:Librato API Token LIBRATO_EMAIL:Librato email account (i.e. username)

class kingpin.actors.librato.Annotation(*args, **kwargs)

Librato Annotation Actor

Posts an Annotation to Librato.

Options

Title:The title of the annotation Description:The description of the annotation Name:Name of the metric to annotate

Examples

{ "actor": "librato.Annotation",
  "desc": "Mark our deployment",
  "options": {
    "title": "Deploy",
    "description": "Version: 0001a",
    "name": "production_releases"
  }
}

Dry Mode

Currently does not actually do anything, just logs dry mode.

kingpin.actors.misc

These are common utility Actors that don’t really need their own dedicated packages. Things like sleep timers, loggers, etc.

Optional Environment Variables

URLLIB_DEBUG:Set this variable to enable extreme debug logging of the URLLIB requests made by the RightScale/AWS actors. Note, this is very insecure as headers/cookies/etc. are exposed

class kingpin.actors.misc.Note(desc=None, options={}, dry=False, warn_on_failure=False, condition=True, init_context={}, init_tokens={}, timeout=None)

Print any message to log.

class kingpin.actors.misc.Macro(*args, **kwargs)

Parses a kingpin script, instantiates and executes it.

Parse JSON/YAML

Kingpin JSON/YAML has 2 passes at its validity. Script syntax must be valid, with the exception of a few useful deviations allowed by demjson parser. Main one being the permission of inline comments via /* this */ syntax.

The second pass is validating the Schema. The script will be validated for schema-conformity as one of the first things that happens at load-time when the app starts up. If it fails, you will be notified immediately.

Lastly after the JSON/YAML is established to be valid, all the tokens are replaced with their specified value. Any key/value pair passed in the tokens option will be available inside of the JSON file as %KEY% and replaced with the value at this time.

In a situation where nested Macro executions are invoked the tokens do not propagate from outter macro into the inner. This allows to reuse token names, but forces the user to specify every token needed. Similarly, if environment variables are used for token replacement in the main file, these tokens are not available in the subsequent macros.

Pre-Instantiation

In an effort to prevent mid-run errors, we pre-instantiate all Actor objects all at once before we ever begin executing code. This ensures that major typos or misconfigurations in the JSON/YAML will be caught early on.

Execution

misc.Macro actor simply calls the execute() method of the most-outter actor; be it a single action, or a group actor.

Options

Macro:String of local path to a JSON/YAML script. Tokens:Dictionary to search/replace within the file.

Examples

{ "desc": "Stage 1",
  "actor": "misc.Macro",
  "options": {
    "macro": "deployment/stage-1.json",
    "tokens": {
      "TIMEOUT": 360,
      "RELEASE": "%RELEASE%"
    }
  }
}

Dry Mode

Fully supported – instantiates the actor inside of JSON with dry=True. The behavior of the consecutive actor is unique to each; read their description for more information on dry mode.

get_orgchart(parent='')

Return orgchart including the actor inside of the macro file.

class kingpin.actors.misc.Sleep(desc=None, options={}, dry=False, warn_on_failure=False, condition=True, init_context={}, init_tokens={}, timeout=None)

Sleeps for an arbitrary number of seconds.

Options

Sleep:Integer of seconds to sleep.

Examples

{ "actor": "misc.Sleep",
  "desc": "Sleep for 60 seconds",
  "options": {
    "sleep": 60
  }
}

Dry Mode

Fully supported – does not actually sleep, just pretends to.

class kingpin.actors.misc.GenericHTTP(desc=None, options={}, dry=False, warn_on_failure=False, condition=True, init_context={}, init_tokens={}, timeout=None)

A very simple actor that allows GET/POST methods over HTTP.

Does a GET or a POST to a specified URL.

Options

Url:Destination URL Data:Optional POST data as a dict. Will convert into key=value&key2=value2.. Exclusive of data-json option. Data-json:Optional POST data as a dict. Will stringify and pass as JSON. Exclusive of data option. Username:Optional for HTTPAuth. Password:Optional for HTTPAuth.

Examples

{ "actor": "misc.GenericHTTP",
  "desc": "Make a simple web call",
  "options": {
    "url": "http://example.com/rest/api/v1?id=123&action=doit",
    "username": "secret",
    "password": "%SECRET_PASSWORD%"
  }
}

Dry Mode

Will not do anything in dry mode except print a log statement.

kingpin.actors.packagecloud

The packagecloud actor allows you to perform maintenance operations on repositories hosted by packagecloud.io using their API:

https://packagecloud.io/docs/api

Required Environment Variables

PACKAGECLOUD_ACCOUNT:  packagecloud account name, i.e. https://packagecloud.io/PACKAGECLOUD_ACCOUNT PACKAGECLOUD_TOKEN:  packagecloud API Token

class kingpin.actors.packagecloud.PackagecloudBase(*args, **kwargs)

Simple packagecloud Abstract Base Object

class kingpin.actors.packagecloud.Delete(*args, **kwargs)

Deletes packages from a PackageCloud repo.

Searches for packages that match the packages_to_delete regex pattern and deletes them. If number_to_keep is set, we always at least this number of versions of the given package intact in the repo. Also if number_to_keep is set, the older versions of a package (based on upload time) packages will be deleted first effectively leaving newer packages in the repo.

Options

Number_to_keep:Keep at least this number of each package (defaults to 0) Packages_to_delete:  Regex of packages to delete, e.g. pkg1|pkg2 Repo:Which packagecloud repo to delete from

Examples

{ "desc": "packagecloud Delete example",
  "actor": "packagecloud.Delete",
  "options": {
    "number_to_keep": 10,
    "packages_to_delete": "deleteme",
    "repo": "test"
  }
}

class kingpin.actors.packagecloud.DeleteByDate(*args, **kwargs)

Deletes packages from a PackageCloud repo older than X.

Adds additional functionality to the Delete class with a older_than option. Only packages older than that number of seconds will be deleted.

Options

Number_to_keep:Keep at least this number of each package (defaults to 0) Older_than:Delete packages created before this number of seconds Packages_to_delete:  Regex of packages to delete, e.g. pkg1|pkg2 Repo:Which packagecloud repo to delete from

Examples

{ "desc": "packagecloud DeleteByDate example",
  "actor": "packagecloud.DeleteByDate",
  "options": {
    "number_to_keep": 10,
    "older_than": 600,
    "packages_to_delete": "deleteme",
    "repo": "test"
  }
}

class kingpin.actors.packagecloud.WaitForPackage(*args, **kwargs)

Searches for a package that matches name and version until found or a timeout occurs.

Options

Name:Name of the package to search for as a regex Version:Version of the package to search for as a regex Repo:Which packagecloud repo to delete from Sleep:Number of seconds to sleep for between each search

Examples

{ "desc": "packagecloud WaitForPackage example",
  "actor": "packagecloud.WaitForPackage",
  "options": {
    "name": "findme",
    "version": "0.1",
    "repo": "test",
    "sleep": 10,
  }
}

kingpin.actors.pingdom

Pingdom actors to pause and unpause checks. These are useful when you are aware of an expected downtime and don’t want to be alerted about it. Also known as Maintenance mode.

Required Environment Variables

PINGDOM_TOKEN:Pingdom API Token PINGDOM_USER:Pingdom Username (email) PINGDOM_PASS:Pingdom Password

class kingpin.actors.pingdom.PingdomBase(*args, **kwargs)

Simple Pingdom Abstract Base Object

class kingpin.actors.pingdom.Pause(*args, **kwargs)

Start Pingdom Maintenance.

Pause a particular “check” on Pingdom.

Options

Name:(Str) Name of the check

Example

{ "actor": "pingdom.Pause",
  "desc": "Run Pause",
  "options": {
    "name": "fill-in"
  }
}

Dry run

Will assert that the check name exists, but not take any action on it.

class kingpin.actors.pingdom.Unpause(*args, **kwargs)

Stop Pingdom Maintenance.

Unpause a particular “check” on Pingdom.

Options

Name:(Str) Name of the check

Example

{ "actor": "pingdom.Unpause",
  "desc": "Run unpause",
  "options": {
    "name": "fill-in"
  }
}

Dry run

Will assert that the check name exists, but not take any action on it.

kingpin.actors.rightscale.api

Base RightScale API Access Object.

This package provides access to the RightScale API via Tornado-style @gen.coroutine wrapped methods. These methods are, however, just wrappers for threads that are being fired off in the background to make the API calls.

Async vs Threads

In the future, this will get re-factored to use a native Tornado AsyncHTTPClient object. The methods themselves will stay the same, but the underlying private methods will change.

The methods in this object are specifically designed to support common operations that the RightScale Actor objects need to do. Operations like ‘find server array’, ‘launch server array’, etc. This is not meant as a pure one-to-one mapping of the RightScale API, but rather a mapping of conceptual operations that the Actors need.

Method Design Note

RightScale mixes and matches their API calls… some of them you pass in a major method and then supply a resource ID to act on. Others you pass in the resource_id and get back a list of methods that you can execute.

For consistency in our programming model, this class relies o you passing in rightscale.Resource objects everywhere, and it does the resource->ID translation.

exception kingpin.actors.rightscale.api.RightScaleError

Raised when the RightScale API returns a specific error

exception kingpin.actors.rightscale.api.ServerArrayException

Raised when an operation on or looking for a ServerArray fails

kingpin.actors.rightscale.base

The RightScale Actors allow you to interact with resources inside your Rightscale account. These actors all support dry runs properly, but each actor has its own caveats with dry=True. Please read the instructions below for using each actor.

Required Environment Variables

RIGHTSCALE_TOKEN:  RightScale API Refresh Token (from the Account Settings/API Credentials page) RIGHTSCALE_ENDPOINT:  Your account-specific API Endpoint (defaults to https://my.rightscale.com)

exception kingpin.actors.rightscale.base.ArrayNotFound

Raised when a ServerArray could not be found.

exception kingpin.actors.rightscale.base.ArrayAlreadyExists

Raised when a ServerArray already exists by a given name.

class kingpin.actors.rightscale.base.RightScaleBaseActor(*args, **kwargs)

Abstract class for creating RightScale cloud actors.

class kingpin.actors.rightscale.base.EnsurableRightScaleBaseActor(*args, **kwargs)

Hacky way to re-use the RightScaleBaseActor but make it ensurable.

kingpin.actors.rightscale.server_array

exception kingpin.actors.rightscale.server_array.InvalidInputs

Raised when supplied inputs are invalid for a ServerArray.

exception kingpin.actors.rightscale.server_array.TaskExecutionFailed

Raised when one or more RightScale Task executions fail.

class kingpin.actors.rightscale.server_array.ServerArrayBaseActor(*args, **kwargs)

Abstract ServerArray Actor that provides some utility methods.

class kingpin.actors.rightscale.server_array.Clone(*args, **kwargs)

Clones a RightScale Server Array.

Clones a ServerArray in RightScale and renames it to the newly supplied name. By default, this actor is extremely strict about validating that the source array already exists, and that the dest array does not yet exist. This behavior can be overridden though if your Kingpin script creates the source, or destroys an existing dest ServerArray sometime before this actor executes.

Options

Source:The name of the ServerArray to clone Strict_source:Whether or not to fail if the source ServerArray does not exist. (default: True) Dest:The new name for your cloned ServerArray Strict_dest:Whether or not to fail if the destination ServerArray already exists. (default: True)

Examples

Clone my-template-array to my-new-array:

{ "desc": "Clone my array",
  "actor": "rightscale.server_array.Clone",
  "options": {
    "source": "my-template-array",
    "dest": "my-new-array"
  }
}

Clone an array that was created sometime earlier in the Kingpin JSON, and thus does not exist yet during the dry run:

{ "desc": "Clone that array we created earlier",
  "actor": "rightscale.server_array.Clone",
  "options": {
    "source": "my-template-array",
    "strict_source": false,
    "dest": "my-new-array"
  }
}

Clone an array into a destination name that was destroyed sometime earlier in the Kingpin JSON:

{ "desc": "Clone that array we created earlier",
  "actor": "rightscale.server_array.Clone",
  "options": {
    "source": "my-template-array",
    "dest": "my-new-array",
    "strict_dest": false,
  }
}

Dry Mode

In Dry mode this actor does validate that the source array exists. If it does not, a kingpin.actors.rightscale.api.ServerArrayException is thrown. Once that has been validated, the dry mode execution pretends to copy the array by creating a mocked cloned array resource. This mocked resource is then operated on during the rest of the execution of the actor, guaranteeing that no live resources are modified.

Example dry output:

[Copy Test (DRY Mode)] Verifying that array "temp" exists
[Copy Test (DRY Mode)] Verifying that array "new" does not exist
[Copy Test (DRY Mode)] Cloning array "temp"
[Copy Test (DRY Mode)] Renaming array "<mocked clone of temp>" to "new"

class kingpin.actors.rightscale.server_array.Update(*args, **kwargs)

Update ServerArray Settings

Updates an existing ServerArray in RightScale with the supplied parameters. Can update any parameter that is described in the RightScale API docs here:

Parameters are passed into the actor in the form of a dictionary, and are then converted into the RightScale format. See below for examples.

Options

Array:(str) The name of the ServerArray to update Exact:(bool) whether or not to search for the exact array name. (default: true) Params:(dict) Dictionary of parameters to update Inputs:(dict) Dictionary of next-instance server arryay inputs to update

Examples

{ "desc": "Update my array",
  "actor": "rightscale.server_array.Update",
  "options": {
    "array": "my-new-array",
    "params": {
      "elasticity_params": {
        "bounds": {
          "min_count": 4
        },
        "schedule": [
          {"day": "Sunday", "max_count": 2,
           "min_count": 1, "time": "07:00" },
          {"day": "Sunday", "max_count": 2,
           "min_count": 2, "time": "09:00" }
        ]
      },
      "name": "my-really-new-name"
    }
  }
}

{ "desc": "Update my array inputs",
  "actor": "rightscale.server_array.Update",
  "options": {
    "array": "my-new-array",
    "inputs": {
      "ELB_NAME": "text:foobar"
    }
  }
}

Dry Mode

In Dry mode this actor does search for the array, but allows it to be missing because its highly likely that the array does not exist yet. If the array does not exist, a mocked array object is created for the rest of the execution.

During the rest of the execution, the code bypasses making any real changes and just tells you what changes it would have made.

This means that the dry mode cannot validate that the supplied inputs will work.

Example dry output:

[Update Test (DRY Mode)] Verifying that array "new" exists
[Update Test (DRY Mode)] Array "new" not found -- creating a mock.
[Update Test (DRY Mode)] Would have updated "<mocked array new>" with
params: {'server_array[name]': 'my-really-new-name',
         'server_array[elasticity_params][bounds][min_count]': '4'}

class kingpin.actors.rightscale.server_array.UpdateNextInstance(*args, **kwargs)

Update the Next Instance parameters for a Server Array

Updates an existing ServerArray in RightScale with the supplied parameters. Can update any parameter that is described in the RightScale ResourceInstances docs.

Note about the image_href parameter

If you pass in the string default to the image_href key in your params dictionary, we will search and find the default image that your ServerArray’s Multi Cloud Image refers to. This helper is useful if you update your ServerArrays to use custom AMIs, and then occasionally want to go back to using a stock AMI. For example, if you boot up your instances occasionally off a stock AMI, customize the host, and then bake that host into a custom AMI.

Parameters are passed into the actor in the form of a dictionary, and are then converted into the RightScale format. See below for examples.

Options

Array:(str) The name of the ServerArray to update Exact:(bool) whether or not to search for the exact array name. (default: true) Params:(dict) Dictionary of parameters to update

Examples

{ "desc": "Update my array",
  "actor": "rightscale.server_array.UpdateNextInstance",
  "options": {
    "array": "my-new-array",
    "params": {
      "associate_public_ip_address": true,
      "image_href": "/image/href/123",
    }
  }
}

{ "desc": "Reset the AMI image to the MCI default",
  "actor": "rightscale.server_array.UpdateNextInstance",
  "options": {
    "array": "my-new-array",
    "params": {
      "image_href": "default",
    }
  }
}

Dry Mode

In Dry mode this actor does search for the array, but allows it to be missing because its highly likely that the array does not exist yet. If the array does not exist, a mocked array object is created for the rest of the execution.

During the rest of the execution, the code bypasses making any real changes and just tells you what changes it would have made.

This means that the dry mode cannot validate that the supplied params will work.

Example dry output:

[Update my array (DRY Mode)] Verifying that array "new" exists
[Update my array (DRY Mode)] Array "new" not found -- creating a mock.
[Update my array (DRY Mode)] Would have updated "<mocked array new>"
with params: {'server_array[associate_public_ip_address]': true,
         'server_array[image_href]': '/image/href/'}

class kingpin.actors.rightscale.server_array.Terminate(*args, **kwargs)

Terminate all instances in a ServerArray

Terminates all instances for a ServerArray in RightScale marking the array disabled.

Options

Array:(str) The name of the ServerArray to destroy Exact:(bool) Whether or not to search for the exact array name. (default: true) Strict:(bool) Whether or not to fail if the ServerArray does not exist. (default: true)

Examples

 { "desc": "Terminate my array",
  "actor": "rightscale.server_array.Terminate",
  "options": {
    "array": "my-array"
  }
}

{ "desc": "Terminate many arrays",
  "actor": "rightscale.server_array.Terminate",
  "options": {
    "array": "array-prefix",
    "exact": false,
  }
}

Dry Mode

Dry mode still validates that the server array you want to terminate is actually gone. If you want to bypass this check, then set the warn_on_failure flag for the actor.

class kingpin.actors.rightscale.server_array.Destroy(*args, **kwargs)

Destroy a ServerArray in RightScale

Destroys a ServerArray in RightScale by first invoking the Terminate actor, and then deleting the array as soon as all of the running instances have been terminated.

Options

Array:(str) The name of the ServerArray to destroy Exact:(bool) Whether or not to search for the exact array name. (default: true) Strict:(bool) Whether or not to fail if the ServerArray does not exist. (default: true)

Examples

{ "desc": "Destroy my array",
  "actor": "rightscale.server_array.Destroy",
  "options": {
    "array": "my-array"
  }
}

{ "desc": "Destroy many arrays",
  "actor": "rightscale.server_array.Destroy",
  "options": {
    "array": "array-prefix",
    "exact": false,
  }
}

Dry Mode

In Dry mode this actor does search for the array, but allows it to be missing because its highly likely that the array does not exist yet. If the array does not exist, a mocked array object is created for the rest of the execution.

During the rest of the execution, the code bypasses making any real changes and just tells you what changes it would have made.

Example dry output:

[Destroy Test (DRY Mode)] Beginning
[Destroy Test (DRY Mode)] Terminating array before destroying it.
[Destroy Test (terminate) (DRY Mode)] Array "my-array" not found --
creating a mock.
[Destroy Test (terminate) (DRY Mode)] Disabling Array "my-array"
[Destroy Test (terminate) (DRY Mode)] Would have terminated all array
"<mocked array my-array>" instances.
[Destroy Test (terminate) (DRY Mode)] Pretending that array <mocked
array my-array> instances are terminated.
[Destroy Test (DRY Mode)] Pretending to destroy array "<mocked array
my-array>"
[Destroy Test (DRY Mode)] Finished successfully. Result: True

class kingpin.actors.rightscale.server_array.Launch(*args, **kwargs)

Launch instances in a ServerArray

Launches instances in an existing ServerArray and waits until that array has become healthy before returning. Healthy means that the array has at least the user-specified count or max_count number of instances running as defined by the array definition in RightScale.

Options

Array:(str) The name of the ServerArray to launch Count:(str, int) Optional number of instance to launch. Defaults to max_count of the array. Success_pct’:(str, int) Optional percent (0-100) to wait for instances to launch before exiting this actor as successful. Default: 100. Enable:(bool) Should the autoscaling of the array be enabled? Settings this to false, or omitting the parameter will not disable an enabled array. Exact:(bool) Whether or not to search for the exact array name. (default: true)

Examples

{ "desc": "Enable array and launch it",
  "actor": "rightscale.server_array.Launch",
  "options": {
    "array": "my-array",
    "enable": true
  }
}

{ "desc": "Enable arrays starting with my-array and launch them",
  "actor": "rightscale.server_array.Launch",
  "options": {
    "array": "my-array",
    "enable": true,
    "exact": false
  }
}

{ "desc": "Enable array and launch 1 instance",
  "actor": "rightscale.server_array.Launch",
  "options": {
    "array": "my-array",
    "count": 1
  }
}

Dry Mode

In Dry mode this actor does search for the array, but allows it to be missing because its highly likely that the array does not exist yet. If the array does not exist, a mocked array object is created for the rest of the execution.

During the rest of the execution, the code bypasses making any real changes and just tells you what changes it would have made.

Example dry output:

[Launch Array Test #0 (DRY Mode)] Verifying that array "my-array" exists
[Launch Array Test #0 (DRY Mode)] Array "my-array" not found -- creating
    a mock.
[Launch Array Test #0 (DRY Mode)] Enabling Array "my-array"
[Launch Array Test #0 (DRY Mode)] Launching Array "my-array" instances
[Launch Array Test #0 (DRY Mode)] Would have launched instances of array
    <MagicMock name='my-array.self.show().soul.__getitem__()'
    id='4420453200'>
[Launch Array Test #0 (DRY Mode)] Pretending that array <MagicMock
    name='my-array.self.show().soul.__getitem__()' id='4420453200'>
    instances are launched.

class kingpin.actors.rightscale.server_array.Execute(*args, **kwargs)

Executes a RightScale script/recipe on a ServerArray

Executes a RightScript or Recipe on a set of hosts in a ServerArray in RightScale using individual calls to the live running instances. These can be found in your RightScale account under Design -> RightScript or Design -> Cookbooks

The RightScale API offers a multi_run_executable method that can be used to run a single script on all servers in an array – but unfortunately this API method provides no way to monitor the progress of the individual jobs on the hosts. Furthermore, the method often executes on recently terminated or terminating hosts, which throws false-negative error results.

Our actor explicitly retrieves a list of the operational hosts in an array and kicks off individual execution tasks for every host. It then tracks the execution of those tasks from start to finish and returns the results.

Options

Array:(str) The name of the ServerArray to operate on Script:(str) The name of the RightScript or Recipe to execute Expected_runtime:  (str, int) Expected number of seconds to execute. (default: 5) Concurrency:Max number of concurrent executions. This will fire off N executions in parallel, and continue with the remained as soon as the first execution is done. This is faster than creating N Sync executions. Note: When applied to multiple (M) arrays cumulative concurrency accross all arrays will remain at N. It will not be M x N. Inputs:(dict) Dictionary of Key/Value pairs to use as inputs for the script Exact:(str) Boolean whether or not to search for the exact array name. (default: true)

Examples

{ "desc":" Execute script on my-array",
  "actor": "rightscale.server_array.Execute",
  "options": {
    "array": "my-array",
    "script": "connect to elb",
    "expected_runtime": 3,
    "inputs": {
      "ELB_NAME": "text:my-elb"
    }
  }
}

Dry Mode

In Dry mode this actor does search for the array, but allows it to be missing because its highly likely that the array does not exist yet. If the array does not exist, a mocked array object is created for the rest of the execution.

During the rest of the execution, the code bypasses making any real changes and just tells you what changes it would have made.

Example dry output:

[Destroy Test (DRY Mode)] Verifying that array "my-array" exists
[Execute Test (DRY Mode)]
    kingpin.actors.rightscale.server_array.Execute Initialized
[Execute Test (DRY Mode)] Beginning execution
[Execute Test (DRY Mode)] Verifying that array "my-array" exists
[Execute Test (DRY Mode)] Would have executed "Connect instance to ELB"
    with inputs "{'inputs[ELB_NAME]': 'text:my-elb'}" on "my-array".
[Execute Test (DRY Mode)] Returning result: True

kingpin.actors.rollbar

The Rollbar Actor allows you to post Deploy messages to Rollbar when you execute a code deployment.

Required Environment Variables

ROLLBAR_TOKEN:Rollbar API Token

class kingpin.actors.rollbar.RollbarBase(*args, **kwargs)

Simple Rollbar Base Abstract Actor

class kingpin.actors.rollbar.Deploy(*args, **kwargs)

Posts a Deploy message to Rollbar.

https://rollbar.com/docs/deploys_other/

API Token

You must use an API token created in your Project Access Tokens account settings section. This token should have post_server_item permissions for the actual deploy, and read permissions for the Dry run.

Options

Environment:The environment to deploy to Revision:The deployment revision Local_username:The user who initiated the deploy Rollbar_username:  (Optional) The Rollbar Username to assign the deploy to Comment:(Optional) Comment describing the deploy

Examples

{ "actor": "rollbar.Deploy",
  "desc": "update rollbar deploy",
  "options": {
    "environment": "Prod",
    "revision": "%DEPLOY%",
    "local_username": "Kingpin",
    "rollbar_username": "Kingpin",
    "comment": "some comment %DEPLOY%"
  }
}

Dry Mode

Accesses the Rollbar API and validates that the token can access your project.

kingpin.actors.slack

The Slack Actors allow you to send messages to a Slack channel at stages during your job execution. The actor supports dry mode by validating that the configured API Token has access to execute the methods, without actually sending the messages.

Required Environment Variables

SLACK_TOKEN:Slack API Token SLACK_NAME:Slack message from name (defaults to Kingpin)

class kingpin.actors.slack.SlackBase(*args, **kwargs)

Simple Slack Abstract Base Object

class kingpin.actors.slack.Message(*args, **kwargs)

Sends a message to a channel in Slack.

Options

Channel:The string-name of the channel to send a message to, or a list of channels Message:String of the message to send

Examples

{ "desc": "Let the Engineers know things are happening",
  "actor": "slack.Message",
  "options": {
    "channel": "#operations",
    "message": "Beginning Deploy: %VER%"
  }
}

Dry Mode

Fully supported – does not actually send messages to a room, but validates that the API credentials would have access to send the message using the Slack auth.test API method.

kingpin.actors.spotinst

The Spotinst package allows you to create, manage and destroy Spotinst ElastiGroups.

https://spotinst.atlassian.net/wiki/display/API/API+Semantics

Environment Variables

SPOTINST_DEBUG:If set, then every single response body from Spotinst will be printed out in the debug logs for Kingpin. This can include credentials, and can be extremely verbose, so use with caution. SPOINST_TOKEN:SpotInst API Token generated at https://console.spotinst.com/#/settings/tokens SPOTINST_ACCOUNT_ID:  SpotInst API Account ID - used optionally when you have multiple accounts under a single Organization. This can also be set on a per-actor basis. http://docs.spotinst.com/#page:api-semantic,header:header-organizations-with-a-single-account

exception kingpin.actors.spotinst.SpotinstException(e)

Base SpotInst exception handler.

This exception handler parses the Spotinst returned messages when an error is thrown. The error message comes back in the body in a JSON formatted blob. This exception handler will parse out the exception message, print it out in a semi-readable log form for the user, and then store it in the Exception body.

See https://spotinst.atlassian.net/wiki/display/API/API+Semantics for more details.

exception kingpin.actors.spotinst.InvalidConfig(e)

Thrown when an invalid request was supplied to Spotinst

class kingpin.actors.spotinst.ElastiGroupSchema

Light validation against the Spotinst ElastiGroup schema.

For full description of the JSON data format, please see: https://spotinst.atlassian.net/wiki/display/API/Create+Group#CreateGroup-JF

This schema handles the following validation cases:

• Only allow a single SubnetID for each availabilityZone object.
• Disallow t2|hc1 instance types for the spot instance section.
• Ensure that the scaling.up and scaling.down arrays are either null or contain at least 1 record.

class kingpin.actors.spotinst.SpotinstBase(*args, **kwargs)

Simple Spotinst Abstract Base Object

class kingpin.actors.spotinst.ElastiGroup(*args, **kwargs)

Manages an ElastiGroup in Spotinst.

Spotinst ElastiGroups act as smarter EC2 AutoScalingGroups that scale up and down leveraging Amazon Spot instances wherever possible. These ElastiGroups are completely configurable through a JSON Blob.

For a fully functional example JSON config, see this one. You can also write your files in YAML if you prefer – Kingpin will handle the conversion.

UserData Startup Script

The Spotinst API wants the instances UserData script to be supplied as a Base64-encoded string – which you can do if you wish. However, there is no need, as Kingpin will automatically convert your plain-text script into a Base64 blob for you behind the scenes.

Rolling out Group Changes

We will trigger the “roll group” API if the roll_on_change parameter is set to True after any change to an ElastiGroup. It is difficult to know which changes may or may not require a replacement of your existing hosts, so we leave this up to the user to decide on the behavior.

Known Limitations

• The Spotinst API does not allow you to change an ElastiGroup scaling ‘unit’ (ie, CPU Count or Instance Count). You can also not change an ElastiGroup’s basic platform (ie, VPC Linux vs Non VPC Linux). We warn about this on each change.

Options

Name:

The desired name of the ElastiGroup. Note that this will override whatever value is inside your configuration JSON/YAML blob.

Account_id:

The SpotInst Account ID that the action is taking place in - this overrides the SPOTINST_ACCOUNT_ID environment variable (if its set).

Config:

Path to the ElastiGroup configuration blob (JSON or YAML) file. Token-replacement can be used inside of your configuration files allowing environment variables to replace %VAR% strings.

This file will be checked against a light-schema defined in ElastiGroupSchema before any authentication is required. The file will be further validated against the Spotinst API during the DRY run, but this requires authentication.

Tokens:

A dict of key/value pairs that can be used to swap in variables into a common ElastiGroup template. These are added to (and override) the Environment variables that Kingpin already uses for variables swapping (as described in the Token-replacement section.

Roll_on_change:

Whether or not to forcefully roll out changes to the ElastiGroup. If True, we will issue a ‘roll call’ to SpotInst and trigger all of the instances to be replaced. Defaults to False.

Roll_batch_size:  

Indicates in percentage the amount of instances should be replaced in each batch. Defaults to 20.

Roll_grace_period:  

Indicates in seconds the timeout to wait until instance become healthy in the ELB. Defaults to 600.

Wait_on_create:

If set to True, Kingpin will loop until the ElastiGroup has fully launched – this only applies if the group is being created from scratch. On updates, see the wait_on_roll setting below. Defaults to False.

Wait_on_roll:

If set to True, Kingpin will loop until the rollout of any changes has completed. This can take a long time, depending on your rollout settings. Defaults to False.

Examples

{ "actor": "spotinst.ElastiGroup",
  "options": {
    "name": "my-group",
    "config": "./group_config.json",
  }
}

Dry Mode

Will discover the current state of the ElastiGroup (present, absent), and whether or not the current configuration is different than the desired configuration. Will also validate the desired configuration against the SpostInst API to give you a heads up about any potential failures up front.

This package provides a quick way of creating custom API clients for JSON-based REST APIs. The majority of the work is in the creation of a _CONFIG dictionary for the class. This dictionary dynamically configures the object at instantiation time with the appropriate @gen.coroutine wrapped HTTP fetch methods.

See the documentation in docs/DEVELOPMENT.md for more details on how to use this package to create your own API client.

kingpin.actors.support.api.create_http_method(name, http_method)

Creates the get/put/delete/post coroutined-method for a resource.

This method is called during the __init__ of a RestConsumer object. The method creates a custom method thats handles a GET, PUT, POST or DELETE through the Tornado HTTPClient class.

Args:

http_method: Name of the method (get, put, post, delete)

Returns:

A method appropriately configured and named.

kingpin.actors.support.api.create_method(name, config)

Creates a RestConsumer object.

Configures a fresh RestConsumer object with the supplied configuration bits. The configuration includes information about the name of the method being consumed and the configuration of that method (which HTTP methods it supports, etc).

The final created method accepts any args (*args, **kwargs) and passes them on to the RestConsumer object being created. This allows for passing in unique resource identifiers (ie, the ‘%res%’ in ‘/v2/rooms/%res%/history’).

Args:

name: The name passed into the RestConsumer object config: The config passed into the RestConsumer object

Returns:

A method that returns a fresh RestConsumer object

class kingpin.actors.support.api.RestConsumer(name=None, config=None, client=None, *args, **kwargs)

An abstract object that self-defines its own API access methods.

At init time, this object reads its _CONFIG and pre-defines all of the API access methods that have been described. It does not handle actual HTTP calls directly, but is passed in a client object (anything that subclasses the RestClient class) and leverages that for the actual web calls.

class kingpin.actors.support.api.RestClient(client=None, headers=None)

Very simple REST client for the RestConsumer. Implements a AsyncHTTPClient(), some convinience methods for URL escaping, and a single fetch() method that can handle GET/POST/PUT/DELETEs.

This code is nearly identical to the kingpin.actors.base.BaseHTTPActor class, but is not actor-specific.

Args:

headers: Headers to pass in on every HTTP request

class kingpin.actors.support.api.SimpleTokenRestClient(tokens, *args, **kwargs)

Simple RestClient that appends a ‘token’ to every web request for authentication. Used in most simple APIs where a token is provided to the end user.

Args:

tokens: (dict) A dict with the token name/value(s) to append to every

we request.

kingpin.actors.utils

Misc methods for dealing with Actors.

kingpin.actors.utils.dry(dry_message)

Coroutine-compatible decorator to dry-run a method.

Note: this must act on a BaseActor object.

Example usage as decorator:

>>> @gen.coroutine
... @dry('Would have done that {thing}')
... def do_thing(self, thing):
...     yield api.do_thing(thing)
...
>>> yield do_thing(thing="yeah man, that thing")

Args:

dry_message: The message to print out instead of doing the actual function call. This string is passed through format(kwargs), so any variables you’d like can be substituted as long as they’re passed to the method being wrapped.

kingpin.actors.utils.timer(f)

Coroutine-compatible function timer.

Records statistics about how long a given function took, and logs them out in debug statements. Used primarily for tracking Actor execute() methods, but can be used elsewhere as well.

Note: this must act on a BaseActor object.

Example usage:

>>> @gen.coroutine
... @timer()
... def execute(self):
...     raise gen.Return()

kingpin.actors.utils.get_actor(config, dry)

Returns an initialized Actor object.

Args:

config: A dictionary of configuration data that conforms to our v1

schema in kingpin.schema. Looks like this:

{

‘desc’: <string description of actor>, ‘actor’: <string name of actor> ‘options’: <dict of options to pass to actor> ‘warn_on_failure’: <bool> ‘condition’: <string or bool> }

dry: Boolean whether or not in Dry mode warn_on_failure: Boolean

Returns:

<actor object>

kingpin.actors.utils.get_actor_class(actor)

Returns a Class Reference to an Actor by string name.

Args:

actor: String name of the actor to find.

Returns:

<Class Ref to Actor>

class kingpin.constants.REQUIRED

Meta class to identify required arguments for actors.

class kingpin.constants.StringCompareBase

Meta class to identify the desired state for a resource.

This basic type of constant allows someone to easily define a set of valid strings for their option and have the base actor class automatically validate the inputs against those strings.

class kingpin.constants.STATE

Meta class to identify the desired state for a resource.

Simple tester for ‘present’ or ‘absent’ on actors. Used for any actor thats idempotent and used to ensure some state of a resource.

class kingpin.constants.SchemaCompareBase

Meta class that compares the schema of a dict against rules.

exception kingpin.exceptions.KingpinException

Base Exception

exception kingpin.exceptions.InvalidScript

Raised when an invalid script schema was detected

exception kingpin.exceptions.InvalidScriptName

Raised when the script name does not end on .yaml or .json

kingpin.schema.validate(config)

Validates the JSON against our schemas.

TODO: Support multiple schema versions

Args:

config: Dictionary of parsed JSON

Returns:

None: if all is well

Raises:

Execption if something went wrong.

kingpin.utils

Common package for utility functions.

kingpin.utils.str_to_class(string)

Method that converts a string name into a usable Class name

This is used to take the ‘actor’ value from the JSON object and convert it into a valid object reference.

Args:

cls: String name of the wanted class and package.

eg: kingpin.actors.foo.bar eg: misc.Sleep eg: actors.misc.Sleep eg: my.private.Actor

Returns:

A reference to the actual Class to be instantiated

kingpin.utils.setup_root_logger(level='warn', syslog=None, color=False)

Configures the root logger.

Args:

level: Logging level string (‘warn’ is default) syslog: String representing syslog facility to output to. If empty, logs are written to console. color: Colorize the log output

Returns:

A root Logger object

kingpin.utils.super_httplib_debug_logging()

Enables DEBUG logging deep in HTTPLIB.

HTTPLib by default doens’t log out things like the raw HTTP headers, cookies, response body, etc – even when your main logger is in DEBUG mode. This is because its a security risk, as well as just highly verbose.

For the purposes of debugging though, this can be useful. This method enables deep debug logging of the HTTPLib web requests. This is highly insecure, but very useful when troubleshooting failures with remote API endpoints.

Returns:

Requests ‘logger’ object (mainly for unit testing)

kingpin.utils.exception_logger(func)

Explicitly log Exceptions then Raise them.

Logging Exceptions and Tracebacks while inside of a thread is broken in the Tornado futures package for Python 2.7. It swallows most of the traceback and only gives you the raw exception object. This little helper method allows us to throw a log entry with the full traceback before raising the exception.

kingpin.utils.retry(excs, retries=3, delay=0.25)

Coroutine-compatible Retry Decorator.

This decorator provides a simple retry mechanism that looks for a particular set of exceptions and retries async tasks in the event that those exceptions were caught.

Example usage:

>>> @gen.coroutine
... @retry(excs=(Exception), retries=3)
... def login(self):
...     raise gen.Return()

Args:

excs: A single (or tuple) exception type to catch. retries: The number of times to try the operation in total. delay: Time (in seconds) to wait between retries

kingpin.utils.tornado_sleep(*args, **kwargs)

Async method equivalent to sleeping.

Args:

seconds: Float seconds. Default 1.0

kingpin.utils.populate_with_tokens(string, tokens, left_wrapper='%', right_wrapper='%', strict=True)

Insert token variables into the string.

Will match any token wrapped in ‘%’s and replace it with the value of that token.

Args:

string: string to modify. tokens: dictionary of key:value pairs to inject into the string. left_wrapper: the character to use as the START of a token right_wrapper: the character to use as the END of a token strict: (bool) whether or not to make sure all tokens were replaced

Example:

export ME=biz

string=’foo %ME% %bar%’ populate_with_tokens(string, os.environ) # ‘foo biz %bar%’

kingpin.utils.convert_script_to_dict(script_file, tokens)

Converts a JSON file to a config dict.

Reads in a JSON file, swaps out any environment variables that have been used inside the JSON, and then returns a dictionary.

Args:

script_file: Path to the JSON/YAML file to import, or file instance. tokens: dictionary to pass to populate_with_tokens.

Returns:

<Dictonary of Config Data>

Raises:

kingpin.exceptions.InvalidScript

kingpin.utils.order_dict(obj)

Re-orders a dict into a predictable pattern.

Used so that you can compare two dicts with the same values, but that were created in different orders.

Stolen from:

http://stackoverflow.com/questions/25851183/how-to-compare-two-json- objects-with-the-same-elements-in-a-different-order-equa

args:

obj: Object to order

returns:

obj: A sorted version of the object

kingpin.utils.create_repeating_log(logger, message, handle=None, **kwargs)

Create a repeating log message.

This function sets up tornado to repeatedly log a message in a way that does not need to be yield-ed.

Example:

>>> yield do_tornado_stuff(1)
>>> log_handle = create_repeating_log('Computing...')
>>> yield do_slow_computation_with_insufficient_logging()
>>> clear_repeating_log(log_handle)

This is similar to javascript’s setInterval() and clearInterval().

Args:

message: String to pass to log.info() kwargs: values accepted by datetime.timedelta namely seconds, and milliseconds.

Must be cleared via clear_repeating_log() Only handles one interval per actor.

kingpin.utils.clear_repeating_log(handle)

Stops the timeout function from being called.

kingpin.utils.diff_dicts(dict1, dict2)

Compares two dicts and returns the difference as a string, if there is any.

Sorts two dicts (including sorting of the lists!!) and then diffs them. This will ignore string types (‘unicode’ vs ‘string’).

args:

dict1: First dict dict2: Second dict

returns:

A diff string if there’s any difference, otherwise None.