Full Module Docs

`kingpin.actors.aws.base`

The AWS Actors allow you to interact with the resources (such as SQS and ELB) inside your Amazon AWS account. These actors all support dry runs properly, but each actor has its own caveats with dry=True. Please read the instructions below for using each actor.

Required Environment Variables

Note

These can be skipped only if you have a .aws/credentials file in place.

AWS_ACCESS_KEY_ID:: Your AWS access key
AWS_SECRET_ACCESS_KEY:: Your AWS secret
AWS_SESSION_TOKEN:: Your AWS session token. Only needed if you are using temporary access credentials

exception kingpin.actors.aws.base.InvalidPolicy[source]: Raised when Amazon indicates that policy JSON is invalid.

class kingpin.actors.aws.base.AWSBaseActor(*args, **kwargs)[source]

async api_call(api_function: Callable[[...], object], *args: object, **kwargs: object) → object[source]

Execute api_function in a background thread.

Wraps a synchronous boto3 call so it doesn’t block the event loop.

api_call_with_queueing(api_function: Callable[[...], object], queue_name: str, *args: object, **kwargs: object) → object[source]

Execute api_function in a serialized queue.

Concurrent calls to this function are serialized into a queue. When any api function hits rate throttling, it backs up exponentially.

The retry loop will always have a pause between sequential calls, and the delay between the calls will increase as recoverable api failures happen.

The api function is assumed to be a synchronous function. It will be run on a concurrent thread using run_on_executor.

The queue_identifier argument specifies which queue to use. If the queue doesn’t exist, it will be created.

Example:

>>> zones = yield api_call_with_queueing(
>>>     ec2_conn.get_all_zones, queue_name='get_all_zones')

class kingpin.actors.aws.base.EnsurableAWSBaseActor(*args, **kwargs)[source]: Ensurable version of the AWS Base Actor

`kingpin.actors.aws.cloudformation`

class kingpin.actors.aws.cloudformation.DateEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]

default(obj)[source]

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return super().default(o)

exception kingpin.actors.aws.cloudformation.CloudFormationError[source]: Raised on any generic CloudFormation error.

exception kingpin.actors.aws.cloudformation.StackFailed[source]: Raised any time a Stack fails to be created or updated.

exception kingpin.actors.aws.cloudformation.InvalidTemplate[source]: An invalid CloudFormation template was supplied.

exception kingpin.actors.aws.cloudformation.StackAlreadyExists[source]: The requested CloudFormation stack already exists.

exception kingpin.actors.aws.cloudformation.StackNotFound[source]: The requested CloudFormation stack does not exist.

class kingpin.actors.aws.cloudformation.ParametersConfig[source]

Validates the Parameters option.

A valid parameters option is a dictionary with simple Key/Value pairs of strings. No nested dicts, arrays or other objects.

class kingpin.actors.aws.cloudformation.CapabilitiesConfig[source]: Validates the Capabilities option

class kingpin.actors.aws.cloudformation.OnFailureConfig[source]

Validates the On Failure option.

The on_failure option can take one of the following settings: DO_NOTHING, ROLLBACK, DELETE

This option is applied at stack _creation_ time!

class kingpin.actors.aws.cloudformation.TerminationProtectionConfig[source]

Validates the TerminationProtectionConfig option.

The enable_termination_protection option can take one of the following settings: "UNCHANGED", False, True

"UNCHANGED" means on Create Stack it will default to False, however on Ensure Stack no changes will be applied.

class kingpin.actors.aws.cloudformation.CloudFormationBaseActor(*args, **kwargs)[source]

Base Actor for CloudFormation tasks

get_s3_client(region)[source]

Get a boto3 S3 client for a given region.

If the CFN template is stored in S3, we need to download it. The bucket may be in a different region than self.s3_conn, so get a connection that is definitely in the correct region.

class kingpin.actors.aws.cloudformation.Create(*args, **kwargs)[source]

Creates a CloudFormation stack.

Creates a CloudFormation stack from scratch and waits until the stack is fully built before exiting the actor.

Options

Name:

The name of the queue to create

Capabilities:

A list of CFN capabilities to add to the stack.

On_failure:

(OnFailureConfig)

One of the following strings: DO_NOTHING, ROLLBACK, DELETE

Default: DELETE

Parameters:

A dictionary of key/value pairs used to fill in the parameters for the CloudFormation template.

Region:

AWS region (or zone) string, like ‘us-west-2’.

Role_arn:

The Amazon IAM Role to use when executing the stack.

Template:

String of path to CloudFormation template. Can either be in the form of a local file path (ie, /my_template.json) or a URI (ie s3://bucket-name/cfn.json).

Timeout_in_minutes:

The amount of time that can pass before the stack status becomes CREATE_FAILED.

Enable_termination_protection:

Whether termination protection is enabled for the stack.

Examples

{
    "actor": "aws.cloudformation.Create",
    "desc": "Create production backend stack",
    "options": {
        "capabilities": [ "CAPABILITY_IAM" ],
        "name": "%CFN_NAME%",
        "parameters": {
            "test_param": "%TEST_PARAM_NAME%",
        },
        "region": "us-west-1",
        "role_arn": "arn:aws:iam::123456789012:role/DeployRole",
        "template": "/examples/cloudformation_test.json",
        "timeout_in_minutes": 45,
        "enable_termination_protection": true,
    }
}

Dry Mode

Validates the template, verifies that an existing stack with that name does not exist. Does not create the stack.

class kingpin.actors.aws.cloudformation.Delete(*args, **kwargs)[source]

Deletes a CloudFormation stack

Options

Name:: The name of the queue to create
Region:: AWS region (or zone) string, like ‘us-west-2’

Examples

{
    "desc": "Delete production backend stack",
    "actor": "aws.cloudformation.Create",
    "options" {
        "region": "us-west-1",
        "name": "%CFN_NAME%",
    }
}

Dry Mode

Validates that the CFN stack exists, but does not delete it.

class kingpin.actors.aws.cloudformation.Stack(*args, **kwargs)[source]

Manages the state of a CloudFormation stack.

This actor can manage the following aspects of a CloudFormation stack in Amazon:

Ensure that the Stack is present or absent.

Monitor and update the stack Template and Parameters as necessary.

Default Parameters

If your CFN stack defines parameters with defaults, Kingpin will use the defaults unless the parameters are explicitly specified.

NoEcho Parameters

If your CFN stack takes a Password as a parameter or any other value thats secret and you set NoEcho: True on that parameter, Kingpin will be unable to diff it and compare whether or not the desired setting matches whats in Amazon. A warning will be thrown, and the rest of the actor will continue to operate as normal.

If any other difference triggers a Stack Update, the desired value for the parameter with NoEcho: True will be pushed in addition to all of the other stack parameters.

Options

Name:

The name of the queue to create

State:

(str) Present or Absent. Default: “present”

Capabilities:

(CapabilitiesConfig, None)

A list of CFN capabilities to add to the stack.

Disable_rollback:

Set to True to disable rollback of the stack if creation failed.

On_failure:

(OnFailureConfig, None)

One of the following strings: DO_NOTHING, ROLLBACK, DELETE

Default: DELETE

Parameters:

(ParametersConfig, None)

A dictionary of key/value pairs used to fill in the parameters for the CloudFormation template.

Region:

AWS region (or zone) string, like ‘us-west-2’.

Role_arn:

The Amazon IAM Role to use when executing the stack.

Template:

String of path to CloudFormation template. Can either be in the form of a local file path (ie, /my_template.json) or a URI (ie s3://bucket-name/cfn.json).

Timeout_in_minutes:

The amount of time that can pass before the stack status becomes CREATE_FAILED.

Enable_termination_protection:

Whether termination protection is enabled for the stack.

Examples

{
    "actor": "aws.cloudformation.Stack",
    "desc": "Manages the state of a CloudFormation stack",
    "options": {
        "capabilities": [ "CAPABILITY_IAM" ],
        "on_failure": "DELETE",
        "name": "%CFN_NAME%",
        "parameters": {
            "test_param": "%TEST_PARAM_NAME%",
        },
        "region": "us-west-1",
        "role_arn": "arn:aws:iam::123456789012:role/DeployRole",
        "state": "present",
        "template": "/examples/cloudformation_test.json",
        "timeout_in_minutes": 45,
        "enable_termination_protection": true,
    }
}

Dry Mode

Validates the template, verifies that an existing stack with that name does not exist. Does not create the stack.

`kingpin.actors.aws.iam`

class kingpin.actors.aws.iam.IAMBaseActor(*args, **kwargs)[source]

User/Group/Role Base Management Class

Managing Users, Groups and Roles in Amazon IAM is nearly identical. This class abstracts that work, so that the actual User/Group/Role actors can be extremely simple and just handle the differences between each type of IAM entity.

class kingpin.actors.aws.iam.User(*args, **kwargs)[source]

Manages an IAM User.

This actor manages the state of an Amazon IAM User.

Currently we can:

Ensure is present or absent

Manage the inline policies for the user

Manage the groups the user is in

Options

Name:: (str) Name of the User profile to manage
State:: (str) Present or Absent. Default: “present”
Groups:: (str,array) A list of groups for the user to be a member of. Default: None
Inline_policies:: (str,array) A list of strings that point to JSON files to use as inline policies. Default: None

Example

{ "actor": "aws.iam.User",
  "desc": "Ensure that Bob exists",
  "options": {
    "name": "bob",
    "state": "present",
    "groups": "my-test-group",
    "inline_policies": [
      "read-all-s3.json",
      "create-other-stuff.json"
    ]
  }
}

Dry run

Will let you know if the user exists or not, and what changes it would make to the users policy and settings. Will also parse the inline policies supplied, make sure any tokens in the files are replaced, and that the files are valid JSON.

class kingpin.actors.aws.iam.Group(*args, **kwargs)[source]

Manages an IAM Group.

This actor manages the state of an Amazon IAM Group.

Currently we can:

Ensure is present or absent

Manage the inline policies for the group

Purge (or not) all group members and delete the group

Options

Name:: (str) Name of the Group profile to manage
Force:: (bool) Forcefully delete the group (explicitly purging all group memberships). Default: false
State:: (str) Present or Absent. Default: “present”
Inline_policies:: (str,array) A list of strings that point to JSON files to use as inline policies. You can also pass in a single inline policy as a string. Default: None

Example

{ "actor": "aws.iam.Group",
  "desc": "Ensure that devtools exists",
  "options": {
    "name": "devtools",
    "state": "present",
    "inline_policies": [
      "read-all-s3.json",
      "create-other-stuff.json"
    ]
  }
}

Dry run

Will let you know if the group exists or not, and what changes it would make to the groups policy and settings. Will also parse the inline policies supplied, make sure any tokens in the files are replaced, and that the files are valid JSON.

class kingpin.actors.aws.iam.Role(*args, **kwargs)[source]

Manages an IAM Role.

This actor manages the state of an Amazon IAM Role.

Currently we can:

Ensure is present or absent

Manage the inline policies for the role

Manage the Assume Role Policy Document

Options

Name:

(str) Name of the Role to manage

State:

(str) Present or Absent. Default: “present”

Inline_policies:

(str,array) A list of strings that point to JSON files to use as inline policies. You can also pass in a single inline policy as a string. Default: None

Assume_role_policy_document:

(str) A string with an Amazon IAM Assume Role policy. Not providing this causes Kingpin to ignore the value, and Amazon defaults the role to an ‘EC2’ style rule. Supplying the document will cause Kingpin to ensure the assume role policy is correct.

Default:

{ "Version": "2012-10-17",
  "Statement": [
      { "Effect": "Allow",
        "Principal": {
            "Service": "ec2.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
      }
  ]
}

Example

{ "actor": "aws.iam.Role",
  "desc": "Ensure that myapp exists",
  "options": {
    "name": "myapp",
    "state": "present",
    "inline_policies": [
      "read-all-s3.json",
      "create-other-stuff.json"
    ]
  }
}

Dry run

Will let you know if the group exists or not, and what changes it would make to the groups policy and settings. Will also parse the inline policies supplied, make sure any tokens in the files are replaced, and that the files are valid JSON.

class kingpin.actors.aws.iam.InstanceProfile(*args, **kwargs)[source]

Manages an IAM Instance Profile.

This actor manages the state of an Amazon IAM Instance Profile.

Currently we can:

Ensure is present or absent

Assign an IAM Role to the Instance Profile

Options

Name:: (str) Name of the Role to manage
State:: (str) Present or Absent. Default: “present”
Role:: (str) Name of an IAM Role to assign to the Instance Profile. Default: None

Example

{ "actor": "aws.iam.InstanceProfile",
  "desc": "Ensure that my-ecs-servers exists",
  "options": {
    "name": "my-ecs-servers",
    "state": "present",
    "role": "some-iam-role",
  }
}

Dry run

Will let you know if the profile exists or not, and what changes it would make to the profile.

`kingpin.actors.aws.settings`

Common settings used by many of the kingpin.actors.aws modules.

`kingpin.actors.aws.s3`

exception kingpin.actors.aws.s3.InvalidBucketConfig[source]: Raised whenever an invalid option is passed to a Bucket

class kingpin.actors.aws.s3.PublicAccessBlockConfig[source]

Provides JSON-Schema based validation of the supplied Public Access Block Configuration..

The S3 PublicAccessBlockConfiguration should look like this:

{
    "block_public_acls": true,
    "ignore_public_acls": true,
    "block_public_policy": true,
    "restrict_public_buckets": true
}

If you supply an empty dict, then we will explicitly remove the Public Access Block Configuration.

class kingpin.actors.aws.s3.LoggingConfig[source]

Provides JSON-Schema based validation of the supplied logging config.

The S3 LoggingConfig format should look like this:

{
    "target": "s3_bucket_name_here",
    "prefix": "an_optional_prefix_here"
}

If you supply an empty target, then we will explicitly remove the logging configuration from the bucket. Example:

{ "target": "" }

class kingpin.actors.aws.s3.LifecycleConfig[source]

Provides JSON-Schema based validation of the supplied Lifecycle config.

The S3 Lifecycle system allows for many unique configurations. Each configuration object defined in this schema will be turned into a boto.s3.lifecycle.Rule object. All of the rules together will be turned into a boto.s3.lifecycle.Lifecycle object.

[
    {
        "id": "unique_rule_identifier",
        "status": "Enabled",
        "filter": {
            "prefix": "/some_path"
        },
        "transitions": [
            {
                "days": 90,
                "date": "2016-05-19T20:04:17+00:00",
                "storage_class": "GLACIER",
            }
        ],
        "noncurrent_version_transitions": [
            {
                "noncurrent_days": 90,
                "storage_class": "GLACIER",
            }
        ],
        "expiration": {
            "days": 365,
        },
        "noncurrent_version_expiration": {
            "noncurrent_days": 365,
        }
    }
]

class kingpin.actors.aws.s3.NotificationConfiguration[source]

Provides JSON-Schema based validation of the supplied Notification Config.

{
   "queue_configurations": [
       {
          "queue_arn": "ARN of the SQS queue",
          "events": ["s3:ObjectCreated:*"],
       }
  ]
}

class kingpin.actors.aws.s3.TaggingConfig[source]

Provides JSON-Schema based validation of the supplied tagging config.

The S3 TaggingConfig format should look like this:

[ { "key": "my_key", "value": "some_value" } ]

class kingpin.actors.aws.s3.Bucket(*args, **kwargs)[source]

Manage the state of a single S3 Bucket.

The actor has the following functionality:

Ensure that an S3 bucket is present or absent.

Manage the bucket policy.

Manage the bucket Lifecycle configurations.

Enable or Suspend Bucket Versioning. Note: It is impossible to actually _disable_ bucket versioning – once it is enabled, you can only suspend it, or re-enable it.

Enable Event Notification. (limited to SQS for now)

Note about Buckets with Files

Amazon requires that an S3 bucket be empty in order to delete it. Although we could recursively search for all files in the bucket and then delete them, this is a wildly dangerous thing to do inside the confines of this actor. Instead, we raise an exception and alert the you to the fact that they need to delete the files themselves.

Options

Name:

The name of the bucket to operate on

State:

(str) Present or Absent. Default: “present”

Lifecycle:

(LifecycleConfig, None)

A list of individual Lifecycle configurations. Each dictionary includes keys for:

id
status
filter (or prefix, which is deprecated)

and at least one of:

transitions (or transition, which is deprecated)
noncurrent_version_transitions (or noncurrent_version_transition)
expiration
noncurrent_version_expiration
abort_incomplete_multipart_upload

If an empty list is supplied, or the list in any way does not match what is currently configured in Amazon, the appropriate changes will be made.

Logging:

(LoggingConfig, None)

If a dictionary is supplied ({'target': 'logging_bucket', 'prefix': '/mylogs'}), then we will configure bucket logging to the supplied bucket and prefix. If prefix is missing then no prefix will be used.

If target is supplied as an empty string (''), then we will disable logging on the bucket. If None is supplied, we will not manage logging either way.

Tags:

(TaggingConfig, None)

A list of dictionaries with a key and value key. Defaults to an empty list, which means that if you manually add tags, they will be removed.

Policy:

(str, None) A JSON file with the bucket policy. Passing in a blank string will cause any policy to be deleted. Passing in None (or not passing it in at all) will cause Kingpin to ignore the policy for the bucket entirely. Default: None

Public_access_block_configuration:

(PublicAccessBlockConfig, None)

If a dictionary is supplied, then it must conform to the PublicAccessBlockConfig type and include all of the Public Access Block Configuration parameters.

If an empty dictionary is supplied, then Kingpin will explicitly remove any Public Access Block Configurations from the bucket.

Finally, if None is supplied, Kingpin will ignore the checks entirely on this portion of the bucket configuration.

Default: None

Region:

AWS region (or zone) name, such as us-east-1 or us-west-2

Versioning:

(bool, None): Whether or not to enable Versioning on the bucket. If “None”, then we don’t manage versioning either way. Default: None

Notification_configuration:

(NotificationConfiguration, None)

If a dictionary is supplised, then it must conform to NotificationConfiguration, type and include mapping queuearn & events

If an empty dictionary is supplied, then Kingpin will explicitly remove any Notification Configuration from the bucket.

Finally, If None is supplies, Kingoin will ignore the checks entire on this portion of the bucket configuration

Examples

{
  "actor": "aws.s3.Bucket",
  "options": {
    "name": "kingpin-integration-testing",
    "region": "us-west-2",
    "policy": "./examples/aws.s3/amazon_put.json",
    "lifecycle": [
       {
         "id": "main",
         "status": "Enabled",
         "filter": {
             "prefix": "/"
         },
         "expiration": 30,
       }
    ],
    "logging": {
      "target": "logs.myco.com",
      "prefix": "/kingpin-integratin-testing"
    },
    "tags": [
      {"key": "my_key", "value": "billing-grp-1"},
    ],
    "versioning": true,
    "notification_configuration": {
       "queue_configurations": [
         {
           "queue_arn": "arn:aws:sqs:us-east-1:1234567:some_sqs",
           "events": [
                         "s3:ObjectCreated:*",
                         "s3:ObjectRemoved*"
                     ]
         }
       ]
    }
  }
}

Dry Mode

Finds the bucket if it exists (or tells you it would create it). Describes each potential change it would make to the bucket depending on the configuration of the live bucket, and the options that were passed into the actor.

Will gracefully fail and alert you if there are files in the bucket and you are trying to delete it.

`kingpin.actors.base`

Base Actor object class

An Actor object is a class that executes a single logical action on a resource as part of your deployment structure. For example, you may have an Actor that launches a server array in RightScale, or you may have one that sends an email.

Each Actor object should do one thing, and one thing only. Its responsible for being able to execute the operation in both ‘dry’ and ‘non-dry’ modes.

The behavior for ‘dry’ mode can contain real API calls, but should not make any live changes. It is up to the developer of the Actor to define what ‘dry’ mode looks like for that particular action.

class kingpin.actors.base.LogAdapter(logger, extra=None, merge_extra=False)[source]

Simple Actor Logging Adapter.

Provides a common logging format for actors that uses the actors description and dry parameter as a prefix to the supplied log message.

process(msg, kwargs)[source]

Process the logging message and keyword arguments passed in to a logging call to insert contextual information. You can either manipulate the message itself, the keyword args or both. Return the message and kwargs modified (or not) to suit your needs.

Normally, you’ll only need to override this one method in a LoggerAdapter subclass for your specific needs.

class kingpin.actors.base.BaseActor(desc: str | None = None, options: dict[str, object] = {}, dry: bool = False, warn_on_failure: bool = False, condition: bool | str = True, init_context: dict[str, str] = {}, init_tokens: dict[str, str] = {}, timeout: str | int | float | None = None)[source]

Abstract base class for Actor objects.

option(name)[source]: Return the value for a given Actor option.

readfile(path)[source]

Return file contents as a string.

Raises:: InvalidOptions if file is not found, or readable.

async timeout(f: Callable[[...], object], *args: object, **kwargs: object) → object[source]

Wraps a Coroutine method in a timeout.

Used to wrap the self.execute() method in a timeout that will raise an ActorTimedOut exception if an actor takes too long to execute.

Note

This method intentionally does NOT cancel the underlying task on timeout. The coroutine continues running in the background — we only notify the caller (via ActorTimedOut) that the deadline was exceeded. This preserves the original gen.with_timeout() contract and is important for long-running actors (e.g. CloudFormation stack operations) that should be allowed to finish even after the caller has moved on. asyncio.shield() prevents asyncio.wait_for() from cancelling the inner future.

get_orgchart(parent: str = '') → list[dict[str, str | None]][source]

Construct organizational chart describing this actor.

Return a list of actors handled by this actor. Most actors will return a list of just one object. Grouping actors will return a list of all actors that are called.

orgchart object:: id: unique string identifying this actor’s instance. class: kingpin class name desc: actor description parent_id: organizational relationship. Same as id above.

class kingpin.actors.base.EnsurableBaseActor(*args, **kwargs)[source]

Base Class for Actors that “ensure” the state of a resource.

Many of our actors have a goal of ensuring that a particular resource is in a given state. This leads to a ton of boiler plate code to “get” the state of something, “compare” that to the desired state, and then maybe “set” the state.

This actor provides a framework allowing the user to simply write the getters and setters (and optionally compare), and lets the rest of the actor handle the order of operations.

Required Methods:

_set_state:

Creates or destroys the resource depending on the ‘state’ parameter that was passed in. Note: The ‘state’ parameter is automatically added to the options. You do not need to define it.

_get_state:

Gets the current state of the resource.

_set_[option]:

A ‘setter’ for each option name passed in.

_get_[option]:

A ‘getter’ for each option name passed in.

Optional Methods:

_precache:

Called before any setters/getters are triggered. Used to optionally populate a cache of data to make the getters faster. For example, if you can make one API call to get all of the data about a resource, then store that data locally for fast access.

_compare_[option]:

Optionally you can write your own comparison method if you’re not doing a pure string comparison between the source and destination.

Examples

class MyClass(base.EnsurableBaseActor):

    all_options = {
        'name': (str, REQUIRED, 'Name of thing'),
        'description': (str, None, 'Description of thing')
    }

    unmanaged_options = ['name']

    async def _set_state(self):
        if self.option('state') == 'absent':
            await self.conn.delete_resource(
                name=self.option('name'))
        else:
            await self.conn.create_resource(
                name=self.option('name'),
                desc=self.option('description'))

    async def _set_description(self):
        await self.conn.set_desc_of_resource(
            name=self.option('name'),
            desc=self.option('description'))

    async def _get_description(self):
        await self.conn.get_desc_of_resource(
            name=self.option('name'))

class kingpin.actors.base.HTTPBaseActor(desc: str | None = None, options: dict[str, object] = {}, dry: bool = False, warn_on_failure: bool = False, condition: bool | str = True, init_context: dict[str, str] = {}, init_tokens: dict[str, str] = {}, timeout: str | int | float | None = None)[source]

Abstract base class for an HTTP-client based Actor object.

This class provides common methods for getting access to asynchronous HTTP clients, wrapping the executions in appropriate try/except blocks, timeouts, etc.

If you’re writing an Actor that uses a remote REST API, this is the base class you should subclass from.

`kingpin.actors.exceptions`

All common Actor exceptions

exception kingpin.actors.exceptions.ActorException[source]: Base Kingpin Actor Exception

exception kingpin.actors.exceptions.RecoverableActorFailure[source]

Base exception that allows script executions to continue on failure.

This exception class is used to throw an error when an Actor fails, but it was an expected and/or acceptable failure.

This should be used for exceptions that are somewhat normal … for example, trying to delete a ServerArray thats already gone.

exception kingpin.actors.exceptions.UnrecoverableActorFailure[source]

Base exception for unrecoverable failures.

This exception class should be used for critical failures that should always stop a set of Kingpin actors in-place, regardless of the actors warn_on_failure setting.

Examples would be when credentials are incorrect, or an unexpected exception is caught and there is no known recovery point.

exception kingpin.actors.exceptions.ActorTimedOut[source]: Raised when an Actor takes too long to execute

exception kingpin.actors.exceptions.InvalidActor[source]: Raised when an invalid Actor name was supplied

exception kingpin.actors.exceptions.InvalidOptions[source]

Invalid option arguments passed into the Actor object.

This can be used both for the actual options dict passed into the actor, as well as if a the wrong options were used when connecting to a remote API.

exception kingpin.actors.exceptions.InvalidCredentials[source]: Invalid or missing credentials required for Actor object.

exception kingpin.actors.exceptions.UnparseableResponseFromEndpoint[source]: Invalid response returned from a remote REST endpoint.

exception kingpin.actors.exceptions.BadRequest[source]: An action failed due to a HTTP 400 error likely due to bad input.

`kingpin.actors.group`

Group a series of other BaseActor into either synchronous or asynchronous stages.

class kingpin.actors.group.BaseGroupActor(*args, **kwargs)[source]

Group together a series of other kingpin.actors.base.BaseActor objects

Acts:: [ <list of kingpin.actors.base.BaseActor objects to execute> ]

get_orgchart(parent='')[source]: Generate an orgchart for all the acts specified.

class kingpin.actors.group.Sync(*args, **kwargs)[source]

Execute a series of kingpin.actors.base.BaseActor synchronously.

Groups together a series of Actors and executes them synchronously in the order that they were defined.

Options

Acts:

An array of individual Actor definitions.

Contexts:

This variable can be one of two formats:

A list of dictionaries with contextual tokens to pass into the
actors at instantiation time. If the list has more than one element, then every actor defined in acts will be instantiated once for each item in the contexts list.
A string that points to a file with a list of contexts, just like the
above dictionary format.

Timeouts

Timeouts are disabled specifically in this actor. The sub-actors can still raise their own kingpin.actors.exceptions.ActorTimedOut exceptions, but since the group actors run an arbitrary number of sub actors, we have chosen to not have this actor specifically raise its own kingpin.actors.exceptions.ActorTimedOut exception unless the user sets the timeout setting.

Examples

Creates two arrays … but sleeps 60 seconds between the two, then does not sleep at all after the last one:

{
    "desc": "Clone, then sleep ... then clone, then sleep shorter...",
    "actor": "group.Sync",
    "options": {
        "contexts": [
            { "ARRAY": "First", "SLEEP": "60", },
            { "ARRAY": "Second", "SLEEP": "0", }
        ],
        "acts": [
            {
                "desc": "do something",
                "actor": "server_array.Clone",
                "options": {
                    "source": "template",
                    "dest": "{ARRAY}"
                }
            },
            {
                "desc": "sleep",
                "actor": "misc.Sleep",
                "options": {
                    "sleep": "{SLEEP}",
                }
            }
        ]
    }
}

Alternatively if no contexts are needed you can use the array syntax.

[
    {
        "actor": "server_array.Clone",
        "options": {
            "source": "template",
            "dest": "%ARRAY%"
        }
    },
    {
        "actor": "misc.Sleep",
        "options": { "sleep": 30 }
    }
]

Dry Mode

Passes on the Dry mode setting to the acts that are called. Does not stop execution when one of the acts fails. Instead Group actor will finish all acts with warnings, and raise an error at the end of execution.

This provides the user with an insight to all the errors that are possible to encounter, rather than abort and quit on the first one.

Failure

In the event that an act fails, this actor will return the failure immediately. Because the acts are executed in-order of definition, the failure will prevent any further acts from executing.

The behavior is different in the dry run (read above.)

class kingpin.actors.group.Async(*args, **kwargs)[source]

Execute several kingpin.actors.base.BaseActor objects asynchronously.

Groups together a series of Actors and executes them asynchronously - waiting until all of them finish before returning.

Options

Concurrency:

Max number of concurrent executions. This will fire off N executions in parallel, and continue with the remained as soon as the first execution is done. This is faster than creating N Sync executions.

Acts:

An array of individual Actor definitions.

Contexts:

This variable can be one of two formats:

A list of dictionaries with contextual tokens to pass into the
actors at instantiation time. If the list has more than one element, then every actor defined in acts will be instantiated once for each item in the contexts list.
A string that points to a file with a list of contexts, just like the
above dictionary format.

Timeouts

Timeouts are disabled specifically in this actor. The sub-actors can still raise their own kingpin.actors.exceptions.ActorTimedOut exceptions, but since the group actors run an arbitrary number of sub actors, we have chosen to not have this actor specifically raise its own kingpin.actors.exceptions.ActorTimedOut exception unless the user sets the timeout setting.

Examples

Clone two arrays quickly.

{
    "desc": "Clone two arrays",
    "actor": "group.Async",
    "options": {
        "contexts": [
            { "ARRAY": "NewArray1" },
            { "ARRAY": "NewArray2" }
        ],
        "acts": [
            {
                "desc": "do something",
                "actor": "server_array.Clone",
                "options": {
                    "source": "template",
                    "dest": "{ARRAY}",
                }
            }
        ]
    }
}

Dry Mode

Passes on the Dry mode setting to the sub-actors that are called.

Failure

In the event that one or more acts fail in this group, the entire group acts will return a failure to Kingpin. Because multiple actors are executing all at the same time, the all of these actors will be allowed to finish before the failure is returned.

`kingpin.actors.misc`

These are common utility Actors that don’t really need their own dedicated packages. Things like sleep timers, loggers, etc.

Optional Environment Variables

URLLIB_DEBUG:: Set this variable to enable extreme debug logging of the URLLIB requests made by the RightScale/AWS actors.

Note

This is very insecure as headers/cookies/etc. are exposed*

class kingpin.actors.misc.Note(desc: str | None = None, options: dict[str, object] = {}, dry: bool = False, warn_on_failure: bool = False, condition: bool | str = True, init_context: dict[str, str] = {}, init_tokens: dict[str, str] = {}, timeout: str | int | float | None = None)[source]: Print any message to log.

class kingpin.actors.misc.Macro(*args, **kwargs)[source]

Parses a kingpin script, instantiates and executes it.

Parse JSON/YAML

Kingpin JSON/YAML has 2 passes at its validity. Script syntax must be valid clean JSON.

The second pass is validating the Schema. The script will be validated for schema-conformity as one of the first things that happens at load-time when the app starts up. If it fails, you will be notified immediately.

Lastly after the JSON/YAML is established to be valid, all the tokens are replaced with their specified value. Any key/value pair passed in the tokens option will be available inside of the JSON file as %KEY% and replaced with the value at this time.

In a situation where nested Macro executions are invoked the tokens do not propagate from outter macro into the inner. This allows to reuse token names, but forces the user to specify every token needed. Similarly, if environment variables are used for token replacement in the main file, these tokens are not available in the subsequent macros.

Pre-Instantiation

In an effort to prevent mid-run errors, we pre-instantiate all Actor objects all at once before we ever begin executing code. This ensures that major typos or misconfigurations in the JSON/YAML will be caught early on.

Execution

misc.Macro actor simply calls the execute() method of the most-outter actor; be it a single action, or a group actor.

Options

Macro:: String of local path to a JSON/YAML script.
Tokens:: Dictionary to search/replace within the file.

Examples

{ "desc": "Stage 1",
  "actor": "misc.Macro",
  "options": {
    "macro": "deployment/stage-1.json",
    "tokens": {
      "TIMEOUT": 360,
      "RELEASE": "%RELEASE%"
    }
  }
}

Dry Mode

Fully supported – instantiates the actor inside of JSON with dry=True. The behavior of the consecutive actor is unique to each; read their description for more information on dry mode.

get_orgchart(parent='')[source]: Return orgchart including the actor inside of the macro file.

class kingpin.actors.misc.Sleep(desc: str | None = None, options: dict[str, object] = {}, dry: bool = False, warn_on_failure: bool = False, condition: bool | str = True, init_context: dict[str, str] = {}, init_tokens: dict[str, str] = {}, timeout: str | int | float | None = None)[source]

Sleeps for an arbitrary number of seconds.

Options

Sleep:: Integer of seconds to sleep.

Examples

{ "actor": "misc.Sleep",
  "desc": "Sleep for 60 seconds",
  "options": {
    "sleep": 60
  }
}

Dry Mode

Fully supported – does not actually sleep, just pretends to.

class kingpin.actors.misc.GenericHTTP(desc: str | None = None, options: dict[str, object] = {}, dry: bool = False, warn_on_failure: bool = False, condition: bool | str = True, init_context: dict[str, str] = {}, init_tokens: dict[str, str] = {}, timeout: str | int | float | None = None)[source]

A very simple actor that allows GET/POST methods over HTTP.

Does a GET or a POST to a specified URL.

Options

Url:: Destination URL
Data:: Optional POST data as a dict. Will convert into key=value&key2=value2.. Exclusive of data-json option.
Data-json:: Optional POST data as a dict. Will stringify and pass as JSON. Exclusive of data option.
Username:: Optional for HTTPAuth.
Password:: Optional for HTTPAuth.

Examples

{ "actor": "misc.GenericHTTP",
  "desc": "Make a simple web call",
  "options": {
    "url": "http://example.com/rest/api/v1?id=123&action=doit",
    "username": "secret",
    "password": "%SECRET_PASSWORD%"
  }
}

Dry Mode

Will not do anything in dry mode except print a log statement.

`kingpin.actors.utils`

Misc methods for dealing with Actors.

kingpin.actors.utils.dry(dry_message)[source]

Async-compatible decorator to dry-run a method.

Note

This must act on a BaseActor object.

Args:: dry_message: The message to print out instead of doing the actual function call. This string is passed through format(kwargs), so any variables you’d like can be substituted as long as they’re passed to the method being wrapped.

kingpin.actors.utils.timer(f)[source]

Async-compatible function timer.

Records statistics about how long a given function took, and logs them out in debug statements. Used primarily for tracking Actor execute() methods, but can be used elsewhere as well.

Note

This must act on a BaseActor object.

kingpin.actors.utils.get_actor(config: dict[str, object], dry: bool) → object[source]

Returns an initialized Actor object.

Args:

config: A dictionary of configuration data that conforms to our v1

schema in kingpin.schema. Looks like this:

{: ‘desc’: <string description of actor>, ‘actor’: <string name of actor> ‘options’: <dict of options to pass to actor> ‘warn_on_failure’: <bool> ‘condition’: <string or bool> }

dry: Boolean whether or not in Dry mode warn_on_failure: Boolean

Returns:

kingpin.actors.utils.get_actor_class(actor: str) → type[source]

Returns a Class Reference to an Actor by string name.

Args:: actor: String name of the actor to find.
Returns:: <Class Ref to Actor>

class kingpin.constants.REQUIRED[source]: Meta class to identify required arguments for actors.

class kingpin.constants.StringCompareBase[source]

Meta class to identify the desired state for a resource.

This basic type of constant allows someone to easily define a set of valid strings for their option and have the base actor class automatically validate the inputs against those strings.

class kingpin.constants.STATE[source]

Meta class to identify the desired state for a resource.

Simple tester for ‘present’ or ‘absent’ on actors. Used for any actor thats idempotent and used to ensure some state of a resource.

class kingpin.constants.SchemaCompareBase[source]: Meta class that compares the schema of a dict against rules.

exception kingpin.exceptions.KingpinException[source]: Base Exception

exception kingpin.exceptions.InvalidScript[source]: Raised when an invalid script schema was detected

exception kingpin.exceptions.InvalidScriptName[source]: Raised when the script name does not end on .yaml or .json

kingpin.schema.validate(config: dict | list) → None[source]

Validates the JSON against our schemas.

TODO: Support multiple schema versions

Args:: config: Dictionary of parsed JSON
Returns:: None: if all is well
Raises:: Execption if something went wrong.

`kingpin.utils`

Common package for utility functions.

kingpin.utils.str_to_class(string: str) → type[source]

Method that converts a string name into a usable Class name

This is used to take the ‘actor’ value from the JSON object and convert it into a valid object reference.

Args:

cls: String name of the wanted class and package. (eg:: kingpin.actors.foo.bar, misc.Sleep, actors.misc.Sleep, my.private.Actor, etc.)

Returns:

A reference to the actual Class to be instantiated

kingpin.utils.setup_root_logger(level: str = 'warn', syslog: str | None = None, color: bool = False) → Logger[source]

Configures the root logger.

Args:: level: Logging level string (‘warn’ is default) syslog: String representing syslog facility to output to. If empty, logs are written to console. color: Colorize the log output
Returns:: A root Logger object

kingpin.utils.super_httplib_debug_logging()[source]

Enables DEBUG logging deep in HTTPLIB.

HTTPLib by default doens’t log out things like the raw HTTP headers, cookies, response body, etc – even when your main logger is in DEBUG mode. This is because its a security risk, as well as just highly verbose.

For the purposes of debugging though, this can be useful. This method enables deep debug logging of the HTTPLib web requests. This is highly insecure, but very useful when troubleshooting failures with remote API endpoints.

Returns:: Requests ‘logger’ object (mainly for unit testing)

kingpin.utils.exception_logger(func)[source]

Explicitly log Exceptions then Raise them.

Logging exceptions and tracebacks while inside of a thread can swallow most of the traceback and only give you the raw exception object. This helper method logs the full traceback before re-raising the exception.

kingpin.utils.retry(excs, retries=3, delay=0.25)[source]

Async-compatible Retry Decorator.

This decorator provides a simple retry mechanism that looks for a particular set of exceptions and retries async tasks in the event that those exceptions were caught.

Args:: excs: A single (or tuple) exception type to catch. retries: The number of times to try the operation in total. delay: Time (in seconds) to wait between retries

kingpin.utils.populate_with_tokens(string: str, tokens: dict[str, object], left_wrapper: str = '%', right_wrapper: str = '%', strict: bool = True, escape_sequence: str = '\\', remove_escape_sequence: bool = True) → str[source]

Insert token variables into the string.

Will match any token wrapped in ‘%’s and replace it with the value of that token.

Args:

string: string to modify. tokens: dictionary of key:value pairs to inject into the string. left_wrapper: the character to use as the START of a token right_wrapper: the character to use as the END of a token strict: (bool) whether or not to make sure all tokens were replaced escape_sequence: character string to use as the escape sequence for left and right wrappers remove_escape_sequence: (bool) whether or not to remove the escape sequence if it found. For example %FOO% would turn into %FOO%.

Example:

export ME=biz

string=’foo %ME% %bar%’ populate_with_tokens(string, os.environ) # ‘foo biz %bar%’

kingpin.utils.load_json_with_tokens(file_path: str | IOBase, tokens: dict[str, object]) → dict | list[source]

Converts a JSON/YAML file to a Python object.

Reads in a JSON/YAML file, swaps out any environment variables that have been used inside the file, and then returns the parsed object (dict, list, or other JSON-compatible type).

Args:: file_path: Path to the JSON/YAML file to import, or file instance. tokens: dictionary to pass to populate_with_tokens.
Returns:: Parsed object (dict, list, or other JSON-compatible type)
Raises:: kingpin.exceptions.InvalidScript kingpin.exceptions.InvalidScriptName

kingpin.utils.order_dict(obj)[source]

Re-orders a dict into a predictable pattern.

Used so that you can compare two dicts with the same values, but that were created in different orders.

Stolen from: http://stackoverflow.com/questions/25851183/how-to-compare-two-json-objects-with-the-same-elements-in-a-different-order-equa

Args:: obj: Object to order
Returns:: obj: A sorted version of the object

kingpin.utils.create_repeating_log(logger, message, handle=None, **kwargs)[source]

Create a repeating log message.

Similar to JavaScript’s setInterval() / clearInterval(). Must be cleared via clear_repeating_log().

Args:: logger: Callable to invoke with message (e.g. log.info) message: String to pass to logger kwargs: values accepted by datetime.timedelta (seconds, milliseconds)

kingpin.utils.clear_repeating_log(handle)[source]: Stops the repeating log message.

kingpin.utils.diff_dicts(dict1: dict, dict2: dict) → str | None[source]

Compares two dicts and returns the difference as a string, if there is any.

Sorts two dicts (including sorting of the lists!!) and then diffs them. This will ignore string types (‘unicode’ vs ‘string’).

Args:: dict1: First dict dict2: Second dict
Returns:: A diff string if there’s any difference, otherwise None.

kingpin.utils.str2bool(v: str | bool | int, strict: bool = False) → bool[source]

Returns a Boolean from a variety of inputs.

Args:: value: String/Bool strict: Whether or not to _only_ convert the known words into booleans, or whether to allow “any” word to be considered True other than the known False words.
Returns:: A boolean

Kingpin version number. You must bump this when creating a new release.