Intro to Toxiproxy

A TCP proxy to simulate network and system conditions for chaos and resiliency testing

Install Toxiproxy via brew

Disclaimer: All steps done with toxiproxy v2.1.2

To keep matters simple, I chose Wordpress as an example application to take toxiproxy for a test drive. I use a classic docker setup consisting of a docker-compose.yml with a vanilla wordpress and a mysql service and then added toxiproxy in between.

Application [wordpress] --> Toxiproxy [proxy] --> Database [mysql]

Before we get into the details, quickly install the toxiproxy server and client in OSX with (we’ll only need the client):

brew tap shopify/shopify
brew install toxiproxy

Setup

Since toxiproxy is the man in the middle it needs to be initialized before our application can connect to its database. The docs say:

When your application boots, it needs to make sure that Toxiproxy knows which endpoints to proxy where. The main parameters are: name, address for Toxiproxy to listen on and the address of the upstream.

This code needs to run as early in boot as possible, before any code establishes a connection through Toxiproxy

The official toxiproxy docker image consists of alpine plus toxiproxy server and client. Since I didn’t want to modify the official image I added an init service to the docker-compose file.

When the stack is started the init container exits after configuring our proxy container. This can be verified with:

$ toxiproxy-cli list
Name                    Listen          Upstream                Enabled         Toxics
======================================================================================
tox_wordpress_mysql_db  [::]:3306       mysql:3306              enabled         None

After finishing the wordpress setup via http://localhost:8080, we log in and can see the default website and the wordpress dashboard.

Time to get the toxics ☠️

Toxics manipulate the way traffic passes through toxiproxy. The docs say:

Toxics manipulate the pipe between the client and upstream. They can be added and removed from proxies using the HTTP API. Each toxic has its own parameters to change how it affects the proxy links.

And include: latency, down, bandwidth, slow_close, timeout, slicer, limit_data

Adding a toxic uses the following syntax:

usage: toxiproxy-cli add <proxyName> --type <toxicType> --toxicName <toxicName> \
    --attribute <key=value> --upstream --downstream

So adding a latency toxic with random jitter looks like this:

$ toxiproxy-cli toxic add tox_wordpress_mysql_db --type latency --toxicName myToxic --attribute latency=300 --attribute jitter=150
# or shorter:
# toxiproxy-cli toxic add tox_wordpress_mysql_db -t latency -n myToxic -a latency=300 -a jitter=150
Added downstream latency toxic 'myToxic' on proxy 'tox_wordpress_mysql_db'

$ toxiproxy-cli list
Name                    Listen          Upstream                Enabled         Toxics
======================================================================================
tox_wordpress_mysql_db  [::]:3306       mysql:3306              enabled         1

$ toxiproxy-cli inspect tox_wordpress_mysql_db
Name: tox_wordpress_mysql_db    Listen: [::]:3306       Upstream: mysql:3306
======================================================================
Upstream toxics:
Proxy has no Upstream toxics enabled.

Downstream toxics:
myToxic:        type=latency    stream=downstream       toxicity=1.00   attributes=[    jitter=150      latency=300     ]