#test strategy

In the my last post, I reposted an article from 2018 about modifying a simple script to make it more testable. It’s been two years since I wrote that, and the way I evaluate testing needs for code has changed. Let’s look at that script again.


The script I wrote emptied AWS S3 buckets of any objects to get them ready to be deleted. AWS won’t delete an S3 bucket if it has any objects (files or subdirectories) in it. Here were the requirements of my script:

  1. We stored buckets in each AWS region, but they all have a similar name: logs-${region}. We need to delete objects in these buckets.
  2. Sometimes there wouldn’t be a bucket for a region, but we need to check them all. If a bucket doesn’t exist or is empty, skip it and continue.
  3. Optimize for speed over quality. This is a one off script.

So, to test it, I’m not sure I need to test it to death. I need to make sure that it:

  • empties every bucket that matches the naming pattern
  • skips (and logs) any buckets that aren’t there
  • doesn’t die on any errors

Changes I’d Make

I want to make sure both the script and test(s) aren’t brittle. AWS updates its regions often enough that I don’t want to bury the region list too deeply. I also don’t want to hardcode the bucket we’re trying to empty.

On the testing side, this will also help me write tests that won’t break unless the requirements for the script change. So here are my changes:

  1. Make the bucket name an argument
  2. Make the region list an argument
  3. Mock AWS S3 to create tests around the three basic requirements:
  • make sure buckets are empty after running the script
  • make sure it skips and logs any buckets that don’t exist
  • make sure a bucket that doesn’t match the pattern is not emptied.

The Script

#!/usr/bin/env python3
import subprocess

def empty_buckets():

    regions = [

    bucket_template = 'fugue-e2e-s3-%s-logging'

    for r in regions:
        bucket_name =  bucket_template % (r)
        print('Verifying logging bucket in %s exists and contains objects.' % (r))
        call = "aws --region {} s3api head-bucket --bucket {}".format(r, bucket_name)

            subprocess.check_output([call, ""], shell=True)

            print('Bucket exists. Emptying.')
            output = subprocess.check_output(["aws --region {} s3 rm s3://{} --recursive".format(r, bucket_name), ""], shell=True)
        except Exception as e:
            print('Error: logging bucket in %s %s' % (r, e))

If you’ve been looking around for information on unit testing and want to know a bit more, or possibly see an example of how to put it into practice, you’re in the right place.

By the end of this blog post, you should be able to:

  • Look over parts of your code where you’d like to add unit tests.
  • Understand how to break your code into smaller functions.
  • Determine what to test.
  • Start creating your tests.

We’ll also cover rudimentary mocking, which is the practice of writing pretend calls to test your code against predictable values.

Read More

“I have an idea to test this.” “Something is wrong here. We need to test it.” “How can we test this so nothing goes wrong?”

I hear statements like this a lot when people come to me wanting to test something. Often it turns out that the problem and the solution are different from what my colleague thinks it is. My goal, as my team’s lead, is to solve the problem in the most efficient way possible. My team is small and the requests for our time are numerous. I’m always trying to find ways to solve testing problems without building automation projects.

Here’s how I do that.

Read More

Quality is something we talk about all the time. We want our work to be good, to make money, to help others, to be awesome and admired. The word quality comes up, especially while planning our work. In the end, we say we don’t test enough, we have to do hotfixes, the app breaks in ways we didn’t imagine.

The dictionary says that quality, in this context, is:

degree of excellence, superiority of kind

Obviously, a quality product goes way beyond just software. We apply similar standards to every item we use, every piece of art or music we consume, and even the people around us. Quality is something we all know when we see it but find it hard to explicitly define. As it relates to software directly, it:

Read More

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now