Using Travis Build Stages to Test Multiple Python Versions and Publish to Pypi

September 02, 2019 #python #software-development

I recently finished a new Python program, wayback-machine-archiver. See my recent post for details. It supports many different versions of Python,¹ which required me to automate the build system. I needed my build system to:

• Install the software for each supported Python version.
• Run tests against each Python version.
• Publish exactly once to Pypi when all the tests in all the Python versions had passed.

Running a bunch of tests, and then deploying your software is exactly what Travis Stages were designed for. There was just one problem: I couldn’t find a good example of how to do it with multiple Python versions. This post will explain how to do it.

The Travis Configuration

Here is the full Travis Configuration file. I’ll go through a simplified version below that covers just the essentials:

Run Tests in Parallel

We start by setting up the Python versions to use for testing:

language: python
dist: xenial # Required for Python >= 3.7
python:
  - "2.7" 
  - "3.7"
  # Also test pypy
  - "pypy3"

This will run tests in parallel on Python 2.7, 3.7, and Pypy3. Python 3.7 is only supported on Ubuntu Xenial, so we set that as the dist. I removed a bunch of version for clarity; to add more, just write them in the list.

Next, we tell Travis how to set up the environment and test the code:

install:
  - pip install -r requirements.txt
script:
  # Unit tests
  - python -m pytest -v
  # Install and smoke test
  - pip install .
  - archiver --help

This installs the dependencies, runs the unit tests, makes sure we can pip install the package, and finally runs a quick “smoke test” on the installed package.

Build and Deploy

After the tests succeed (and only after) we build the Pypi package:

jobs:
  include:
    - stage: build
      python: "3.7"
      script: echo "Starting Pypi build"
      deploy:
        provider: pypi
        user: alexgude
        password:
          secure: Bq6I8x...sqslR  # Hashed password
        distributions: "sdist bdist_wheel"
        on:
          tags: true
          branch: master
          repo: agude/wayback-machine-archiver
        skip_existing: true

This defines a new stage to build the package in 3.7 and then deploys it to Pypi, but only if it is the master branch on a tagged (from git tag) release.

Which gives us this:²

I hope that helps you set up your own Python packages for testing and deployment! In the future, I hope to migrate to Github Actions, but that is for another time.

---

1. Currently 2.7, 3.4 through 3.7, the development versions of 3.7 and 3.8, the nightly release, and pypy 2.7 and 3.5. ↩

2. I took out a bunch of the versions in the example YAML configuration; the screen shot shows all the versions I test against. ↩