Bitbucket Branches & Variable Handling in Runners

So you’ve got a Bitbucket project that is using branches, but you also need to either create or substitute some vars for a given branch, all within a pipeline and preferably on-prem. Well lucky you, I’ve got some example code that will show you how to handle Bitbucket branches in conjunction with runners and variables. As a bonus, and because it looks cleaner, we’ll see how to pass variables from one step to another in the pipeline. Toss on some new Tennyson and let’s jump into it!

Bitbucket Branches

Like most any code repository, Bitbucket offers a branch feature so that you can test out code in your lower environment before merging to your production environment. Now, since you won’t have a separate pipeline file for each branch, you need to figure out an approach to variables and steps to ensure those variables and steps match what you want to run for each branch.

There are a few options for how you define steps within your pipelines. Usually, you would set step definitions up front and then simply call the steps you need in a given branch. However, you can also define a given step within the branch as it runs. I’ll show a bit of each of these configurations.

The Use Case

These ideas can be used for any project where you want to use some logic to define a variable based on the branch being run. You could create repo variables to be used for each environment and then use the sed command to swap those in. However, there may be times when you need to create or modify a variable and then pass that variable between steps. My example doesn’t do that directly, but it shows you how it would work using echo, artifacts, and cat.

How It Works

So how the heck does this work? Take a look at the example repo I’ve got if you want a better idea of how all the pieces fit together. Otherwise take a look at the first half of the bitbucket-pipelines.yaml file below. This first section is used to define all the steps we’re going to call later in the pipelines file, so let’s break that all down.

Our first step (set-env) uses some very simple logic to define a message variable depending on the branch being run. This variable is then echoed into a text file and that file is made into an artifact, so that it can be passed along to further steps. The next step (set-template) takes that artifact, reads it back into a variable, and then uses that newly created variable to replace a variable in a jinja template.

definitions:
  steps:
    - step &set-env
      name: Set Environment
      runs-on:
            - "self-hosted"
            - "ansible"
      image: quay.io/ansible/ansible-runner:latest
      script:
        - if [ "BITBUCKET_BRANCH" = "develop" ]; then MESSAGE="This is the dev deployment!"; fi
        - if [ "BITBUCKET_BRANCH" = "master" ]; then MESSAGE="This is the prod deployment!"; fi
        - echo "$MESSAGE" > message.txt
      artifacts:
        - message.txt
    - step &set-template
      name: Set Template
      runs-on:
            - "self-hosted"
            - "ansible"
      image: quay.io/ansible/ansible-runner:latest
      script:
        - MESSAGE=$(cat message.txt)
        - sed -i 's/#MESSAGE/'"${MESSAGE}"'/g' ./template_config/template/message.j2
    - step &deploy-template-dev
      name: Deploy Template Dev
      runs-on:
            - "self-hosted"
            - "ansible"
      image: quay.io/ansible/ansible-runner:latest
      script:
        - sed -i 's/#AdminUser/'"${AdminUser_Dev}"'/g' ./template_config/main.yml
        - sed -i 's/#DevHost1/'"${DevHost1}"'/g' ./devhosts.yml
        - sed -i 's/#DevHost2/'"${DevHost2}"'/g' ./devhosts.yml
        - sed -i 's/#AdminUser/'"${AdminUser_Dev}"'/g' ./devhosts.yml
        - sed -i 's/#AdminPass/'"${AdminPass_Dev}"'/g' ./devhosts.yml
        - ansible-playbook ./ssh/main.yml -i ./devhosts.yml
        - ansible-playbook ./template_config/main.yml -i ./devhosts.yml

The very last step seen here (deploy-template-dev) defines the process we will run on the dev branch. I used the sed command and repo variables to replace all the variables I’ve placed in my code. That last playbook takes the jinja template that we inserted our environment message into and lays it down on the host as a script. It also calls that script to ensure our variable was correctly assigned per branch.

Now let’s look at defining how the pipeline runs. We first tell Bitbucket we’re using branches and then define each branch. In the dev branch process, I’m simply calling each of the steps I defined above. For the master branch you can see I’m calling the first two steps, and then defining a step that does the same as the “deploy-template-dev” step above, but this time using the production variables.

pipelines:
  branches:
    develop:
      - step: *set-env
      - step: *set-template
      - step: *deploy-template-dev
    master:
      - step: *set-env
      - step: *set-template
      - step: 
          name: Deploy Template Prod
          script:
            - sed -i 's/#AdminUser/'"${AdminUser}"'/g' ./template_config/main.yml
            - sed -i 's/#ProdHost1/'"${ProdHost1}"'/g' ./prodhosts.yml
            - sed -i 's/#ProdHost2/'"${ProdHost2}"'/g' ./prodhosts.yml
            - sed -i 's/#AdminUser/'"${AdminUser}"'/g' ./prodhosts.yml
            - sed -i 's/#AdminPass/'"${AdminPass}"'/g' ./prodhosts.yml
            - ansible-playbook ./ssh/main.yml -i ./prodhosts.yml
            - ansible-playbook ./template_config/main.yml -i ./prodhosts.yml

Feel free to use any of the code I’ve got in my repo for this. I don’t have a way to test it, as I don’t have branches or pipelines setup on my personal repo, so it’s likely there are some small bugs to work through. Hopefully this gives you an idea of how you can play with branches, variables, and some fairly basic logic!

Gotchas & Notes

There are a few things to look for here. First, the SSH playbook I’m using is slightly modified from what I found on the interwebs. Note specifically that I’m using the updated ansible_host variable to pull the hostname/ip we’re connecting to for each host from the provided inventory file. Also of note is that I’ve included “delegate_to: localhost” for each task in the SSH playbook.

There are a tooooon of variables flying around in this setup, so ensure you’ve got your formats and naming correct throughout your code and repo as well. Another thing to keep in mind is that each step is it’s own container, that’s why we’ve got the runs-on & image tags defined in each step.

The main playbook in this repo is a file replacement playbook that uses a jinja template and the ansible.builtin.template function. The template feature can be super useful in a few ways. For example; you could place the same file with different configurations for dev and prod as in this example, or maybe just replace your sudoers file across your estate with a new, uniform file.

Have any questions? Feel free to drop them below and I’ll try to answer or point you in the right direction!