Pushing to Production

Code Release Management @ ContextLogic

At ContextLogic we like to iterate quickly and do a major code release every day. That means each and every day we push new features to Wish. Releasing code into production this often requires that we have a code release process that is efficient and stable.

Our Setup

We use Git for source control and deploy our services to AWS. For deployment we use Fabric scripts and AutoScale.

Each day an engineer spends time preparing, testing, pushing and monitoring the release. Since this engineer has many other tasks to complete, it is paramount that each of these steps are fast and reliable.

Our Goals

When designing our release process we had a few goals in mind:

  • Easy to use, so any engineer can push
  • Catch bugs before they appear in production
  • Keep the amount of time pushers spend pushing to a minimum
  • Automate as many of the steps as possible

How We Do It

Each night, a cron job runs that cuts the release, pushes it to a staging area, and sends out summary emails. Automated and manual testing can then be run on the release. The next day the pusher will merge the production branch into the release. The release is then pushed to a testing environment for verification. After verification the release becomes the new production branch and is pushed live to wish.com.

Visual Overview

1. Cutting a new release

Every evening a Teamcity job runs and uses traditional git branching and merging techniques to create a new release branch from the current master branch.

2. Pushing a New Release

To ensure that any hotfixes from the previous night are included in the release, the production branch is merged into the release branch.

Any necessary fixes or last minute commits are then cherry picked into the release branch.

The release is then pushed to a testing tier of front end machines. Functional and manual testing is then run on the tier. If no bugs are found, then the code is pushed live.

The Benefits

  • Many steps are automated, saving the pusher’s valuable time
  • By cutting the release at the same time every night, developers are less compelled to rush code into releases
  • We can dogfood each release by switching our DNS for http://wish.com to the staging environment
  • The time between cutting the release and pushing it allows us to run many automated functional and unit tests
  • There is a very low probability of merge conflicts, making the pusher’s life easy

Conclusions

Like everything at ContextLogic this process is evolving all the time as we continue to optimize and improve. If you know a better process or have suggestions, hit me up josh@contextlogic.com.

~ Josh Kuntz

Joshua Kuntz

ps. We’re hiring. Careers @ ContextLogic