Git Push Heroku Master: Now 40% Faster

Flow is an important part of software development. The ability to achieve flow during daily work makes software development a uniquely enjoyable profession. Interruptions in your code/test loop make this state harder to achieve. Whether you are running unit tests locally, launching a local webserver, or deploying to Heroku there's always some waiting and some interruption. Every second saved helps you stay in your flow.

We’ve been working on reducing the time it takes to build your code on Heroku. Read through this post for details on the process we used to make builds
fast, or check out the end result from the graph below:

Let's take a look at our process in delivering these improvements further.

It all starts with instrumentation

Every speed improvement effort starts with visiblity. We collected detailed
metrics about every part of our build infrastructure and in each of our
buildpacks. These metrics allowed us to see where time was being spent. We
were also able to measure how each update we made impacted the build phase it
was meant to improve.

Improvements to our build infrastructure

With detailed timing data in hand, we were able to make changes to how our
build fleet and git service work. These included additional caching, improved
mechanisms for file transfers and storage,
and providing better user feedback to let you know what is happening during your builds.

One crucial step in the build process that had room for improvement was the
creation of the slug
archive. We found this was taking an appreciable amount of time and were able
to improve it by using pigz which is a parallelized
implementation of gzip.

Improvements to the efficiency of individual buildpacks

In addition to the changes to the build process, we had an opportunity for
each of our language specialists to dig in and optimize their specific
buildpacks. This included a range of improvements from being more efficient
about how dependencies are downloaded, to making better use of the build
cache, to pre-fetching common dependencies.

This process involved many changes across all buildpacks. Some of the most
significant are highlighted in the Heroku
changelog:

• Rails 4 asset pipeline speed improvements
• Play 2.x slugs exclude SBT compile-only artifacts
• New Node.js Buildpack
• Improved build caching for apps using multiple buildpacks

The results

We've achieved improvements in build time across all languages on Heroku.
Although we looked at much more detailed metrics while changes were being
made, our target was improving build times at all percentiles on a per
language basis. We’ve showed you the 50th
percentile or median because it
proved to be a good proxy for the other metrics in this case.

Being able to deploy code and iterate on ideas quickly is a big part of
developer happiness. We will continue to use the metrics we’ve created to look
for more ways to improve build and deployment speed. As always, we welcome
your feedback on how we can improve your experience and help you maintain your
flow.