pyvideo.org: Videos of Erik van Zijsthttp://www.pyvideo.org/speaker/1047/erik-van-zijst/rssen-usThu, 24 Jul 2014 00:00:00 -0500500The inner guts of Bitbuckethttp://www.pyvideo.org/video/2964/the-inner-guts-of-bitbucket<p>Abstract</p>
Today Bitbucket is more than 30 times bigger than at the time of acquisition almost 4 years ago and serves repositories to over a million developers.
This talk lays out its current architecture in great detail, from Gunicorn and Django to Celery and HA-Proxy to NFS.
<p>Description</p>
This talk is about Bitbucket's architecture. Leaving no stone unturned, I'll be covering the entire infrastructure. Every component, from web servers to message brokers and load balancing to managing hundreds of terabytes of data.
Since its inception in 2008, Bitbucket has grown from a standard, modest Django app into a large, complex stack that while still based around Django, has expanded into many more components.
Today Bitbucket is more than 30 times bigger than at the time of acquisition almost 4 years ago and serves Git and Mercurial repos to over a million users and growing faster now than ever before.
Our current architecture and infrastructure was shaped by rapid growth and has resulted in a large, mostly horizontally scalable system. What has not changed is that it's still nearly all Python based and could serve as inspiration or validation for other community members responsible for rapidly scaling their apps.
This talk will layout the entire architecture and motivate our technology choices. From our Gunicorn to Celery and HA-Proxy to NFS.
Erik van ZijstThu, 24 Jul 2014 00:00:00 -0500http://www.pyvideo.org/video/2964/the-inner-guts-of-bitbucketLimiting Execution Time Through Interrupt Driven Programminghttp://www.pyvideo.org/video/2209/limiting-execution-time-through-interrupt-driven<p>Abstract</p>
This talk explores the challenges of ensuring responsiveness of applications under varying conditions like suddenly increased load, code regressions and problematic user data that reveal code paths with unusually high time complexity.
I'll be looking at interrupt-driven techniques to help bring the 95 percentile of the response times of your application closer to the (usually much lower) mean.
In this talk, we'll go beyond traditional tricks like caching, sharding and data denormalization and instead look at tools that can interrupt execution of overly expensive code paths, such that you can guarantee an upper bound in response time.
Interruptingcow and django-timelimit will be some of the tools that will be covered in this talk.
The context for most of this is web applications, and I'll be drawing many examples from our ongoing experiences with running and scaling Bitbucket, which is entirely written in Python.
Having said that though, many of the tools and techniques demonstrates will apply just as well to other types of applications and situations.
Erik van ZijstSat, 06 Jul 2013 00:00:00 -0500http://www.pyvideo.org/video/2209/limiting-execution-time-through-interrupt-driven