Lecture: Solving your Big-Data problem before it arises, using Django
Summary
More and more websites are collecting huge amounts of data and developers often don't think about this data wave when developing their apps or sites. In this talk I want to describe how thinking about sharing your data will not only make your app scalable, but also faster and the code will be better.
Description
This talk is structured in two parts. The first is, about sharding and different strategies that can be used in solving a typical big data problem for various projects. The second part will focus on a Django implementation on how to implement a sharding technology and create a fail over website without relying on any "cloud" providers. We will make the argument, that thinking about how your data will perform and testing these assumptions will make your code better and faster, even if you don't have too much data at the beginning. We will also show these assumptions on our real live data and describe how we shard our data and what motivated us.
