Kartones Blog

Be the change you wanna see in this world

Rate limits with Python

Rate Limiting is something that most projects get as a feature late, but that the earlier it comes the better for everyone. Any non-trivial service you use will have rate limits, whenever is a soft limit ("your XXXXX service can only handle Y operations per second on your current pricing tier") or a hard limit ("any table on the database cannot have more than 12k columns per table"), and this is good, because unbounded resources are points of failure. Without restrictions on an HTTP API, you're not only allowing abusive clients to DOS the platform, you're also risking any internal developer mistake to take it down, any big process (like a batch update or a yearly report).

So basically we can agree that every system should have resource limits. There are many different ways to put them in place, but commonly either the software you build (e.g. Python services) will use some component(s) (or implement their own), or use features of the web servers to limit certain type of actions based on some criteria.

Recently at work we wanted to build an internal REST API that would perform small tasks like Google Cloud Tasks (where you queue a task and when dequeued it calls an HTTPS endpoint and you're the one in charge of executing the task on one of the instances behind that hopefully load-balanced URL). To simplify the scenario, let's say we wanted to perform lots of jobs that individually should execute quickly but if massively batched could hurt a database. The best way to avoid problems is making hard for them to happen, so I wanted to put a limit to the amount of requests that the endpoint can receive, so the resources "breathe" enough to never reach too high values.

A good summary of choices regarding rate limit algorithms can be found in the following article: https://konghq.com/blog/how-to-design-a-scalable-rate-limiting-algorithm/

For example, to us didn't mattered much if we implemented a fixed or sliding window algorithm, we don't need that much precision, but one aspect was important: It had to be distributed, because the hosts are load balanced and sometimes there are few instances, but other times there are around a dozen, so leaving 2 tasks per second with 12 instances consumes more resources than when having 3 and could cause system instability. We prefered to be more accurate with load/usage predictions, so that ruled out alternatives like implementing rate limiting at Nginx.

Checking Python libraries the main requisites were for the chosen one to be distributed and easy to use (a decorator being the best option). After some digging, the winner was django-rate-limit, which offered:

  • very easy setup as a django 1.11 middleware
  • a fixed window distributed rate limit (using Redis for the distributed storage)
  • a simple yet configurable decorator to mark http endpoints at the django views. As an added bonus it automatically returns 429 HTTP responses when the rate is exceeded, so no manual handling of exceptions!
  • request-path rate-limit key, which while not perfect (no way to rate limit by ip or other custom mechanism like user_id or cookie), was good enough as a staring point and could be implemented in the future without much effort

The library implements one of the two official Redis recommendations of building a rate limiter pattern with INCR so it was good enough and race conditions small enough to not pose an issue.


We already have it working and I even did some quick django tests and confirmed everything works as expected.

As a fun fact, as the library requires Python 3, this was the main reason that I decided to give a try to migrating ticketea.com to Pyton 3.

Posted on 2018-02-01

Migration of ticketea.com website to Python 3

Today I wrote a new blog post at ticketea's engineering blog, this time about how we migrated ticketea.com to Python 3 in two weeks.

It was an interesting task, and I think went well because "ignorance is bliss", as I hadn't used Python 2 I wasn't biased by fear nor knew the real reach of possible failures, so I just went on, fixing every single error and testing and testing and testing again until all looked fine, and then more tests.

Some libraries already had Python 3 branches (outdated but with quite some work done) and QA helped me so the merit is not mine alone, but I'm quite happy with the result and it is a pleasure to focus on just Python 3.6 (plus now we can start adding type hinting and other fancy stuff).

Fun fact: All of this originated with the desire to add rate limiting to a new (internal) API I was building. I will actually write here a small blog post about rate limits because I found two very good links with nice explanations and I think can be of use for others in search of similar features.

Posted on 2018-01-12

Newsletters I read

Apart from hearing podcasts frequently (e.g. commuting to work), reading books and lots of RSS and articles (it is hard to take out the FOMO anxiety, but I'm now below 100 feeds), I discovered a while ago that subscribing to some weekly newsletters provides me with nice summaries of topics in different areas. I might read all the articles, or maybe just give a quick glance of what's new.

At the time of writing this blog post, I'm subscribed to the following newsletters related with tech:

English
Spanish
  • Bonilista: Miscellaneous topics, from tech and business to political rants
  • Mixx.io: Tech and tech-related business news

I'm open to suggestions of additional (tech) sources so feel free to drop me a comment ;)

Posted on 2018-01-03

2017 Recap

I don't like to predict or set expectations for following years, so instead this is just a small recapitulation of the key aspects of my 2017.

I switched job again (sigh). There's a saying, "things don't change unless you make them change". Sometimes change is hard, so as the company wasn't going to change to my expectations/desires I was the one that changed. And it is a pity as technically I cannot be happier to work there and what we already had but... At least I'm betting strong on the current one and so far I'm really happy (fingers crossed!). Also was a good exercise to do some CV cleanup and simplification, plus the small transparency exercise of adding the reasons of leaving each past position.

I'm in love with Python: After two years using it almost daily, it has some not-so-beautiful things but is so simple, powerful and yet nice to write into. Plus the huge ecosystem, the nice community around it... I'm doing all new experiments and pet projects using Python 3 and even started to give talks and participate at meetups again!

My grandfather died. After months with weekly visits to see him, the inevitable happened. But hey, I really wish to reach 95 as he did (working a garden until being around 90 years old!). This also affected my decision of changing jobs, I needed peace to settle my mind and for the first time in my life I gave my resignation letter with the idea of going home and then doing more interviews.

Talk less, do more. My twitter activity keeps at low levels, but I've done more pet projects than past years and managed to at least write one blog post per month, also tending to be of more appeal or at least have more techy content. I still have lots of ideas but at least some of them materialize now.

Letting go of things. I had hundreds of books, dozens of boardgames and book RPGs and lots of other "tangible decorative things", ranging from a LEGO Death Star to old videogame consoles. Now I keep a few dozen books, very few boardgames and excepting my miniatures (painting is really relaxing when I can spend some time) most decorative things are gone. I even reduced my hobbies both in number and in scope. I feel much better, like I have time again both to do geeky things and to spend time outdoors, with the family and the pets, etcetera.

Read more. Here is a partial ok only. Had a few periods along the year that I couldn't focus on reading as much as I'd wanted, or just needed distractions to ease my mind (videogames, movies, ...). I am hearing a lot of podcasts, watching some talks every month, and doing some online video courses (AWS, now a Google Cloud Platform ongoing one), but one of the points to improve for next year is reading a lot more.

Positive stance: Less complaining, more trying to be part of the solution instead of the problem, or else shutting up and moving on. The attitude of removing toxicity from my surroundings keeps improving things (even if I create a kind of echo chamber). Life is too short to spend my energies with bullshit.

Posted on 2017-12-17

Recommended Articles - 2017/12/15

Posted on 2017-12-15

Previous entries