Kartones Blog

This post is a mere declaration of intentions and brief explanation of where I spend lately not-tiny-but-neither-huge blocks of spare time when at home. Still, can be of some interest if you do Python and Django.

I won't enter into details of the project purpose (you can read it) other than mentioning that Finished Games is a small website to catalog and track your videogames playing history. You can also check a live demo at https://finishedgames.kartones.net (to me the most useful part is the user catalog). Note: Game and Platform catalogs are not totally imported yet, just a few hundred samples to do some real usage scenarios.

The tech stack is mainly Python 3.5, Django 2.1, Docker & Docker-compose (for testing and development, not required for production). It only has at the time of writing this blog post less than 50 lines of Javascript so I wouldn't consider it a "main" technology (be advised it uses ES2015 without polyfills on purpose so no old browsers neither IE). In local it uses SQLite for storage.

Everything is dockerized to ease testing and not polluting your computer with Python packages you might not desire to keep. Initial setup requires you to run the migrations (for now, I want to add a first-run aware bootstrapper) but other than that, [commands are as simple as make run, make test, make coverage, etcetera. Check the README to see many of them in action.

Regarding Django the project includes installed django-debug-toolbar but with just a few modules, mainly to be used to detect unoptimized queries. I would recommend it to be switched off in production, it's easily configurable.

Testing is done with Pytest, Coverage + Pytest-cov, plus the code is fully type-annotated so there are MyPy & Flake8 "linter tests" so tests won't pass if you don't type the code or have it clean (almost no custom rules except 120 chars line length and default double quotes). Continuous integration is using CircleCI and just rans the same tests as make test.

I'm doing my best to build the code in a legible, pythonic and simple (I'd add "clean" but not in the craftmanship meaning) way. It is an ongoing effort, and as I'm iterating quite fast from an initial MVP sometimes a piece of code might be pending some deeper testing or refactor, but in time it will come.

The code includes as many Django best practices as I'm being able to apply, from avoiding N+1 query problems using .select_related(), how to properly instantiathe the defined User auth model (did you know you it can be overriden so you should never directly import the model?), statics configuration for development and production, class-based views for non-trivial views (or multi-verb HTTP endpoints), named url paths...

I'm very pragmatic so I didn't wanted to reinvent any wheel if not needed. That's why I've chosen simple yet beautiful CSS:

But also by trying to do as less as possible I'm heavily relaying on Django Admin to all the CMS management, only extending it with extras I really need to build: Custom actions, custom filters (including context-based smart filtering), field decorators, hand-made views...

Everything that can be configured using existing ModelAdmin features is done, like list_display, list_filter, search_fields, readonly_fields or ordering.

Instead of the classic scripts folder so prone of becoming a dumpster, I'm sticking to Django commands for any operation, django migrations for any database change (even if is a mere DB field description/comment update):

Settings files use a simple hierarchy: base.py defines things, then dev.py, test.py or prod.py (not included, but provided a prod.py.sampl placeholder) inherit from it, and optionally if you define a local.py it can override dev.py values (handly for testing some production setting locally, careful to not abuse or pollute it!).

There are some tests and many more will come once I reach the point of being able to import hundreds of games quickly (really only remains a batch import feature, as one by one import is already implemented), after all code coverage wasn't added for nothing. Pytest plugs correctly to django so coverage is accurate (at least more accurate than not properly configuring it). As the first catalog source adapters are defining both the fetch/import API and the database structure in this case I'm prioritizing quick iteration and manual verification over tests. Once settled tests will harden the code as much as possible. Plan is not to do hundreds of view integration tests but a few ones will be provided to showcase how to test Django views (so my future self can easily remember how it's done). Similar scenario for Django Commands, I'll do some basic testing but won't mock the whole universe regarding data sources, instead focusing on testing FetchedGame and FetchedPlatform entities and their interactions with the import process.

The code doesn't contains any super-algorithm nor does anything clever, the coolest piece being a token bucket rate limit implementation (sample usage as a method decorator here) that mostly comes from StackOverflow and Wikipedia (at least the tests are built from scratch by me), but I've never been a brilliant mind creating amazing algorithms, I just "get shit done".

As the README mentions, there's a roadmap but is the only thing I'm keeping private for now, as I want to work on this project at my pace, with my rules and as much as I want/decide to. For example, initially I was decided to migrate it to Django REST framework and React/Redux for a version 2.0, but now I'm not totally sure I'm going to do it, as a classic/non-SPA approach is working so nice for now (and I can just do any other side project focusing on Javascript). That doesn't means that if somebody leaves issues, pull requests or comments I will ignore them, far from it, but my priorities might not be the same as others.

The license is Unlicense so you can do whatever you want except helding me liable for any problem :)

Ok, enough talking, show me the code and leave me alone! https://github.com/Kartones/finished-games

Closing up, I'm having a great time with the project as I'm combining building a tool I really want to use with being able to go deeper into some Python packages/libraries I don't use so much in my daily work. And for once it'll be a non-totally useless project to showcase on my Github profile.

Posted on 2019-03-16

I was running out of intermediate or advanced english learning content at the Udemy for Business account, so I decided to try an "essential" course, Business English Vocabulary: Learn Essential Business Words. And oh, what a discovery!

The course is 5 hours long and contains around 150 words separated by topics (engineering, business, finances, law...), but not only is each word carefully explained and presented with multiple examples of use, what I liked the most is synonyms or at least closely related workds (between 5 and 7 per word), that's a gold mine to me. The author also speaks very clearly and at a good cadence.

Definetly recommended.

Posted on 2019-03-10

I recently did another small "knowledge pill" course at Udemy, Vagrant Quick Start: Virtualized Development Environments.

I knew it would be introductory and didn't expected much, and in that it delivered: You get to know what's Vagrant, how to install it, and the basic operations of managing VM instances, SSHing into it, provisioning and updating boxes, and where to find baked up VMs. Just beware that out of the 2 advertised hours, almost half of the time goes in basic and advanced versions of "how to install" on Mac, Windows and Linux.

Other than that, professionally build, really well explained (one of the few times the speaking cadence was so nice I chose not to speed it up!), big visible fonts and easy to follow examples. The author has a more advanced course which I'll definetly consider doing.

Posted on 2019-03-03

Since I left .NET development I've tried using a few IDEs, but mostly sticked to Jetbrains suite for work and Sublime Text for personal projects. Jetbrains products require some setup for non-trivial projects, and Sublime's power came with its extensibility via packages (e.g. it's very easy to setup Python linting). In 2018 I decided to test Visual Studio Code after reading so much good feedback about it. At first I tried it at home, just for Python and basic web development, but after a few weeks, not only I uninstalled Sublime, but even decided to try it at work (instead of PyCharm). The results cannot be more satisfying.

I've come to terms with myself regarding what I can get from IDEs other than the real Visual Studios; In my opinion nothing will ever get as good as Visual Studio 2008 upwards (or even as the old Turbo C. But if I at least can have a tool that is fast, does the job, helps me being productive and has a few shiny extras, then it's fine.

I've installed nice extensions which allow me to just keep minimized CSS for my websites, and then re-format it in a second, change whatever I need and then re-compress & minimize. I can navigate through Python object definitions and have mypy type hint warnings. I can even now do live code editing and pair programming with the just discovered VS Live Share extension! I'm so glad I tried that last one, as in the past I've had to rely on the basic tmate + vim combo.

I might still sometimes suffer to attach to a debugging statement and have to rely on SSHing and a terminal to do it instead, but while Pycharm took ages to index and reindex and index again, VS Code just boots up and I get my last opened files and get back to coding in no time.

As a final note, I've setup a extensions and configuration blog page, so I can keep track of which extensions I use and keep at hand my customized editor config in JSON (to just copy & paste). It is nothing special, just a mixture of React, Python, Docker and miscellaneous web related plugins.

Posted on 2019-02-21

Although since mid-2018 I haven't touched much AWS except for personal projects (and just a few services), I had the AWS Certified Developer - Associate 2018 purchased and forgotten, so as a way to refresh some of the services and learn new ones I took it during the past weeks.

Note: I confess I either skipped or quickly went through most of the labs as I have already done the AWS Certified Solutions Architect - Associate course and many are very similar. There are some differences like an in-depth multi-video section about Elastic Beanstalk.

I learned about a few services, like X-Ray (tracing and some metrics, like a lightweight DataDog), Step Functions (Lambda "ui"), about the fact that API Gateway supports importing and updating API definitions from Swagger definition files, DynamoDB Accellorator and DynamoDB Streams (too bad events data is only kept for 24h). SQS FIFO queues (new from 2018) also sound cool as they guarantee order and exactly-once processing, too bad they are restricted to 300 transactions per second; still really great to have such power in an easy to use service. More interesting topics were an intro to CI/CD using AWS services (CodeCommit, CodeBuild, CodeDeploy and CodePipeline), how to build Docker containers with CodeBuild and ECR, Cognito and Identity Access Management (IAM) services.

As usual you probably won't use all of them but the amount of pieces you have available as services is incredible, if you don't care about vendor/cloud lock-in, you can perfectly build your whole business on AWS probably for years to come until you might need something more "powerful".

The quiz questions good to practice, as many depict real world-like scenarios and ask you the best service or proper configuration to solve a problem or match existing (but non-AWS) requirements, but I found some of the questions quite bad, as they are as dumb as asking you the exact name of an API method call, or some numbers which might get asked in the certification exam, but also change as Amazon softens some limitations.

Overall, a great and interesting course, with more than 25 hours of videos, exercises, labs, exam tips, and fresh & updated content whenever something new appears. I still recommend going first for the Solutions Architect Associate one if you want breath-first services knowledge, but I enjoyed both.

Posted on 2019-01-29