Kartones Blog

Review

Title: Writing Idiomatic Python

Author: Jeff Knupp

A small yet very useful book to teach you how to do things in a proper "Python way". It assumes you already know the language, and uses a very simple yet effective system of 1) describing the topic, 2) showing the harmful/wrong/typical way of doing it and 3) showing the idiomatic/best way to do it.

Comprehension lists, generators, choices of function arguments, default value caveats, even when to discard using objects in favor of simpler structures like named touples or just modules with functions.

Some of the examples were very revealing for me, as I come tainted from other object oriented languages and things are different in Python. You can read it in one or two afternoons so there's really no reason not to do it if you work with this programming language.

Notes

I updated my Python Gist with some notes from the book.

Posted on 2017-08-01

My latest bag of interesting articles and talks I've recently read, watched or listened too (you can check which podcasts I listen to here).

• #NoEstimates (Allen Holub): One of those inspiring talks I wish every manager, CTO and CEO watched, digested and understood. Do yourself a favor and watch it

• AWS Lambda - 3 pro tips for working with Kinesis streams: I found this article very appropiate, because now seems everybody wants to "lambda-everything" and there are good and bad scenarios. Also good tips for kinesis, dead letter queues, etc.

• UUID or GUID as Primary Keys? Be Careful!: I still think UUID benefits outweight the caveats, but probably I'm biased due to past troublesome DB migrations (in which UUIDs saved the day).

• 10 top talks of PyCon 2017 reviewed

• Terraform gotchas and how we work around them: Some colleages at work tried Terraform and found some of the advices useful.

• How does language, memory and package size affect cold starts of AWS Lambda?: Nice comparison, although as you can imagine Java and C# are the slowest ones due to their VM, related booting up.

• Zachary Johnson & Andrew Reitano:NESpectre: The Massively Multi-Haunted NES | JSConf EU 2017: A modded NES that allows multiplayer inputs and hacks, impressive.

• The Ultimate Game Boy Talk (33c3): If you're into Gameboy development, this talk is a rollercoaster of details and features (and a few hints). It is so packed you'll probably watch it twice.

• The Ultimate Commodore 64 Talk: Same as previous but regarding the Commodore-64. Amazing how the visual mode hacks allowed for higher amount of colors, extra sprites and even drawing outside of the "main" screen!

• Securely access AWS Parameter Store from your Elastic Beanstalk Docker containers: Accessing AWS Parameter Store from Docker containers is not straightforward, so this post gives a brief but working approach to do it.

• ev3dev is a Debian Linux-based operating system that runs on several LEGO MINDSTORMS: Great if you own an EV3 and wish to change the OS (no need to override the original one, just format an SD-card).

• With new browser tech, Apple preserves privacy and Google preserves trackers: Sad truth, one sells hardware and the other sells Ads, so privacy is secondary to ad serving and user tracking.

• Scroogled no more: Gmail won’t scan e-mails for ad personalisation: It is probably to avoid more suing and possible fines for privacy violations and regulations, but in the end we "win".

• Postgres full-text search is Good Enough!: If you use Postgres, really worth read about how to properly do full-text search.

• Latest massive ransomware attack was actually something much worse: Interesting details, like it was created to actually destroy data but fake the intentions as ransomware.

• Twitter says Trump's tweet doesn't violate its rules: because we're equal... until we're not and it is convenient to have people breaking the rules in the system (as increments platform engagement)

• Is it possible to host Facebook on AWS?: an attempt to calculate if and how could FB move to AWS

  • calculations look a bit broken to me as assume all servers have the same role and this cannot be further away from the truth: everyone uses specialiced servers with different roles.
  • Also I directly don't understand the employees per server "metric" ¯_(ツ)_/¯
  • Links provided are quite interesting but the article in general is very high-level
  • Calculations at the end are more useful to reuse at other scenarios

• You can use the new Chrome 59 dev tools to see what scripts aren't being used: Love this useful new feature!

• Github recently added code owners: Nice feature, although more than owners I prefer "guardians", because code is not mine, but I can be the "reference guy" due to my knowledge about it.

• Hard truths about tech: I recommend reading the full article but this is the list of points:

  • Learning programming is hard
  • Self-directed learning is hard
  • Attending one workshop or a couple won't turn you into a professional developer
  • It takes time: You won't become a developer in 3 months
  • Finding your first developer job is hard
  • Finding any kind of first job in tech is hard
  • Tech interviews are terrifying
  • Job search in tech is extra long and frustrating
  • Some people won't make it
  • You may not become a programmer but your skills are needed elsewhere in tech
  • Not everyone has to become a tech speaker
  • It's all about connections
  • Volunteering won't get you a job
  • You will often feel really stupid, you will often be really frustrated
  • It's a struggle
  • Not everyone has the money to attend a bootcamp or wants to attend a bootcamp
  • It takes more than good code for a project/company to succeed

• What Really Happened with Vista: Really interesting article on why Windows Vista was such a disaster despite being a full codebase rewrite and having such high hopes.

• Want a happier workplace? Studies say the best companies do these 5 things every single day:

  • Identify what makes your workplace a good place to work
  • Establish a learning spirit within the organization
  • Establish a culture of open communication and frequent feedback
  • Establish a culture that values greater work-life balance
  • Establish a culture of praise and recognition

• Amazon's SQS performance and latency: Interesting numbers to at least grasp how fast is SQS.

• Lip-syncing Obama: New tools turn audio clips into realistic video: Really scary article... makes me fear in few years we won't be able to trust the media at all...

• What does your car know about you and will it share your secrets?: And more scary stuff, in this case the telemetry that self-driving cars is storing about everything.

• Mac as a first-class gaming platform: Are we there yet?: TL;DR no, but it's on the right path.

• The Secret of High-Performing Developers: Best highlight: "if you need to debug — you've already lost your way"

• What happened to John Carmack after the book Masters of Doom?: Interesting summary of the thinking of this genious. You don't have to agree with everything he did/promotes, but he's still an incredible developer and engine creator.

• Facebook wants to analyse your emotions as you browse: It is a patent to discreetly take control of the phone/laptop camera to analyse your emotions while you browse the site and serve ads tailored to your reactions.

• China forces its Muslim minority to install spyware on their phones: Scary country, I wouldn't like to live there, there are so many privacy cuts.

• Videos from the recent EuroPython 2015 Italy: Too bad they are not split but per room and day instead :(

• (Now More Than Ever) You Might Not Need jQuery: Interesting to see more and more vanilla JS "powers".

• A hacker stole $31M of Ether — how it happened, and what it means for Ethereum: Very long but instructive post about Ethereum, how it works, and how it was hacked to steal that money. Key summarizing point: "this ecosystem is young and immature. [...] But we're going to have to get there for blockchain to be successful in the long run".

• Games software/hardware over $150B in 2017, $200B by 2021: Incredible numbers, I've been recently reading old videogame magazines from the late 80s and the 90s and it is amazing how they grew from a niche to now being almost everywhere, from a children toy to now even more appealing for adults.

• What Does It Really Take To Track A Million Cell Phones?: Scary conclussions (it is really easy and quick).

Posted on 2017-07-25

I love videogames and I grow up with what now is called retrogaming. I also switched from Windows to Linux a while ago and, despite having a dedicated gaming desktop PC, it is mainly for recent titles. Taking advantage of some holidays I decided to setup some emulators at the laptop. It wasn't always easy so I decided to write this post, both for myself in case I need to repeat it and just in case it is of use to anybody else.

Note: At the time of writing this blog post, my system is an Ubuntu 16.04 x64. Based on my experience, Linux software is very sensitive to operating system versions (way more than Windows), so I can't guarantee that everything will run for example at Ubuntu 17.04.

For Arcade machines I use MAME, I use the oficial Ubuntu Software Center MAME Arcade build/binaries plus the GNOME Video Arcade GUI (available at Ubuntu Software Center too). The main issue is that it is a barebones GUI, missing many many features from things like MAMEUI, so I also keep MAMEUI inside a virtualized Windows XP SP3.

Before continuing, a small intermission to explain the reasoning behind that Windows XP virtual machine. VirtualBox has come a long way regarding Virtualization, and even under Linux (where I haven't been able to make work 3D emulation) it works quite nicely and I use it mainly for three tasks that have to do with videogames (among other unrelated tasks):

• Playing old Windows games that don't work with Windows 7 and don't require Direct3D, like Civilization II.
• Access advanced VirtualBoyAdvance features like object and map memory visualizers (GameBoy development related). It works really well inside VirtualBox, and Wine wouldn't load it.
• Launching MAMEUI to see game snapshots (screenshots). Initially only until I finish doing the cleanup but I actually want to try running a virtualized MAME32. Note that I haven't tried running it with Wine, so it might work. From a host SSD it boots up in less than 3 seconds and just has configured a few shared folders to not need to move things in & out. And yes, XP is really old, but precisely is that version the one chosen for conflicting games (with Vista and 7 more than a few old Windows games stopped working). Oh, and for USB 2.0/3.0 and other goodies support, install also the VirtualBox Extension Pack.

For old Nintendo systems (Gameboy, GameBoy Color, Gameboy Advance, NES/Famicon and SuperNintendo/SuperFamicon) I use Higan. You have recent versions at PlayDeb2, but wherever you grab it, should at least be v103. The main reason (apart from typical better emulation and speed) is that GameBoyAdvance BIOS ROM loading was mostly broken under Linux and got fixed around version v100. One remark, to run GameBoy Advance ROMs, you need the BIOS ROM.

To play the SEGA MegaDrive/Genesis, I used DGen/SDL, but you need to compile it and it is command-line based, so in the end I tried and am happy with Gens, which can be obtained at PlayDeb2.

My beloved Commodore AMIGA 500 still works nicely, but floppy disk loading times and the like are tiring, so I also play this great computer via emulation. Especifically, using FS-UAE + FS-UAE Launcher. A few notes/tips also here:

• RTFM. There are many options and some "flows" are really clunky, like disk swapping (you need to "multi-select" all disks to be able to F12 and switch them, but the UI doesn't mentions this anywhere). The documentation is almost a mandatory read in this case
• To run anything you need the desired AMIGA firmwares (e.g. I needed Amiga 500 one). In this case there are even official commercial compilations of AMIGA software which include them
• To run Workbench tools you need AMIGA Workbench, and to run hard disk programs you need to first create an AMIGA hard drive (via the configuration)
• Combining that the AMIGA was prone to give Guru meditation errors either with cheats or with obscure unrecoverable errors, plus other mysterious hangups that make my whole Ubuntu freeze to death once or twice while running in fullscreen, this is the only emulator that didn't felt 100% stable. Still, in general works nicely and I'm not entirely sure it is not due to my graphics card, as in windowed or borderless maximized window I had no such issues.

And finally, to play old MS-DOS games either not available at Good Old Games or that I already own, nothing beats the great DOSBox, which can for example be found at Ubuntu Software Center. This is a generic operating system virtualization so each game might need individual tweaks, but many work perfectly out of the box.

One thing that I haven't tried yet is Playstation/PSX emulation. PS2 is still not 100% emulated under Windows so I don't even care, but I have pending to check for Linux PSX emulators, there should be something decent already...

Once setup, all this software works nicely, but it is not an easy task (at least not without this post summarizing it ;). There is a great all-in-one solution that I tried, RetroArch. It is a multi-emulator GUI-software-thingy that supports plugins to run many many systems, from legacy ones to really recent stuff like Wii's Dolphin emu. The reason I wasn't convinced by it is that at least when I tried it a few months ago, the Linux build was unstable and only worked with some systems. Windows build looked way more robust (I tested it) but as wasn't my plan uninstalled. It is the base system used at the RetroPie distribution, so the distro is correctly setup and already contains many basic features I probably missed out, but it wasn't as trivial as I thought.

My laptop is an old 2012 Dell XPS but runs perfectly the systems mentioned above. I know a Raspberry Pi can even run now Neo-Geo games at a decent framerate so one day I'll get one, but my two main reasons for waiting are a) I wanted to do this learning experience before grabbing a quick'n easy solution, and b) I still want to wait a bit further until commodity hardware evolves and runs more powerful machines like the PSX or GBA without frame skipping (probably something like a Raspberry pi 4 will do).

Hope this list helped you out!

Posted on 2017-07-24

This is a small post just to write about what we use at work and what I'm starting to use at home too for personal experiments. I thought would be interesting to share as at least two friends showed interest about the topic.

First, we're using Python 3.6. I've been using Python for around a year and a half only, and I have almost fully skipped python 2.x, so I am not biased with the "2 is better, don't migrate to 3!" war. I just like a lot all the new features and the way better encoding handling so my ignorance makes me not understand why would people keep with an old and worse version... ¯\(ツ)/¯

Then, we use flake8 as our linter, with every restriction on except the line size rule (which we've upped to a more reasonable 120 characters). But as some people have tendency to drift away from coding standards, to make sure everybody follows it my colleages have setup an integration test that uses flake8.api.legacy to run the checks and make sure there are no violations. It can look silly sometimes, but helps a lot to maintain a uniform codebase.

And finally, again thanks to my colleages we're doing typed Python using mypy. It adds optional type checking both to Python 2 and 3 and provides a linter-check call which reports any error (so can be actually made mandatory). Added to the same battery than the normal linter test, it means all new code must be fully typed or you won't be able to push a build. It is quite robust and you have typing hints for everything, from basic types to optional parameters, callable function handlers, generics and multiple return values (Union type, you still need to specify to it which values are allowed). It stings a bit when you begin using it but after a while it's so nice to forget about weak typing errors (one of my main complaints of scripted languages).

I highly recommend you to watch this PyCon 2017 talk of the creators of mypy to both see it in action and learn about its internals:

All of this combined with some decent tests (kind-of-TDD-without-being-always-strict) means I can do non-trivial changes and refactors without worrying of breaking unexpected things. If you do python, you should try too, the absence of fear makes you feel really nice.

Oh, and just in case you want to check it out, I keep a small gist where I from time to time write notes and miscellaneous things regarding Python that I learn and wish to keep for the future.

As a tiny sample, I wrote a Python implementation of a double linked list that you can check at my GitHub. It has both flake8 and mypy "linter tests" that check the code for errors or missing typings. Sadly, variable type hinting is only available from Python 3.6 onwards so I've used comment annotations at two places I needed to, as I'm for now using Python 3.5.

And finally, if your IDE is Sublime Text as it is mine, I wrote a post about installing a few linters on the application to directly code clean and best-practices approved Python.

UPDATE #1: Added Youtube talk video.

UPDATE #2: Added my double linked list example.

UPDATE #3: Added link to my post about Sublime Text Python linters.

Posted on 2017-06-20

Latest bunch of posts, articles and links I found out interesting.

• Designing robust and predictable APIs with idempotency: About building idempotent API endpoints for certain operations (e.g. PUTs and DELETEs), by generating on the client a unique id sent with the request, so that id forces the server to not repeat multiple retries of the same operation but instead keep (somehow) the state of that id's operations and fullfil it.
• Scaling Amazon Aurora at ticketea: Some AWS Aurora (special MySQL) scaling challenges and how do we overcome them at work.
• Talks from Serverlessconf available
• Why Amazon is eating the world: Two key points I take:
  • each piece of Amazon is being built with a service-oriented architecture, and Amazon is using that architecture to successively turn every single piece of the company into a separate platform
  • they’re permanently dogfooding. There is nowhere for poor performance to hide
• The Internet Is Broken: Interesting interview to one of the creators of Blogger, Twitter and now Medium.
• [Spanish] La prohibición de Google News en España recorta 9 millones de beneficios a la prensa online: Big backslash to spanish news media, who threatened Google and asked for money, and Google simply removed News from Spain.
• Announcing JSON Feed: Intersting proposal of RSS feeds in JSON format. I do love RSS and still widely use them so it'd be interesting to have a JSON version for current times (and not needing XML anymore).
• CGA Post-Processing in UE4: Wanna give your 3D game a CGA graphics look? then check this post
• The first generation of CDs is already rotting and dying: TLDR:
  • In general, CD formats start to fail in significant numbers inside 20 years – on average [...] some of these failures occurring in less than ten years [...] writeable CD media ... are even more susceptible to failure
• Building a Backend for Frontend (BFF) For Your Microservices: Good article explaining this pattern
• 20+ Grafana dashboards for AWS CloudWatch metrics
• How AWS Cloud is demolishing the cult of youth: Interesting point of view about age vs experience.
• Mosaic: processing a trillion-edge graph on a single machine: Because sometimes optimizations do matter. And having infinite money and engineers to invest on them, too ;)
• My kind of contract: Just read it. Quick, to the point and great.
• DHH at Twitter about workalcoholics: Funny discussion
• The greatest website to ever fit in a single TCP packet: Incredible
• Don’t Let Your (Technology) Tools Use You: About loss of focused attention due to mobile devices, social networks and the like.
• Notebook for Creativity, Inc. book: Notes from my friend @javisantana about managing creativity, Lucasfilm and Pixar.
• Introductory bullshit detection for non-technical managers: Very interesting points that not only managers but even engineers should ask themselves.

And, while not news, I also wanted to mention (as I'm recommending to my friends) anybody to do the following (free) Coursera course: Learning How to Learn: Powerful mental tools to help you master tough subjects. It is from the University of California and, despite being a 4 weeks course, you can do it more or less in half the time (if you have enough spare time). It not only teaches you how your brain works but really provides good tips to improve your learning lessons. I really enjoyed it and highly recommend it. I'll probably mention other online courses when I finish them.

Posted on 2017-06-12