Kartones Blog

Review

Title: Game Engine Black Book: Wolfenstein 3D

Author: Fabien Sanglard

I've been following Fabien's blog for many years, with his excellent videogame source code reviews and in general great articles, so when he announced he was finishing a book about Wolfenstein 3D I got so excited and actually preordered it (on Google Play/Android, read from a 9" tablet).

You can actually grasp the contents of the book not only by reading the whole author's blog, but with certain examples, like how the fizzlefade effect was built or how to understand floating point. But to give a general overview, it is a quite specific technical book: You get history about 90s hardware (the 80386, VGA graphic cards, sound systems and the dreaded MS-DOS RAM handling), some background about the team and who did what regarding Wolf3D, but then it is a dive into the source code, analyzing, explaining and commenting the most relevant parts of the game internals, from how the raycaster works to how the sprites were stored, the different subsystems (sound, input, memory, cache, ...). It feels short but is +300 pages long, and also contains appendices and sequels & ports chapters to learn miscellaneous things related with the game.

I loved reading it, mostly because Wolf3D was one of the first PC games that actually made me start enjoying my father's 386DX instead of only my beloved AMIGA 500, and while it wouldn't be until 1994 that I started learning to code and grasped VGA graphics programming and assembly, I love the amount of tiny but interesting details that I've learned, from how a digital joystick worked and would be read its inputs, to why there were different sound formats or how clever were the guys at id storing the sprites to speed up even their reading and writing with VGA planar modes.

A great technical and nostalgic walkthrough the source code of a game that changed everything, but also a reading about how at the early 90s and the old MS-DOS days everything was harder, from playing videogames (with the conventional memory limitations) to developing them (way harder than being able to play them :)

Notes

As I was reading a book about generating computer mazes I read the book announcement and couldn't resist to try to adapt the maze generators to export Wolfenstein 3D maps. The result can be seen here, as Wolf3D doesn't supports mods or external maps easily, I decided to reverse-engineer NMap's (an unnofficial mapping tool) .LEV file format and generate those files with simple data.

After reading the book now I want to do a raycaster (probably using PyGame for drawing :_)

Posted on 2017-09-19

Quick post with some personal thoughts regarding the last Code on the rocks podcast, which was about recruiting and human resources, but it mentioned the topic of technical code tests for candidates.

First, a confession: I am quite terrible at doing code tests, due to two main reasons:

• I tend to get nervous, so I try to complete them in one sit even if given more time (like "the weekend")
• I always time-box them if they aren't limited in time

When I start a code test, I get into a state I can only think about it, so my normal life gets disrupted until I'm done. I'm unable to take long breaks, or sleep and then continue. I also know it can be harmful for me to time-box when I could instead take advantage, because I could for example do multiple iterations using the extra time to improve something, but my reasoning here is:

• This is a test, so for me the rule "is take it as such, a test": not strive for perfection but "what you'd do in a few hours most"
• My time is precious. I'm not going to spend a whole weekend with an unpaid code test, I'm sorry

I've only done 4 code tests in my life (out of 10 companies I've worked/work at):

• one I nailed it
• other I did well enough to get hired
• one I failed (there were external factors but I still had mistakes)
• the 4th wasn't evaluated as I got hired elsewere and they stopped the process. I'd like to think was going to be ok

Asking for a code test it's a standard practice, and in some scenarios it is one of the few alternatives (e.g. remote work), but I dislike them as it is hard to be trully fair/objective evaluating them. It is also quite hard to come up with unbiased coding challenges, too easy to fall into either algorithmic ones, or too specific/narrow problems that while you might actually be working at if hired, you might as well not have faced before (so is not fair to ask about them). And most times there's a subjectivity factor upon the reviewers: They might not like your style of code, they might have their biases and/or preferences, and sometimes happens that they might be reviewing the code for the first time and think there is only one "correct approach".

I instead prefer other alternatives:

• do an actual job task (or bugfix): getting paid for it as will take some time, but is a nice way to get the feeling. The main problem of this approach is it's really hard to do while working
• "work with us for a week": Similar to previous one, but becoming a member for a certain time. Best way to see (both sides) if there's a match, you're part of meetings, etc. and not just of a specific task
• doing a real task while pairing with an actual engineer from the company, so you don't get overwhelmed by doing something without knowledge of the scenario/context
• if not the previous one, at least doing a code test but live with a peer from the company, in a constructive way. Probably will make you nervous but at least you're able to explain your reasonings and the other side can also ask you why you take some decisions

Also, one thing I think it is critical is to give feedback to the candidate. I'm not saying you should reply with the failure points, but at least give some general feedback so they can improve. After all, they gave you time and effort by doing it.

And finally, coding tests are a cheap way to filter candidates for the recruiting deparment, but they do take effort both from candidates and from the technical side of the company (preparing and reviewing them). Effort that sometimes the recruiters could also spend filtering their targets instead of blind-shooting everywhere. This is a complaint because I've seen and still see so many bad examples.

For me code tests just work as a filter but act like a double-edged sword. I know of a few good engineers who have failed code tests, and I've met other people who probably were good "coding" but were not somebody I'd be happy to work with. As usual, there is no silver bullet.

Posted on 2017-09-11

Vacations delayed this post a lot, but on the other hand lots of interesting articles gathered during this weeks.

• Roomba vacuum maker iRobot betting big on the 'smart' home: but betting on a sad area... selling your house's map to Amazon, Google or other similar big company.
• How Microsoft brought SQL Server to Linux: Quite interesting because I thought it was a mere VM-like wrapper and while uses virtualization, it is more advanced.
• High-Resolution Custom Metrics and Alarms for Amazon CloudWatch: Finally, 10-second precision alarms at AWS.
• piprot: Small Python library that checks how outdated are your requirements (libraries/components). Really handy.
• History of iPhone pricing, including the average selling price: Interesting that there is a more or less stable baseline, despite going higher and higher (into indecent prices for a phone).
• "The most popular software for writing fiction isn't Word. It's Excel" @brianalvey
• "If you don't run experiments before you start designing a new system, your entire system will be an experiment" -Mike Williams (via @compscifact)
• Instead of hacking self-driving cars, researchers are trying to hack the world they see: Clever and scary in equal parts, demonstrates how advanced our sight and brain are compared with automated cars. Related article on how to hack signs.
• How I didn't become a SoundClouder: Sad example of why companies shouldn't lie to applicants and probably directly not hire anybody if they are having money issues. There was a similar situation in Spain around end of 2016 with a known company hiring people and sometimes weeks after firing them due to layoffs. The solution is simple: Talk to HR and ask them to stop any hiring procedure, you're playing with people's professional careers, not with mere spreadsheet numbers that you can adjust instantly.
• LinkedIn: it's illegal to scrape our website without permission: It was nice that they lost, because it feels fully anti-nature to on one side provide public information and on the other say it is illegal to scrape/gather it. Still makes me sad to see how the nice and public and free internet gets more and more attempts to put bars, borders and walls.
• The Android Browser: Slidedeck about how terribly fragmented seems to be Android's browser.
• How to Host a Website on S3 Without Getting Lost in the Sea: Nice summary, saving it for the future me that will probably want to host some static site.
• Operation Luigi: How I hacked my friend without her noticing: Scary tale about social engineering and forgotten account dangers.
• No, Facebook Did Not Panic and Shut Down an AI Program That Was Getting Dangerously Smart: Same crap I avoided at TV from unconfirmed, flamboyant and sometimes fake news, seem to be happening lately a lot.
• Slides and links for 2017 talks of the GopherCon
• Ethereum miners are renting Boeing 747s to ship graphics cards and AMD shares are soaring: Personally I think this is getting absurd and is another tech bubble waiting to explode, so much electricity wasted on cryptocurrencies...
• Investors poured millions into a storage network that doesn't exist yet: And another example, a "blockchain-based cloud storage technology" that has raised 52$ million and is just an idea...
• Why is ARKit better than the alternatives?: Huge but really interesting article on augmented reality, not just on Apple's ARKit.
• Google fires engineer who "crossed the line" with diversity memo: There's still hope in humanity, because I thought it was going to be let go just for the shake of "freedom of speech", but no, I'm not the only one thinking there are lines you cannot cross "despite you can". Some things are plainly wrong.
• If you want to debate the Googler's Manifesto and you're also a good person: Because we confuse freedom of speech with "anything goes", and as the article points out, some discussions cause harm, are not just a discussion.
• When tech firms judge on skills alone, women land more job interviews: Speaking of inequality...
• Queues vs Distributed logs (tech pill): Nice brief but well explained summary of what both are and when to use each.
• Blizzard and Google betray humanity with StarCraft 2 tools to train artificial intelligences: It is so nice to see official APIs!
• Inside the crash of Fling, the London startup whose founder partied in Ibiza while his company burned through $21 million: Self-explaining title, a typical burned startup story.
• [Silicon Valley's wealthy elite have made social inequality worse](https://www.spectator.co.uk/2017/08/silicon-valleys-wealthy-elite-have-made-social-inequality-worse/: I knew things at SF are crazy, but this is really sad.
• Learning at work: Nice point of view and suggestions, I don't agree with all of them but the general message is relevant.
• Block-breaking game in vim: Intelligent and fully playable, kudos to the author!
• Ad blocking is under attack: Sad that lawyers and DMCAs are pouring even at Ad-blocking level. Instead of curing a disease, they force their way through mostly the only visitors protection...
• Scalable design pattern sample (tech pill): Another "tech pill" of an example where a combination of distributed log + queue provides a great solution. One that I've been part of at a past job and it indeed worked great :)
• SQL Tutorial: How To Write Better Queries: Really excellent summary of general and less general tips and improvements to apply when writing SQL.
• Why Github can't host the Linux Kernel Community: A nice article on the whys, pros and cons of both Github and the Linux Kernel community itself. You might agree or not with how they work but at least this is a polite, fundamented explanation.
• Tesla workers and the Model 3: To hell and back again: Sad to read that such a revolutionary car, manufacturing plant and general vision comes with an unsafer environment for the workers (higher tan average auto factories injury stats).
• Chess Is Being Forever Changed by Technology: Interesting how not only computers are able to win by "brute force" or raw computation power, but also provide powerful analytics to help chessmasters analyze past champions and improve their techniques.
• FP vs. OO: I wish there were more discussions like this article, trying to balance both sides and not being radical on any.
• Linux load averages: "system load averages" is much better, as it includes disk/NFS I/O and wait locks. Reasoning makes sense once you think this requires resources too. Still dividing the average between amount of CPUs works as a rough measurement, just don't take it as 100% precise.
• About React licensing, how to hide things in an apparently simple language: React license caused quite some stir this past weeks, but is nice that raised awareness of being careful with not-so-hundred-percent-free licenses.
• security.txt: A "standard" that allows websites to define security policies. Looks interesting, let's see if gets some adoption.
• How long employees are staying at the 10 biggest companies in tech: Quite revealing that people still switch "early" to another company despite being at one of the top/best ones. Whenever it is for getting more money or better conditions or simply more interesting projects, in the end is what happens at lower levels too.
• Why Even the Hyperloop Probably Wouldn't Change Your Commute Time: Interesting reading about city growth. Also learned about Marchetti's constant. TL;DR: average travel time stays approximately constant.
• Designing a Microservices Architecture for Failure: Some highlights (but you should read it all):
  • careful with network failures
  • 70% of the outages are caused by changes in a live system
  • Reverting code is not a bad thing
  • skip unhealthy instances from load balancers
  • self-healing can cause trouble
  • failover caching -> two different expiration dates (short for normal situation, longer during failure) -> better outdated data than nothing. stale-if-error HTTP header useful for HTTP-based communications
  • a larger amount of retries can make things even worse (cascading effect). Also should be abke to handle idempotency (using idempotency-keys)
  • rate limiters and load shredders both for traffic peaks and traffic control to allow critical operations to complete
  • to fail fast and separately -> bulkhead pattern (segregate resources, protect limited ones). using timeouts is an anti-pattern -> circuit-breaker pattern (protect resources by checking success / fail statistics of operations)
  • test your system against common issues (and frequently) -> survive various failures
• AWS Glue Now Generally Available: One of those AWS services you might have never heard of, it is an apparently simple ETL service. Having done some non-trivial ETL work at a past job I'm curious about how powerful it really is.
• Expanding the Medium Partner Program: aka "Expanding the paywall". I firmly believe on openly accessible information so I'm glad I never had the urge to move to that platform. I'm just sad the monetization attempts are pushing it into a more closed system...
• Tesla's Push to Build a Self-Driving Car Sparked Dissent Among Its Engineers: Again better to wait until technology matures, but also I think this metric is quite... relevant: "Human error causes 94% of crashes, according to [american] government statistics".
• VW engineer sentenced to 40 months in prison for role in emissions cheating: So, now not only you do have moral decisions to take, but also can be legally held accountable if you pick the wrong one. The guy lied more than once, not just "wrote some lines of code" but still it sets a precedent for the future.
• On Quiet Developers: Interesting read, especially now that all seems to orbit around GitHub profiles and contributions.
• Django Migration Don'ts: Pretty similar solution for delicate migrations than the one we built at CartoDB. We implemented a quick rollback feature that checked if there were RoR migrations and if so wouldn't let you do the quick one (so you do the full rollback and check data is still fine),
• In Silicon Valley, Working 9 to 5 Is for Losers: I have no interest on working in the United States, but Sillicon Valley is becoming quite unfriendly. Disgusting is the best summary for the article, but go read it.
• On MongoDB: Nice 3 part detailed walkthrough of Mongo's 5 years of life, with mistakes and success reasons. I like the mentioned simple three part process (PAT) when assessing hyped technologies:
  • Problem: Understand your problem deeply
  • Assess: Critically assess claims in potential solutions
  • Tradeoffs: Weigh tradeoffs in the short and long term, rather than thinking about good vs bad
• Cloud Native Landscape: Nice visualization with a summary of many players on each of the areas regarding "clouds".

Posted on 2017-09-05

Review

Title: Writing Idiomatic Python

Author: Jeff Knupp

A small yet very useful book to teach you how to do things in a proper "Python way". It assumes you already know the language, and uses a very simple yet effective system of 1) describing the topic, 2) showing the harmful/wrong/typical way of doing it and 3) showing the idiomatic/best way to do it.

Comprehension lists, generators, choices of function arguments, default value caveats, even when to discard using objects in favor of simpler structures like named touples or just modules with functions.

Some of the examples were very revealing for me, as I come tainted from other object oriented languages and things are different in Python. You can read it in one or two afternoons so there's really no reason not to do it if you work with this programming language.

Notes

I updated my Python Gist with some notes from the book.

Posted on 2017-08-01

My latest bag of interesting articles and talks I've recently read, watched or listened too (you can check which podcasts I listen to here).

• #NoEstimates (Allen Holub): One of those inspiring talks I wish every manager, CTO and CEO watched, digested and understood. Do yourself a favor and watch it

• AWS Lambda - 3 pro tips for working with Kinesis streams: I found this article very appropiate, because now seems everybody wants to "lambda-everything" and there are good and bad scenarios. Also good tips for kinesis, dead letter queues, etc.

• UUID or GUID as Primary Keys? Be Careful!: I still think UUID benefits outweight the caveats, but probably I'm biased due to past troublesome DB migrations (in which UUIDs saved the day).

• 10 top talks of PyCon 2017 reviewed

• Terraform gotchas and how we work around them: Some colleages at work tried Terraform and found some of the advices useful.

• How does language, memory and package size affect cold starts of AWS Lambda?: Nice comparison, although as you can imagine Java and C# are the slowest ones due to their VM, related booting up.

• Zachary Johnson & Andrew Reitano:NESpectre: The Massively Multi-Haunted NES | JSConf EU 2017: A modded NES that allows multiplayer inputs and hacks, impressive.

• The Ultimate Game Boy Talk (33c3): If you're into Gameboy development, this talk is a rollercoaster of details and features (and a few hints). It is so packed you'll probably watch it twice.

• The Ultimate Commodore 64 Talk: Same as previous but regarding the Commodore-64. Amazing how the visual mode hacks allowed for higher amount of colors, extra sprites and even drawing outside of the "main" screen!

• Securely access AWS Parameter Store from your Elastic Beanstalk Docker containers: Accessing AWS Parameter Store from Docker containers is not straightforward, so this post gives a brief but working approach to do it.

• ev3dev is a Debian Linux-based operating system that runs on several LEGO MINDSTORMS: Great if you own an EV3 and wish to change the OS (no need to override the original one, just format an SD-card).

• With new browser tech, Apple preserves privacy and Google preserves trackers: Sad truth, one sells hardware and the other sells Ads, so privacy is secondary to ad serving and user tracking.

• Scroogled no more: Gmail won’t scan e-mails for ad personalisation: It is probably to avoid more suing and possible fines for privacy violations and regulations, but in the end we "win".

• Postgres full-text search is Good Enough!: If you use Postgres, really worth read about how to properly do full-text search.

• Latest massive ransomware attack was actually something much worse: Interesting details, like it was created to actually destroy data but fake the intentions as ransomware.

• Twitter says Trump's tweet doesn't violate its rules: because we're equal... until we're not and it is convenient to have people breaking the rules in the system (as increments platform engagement)

• Is it possible to host Facebook on AWS?: an attempt to calculate if and how could FB move to AWS

  • calculations look a bit broken to me as assume all servers have the same role and this cannot be further away from the truth: everyone uses specialiced servers with different roles.
  • Also I directly don't understand the employees per server "metric" ¯_(ツ)_/¯
  • Links provided are quite interesting but the article in general is very high-level
  • Calculations at the end are more useful to reuse at other scenarios

• You can use the new Chrome 59 dev tools to see what scripts aren't being used: Love this useful new feature!

• Github recently added code owners: Nice feature, although more than owners I prefer "guardians", because code is not mine, but I can be the "reference guy" due to my knowledge about it.

• Hard truths about tech: I recommend reading the full article but this is the list of points:

  • Learning programming is hard
  • Self-directed learning is hard
  • Attending one workshop or a couple won't turn you into a professional developer
  • It takes time: You won't become a developer in 3 months
  • Finding your first developer job is hard
  • Finding any kind of first job in tech is hard
  • Tech interviews are terrifying
  • Job search in tech is extra long and frustrating
  • Some people won't make it
  • You may not become a programmer but your skills are needed elsewhere in tech
  • Not everyone has to become a tech speaker
  • It's all about connections
  • Volunteering won't get you a job
  • You will often feel really stupid, you will often be really frustrated
  • It's a struggle
  • Not everyone has the money to attend a bootcamp or wants to attend a bootcamp
  • It takes more than good code for a project/company to succeed

• What Really Happened with Vista: Really interesting article on why Windows Vista was such a disaster despite being a full codebase rewrite and having such high hopes.

• Want a happier workplace? Studies say the best companies do these 5 things every single day:

  • Identify what makes your workplace a good place to work
  • Establish a learning spirit within the organization
  • Establish a culture of open communication and frequent feedback
  • Establish a culture that values greater work-life balance
  • Establish a culture of praise and recognition

• Amazon's SQS performance and latency: Interesting numbers to at least grasp how fast is SQS.

• Lip-syncing Obama: New tools turn audio clips into realistic video: Really scary article... makes me fear in few years we won't be able to trust the media at all...

• What does your car know about you and will it share your secrets?: And more scary stuff, in this case the telemetry that self-driving cars is storing about everything.

• Mac as a first-class gaming platform: Are we there yet?: TL;DR no, but it's on the right path.

• The Secret of High-Performing Developers: Best highlight: "if you need to debug — you've already lost your way"

• What happened to John Carmack after the book Masters of Doom?: Interesting summary of the thinking of this genious. You don't have to agree with everything he did/promotes, but he's still an incredible developer and engine creator.

• Facebook wants to analyse your emotions as you browse: It is a patent to discreetly take control of the phone/laptop camera to analyse your emotions while you browse the site and serve ads tailored to your reactions.

• China forces its Muslim minority to install spyware on their phones: Scary country, I wouldn't like to live there, there are so many privacy cuts.

• Videos from the recent EuroPython 2015 Italy: Too bad they are not split but per room and day instead :(

• (Now More Than Ever) You Might Not Need jQuery: Interesting to see more and more vanilla JS "powers".

• A hacker stole $31M of Ether — how it happened, and what it means for Ethereum: Very long but instructive post about Ethereum, how it works, and how it was hacked to steal that money. Key summarizing point: "this ecosystem is young and immature. [...] But we're going to have to get there for blockchain to be successful in the long run".

• Games software/hardware over $150B in 2017, $200B by 2021: Incredible numbers, I've been recently reading old videogame magazines from the late 80s and the 90s and it is amazing how they grew from a niche to now being almost everywhere, from a children toy to now even more appealing for adults.

• What Does It Really Take To Track A Million Cell Phones?: Scary conclussions (it is really easy and quick).

Posted on 2017-07-25