Down The Low-Level Rabbit Hole

November 7, 2020

This is the draft transcript of a talk I gave for Google Developer Group Sudbury’s DevFest 2020.

Intro

Hello, my name is Victoria Lacroix and I am a computer programmer located in Sudbury, ON. My current job makes use of software development in various capacities, from embedded firmware development in C all the way up to writing web applications in Javascript using React. Like it or not, I do it all.

Originally, this talk was going to be about why and how I decided to write a JRPG from scratch - without being based on an existing game engine - entirely using the C programming language. That game isn’t complete, and unforunately isn’t even in a state where there’s much of a “game” to it. I’ve instead decided to focus this talk on a small game I wrote after starting this talk. The game is more complete and I consider it a good case study of the merits of software development at the low level.

Though I am considered a highly technical person, this isn’t a technical talk. My intention is to be persuasive rather than instructional.

Motivation

My programming style feels unusual. At all times, my primary desire is to have an understanding of how the entire system works. This runs contrary to how most programmers operate - or at least how most software projects operate. New programmers are taught not to solve problems but to use “libraries”, “frameworks”, and “engines” which act as generalized solutions for their current problem space. This type of solution is known as an “abstraction”. Emphasis is placed not on comprehension, but in the ability to assemble pre-fabricated, highly complex parts into new software. This type of top-down development is what we call ‘high-level’.

Today’s software development is high-level in nature. Web apps in browsers like Chrome or Firefox allow the user to ignore how the computer will run their code. While this has allowed lots of code to converge on one standardized programming language (ECMAScript), this reliance on a class of applications has led to numerous adverse effects. Chrome has monopolized the browser space, and so any bug in it will also affect an untold number of web applications. On the server side, projects using Node have a tendency to accumulate thousands and sometimes tens of thousands of dependencies. Even compiled languages such as C# will ship with many dependencies, not to mention the extreme complexity of that language’s .NET framework.

Most new computer programs created by new programmers tend to be slow, and take up lots of system resources (CPU, RAM). It’s obscenely wasteful, but I have another reason to dislike this kind of project: I can’t understand its inner workings. Most “frameworks” conceal some low-level details that library authors describe as “magic”, but the reason they’re described as magic is due to how complex everything under the hood is. Despite the complexity, I don’t think computers are magic. They’re machines that do exactly what we command them to. The problem with high-level programming is that we lose sight of what precisely we are asking the computer to do, which gives the appearance of computers being “magic”.

On the other end, low-level programming is the type of programming where the developer does concern themselves with how the computer will run their code, how the computer will manage resources, and even how other developers will understand the code in its entirety. This isn’t to say it is never complicated, but rather that by its nature, it is easier for low-level code to be simple without hiding an enormous, Lovecraftian foundation.

This talk is titled “The Low-Level Rabbit Hole”, but that name betrays what I really think is going on. I liked the modern, commercial software landscape to the biblical Tower of Babel: a construction so complex and foolhardy that anyone who endeavours to undertake it is doomed to failure by the limitations of the universe itself. This escape from the Tower only looks like Alice’s fall into the Rabbit Hole because we have collectively been trapped in the here for so long that we’re becoming accustomed to the insane complexities of our own making.

Resources

So we’ve decided to strip away all of the non-essentials from our code. Our task is not to deliver something quickly, but to deliver something concise. Simplicity may be our M.O., but that is something that is much easier said than done. Reduction of complexity is extremely difficult because it is nearly impossible to remove complexity from existing software projects without breaking some kind of expected behaviour, and the process of removing complex behaviour requires that you understand highly complex code so you can cover all of the “gotchas” present within.

Because of the enormous costs of simplifying software, we need to instead think about building simple, comprehensible products from the ground up. To build a comprehensible program, we need to carefully think about every line of code and dependency we add to it, and what the overall design will look like. This is bottom-up design.

Of course, constructing your own software without any libraries is a ridiculous task. So instead, we want to prioritize system libraries that have single concerns, have been around for a long time, and are generally reliable.

Once you have written the basic code for interfacing with your libraries, you are left only to worry about writing game logic that is both highly performant and internally simple. For our base libraries, we want to avoid anything too big and complex, and if we can’t, then we want to limit ourselves to the smallest, boringest subset of features we need from our dependencies. A fun fact about C is that your compiler will only ever bring in parts of a library that your program actually uses. The rest is left out, and so even massive dependencies like our first example will only have a tiny portion active.

Onto the example, OpenGL is by far the best way to draw graphics onto a computer screen. The library is extremely well-defined by open standards, and available on nearly every platform. Its origins can be traced all the way back to Silicon Graphics and early 3D game consoles. Three commands - glBegin(), glVertex2f(), and glEnd() - are enough to draw shapes onto the screen. A few more: glColor3f() and glClear() are enough so you can re-draw the screen every frame of gameplay. Just five small function calls is enough to do the most important part of a video game: video!

GLFW is a library that takes your OpenGL context, and draws it on a Window. It also exposes keyboard controls, and fixed timings for re-drawing the OpenGL context. Unlike many other libraries that provide these capabilities, GLFW doesn’t provide much more than that.

LibPNG can decode PNG images. Once decoded, they can be bound to OpenGL as a texture, which thus can be used in your graphics.

With those libraries in play, the rest of the programming process involved writing some game logic (player control, basic physics, scoring) in C with a few functions that leveraged the libraries. The result is a simple single-player game written with less than 500 lines of C code, all in one file.

Demonstration

The source code for this game is available on Github, at https://www.github.com/VictoriaLacroix/cpong.git

Conclusion

Low-level programming is worth learning. Most of today’s programmers fear working at the low level, and thus lack any real knowledge of how their computers work, and in my opinion that fear is unfounded. Even a basic understanding of assembly code is enough to grant the programmer the ability to grasp C and its nuances, understand its pitfalls and how to avoid them.

Lastly, if this appeal to simplicity isn’t enough to sway you, consider this: one of the primary justifications for writing slow code using scripting languages like Python or Javascript is the observation that computers are always getting faster. This observation is known as Moore’s Law, and by all accounts - including Gordon Moore, the law’s inventor - this is no longer true.

According to many predictions, we’re about 5 years away from a future where consumer-grade computers will no longer become more performant. To put it broadly, this means the programmer of the future will be a lot like the programmer of the past: one who can to squeeze every last ounce of performance from a limited resource pool. The only big difference is that developers will intentionally set their own constraints, rather than have their constraints imposed by the hard limits of the systems themselves.