Abstraction Hell

Table of Contents

Cloud is just someone else computer

If you work or are interested in information technology (IT), you’ve probably seen this joke on memes, shirt and mugs… In fact it’s quite far from reality, your implementation of an Infrastructure as Code (IaC) abstraction involves someone else’s API, which calls various services from your cloud providers; these services, in turn, deploy applications developed by someone else running on someone else’s computer. I’ve vonlountrily shortenthis sentence before it became too long but you got my point.

This article represents my personal opinion on the topic, and while it’s based on my experience and observations, it’s not a scholarly or research-driven piece. My aim is to delve deeper into an issue I believe hasn’t received enough attention, which could potentially lead to significant challenges in the future.

What’s abstraction

In IT, abstraction is a fundamental concept that helps developers manage complexity by hiding intricate implementation details and exposing only what’s necessary. It simplifies complex systems, allowing developers to focus on higher-level functionality without getting lost in low-level technicalities.

To illustrate, here’s what happens (roughly speaking) when you write code in a high-level language and how it eventually reaches the processor:

%%{init: {'theme': 'dark', 'arrowMarkerAbsolute': 2}}%%
graph TD
    A(High-Level Language) --> | Source code | B(Compiler)
    B --> | Translates to machine code | C(Assembly Language)
    C --> | Low-level instructions | D(Machine Code)
    D --> | Binary instructions | E(Processor)

Let’s be honest: writing everything in assembly or machine code would be a nightmare. So, lucky for us, compilers translate human-readable code into processor-friendly instructions. The trade-off? We now depend on layers of someone else’s code to make our programs work.

The Problematic Side of Abstraction

Let’s get back with our first example, “Cloud is just someone else’s computer” oversimplifies the reality of modern IT infrastructure. Your implementation of Infrastructure as Code (IaC) involves multiple layers of abstraction, from APIs to cloud services, each calling upon various components developed and managed by others. While this enables rapid deployment and scalability, it also introduces significant challenges:

  1. Loss of Expertise: As abstraction layers multiply, fewer developers understand the underlying systems. With modern architectures continuously stacking new layers on top of each other, deep technical knowledge of the lower layers is slowly fading away.
  2. Resource Consumption: The cumulative overhead from these abstraction layers significantly increases resource consumption. Running a pod that executes IaC code and makes calls to your favorite cloud provider’s API to deploy a Docker image on a Kubernetes cluster will inevitably consume more resources than simply deploying that Docker image on your own server.
  3. Security Risks: Each additional layer introduces potential security vulnerabilities. The more interconnected and complex a system becomes, the harder it is to secure the entire stack.

And that’s just scratching the surface. Add to this the hidden costs, maintenance complexity, and potential performance bottlenecks, and it becomes clear that abstraction, while useful, comes with significant trade-offs.

Finding a Balance

So, should we abandon abstraction altogether and return to writing raw machine code on bare metal? Of course not—we’re not trying to reinvent the Stone Age of computing. Abstraction is essential; it enables efficiency, scalability, and rapid innovation. The real challenge is making sure we use it wisely.

But of course, in the wonderful world of tech, there’s always some “visionary” in a Patagonia vest fresh out of business school who sees an opportunity to create yet another revolutionary SaaS that will “simplify” something that was already working just fine. And before you know it, your product manager has negotiated an “amazing price” for this new machine-learning-powered orchestration platform. And in the end, you’ve just added another layer of abstraction, making your stack about as pleasant to deal with as a mandatory rooftop networking event.

As we move forward, we need to be more mindful of the layers we add. Not every abstraction is necessary, and some may be too costly in terms of performance, security, or energy consumption. With growing concerns about environmental impact, we should aim for more sustainable computing practices—optimizing every layer, reducing unnecessary complexity, and making our systems as efficient as possible.

At the end of the day, abstraction is here to stay. The key is to use it consciously, not blindly stack layers until systems become unmanageable monsters. Because, let’s be honest, debugging something eignteen layers deep in an opaque stack is nobody’s idea of fun.