Where asynchronous task execution fits into a larger system is something I’ve been pondering for a while. It is an interesting type of problem, which exists in an unusually strong intersection of operability, guarantees, and fundamental system design principles. Much like system design itself, two extremes bookend the spectrum of common solutions.

On one end, we treat asynchronous task execution as a purely local concern. In this model, every framework, subdomain, or even service implements its own asynchronous task execution that is just a part of how the subsystem runs. Operations equally become a local concern.

On the other end, asynchronous execution is a fundamental property of the entire system. We tightly integrate frameworks and use them as first-tier hosts of the system. If that description is too abstract, Celery is the Python world’s canonical example. This design is familiar to most as the de facto approach in most monoliths¹. While tightly coupled, it does offer some benefits as compared with the localized design. Most prominently, it makes centralizing operations and tooling natural².

In between these two extremes lies a myriad of possible designs. In reality, though, trade-offs become very nuanced. This work is therefore rarely released, and little has been published about this topic. That only makes it altogether more exciting that Dropbox yesterday published a design paper on their solution to this problem at scale. More exciting still, they went with my current personal local optima: a loosely coupled service that relies on well-defined boundaries rather than implementations to execute tasks as if they were any other type of invocation.

This design beautifully centralizes operations of scheduling and invocation while leaving decisions about and operations of execution localized — in other words, localizing operational expertise and system impact. Specific implementation details, such as the polling-heavy design, clearly result from a particular set of trade-offs. But, at a high level, I genuinely believe this design offers a clear improvement over most approaches I’ve seen in the wild, namely by separating concerns, both technically and organizationally. While nuances remain nuances, it would be truly exciting to see this shape of solution generalized well and open-sourced as a component in our shared toolbox.

Regardless, it’s incredibly exciting to see companies publish their solutions to very big design problems.

For the majority of my career, more engagement, more stickiness, more data, more manipulation, and more breaking things were unequivocally more better. The last few years have been a collective sobering experience. As with any other advance, when unbounded, there are unintended¹ consequences. Our manipulation, data collection, broken practices, and starry-eyed belief in the greater good of technology have slowly but surely caught up with us².

The consequences are many. They run the gamut from deteriorating mental health through perpetuating socioeconomics to influencing elections. And, the world has woken up to it. As the people who build technology, it’s time we wake up as well.

My friends in the Copenhagen tech community have for the last three years hosted Techfestival., a celebration and discussion of the intersection of technology and humans. The first proud product was The Copenhagen Letter, a manifesto on ethical tech.

This year, the community shaped and published The Tech Pledge. A Hippocratic Oath of technologists, it’s a beautiful summary of the human values, we should all hold dear as we do our work. Of the twelve pledges, three resonate extra strongly with me:

… to take responsibility for what I create.

… to only help create things I would want my loved ones to use.

… to never tolerate design for addiction, deception or control.

Let’s move technology into the ethical era, together. Make The Tech Pledge.

I have referenced this story to a few people over the last month or so. There isn’t any one passage that stands out. It’s just a simultaneously incredibly frightening and heartwarming story about the nature of humans.

Going through some old notes on technology choices, I came across Jason Moiron’s excellent commentary on iron.io’s performance gains from their switch to Go from a few years ago:

People have commented that these savings could have been gained from writing critical sections in C or by going over the original code with an eye for performance. Putting the obvious parallelization benefits that Go has over most other languages aside, the point they’re missing here is that you more or less achieve these results by writing normal Go code.

While I still think Go lacks a lot in terms of language features to make productivity a priority, it’s hard to dispute the fact that for a number of cases, Go is better by default. While my notes took Moiron’s point to heart, I still chose Python for the bulk of the backend work of Audacious for now, for that exact reason. There are presently a few services written in Go for critical paths, but for the most part, Go is incredibly unproductive, especially in the context of simple Web service related things, which are, by and large, CRUD with one or two bells or whistles.

Zach Holman makes a very solid point in the wake of the most recent wave of realisations that the tech industry is filled with misogynistic shitheads. The “throw women at the problem” approach to fixing the tech industry simply isn’t the right way about it, and it obviously isn’t working. It has to start somewhere else.

Fundamentally, the assholes who ruin it for everyone else need to either go, or change their behavior in major ways and own up to their past mistakes in a genuine manner. I must say, I’m pretty pleased to see a few of the most gnarly examples I’ve had the chance to interact with get theirs in recent time – including a few who have been smelling their own farts hard in public.

However, while this is all well and good, it shouldn’t be the women’s job to make this happen. There are enough “good guys” in this industry, that we should be able to call these people out and solve it within our own “ranks¹.” Or at least, that’s how it should be – and we should kind of all be embarrassed that this hasn’t been the case. Sure, that gun-wielding founder is mercurial and scary and all, but whatever happened to doing the right thing? The industry as a whole spends so much time talking about “changing the world for the better,” that surely, this must be part of the agenda too.

I mean, the whole industry can’t possibly be this deranged… right?