In a recent discussion on
Hacker News, a commenter posted
the following question:
Okay, so, what do we think about TarSnap? Dude was obviously a genius, and
spent his time on backups instead of solving millennium problems. I say
that with the greatest respect. Is this entrepreneurship thing a trap?
I considered replying in the thread, but I think it deserves an in-depth
answer — and one which will be seen by more people than would notice
a reply in the middle of a 100+ comment thread.
First, to dispense with the philosophical argument: Yes, this is my life,
and yes, I’m free to use — or waste — it however I please; but
I don’t think there’s anything wrong with asking if this is how my time
could be best spent. That applies doubly if the question is not merely
about the choices I made but is rather a broader question: Is our society
structured in a way which encourages people to make less than the greatest
contribution they could?
That said, I do object somewhat to the premise of the question —
specifically the statement that I “spent [my] time on backups”.
On one hand, it is true: Tarsnap has been my full time job since 2006. I
do occasional consulting — more in the early years than I have
recently — but financially speaking that’s a rounding error; it’s
Tarsnap which pays the bills (including allowing me to buy the house which
I’ll be moving into next week). On the other hand: My work on Tarsnap has
extended considerably into adjacent fields.
In 2009, having had many users ask for passphrase-protected Tarsnap key
files, and having determined that the current state of the art of password
based key derivation was sorely lacking, I invented
scrypt — and in the
process, opened up a whole new field of cryptography. Sure, I was doing
this because it was something I could do to make Tarsnap more secure; but
it would be a stretch to place this under the umbrella of “spending my
time working on backups”.
In 2011, wanting to connect daemons on disparate hosts together securely,
and not being happy with the existing TLS-based options, I wrote
spiped. While I think
it remains largely underappreciated by the world at large, I nonetheless
consider it a significant contribution to computer security — and,
like scrypt, while I created it to serve Tarsnap’s needs, it would be a
stretch to place such a general-purpose open-source tool under the narrow
umbrella of “working on backups”.
Around the same time, I started working on
kivaloo, my high
performance key-value data store. This may be the least used of all of
the software I’ve written — I’m not aware of anyone else using it
at present (although being open source software that doesn’t necessarily
preclude the possibility) — but I consider it to be some of my best
code and I think it may become used in more niches than merely Tarsnap in
the future.
Starting in 2006, and accelerating significantly after Amazon launched the
“M3” family of HVM-enabled EC2 instances in 2012,
been creating and maintaining the FreeBSD/EC2 platform. While I don’t have
any precise statistics on its usage, a survey last year found that
44% of people
running FreeBSD in the cloud use Amazon EC2; so — despite the fact that
only 22 people currently
provide sponsorship for my efforts
— it’s clear that my work here has been productive. Again, while I was
working on this because I wanted to run FreeBSD in EC2 for Tarsnap, I don’t
think it can be placed entirely into the category of “working on backups”.
Of course, the question at hand isn’t whether I’ve done anything useful, but
rather whether this was the most useful way I could have spent these
years. Judging by the reference to the
Problems, I imagine that the specific alternative they had in mind was a
research career; indeed, between my Undergraduate studies in number theory
under the late Peter Borwein and my Doctoral studies in Oxford I might have
considered seriously working on the
and Swinnerton-Dyer conjecture had my life taken a different path. (A very
different BSD from the one with which I am currently involved!)
So why am I not an academic? There are many factors, and starting Tarsnap is
certainly one; but most of them can be summarized as “academia is a lousy
place to do novel research”. In 2005, I made the first publication of the use
of shared caches in multi-threaded CPUs as a cryptographic side channel, and
in 2006 I hoped to continue that work. Having recently received my doctorate
from Oxford University and returned home to Canada, I was eligible for a
post-doctoral fellowship from Canada’s
National Sciences and Engineering
Research Council, so I applied, and… I didn’t get it. My supervisor
cautioned me of the risks of doing work which was overly novel as a young
academic: Committees don’t know what to make of you, and they don’t have any
reputational prior to fall back upon. Indeed, I ran into this issue with my
side channel attack: Reviewers at the Journal of Cryptology didn’t understand
why they were being asked to read a paper about CPU design, while reviewers
at a computer hardware journal didn’t understand why they were being asked
to read about cryptography. It became clear, both from my own experiences
and from advice I received, that if I wanted to succeed in academia I would
need to churn out incremental research papers every year — at very least
until I had tenure.
In many ways, starting my own company has given me the sort of freedom which
academics aspire to. Sure, I have customers to assist, servers to manage (not
that they need much management), and business accounting to do; but professors
equally have classes to teach, students to supervise, and committees to
attend. When it comes to research, I can follow my interests without
regard to the whims of granting agencies and tenure and promotion committees:
I can do work like scrypt, which is now widely known but languished in
obscurity for several years after I published it; and equally I can do work
like kivaloo, which has been essentially ignored for close to a decade, with
no sign of that ever changing.
Is there a hypothetical world where I would be an academic working on the
Birch and Swinnerton-Dyer conjecture right now? Sure. It’s probably a world
where high-flying students are given, upon graduation, some sort of
“mini-Genius Grant”. If I had been awarded a five-year $62,500/year grant
with the sole condition of “do research”, I would almost certainly have
persevered in academia and — despite working on the more interesting
but longer-term questions — have had enough publications after those
five years to obtain a continuing academic position. But that’s not how
granting agencies work; they give out one or two year awards, with the
understanding that those who are successful will apply for more funding
In short, academic institutions systemically promote exactly the sort
of short-term optimization of which, ironically, the private sector is
often accused. Is entrepreneurship a trap? No; right now, it’s one of
the only ways to avoid being trapped.
blog comments powered by Disqus