I'm going to try and post something about a project I've worked on each week, most likely on Tuesday mornings. Today's post is about "Preposterous".
Way back when there was a blogging platform and website called "Posterous". Posterous was based on the beautifully simple idea of using email as the primary method of creating and maintaining a blog.
Eventually Posterous, like many companies wandered from this path and began adding features and complexity but more-or-less preserved the "email only" interface until the end.
Yes, the end. After a few years Posterous was bought by Twitter, who then dismantled it and took it offline. Members found other homes for their posts, but the elegance and simplicity of the original Posterous was now completely gone.
I was an avid user of Posterous, and made it the hub of my online presence. Posterous had simple integration with broadcast-based social networks like Twitter and Facebook, so it was easy to use Posterous as a central publishing point and then maintain discussion across a wide range of other platforms. Since all it took to post something was an email, I could share information, pictures, video, etc. from almost anywhere using almost any device, and due to the way that email gracefully handles intermittent connectivity I could post things immediately and know that the information would make its way to the net eventually, whenever connectivity resumed.
I searched for an alternative, and while there were countless blogging platforms, many capable of accepting posts via email, none of them could be completely managed that way the way the original Posterous worked, and they were all more complex to operate and maintain. So I kind of gave up.
I gave some thought to what it would take to build my own Posterous clone, but it was hard to justify the time, and to some degree I thought I might just get over it, since no one else seemed to be so hung up on the idea of an email-driven blog system.
Some time later (years?) I was discussing email-based online systems with some friends and Posterous came up. To my surprise I wasn't the only person who lamented its fall, and I mentioned that I had considered building a replacement. The idea of building a direct replacement was said to be "preposterous".
So I went to work
That night I set out to build the simplest possible replacement for Posterous. I chose Python because I had been working with it recently writing "systems" code (command line and batch processing tools for Linux) and I knew it would be able to talk to things like mail servers, etc. without a lot of external tools. I also wanted it to be open and portable so a Python program with minimal or no external libraries seemed like something that could run just about anywhere. I also gave myself an arbitrary constraint of keeping the whole thing under 100 LOC, partially to manage the scope and partially to make it interesting.
Later that night Preposterous went live with this post:
The first public release was ready the next night, commemorated with this post:
At this point the architecture was set, and it's rather clever if I do say so myself. Unlike most blogging platforms, Preposterous doesn't use a database in the traditional sense. Instead it leans on LDAP, as all input to Preposterous is via email. Doing this the way a typical web application works, by reading the database every time a page is requested would beat up the LDAP server pretty hard so instead Preposterous renders all content in advance into simple HTML (and later, JSON and XML) files. This means that it scales well, even on constrained hardware (think Raspberry Pi), and it's elementary to scale horizontally as well since it is, essentially, at any given moment, a 1990's static website. Copy the files, turn on an HTTP server and bam, you have another node in the cluster.
The LDAP back-end has other advantages. LDAP is of course a tried-and-true technology, and making LDAP scale and perform is well-known territory, so even though Preposterous currently leans on a 3rd-party service for its LDAP back-end, breaking away from that is not a daunting task like moving a database from one host to another (or from one database platform to another). The LDAP model also serves as a sort of "transaction log", since all posts are stored chronologically in the LDAP server. If something goes wrong, re-creating all the static content is just a matter of running Preposterous once with a parameter to ignore the "read" flag and all content will be recreated automatically.
That brings up another aspect of Preposterous that is interesting: everything it creates is deterministic based solely on the content of the email posts it receives. This includes all things created when a new account signs up (hence the creative URL's). This allows the Preposterous server to be run as a simple cron job, against a specified LDAP server and account, and always produce the same output.
Over time, I expanded Preposterous's capabilities but with a focus on hiding any complexity. Eventually the code grew beyond 100 lines, but added the ability to post pictures, videos and audio as well as generate RSS feeds and potentially support custom domains. Additionally all posts are rendered as JSON documents as well so one could conceivably consume the content using other programs with little effort.
Features aside I maintained singular focus on making Prepostrous support an "email first" philosophy, even going so far as leaving seemingly critical features unimplemented until I found a good way to handle them via email. Account creation is simply a matter of sending an email to email@example.com and following the instructions in the reply. I used Preposterous exclusively for some time to force myself to make it better, "eating my own dogfood" until it was usable.
Of course there were (and are) challenges. Fishing information reliably out of emails is not a science, and over time things that worked stopped working and required continual effort to keep Preposterous functional in light of evolving email clients. None of these challenges are insurmountable, but allocating time to work on them becomes challenging, especially as the difficulty of the problems begins but bump against the ceiling of my Python & email/MIME knowledge.
At this point I want to continue to develop Preposterous and I've noodled on a number of ways to support the effort. Preposterous is of course Open Source and can be found on Github here: https://github.com/jjg/preposterous/ . Pull requests are welcome and there are a few immediate improvements listed in the Issues on this repository.
However if there is interest I think that Preposterous could be made self-sufficient with a very small dedicated user base. "1000 True Fans" could easily support future development operation via subscriptions, and this would be my preferred route vs. the typical ad-supported or venture capital models so common online.
My box has turned yellow, which I take to mean it's time to wrap up this post. I'd love to hear your thoughts on Preposterous and stay tuned for another project next week!