On March 7th, 2012, I announced the launch of Pushover, a simple mobile notification service with device clients available for Android and iOS. I kept some notes during the development process, which mostly occurred in the evenings and weekends around my other work.
I had been using Notifo for a year or so to receive push notifications on my phone from my custom network monitor, but last year the free service announced it was shutting down. When I switched back to my Android phone a few months ago, I was unable to download Notifo's Android app which never made it out of beta.
I asked the lazyweb for suggestions on what to use to replace Notifo. I had used Prowl in the past, but it was iOS-only and focused on integrating with the (now non-free) Growl system. My question on Twitter didn't get any responses, so I started thinking about creating a new solution. The product was already pretty clearly defined: an iOS client, an Android client, an HTTP API, a website, and some plugins to make adoption easier. I would charge a few dollars for the iOS and Android apps to support the cost of operating the service. API usage would be free for most use cases, but if any large companies or websites wanted to use it, their high-volume use would cost a monthly fee.
January 13: What's in a Name
The first step of the project was to come up with a name. My initial idea was "Knowtify" but I thought about telling someone about the product and having to say "Knowtify… er, k-n-o-w, like, know, then…" and scrapped it pretty quickly. The name "Pushover" came to mind soon after, as a distinctive name that played on the service's basic functionality of push notifications.
Satisfied with the name "Pushover", I found that pushover.com and pushover.net were both taken and squatted on. pushover.net was available at a domain resale service for $688 but pushover.com was behind a whois privacy service with no website. I sent a message to the owner of pushover.com but figured that even if it were for sale, it would be pretty expensive.
After a quick live-chat negotiation with the sales team of the company owning the .net, I purchased the domain pushover.net for $500. (A few weeks later, the owner of the .com domain finally responded that he had put it on Sedo with an initial asking price of $25,000.)
January 13: Beginning Android Development
With the name picked out and the domain name secured, I started work on the Android application. I was pretty familiar with Android internals due to my work with Blandroid, but my only major work on applications was the low-level encryption key handler and hacking up the CMUpdater app that would be Blandroid's System Updater. Writing a modern application from scratch to be compatible with Android 4 and 2.3 was new to me.
Wanting to stay away from Eclipse and with Android's emulator being painfully slow, I chose to do all application development in Vim and deploy to my Nexus S from the command line:
ant debug install &&
adb shell am start -a android.intent.action.MAIN \
-n net.superblock.pushover/.MessageHistoryActivity && \
adb logcat
adbWireless made it easy to use adb over a wireless connection to avoid having to be plugged into my laptop all the time. Having 3 Android phones made it easy to test among different Android versions and screen sizes, with my main Nexus S running Blandroid 4.0.3p0, my Nexus S 4G test device running Blandroid 2.3.7p0, and my smaller-screened Nexus One running 2.3.7p0.
Android development tip: registration for and use of Google's C2DM service is tied to one's Google account, the password of which has to be stored in your server code to be able to establish a login cookie and send C2DM messages. Create a new Google account that will be used only for this service to avoid having your main Google account password stored in plaintext somewhere.
January 18: Branding
While developing the Android application and waiting for the domain to transfer, I started working on the logo. I needed a wordmark for the website as well as a simple icon for the iOS and Android apps.
Having just paid $500 for the domain name, I was reluctant to pay for a designer to create a logo, so I worked on it myself. Using Google's Web Fonts site, I found the Lobster Two font which was freely available for download and modification. I edited it in a font editing application, tweaking it to be more italicized to make "Pushover" look more "pushed over".
Since the icon would just use the capital "P" from the modified font over a circle or square, I had to pick a distinctive color. Path and Pinterest were two things that came to mind that also had an icon with a "P" that I wanted to avoid confusion with, but since they were both red, I opted for a turquoise color. I created a 512x512 icon in Gimp that would be used for the Android Market site and could scale down to the iOS sizes 114x114, 72x72, 57x57, and a 16x16 favicon for the website.
January 19: Android Application Working
After a flurry of commits, I had the basic functionality of the application finished, including C2DM receiving. Google's Android development documentation and a quick compilation/deployment/test cycle on a real device made it easy to develop the application in a comfortable feedback loop.
January 20: API Development and Rails 3
My initial work on the Android application used a simple script to send test messages directly to the Google C2DM servers. Since Pushover's basic premise was using an HTTP API to queue and send messages, it was time to start working on it.
I've been developing in Ruby on Rails since 2005 (before version 1.0, I think) but everything I had been reading about Rails 3 had kept me using 2.3 for my apps. A lot of things had changed like the asset pipeline and its dependencies on Ruby 1.9, bundler, V8 (for JavaScript minification), and Java (for CSS minification) had scared me off of using it. However, starting a new application on 2.3 seemed kind of futile, so I finally bit the bullet and dove into Rails 3.
After installing Ruby 1.9, fighting with integrating RVM into tcsh, figuring out how to disable CSS and JavaScript minification (to avoid installing V8 and friends), and getting a matching deployment setup on my OpenBSD server, I was finally able to work on the actual API code. Rather than use a fancy queue setup like Active/ZeroMQ or a NoSQL storage solution, I went with a boring-but-reliable MySQL setup. Messages are written to a queue table and a separate process polls the queue table every second and dispatches them to Google's C2DM (and later, Apple's APNS) server. As Pushover grows, I'll work on threading the message dispatching and maybe switching to a different queue once the need arises.
The basic API functionality was easily completed, including user and device registration needed by the Android application. I wrote a small daemon to read the message queue and push out messages every second. The pushover.net domain finally finished transferring so I was able to request a free SSL certificate from StartCom for api.pushover.net with pushover.net as an alias. I setup the DNS, mail, and Apache/Rails app on one of my OpenBSD servers to start making the Android application talk to api.pushover.net over SSL.
January 20: Venturing into iOS Development
My prior experience with Xcode was limited to doing xcodebuild from the command line for things like Adium plugins, and while I opted to do Android development from the command line, iOS development seemed to require the Xcode IDE for a bunch of things. I'm usually not a fan of IDEs because the editors are terrible, but iOS development with Xcode is pretty well integrated.
Unlike Google, which allows C2DM use without an Android Market account, Apple requires a full iOS developer account for access to its APNS servers for push notifications. I registered for an iOS developer account which cost $100, and would subsequently cost $100 every year to keep my app in the App Store.
Since I had recently sold my iPhone 4S and gone back to Android (which started this whole thing), and the iOS simulator was much faster than Android's emulator, I was planning to do all development work in the iOS simulator. Though as I soon found out, push notifications only work on real devices, so I walked down to Radio Shack and picked up the cheapest iPod Touch for $218.99. The iPod Touch does not have a vibration motor so I would not be able to test that part of the alerts, but would otherwise work fine.
January 21: iOS Changes my Plans
On Android, when a C2DM push notification is received by the operating system, the corresponding app is woken up and is passed the message to do whatever it needs to, which may be handling it internally or turned into an audible/visual notification to show the user. My (naive) plan for Pushover was to send a message to Google's and Apple's servers and let them handle queuing, then store the alert in the app's local database once it got the notification. As I soon discovered, this is completely different from how iOS handles APNS notifications and would require a change in infrastructure.
On iOS, a notification is only passed to an application if it is running in the foreground. If it isn't, the OS handles the alert natively and the app is never woken up. If the user taps on that notification, it is passed to the application but if the application is started directly, that notification cannot be seen by the app. This meant that the app would not be able to reliably receive and store all notifications.
This difference in design meant that I had to switch to a synchronization system, where messages would stay on the server and only be deleted once the application downloaded all of them over HTTP and synched its local database. Since the iOS application could not intercept all notifications, I would also not be able to implement message encryption, a feature I thought about implementing at a later date that could differentiate Pushover from other push notification apps.
After adding the message synchronization calls to the Rails API and modifying the message sending daemon, I implemented message synchronization over SSL in the iOS app and modified the Android app to do the same.
January 21: An Adium Plugin
I use Adium for instant messaging on my laptop with a Jabber account on my server. Since I often wander away from my desk, I like receiving messages on my phone while I'm away. Long ago, I did this with Prowl by integrating with Adium's Growl notifications, but that resulted in messages being forwarded while I was still at the computer. I later switched to Notifo and wrote a plugin for Adium to pipe messages to a Ruby script, which would selectively forward messages.
For Pushover, I created a native plugin that can forward messages when away and/or when the screen is locked (I am frequently "away" even though I am here, so I consider my screen being locked as really being away). Creating the plugin was pretty easy since I was able to use my pipe-event plugin code as a base.
While this plugin was mostly for my own use case, it would also increase the visibility of Pushover and attract potential users that discover the plugin through the Adium site.
January 23: Android Market Account and iOS Progress
I paid the $25 one-time fee for an Android Market account. The Android app is not ready for release yet, but if there was any paperwork involved, I wanted to get an early start. There was no paperwork.
The iOS app is looking more polished. Using a UITableView view with custom cell heights (due to messages being of different lengths and expanding/contracting when tapped on) was challenging and had a bunch of quirks, but I've got it working.
January 26: A Custom Alert Tone
While the iOS app may not be able to control the display of messages, it can control the sound that is played. I thought about having a custom, distinctive tone made that I could use as the default sound on the Android and iOS apps to unify them. While I probably would not have sprung for the cost of it otherwise, I have a few friends that are music producers, so I met up with one of them over dinner to discuss making a custom tone for the project. He was enthusiastic about it and explained to me the differences in various phone speakers that can affect the way a particular tone sounds on them.
January 29: Website Work
Since the Android and iOS apps were approaching a releasable state and I was waiting for the custom tone to be made, I started building the website with Twitter Bootstrap. I've previously criticized the over-use of Bootstrap due to it making websites look the same, but Bootstrap does make it quite easy to get a structured cross-browser layout.
I worked on the splash page and the logged-in functionality, such as enabling/disabling devices, creating new apps, sending messages from the site, and writing up the API documentation.
February 11: Restarting iOS Development
I had not been able to work on the project much lately, but now something is broken with my iOS app. Reverting to older versions of the code didn't fix it, and I'm frustrated. My notes say something about "Cannot create an NSPersistentStoreCoordinator with a nil model" errors. I started clean with a new Xcode project and added most of my old code to it and magically it all worked again. While restarting, I opt for a master/detail split controller to work better on the iPad, but later scrap it in favor of matching the Android behavior of expanding messages inline.
February 19: Custom Tone is done
After some back-and-forth e-mails about the direction of the custom alert sound, I pick the one that I like the most and add it to the apps running on my test devices. After using it for a few days and hearing the alert at random times, I am happy with the final version.
February 25: Pushed to Android Market
While I continued working on the iOS code, fixing crashes and diagnosing random iOS/Xcode error messages that apparently plagued other developers according to StackOverflow, the Android version was pretty much done. I pushed it to the Android Market with no fanfare, just to get it out there and stay focused on the iOS app. Good developers ship, right?
February 26: iOS 4 Compatibility
My iPod Touch is running iOS 5, but to stay at least partially backward compatible to support more users, I opted to support iOS 4 in my Xcode project. This can be a bit tricky as iOS 4 doesn't support things like JSON and Objective C's Automatic Reference Counting, which was not only new, but being turned on by default for new Xcode projects. It took me a while to track down a bug in communicating with Pushover's API from an iOS 4 device, where response messages were not parsed correctly. As it turns out, doing JSON decoding on iOS 4 didn't cause an error or even a warning, it just silently did nothing. I had to import JSONKit for backward compatibility, but JSONKit didn't support ARC, which meant it didn't build for iOS 5.
There would ultimately be a few more of these situations, where backward compatibility was needed so some 3rd party code had to be imported (properly licensed, of course) to support it, but that code didn't support newer build settings in the iOS 5 SDK.
February 27: Submitted to Apple App Store
The iOS app was finally finished, so after going through all of Apple's ridiculous legalese and distribution procedures, I finally submitted the first version of the app to the App Store and prepared for a long wait.
Using Bootstrap's responsive functionality, I was also able to make the Pushover website look nicer on mobile phones without many changes.
February 28: Load Testing and Monitoring
I wanted to be sure the Rails site would handle the load if it were announced anywhere popular, so I played with blitz.io. Since most traffic would be hitting the static splash homepage, I added a simple step to the deployment process to fetch the splash page from Rails (which would change every deployment due to the asset pipeline stuff) and cache it to a static file that Apache would look at.
However, since the "/" URL would need to show differently for logged-in users, I needed a way to avoid having Apache serve that static file in those cases. A cookie-based mod_rewrite rule was used so that if the request has no cookie, Apache serves the static HTML file directly. If the cookie is present, the request is passed to Rails which can then verify the cookie and show the logged-in view or just present a dynamically generated splash page. With this caching setup, I was able to handle a respectable amount of traffic.
I also setup some monitoring, adding an authenticated API call that returns the amount of messages in the queue. My network monitor was setup to poll that value, in addition to polling the static and API sites to check for failures. Ironically, the entire reason I started making Pushover was to use it with my network monitor, but now that my network monitor is actually monitoring Pushover itself, I couldn't use Pushover to receive alerts. For the Pushover-specific components, I made the monitor fall back to sending me text messages. I also added some SNMP hooks to export the amount of messages received through the API per minute to be graphed on my desktop monitor.
March 5: A Response from Apple
After waiting a week in App Store purgatory, Apple's reviewers finally responded… and rejected the application.
The reviewer didn't understand what the application did, and had some questions about whether there was a fee for using the API (you know, so Apple could get their cut). I responded to their questions and the app went back into pending state.
March 6: Rejected Again
The iOS app was rejected again, this time for:
3.3
We found that your Application Description includes information that is not relevant to the application content and functionality, and is therefore not in compliance with the App Store Review Guidelines.
It would be appropriate to remove or revise the following content:
transmission entry in screenshots for a torrent application.
Please refer to the attached screenshot.
The app was rejected because one of the screenshots showed an example message from Transmission, with a (lame attempt at a joke) filename "Not illegal copyright infringement in any way.mkv". Despite a notification from Transmission actually being "information [..] relevant to the application content and functionality", I figured the soulless app store reviewer just didn't find copyright infringement funny, so I edited the screenshot image to make the entry just show "Download complete" and resubmitted the app.
March 6: Still not Funny
The app was rejected a 3rd time with the note "transmission is still in the screenshots". Apparently just mentioning BitTorrent is against App Store policies, regardless of whether it's encouraging copyright infringement.
I removed the Transmission alert completely from the screenshot and re-submitted it, and it was approved shortly after.
March 7: Announcement Time
With both apps live in their respective app stores, the website completed, and the Adium plugin completed, I announced the project on Twitter. I submitted the site to Hacker News, but either due to the iPad announcement that happened that day, or just submitting with a title that didn't have enough linkbait, the announcement quickly moved off the /newest page with only one upvote, so I deleted the link to be able to resubmit it later.
March 8: iOS Bugs Found
After some feedback from Pushover users, I found and fixed some bugs in the iOS app, as well as adding a "delete all messages" button in the settings. I submitted the new version to the App Store, but it will take another week to get approved.
March 9: Suddenly, a Wild Competitor Appears
14:20:26 sma: did you happen to see that airgram app on hacker news?
14:21:03 me: heh
14:21:20 sma: is that pretty much like pushover?
14:21:24 me: yeah
14:21:30 sma: oh noes
14:21:45 sma: what horrible timing
14:35:54 me: oh well
As (bad) luck would have it, between my initial announcement and waiting to resubmit it, another app was announced on Hacker News that did basically the same thing as Pushover: Android and iOS clients and an HTTP API. (Side note: maybe submitting to Hacker News with "Show HN" is the key to getting something upvoted.)
While their app looks more targeted towards general users that want Twitter notifications and RSS feeds, it's still pretty similar to Pushover in functionality. I replied to the creator of the app, trying not to come off as spamming for my app in their thread, but I wanted to know how they thought they were going to end up different than Notifo since they weren't charging for their apps or their API. As other users pointed out in that thread, their app and target audience is not much different than Boxcar's (though it does have the ability to "Geo-target and geo-fence your notifications to be hyper-relevant to users", whatever that means). Since Pushover has an actual business plan to sustain itself (which users prefer), I'm not too worried about it.
March 13: Resetting the Wait Period
I rejected my previous iOS app update before it was reviewed due to another bug found and fixed, which unfortunately means it goes to the back of the line for reviewing. While trying to prepare for a new binary to be uploaded on iTunes Connect, I kept getting a generic error from the site. Fun tip: if you ever get these errors on Apple's website, look at the HTML source code, there is a ridiculously long Java-looking stack trace in an HTML comment in the code which shows all kinds of neat things about the underlying app. In my case, the error was about not being able to create a directory in /tmp. It took a day for Apple to clear this out, but I was finally able to submit a new binary and will have to wait another week for a review.
March 16: Summary
The costs for the development and publishing of Pushover were $500 for the domain, $0 for the SSL certificate, $218.99 for the iPod Touch, $0 for the already-owned Nexus S, $25 for the Android Market account, $100 for the Apple App Store account, and a small amount for the custom alert tone. The server and development laptop were already owned.
As of right now, there have been 21 purchases of the app on the two app stores. The iOS app update is still pending review, which is frustrating and it makes me anxious to have new users downloading an old version of the app that has known bugs. I have received positive feedback and feature requests from users, which I will work on incorporating over time. As Pushover grows, it will probably be moved to a new server and maybe a multi-threaded message dispatch daemon, but for now it is handling the user load without blinking.
Questions or comments?
Please feel free to
contact me.
