Category Archives: Software

There’s a lot of people out there who have their favorite technologies and there are few more passionate debates than the issue of using HTML5 or native development for mobile apps.

It’s still early days for HTML5 on a mobile platform, the technologies are still maturing and so there are fewer libraries out there for doing things like writing games or interfacing with the hardware. Due to the nature of HTML and interpreted Javascript, performance will never match that of a native app, but with a fast enough phone and a good performing renderer performance issues will eventually not be issues at all.

Native development allows apps to build on the look and feel that users are used to on their devices. Usability should therefore be better on a native app because buttons will be using standard (tested) sizes and provide feedback that the user expects. Because native apps are compiled and optimized for the hardware they naturally have a performance advantage over HTML.

Personally I prefer to use native stuff whenever possible. But sometimes it is just far easier to hook into a web server to deliver some dynamic content in which case it can be simpler just to display it in it’s HTML form.

The current feature I’m working on is to add a commenting system to a bunch of my apps. Using the django commenting app and jquery mobile I was able to get a pretty good web based commenting system in place in a few hours. Translating that to a native form in Objective-C would have taken much longer. So I’m just going to throw up a web rendering widget in the app to load the webpage.

I see the lines blurring over the next few years in this way. HTML is catching up to native in a lot of ways but native will always have some natural advantages and is also improving. Any new hardware features will first be accessible to native APIs before they are wrapped up by webkit and exposed for use in HTML. And differing platforms will always result in HTML5 apps that have to things like “if iPhone then …”. The main benefit for HTML5 then is centralizing to servers that can be quickly updated with bug fixes and that people have HTML/JS/CSS skillsets and want to work on mobile apps.

Right now is simply too early to jump into a pure HTML5 mobile app for anything serious though. As seen by the HTML5 based Facebook App which is terribly slow on both iOS and Android and got bad user reviews as a result of those performance woes. A native Facebook app is in the works to address those shortcomings. But I would predict that it won’t be until iPhone 7 comes out that HTML5 will be able to compete from a user experience perspective with native apps.

Recently I’ve been building out my iPhone App server to provide a business dashboard with all the relevant services and numbers that I care about available at a glance. It avoids me having to sign in and out of many different sites to get the information and makes it easier to push things together – for example charting both Admob and iAd data on the same graph.

Thankfully the web is becoming more programmable every week and these things are becoming easier to put together quickly.

This is a chart I built last night to display the downloads and updates across all my apps for the past 31 days:

You can see the jumps in downloads that correspond to when I released updates to iTunes.

With these sorts of things I’m finding that there is a tipping point. If the custom page I have created is only 90% as good as going to the original source then I’ll just opt to login there but once it becomes as good as or better than that you’ll quickly forget about the 10 different logins you needed to get all those numbers.

Being in charge of it is even better. I use iAd and Admob for advertising and can pull those numbers in and compare them appropriately. On the same page I display data from Apple, Google, Linkshare, as well as numbers I collect myself such as traffic, link clicks and ad impressions. I only have to login to iTunes Connect to release new Apps.

I will continue open sourcing the components for this system over the next few weeks.

One of the ways to make money on the itunes store is to sign up for the itunes affiliate program. Linkshare runs the program in the USA and they will give you a 5% commission on all sales that you refer. It works through cookie based tracking that is valid for 72 hours… Meaning that if you follow one of my links (even to a free download) and then buy something 2 days later then I get credited for the referral and make a few cents.

Linkshare links currently make up about 15% of the revenue in my app business. It’s an extra little bit of money that takes very little effort to add to the bottom line.

I’m building up my back end platform for reporting on all the various numbers I get for the business and putting them in one place. This past friday I got an email about the new web services REST api for Linkshare which I can use to generate various reports. It took just a couple hours to put together a django app that can download the month to date numbers and store them in the database for reporting.

There might be a handful of people out there interested in using this sort of thing so I put the code up on github.

LinkShare_336x280

Fabric is a pretty awesome tool for deploying projects. But it turns out that it’s also pretty awesome for lots of other stuff.

At PyCon 2012 there was a talk given by Ricardo Kirkner (which you can watch here) that inspired me to play around with fabric in some new ways.

It’s possible to use fabric as a wrapper around the standard django ./manage.py script, to help setting up virtualenvs and install packages. Using fabric to script around these things means that there are fewer tools that new developers will need to get set up and know how to use. Scripts that normally might have been loose bash files can now be collected, organized and documented.

I’m currently working on a large django project with 4 other developers who are new to python and django. Getting everyone’s development environment working was a big pain since there were multiple platforms (Mac and Linux) and different configurations for base packages. If I had thought of this sooner I might have been able to create a reliable fabric script so that “easy_install fabric; fab init_project” got them from zero to running django app.

There’s also several oneliners that I run fairly regularly which can be saved in fabfile.py and called much easier. For example:

def clean():
    local('find . -name "*\.pyc" -exec rm -r {} \;')

Now

$ fab clean

will clear out any .pyc files in the project.

It’s also possible to manage the virtualenv environment through fabric:

VIRTUALENV = '.virtualenv/'
def setup_virtualenv():
    created = False
    virtual_env = os.environ.get('VIRTUAL_ENV', None)
    if virtual_env is None:
        if not os.path.exists(VIRTUALENV):
            _create_virtualenv()
            created = True
        virtual_env = VIRTUALENV
    env.virtualenv = os.path.abspath(virtual_env)
    _activate_virtualenv()
    return created
 
def _activate_virtualenv():
    activate_this = os.path.abspath("%s/bin/activate_this.py" % env.virtualenv)
    execfile(activate_this, dict(__file__=activate_this))
 
def _create_virtualenv(clear=False):
    if not os.path.exists(VIRTUALENV) or clear:
        args = '--no-site-packages --distribute --clear'
        local("%s /usr/local/bin/virtualenv %s %s" % (sys.executable, args, VIRTUALENV), capture=False)
 
def virtualenv_local(command, capture=True):
    prefix = ''
    virtual_env = env.get('virtualenv', None)
    if virtual_env:
        prefix = ". %s/bin/activate && " % virtual_env
    command = prefix + command
    return local(command, capture=capture)
 
def manage(command, *args):
    virtualenv_local("python manage.py {0} {1}".format(command, ' '.join(args)), capture=False,)
 
def runserver(*args):
    manage('runserver', *args)

These functions let you create the virtualenv and run commands in the virtualenv (without having manually activated it). virtualenv_local wraps the call to fabric’s “local” function and sources the activate script before launching the command specified. the manage function provides a way to call django’s manage.py script, and the runserver function gives a way to use fabric to launch the server (in the virtualenv). So now the fab command can be used to consolidate both virtualenv tools and manage.py script into one document-able fabfile.py with a consistent command-line interface.

I was surprised that there wasn’t already a script out there to download iAd reports from Apple’s iTunes Connect website.

Apple released a Java based command line tool to download the sales reports for Apps but neglected to provide something similar for iAd publishers. Some Googling around I was further surprised that I couldn’t find any 3rd party scripts to download this data.

I did however find a python script called appsalesdaily.py which is a web scraper that downloads the sales reports. It handled a lot of the nasty http cookies and login stuff that is usually very tricky to do with a script. I modified that script and extended it with a function to download daily iAd reports.

The rather fascinating thing was just how complex the single page of the iAd publisher dashboard is. It was built with GWT which is perhaps the worst thing ever developed. It produced a webpage that contains 40,000+ lines of javascript and all it does is draw a few graphs. The code was terribly convoluted and nearly impossible to reverse engineer. But 6 frustrating hours later I was able to download the files I wanted.

I don’t wish this sort of struggle on anyone so I’m making the code available. https://github.com/mfwarren/appdailysales

I am continually in awe of the power of Python code. The readability of the language combined with the lack of magic, and the massive number of easily obtainable libraries out there make getting things done insanely fast. It is the only language where I consistently experience writing code that works the first time.

Last night I added a geo-location aware link redirector with click tracking to my django web app. It took about 1 hour to code while I was listening to a panel discussion. Within that hour I had written, tested, added additional models, migrated the production database and deployed the new application.

At my day job, we have decided to migrate our web app from grails to django. The benefits have been numerous. The Groovy to Python conversion has resulted in significantly less lines of code, unit test runtime has dropped from 15 minutes to 3 seconds. The delay in starting the dev server dropped from 10 seconds to instantaneous. The functional style of code makes it far easier to avoid copy/pasting logic between subclasses. Python’s flexible import statement allows us to structure the code so that it makes more sense.

If you’re not using Python. You should be.

over the last week I have been seeing an odd error usually just once or twice per day out of the 10,000+ requests per day that hit my django web app backend for the iPhone apps.

It appears as though the GET parameters get dropped and I can’t explain why. Hoping someone out there has some suggestions.

The code in my iPhone apps check in to get data from the webserver. This is new content for the app. For each request it needs to have a parameter to determine which app is requesting the data – since there are a few using the same server.

The app code calls a statically defined url string like “http://appserver.com/app_data?app_id=aiuwbev”

However, about one in every 10,000 requests that come though create an exception because that app_id is not present.

The other bits seem to be on the mark. They don’t seem to be coming from a web browser manually.

So the most likely culprit is either that some small number of users are doing something sneaky, or there is a bug somewhere.

I’m curious if anyone has seen this issue before, or knows what might be causing it.

Edit:
Solved – the problem was due to the string encoding on iOS side of things. Needed to enforce a utf8 encoding for all urls.

My first thoughts yesterday when I started trying to add a lookup for a user’s country based on IP address was that this was going to be tricky. I figured I would have to create some models, fill them with data fixtures and do some manual queries against the database.

Turns out it was fairly trivial to do.

Django comes with some handy modules for doing it all for you. It just requires installing a C library from MaxMind and downloading their free Country data file.

To install the GeoIP MaxMind C Lib on Mac I used homebrew

brew install geoip

on the server I had to do the same thing on Linux:

sudo yum install geoip

Then I downloaded the free country data file from MaxMind and put it in my django project’s ‘geo’ directory:

$ curl -O http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
$ gunzip GeoIP.dat.gz 
$ mv GeoIP.dat <project_root>/geo/</project_root>

To finish the setup I needed to add a line to the settings.py file:

import os
PROJECT_ROOT = os.path.dirname(__file__)
GEOIP_PATH = os.path.join(PROJECT_ROOT, 'geo')

Getting the country of a connecting user was then rather simple.

from django.contrib.gis.utils import GeoIP
g = GeoIP()
country = g.country_code(request.META['REMOTE_ADDR'])

For the complete documentation on GeoIP check out the official documentation.

I usually use Linux for doing python and django development.   However last night my Linux PC choked up yet again due to bad video drivers and I was forced to do a hard reboot.

That was the final straw that made me switch over to using my Mac for most of my development work going forward.

I keep my current projects in Dropbox so that they are always up to date across all the computers I use day to day.  So there was nothing to do to get those files migrated over to the Mac.

My python and django development environment is pretty light weight. I use:

  • virtualenv
  • vim
  • textmate
  • terminal with zsh
  • mercurial

I don’t do django stuff in an IDE.  I find them a bit too heavy for coding.  Instead I opt for using either Vim or TextMate.  Very simple, text editing with little clutter or UI to get in the way.

I use the standard Mac Terminal app but I’ve changed the standard bash shell over to zsh using Oh My zsh. Which can be installed with this one line:

curl -L https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh | sh

Virtualenv is a necessity.  It keeps all the projects I work on isolated so that I can set versions for all the libraries and come back to a project a year later and still have it work immediately.  Setting up virtualenv and virtualenvwrapper on the Mac was fairly easy:

$ sudo easy_install pip
$ sudo pip install virtualenv virtualenvwrapper
$ mkdir ~/.virtualenvs
$ echo "export WORKON_HOME=$HOME/.virtualenvs" >> ~/.zshrc
$ echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.zshrc

Restart the terminal after that. Now ‘workon’ and ‘mkvirtualenv’ will be available for working on and creating virtualenv python environments.

Mercurial is what I use to version control all my projects. I just find it makes more sense than git and the command line is much simpler and cleaner.

That’s pretty much it.

My mobile app control server is turning into a bit of a powerhouse.  The latest and perhaps most exciting addition to the server has been support for sending Apple Push Notifications and registering devices for those notifications.

The goal of this is to make use of the push notifications for updating the apps, and cross promoting new apps to help build bigger and bigger launches.

One of the things that’s possible to do (but which I have not seen very many examples of yet) is having an alert open a link.  It’s a very powerful feature when you consider how easy it is to link directly to the App Store, – With a LinkShare affiliate link (aff link).  So when I find out about any hot new game release, or sale event I can let my users know about it.

It took a while to figure out how to handle the notification payload so here’s the code snippet:

- (BOOL) application:(UIApplication*)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    NSString *link = [[launchOptions objectForKey:@"UIApplicationLaunchOptionsRemoteNotificationKey"] objectForKey:@"link"];
    if (link != nil) {
        //abort app startup and goto link
        NSURL *url = [NSURL URLWithString:link];
        [[UIApplication sharedApplication] openURL:url];
    }
...

With that bit of code in the App Delegate I can send a notification with a {“link”:”http://halotis.com”} custom payload and when the user clicks the alert it will direct the user to the website. Because the openURL call can handle lots of custom URL types it becomes possible to link directly into the Facebook App (if it is installed), or link to the app rating page to ask for the user to rate the app in iTunes, open a map location, or even dial a phone number.

Why not use something like urban airship to do all this? Well mostly because I want to consolidate app management as much as possible. The more features I put into my custom server the more likely I will leave it open and make use of it.