Oct 17, 2016

Useful, but rarely used git commands

Some useful, but rarely used and hence easy to forget git commands. Recording these here as a note to myself.

Purpose Command
Delete a Remote Tag git push --delete origin <tagname>
Filenames diff between two commits git diff commit commit2 --stat
List all tags, sorted by name, in reversegit tag -l --sort="-v:refname"

Oct 14, 2016

How to Create a Random String

Random strings are used everywhere in computing -- mostly as keys for cryptographic signing sensitive data. So how does one go about creating one?

My first thinking was to use Python code and its random library. I came up with this

import string
import random

def gen_random_string(length=32, charset=string.printable):
    A function to generate a string of specified length composed of random
    characters. The characters with which the string is composed can be
    customized thru the second parameter.

        length - int, length of the string. Defaults to 32.
        charset - array of characters from which the letters to compose
                 the string are randomly chosen. Defaults to string.printable


    num_chars = len(charset)
    random_string = ''
    for i in range(0, length):
        random_string += charset[random.randint(0, num_chars-1)]
    return random_string

It does produce a random string of the required length.

However, I found out a better way:

$ openssl rand -base64 48

This generates a random string of length 48 characters.

Not only is the latter more succinct (though it requires OpenSSL to be installed), I would think the generated string it's more random than one based in python random library.

Oct 13, 2016

Script to Create Secure Django Site Deployment Environment

My first production level shell script is out. Yes, after two decades spent in the depths of code, I have finally published a shell script that is worthy of being called 'code'.

It's something that started as a few lines hack to update a Linux box with a series of packages that needed to be installed to deploy a Django web app that eventually grew to 300+ lines of madness (I can't find anything else to describe shell scripting language) that I have used to create consistent configuration across a few servers.

Without much ado, here it is:

Script to Create Secure Django Site Deployment Environment

My first production level shell script is out. Yes, after two decades spent in the depths of code, I have finally published a shell script that is worthy of being called 'code'.

It's something that started as a few lines hack to update a Linux box with a series of packages that needed to be installed to deploy a Django web app that eventually grew to 300+ lines of madness (I can't find anything else to describe shell scripting language) that I have since used to create consistent configuration across a few servers and apps.

Without much ado, here it is:

Sep 19, 2016

Token Based Authentication in a Cordova App

Any web based service that provides a mobile client requires some form of authentication before the user can access their permitted resources. The simplest approach is to require the user to enter their username and password for every session (or after the session expires). Simple to implement, but not the most friendly approach. User’s are notorious for forgetting their login credentials more so now than ever as more and more services require login ids and their own passwords.

A better approach is to use token based authentication whereby user is issued a signed token that they can then use to access their data from the online service. This introduces its own security challenges but this can be mitigated quite nicely with proper server side measures such as serving your entire site in HTTPS only. Of course, token based approach requires that your back-end implements an API that is suitably protected and accessible only through authenticated clients. But that’s a given and I’m not going to delve into that in this post.

Tokens however have a validity period attached to them after which they expire and are no longer useful. One way to extend their validity is to refresh them using a separate request typically provided through a separate API endpoint. Once this is setup, you can create a background task in your mobile app that would periodically keep refreshing your token so that it stays valid for prolonged periods of time providing user with a seamless experience a la Facebook app.

This is where the problems start. Creating a background task is typically achieved through a service which is a special kind of task (in Android at least) that remains active throughout the power cycle of your device. This means even when your soft turn off your device, the service runs in the background periodically refreshing the access token. Drains the battery a little, but who cares as long as your app provides the necessary user experience?

With this approach in mind, I started implementing a background service in my Cordova app. Searching the web, I found a few Cordova plugins for this and out of these, I chose the most popular one. Now the author does warn that some app stores may not certify apps that use the plugin and that it is only for enterprise apps, meaning apps that are distributed outside of legitimate app stores and run within a controlled environment. I still went ahead and tried it. Unfortunately my experience has not been all too great. Try as hard as I may, the background task fails to run even in my Android emulator.

Then there’s another limitation, on Android, for the background task to run reliably, the platform requires you to create a notification to the user highlighting to them that the task that they just closed is indeed still active. Quite rightly so too as the user needs to be informed that there’s a process that is draining their precious power bank, no matter however minuscule that consumption may be.

This caused me to pause a moment and think — perhaps my solution to the problem requires rethinking. Is there a way by which we can eliminate the need for periodic background token refresh altogether?

This is the approach that I can think of right now:
  • Extend the token validity period to one that covers the duration of use of even the most sporadic of your users. So if the most sporadic user’s interval of access is 3 months, make the token validity period so a comfortable range such as 6 months.
  • Store the token in offline storage and then upon application startup, look for this token. If it’s present, take the user to the app page that you would direct user to upon successful login within the app.
  • Refresh the token whenever the app is in the foreground and when the preset token refresh interval has elapsed. This would require that token acquire/refresh times to be saved in offline store such as localStorage of IndexDB.
  • Some token standards such as JWT provide a separate configurable interval for the maximum duration of token refresh. Extend this to a very long period — using the above as an example — 2 years or even longer.
  • To mitigate user losing their device, provide a web page listing all the devices using their account and allow them to revoke any device from that list. Once revoked the accessing the service from that device would require user to entire the username and password again.
Come to think of it, this seems to be reason why services like Facebook and Google provide the device list and revoke features for their services.

  • This approach is only recommendable for such services where the server does not hold highly sensitive information such as credit card information, financial account details, etc. If your service keeps track of user’s diet calorie intake that they enter after every meal, I guess then it’s not too big a loss if it gets into the wrong hands (unless you’re a celebrity of course)
  • The solution employs a blend of client side and server side code for what is perhaps the best user experience. High usability owing to the seamless experience and zero power impact as there’s no background service is running.
  • High usability provided by the seamless experience
  • Zero power impact as there’s no background task constantly running and draining power, however little.
  • Slightly lower security as losing a device can allow a malicious user to access the user data until the user revokes the device.
  • Additional development effort as the solution requires server side infrastructure to be built. Server side work involves changes to backend database to store the device-to-token association as well as a front end where the user can list and revoke their devices.
Perhaps there are ready made solutions out there that provide this whole thing as a pluggable component, but if there is one, I’m yet to discover that. Until then I have no choice but to embark on building one myself.

Mar 1, 2016

How to setup PostgreSQL on Linux for Django

  1. First step is to install PostgreSQL on Ubuntu. In its default installation, Ubuntu does not included PostgreSQL. So uou have to install it separately.
    sudo apt-get install postgresql
  2. Now install the python module for accessing PostgreSQL -- psycopg2. You have two choices here -- either install this from the OS package manager or manually using pip. Which approach to choose depends on your python development environment.

    i. If you're using the system python as development environment, you may install psycopg2 from the OS package manager. You can do this by
    $ sudo apt-get install python-psycopg2

    ii. However, if your python development environment is based on a virtual environment, the above approach will not work. This is because the above installs psycopg2 on the system python dist-packages, which does not copied into the virtual environment when it's created. This is the behavior even if the package is installed before the virtual environment is created. So in this case you need to revert to installing psycopg using the pip from the virtual environment.

    This is where it gets a bit interesting. Pip base installation of psycopg, essentially installs the source code and then uses the OS compiler and libraries to build the python extension modules. One of the essential libraries for this is the PostgreSQL client development libraries. So you need to install this first.
    $ sudo apt-get install libpq-dev

    Buidling a python extension module also requires python development environment, which is not installed by default. You have to install this by:

    $ sudo apt-get install python-dev

    Now install psycopg2 using pip.

    (virtenv) $ pip install psycopg2

    This will download the C source for the python extension module and will trigger these modules to be built. You should a series of messages where gcc is invoked to compile the C source. If all goes well, psycopg2 should be installed to your virtual environment.
  3. Verify that psycopg2 was properly compiled and installed by issuing the following from python shell:
    $ python
    Python 2.7.6 (default, Jun 22 2015, 17:58:13)
    [GCC 4.8.2] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import psycopg2

    The next step is to configure PostgreSQL and then your Django project.
  4. When PostgreSQL is installed, it will create the user account 'postgres' in the OS that will be the administrator account for managing PostgreSQL. To administer PostgreSQL, you need to login as this user and then start the PostgreSQL console using 'psql'. But before we go into PgSQL administration tasks, we need to set a password for the user 'postgres'. Do this by switching to root and then using the passwd command

    $ sudo su
    # passwd postgres
    Enter new UNIX password: <password>
    Retype new UNIX password: <password>
  5. Create the database that you want to use with your Django project from PostgreSQL shell.

    $ su - postgres
    Password: <password>
    postgres@hostname:~$ createdb <dbname>

    Note how we changed the user context to postgres, thePostgreSQL administrator before we created the database.

    Another point here is that createdb is actually an external Perl script that is installed when PostgreSQL is installed. This, along with a few others such as createuser, are convenience wrappers to ease PostgreSQL administration directly from the bash rather than having to launch the PostgreSQL shell and then issue relevant commands. 
  6. Next step is to create a role that will be used to access this database. A role is a PostgreSQL internal user that is independent of the operating system user. You can assign access privileges to roles to different database objects. Every connection to the database server is made using the name of some particular role, and this role determines the initial access privileges for commands issued in that connection. For example, creating new databases requires a different privilege from the login privilege.

    Only roles that have the LOGIN attribute can be used as the initial role name for a database connection. Without going too deep into the internals, you can use the helper script, createuser, to create a role with login privilege.

    postgres@hostname:~$ createuser -P 
    Enter password for new role: <password>
    Enter it again: <password>
  7. Grant the necessar privileges on the new database to the newly created user. You do this from psql prompt from PostgreSQL administrator.

    postgres@lubuntu1:~$ psql
    psql (9.3.11)
    Type "help" for help.
    postgres=# GRANT ALL ON DATABASE  to 

    Note that you don't have to grant all privileges. You may be selective.
  8. Now configure Django to use the newly created PostgreSQL by defining this in the project

        'default': {
             'ENGINE': 'django.db.backends.postgresql_psycopg2',
             'NAME': '<dbname>',
             'USER': '<dbuser>',
             'PASSWORD': '<dbuser password>',
             'HOST': 'localhost',
             'PORT': '',
  9. From your Django project, issue

    $ python ./ migrate

    If your installation was successfully completed, various tables used by the Django project should be created in your newly minted database.