Marcin Swierczynski's blog

In the Java world there was always a problem with dependencies. All the jars we had to mange by hand – quite awful. Thankfully there is a Maven project which helps us in requirements management. But what about Python? Is there any way to handle necessary libraries? Of course, there is! It is a tandem of virtualenv and pip.

Virtualenv is a tool to create isolated Python environments. I’m sure you can imagine a situation when one project needs a library in version x, while another one needs the same library but in version y. Then, you can’t install both libraries to your default site-packages directory. Another example: let’s assume that new version of Django has been released and you want to test your application with this version. It’d perfect to create independent environment for the new release, wouldn’t it? So, this is a place for virtualenv.

Pip is simply the Python packages installer. It can be called easy_install successor. Here, you can read pip and easy_install compare. From our point of view, pip has one extremely useful feature. It can save all requirements information to the text file and use it later to download necessary libraries. The example file is listed below.

Django==1.2.1
Fabric==0.9.1
PIL==1.1.7
South==0.7.2
...

OK, but how can we use it together? Here comes how. Let’s assume, we’ve virtualenv installed. Then, just do the following steps.

1. virtualenv --no-site-packages environment_name – this command creates new environment with given name. The no-site-packages option ensures that this environment won’t inherit any libraries previously installed in your system.
2. cd environment_name – virtualenv has created a new directory, so we’ve to go to it
3. Now, we’ve to install all requirements into a new environment. As a bonus, virtualenv contains pip package and puts it into each environment. So, all you’ve to do is bin/pip install package_name. We need to repeat this command for each library our project depends on.
4. It’d be useful to export all the dependencies into to the file to use it in the future. To do that just run bin/pip freeze > requirements_file.txt
5. You can use this file to build your environment later, perhaps on another machine. It’s simple: bin/pip install -r requirements_file.txt
6. It’s almost done. The last thing is to activate our new virtual environment. We need this to run all subsequent commands within its context. Use source bin/activate and notice that your command prompt got a prefix – an environment name. To deactivate the environment, simply write deactivate.

Of course, the possibilities of virtualenv and pip are a lot more sophisticated. For example, you can create your own bootstrap script with virtualenv to setup a whole web application. On the other hand, you can use pip to download dependencies from version control systems directly or even create your own libraries bundle to use it when you’re offline. The use case I’ve described above is just an introduction. But I hope, it is a useful one.

Last week I’ve completed an interesting task within our project: building internal link-shortening system based on persisted objects, not on constant URLs. The main requirement was to get as loosely-coupled design as possible. In the perfect world the “shortenable” classes should know absolutely nothing about link-shortening mechanism. How to get such a flexibility in Django? I’ve used the ContentTypes framework with generic relations and it works pretty well!

First of all, we need to prepare the model class which will keep references to every “shortanable” object and its shortening key which will be used in URLs. The key-related part is quite easy, but what about the former requirement? How it is possible to keep a relation to an object of type we don’t even know? The answer is: ContentTypes framework. It gives us an ability to get an identifier of every class within Django-based application. So, we’ve got all we need now. Just take a look into a small code snippet to see this feature in action:

class ShorteningKey(models.Model):
    """ Contains a key for a generic object used in short link resolving """
    key = models.SlugField(unique=True)
    content_type = models.ForeignKey(ContentType)
    object_id = models.PositiveIntegerField()
    content_object = generic.GenericForeignKey()

Quite neat, isn’t it? Now it is possible to get a shortening key for a given object. We’ll do that with help of the following method.

    @classmethod
    def get_shortening_key_for_instance(cls, instance):
        """ Return ShorteningKey object associated with given model object.
            May throw DoesNotExist exception """
        type = ContentType.objects.get_for_model(instance)
        return ShorteningKey.objects.get(content_type__pk=type.id, object_id=instance.id)

The last thing to do is to generate a unique key for every object. As I’ve mentioned, it was extremely important to keep “shortenable” classes as clean as possible, so hard-coding shortening key generating method within the scope of each relevant class wasn’t a good idea. Instead of this, I’ve used Signals. Now, we have to write key-generation method and associate it with post_save signal. Why do we use the post_save? Because we need persisted instance to use it within ContentTypes framework. The described function is listed below while process of attaching it to the signal will be shown later.

def generate_shortening_key(sender, **kwargs):
    """ Generates shortening key for given object """
    instance = kwargs['instance']
    new_instance_created = kwargs['created']

    if new_instance_created:
        attempt = 0
        while attempt < 100:
            try:
                shortening_key = ShorteningKey(content_object=instance, key=generate_key())
                shortening_key.save()
                break
            except DatabaseError:
                attempt += 1
        else:
            logging.warning("Cannot create link shortening for " + str(instance))

We'll also need a function which will delete unnecessary key after object deletion. It's quite simple and similar so I won't put its code here.

The whole thing is almost ready, we just need to provide a function which is responsible for retrieving shortened URL.

def get_shortened_url(instance):
    """ Returns shortened url to given instance. If there is no shortened url,
        it tries to return full url to object. If there is no one, it returns empty string.
        This function is dynamically attached to any class with link shortening enabled """
    try:
        shortening_key = ShorteningKey.get_shortening_key_for_instance(instance)
        return shortening_key.get_shortened_url()
    except ShorteningKey.DoesNotExist:
        try:
            return instance.get_absolute_url()
        except (TypeError, AttributeError):
            return ''

It will be perfect if the function would be a member of each "shortenable" class, won't it? We will attach it dynamically to each of them in the same loop which is responsible for signals attaching.

LINK_SHORTENING_ENABLED_CLASSES = (Class1,Class2,...)
for cls in LINK_SHORTENING_ENABLED_CLASSES:
    post_save.connect(generate_shortening_key, sender=cls)
    post_delete.connect(delete_shortening_key, sender=cls)
    cls.get_shortened_url = get_shortened_url

That's all! We've got fully featured link-shortening system with loosely-coupled classes. You can easily adjust this example and make it appropriate for your particular needs. For example, you can build a comment system, just like Django team did