Yes, I could run my Django application on Google App Engine, but that requires all sorts of hackery and my app ended up an abomination of the original… too unnatural for my tastes.

I sensed a small shift in the Earth’s rotation on Sept 28, 2011. This is when Heroku added support for Python/Django on their platform. I needed to give it a test drive and I was amazed how simple it was. The following is an account of what I did to port one of my existing Django sites to Heroku…

Initial setup

Heroku already provides a decent quick start guide for Python. That’s a great place to begin. Check out the Prerequisites and Local Workstation Setup which will get you up and running quickly. It helps if you’re already familiar with Git, virtualenv and pip. If you’re not, then now is an excellent time to learn!

First things first. Assuming your Django project is already in Git, change directory to the project root. Then…

$ heroku login
... output omitted ...
$ heroku create --stack cedar
... output omitted ...

Believe it or not, we’re almost there!

Database config

Now, my existing app is running a MySQL database. I’m going to use the built-in Postgres DB on Heroku. So I need to update my Django settings…

# Database config
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
    }
}

Note, that I didn’t supply any database info or credentials. Heroku auromatically injects this info into your settings.py file. I also need to dump out my current database which I can later import into my new app (this could also be done with fixtures).

$ python myapp/manage.py dumpdata > db.sql

I’m already storing my Python dependencies in a requirements.txt file. If you’re not, you’ll need to create this file at the project root.

$ pip freeze > requirements.txt
$ cat requirements.txt
Django==1.3
feedparser==5.1
gunicorn==0.12.2
lxml==2.3.3
psycopg2==2.4.2
python-dateutil==1.5
python-sunlightapi==1.1.0

Don’t forget to commit your changes! Then push to Heroku.

$ git commit -a -m 'Mods to run on Heroku.'
$ git push heroku master

That second line above is really something to be admired. Not only does it push the code to Heroku’s repository, but then it triggers a real deployment. Heroku automatically copies the files to the stack, installs all the dependencies (via requirements.txt), detects that this is a Django application, and runs the application with “runserver”. Poof! Done.

Well, not quite done. But really that’s bulk of the it and my app is in fact running. You can confirm this by using the “heroku ps” command.

Running your app

Now I need to do all the normal Django setup stuff, like syncdb and loaddata…

$ heroku run python myapp/manage.py syncdb
$ heroku run python myapp/manage.py loaddata < db.sql

If all goes well, I should be able to hit my site with a web browser at the wacky hostname provided by Heroku when I created the stack. Herou provides a shortcut…

$ heroku open

Final thoughts

We’ll that was pretty darn easy. A few other things to note if you’re trying this yourself.

• You’ll need to serve static media from somewhere. I used django.contrib.staticfiles in this example, but that’s probably not idea for production. Though the output does get cached… so it’s also not too bad.
• You don’t want to use the built-in Django runserver. I prefer gunicorn and it’s easy to configure that!
• Enjoy yourself. This is cool stuff!

The “Stop Online Piracy Act” (H.R. 3261) and the “Preventing Real Online Threats to Economic Creativity and Theft of Intellectual Property Act of 2011″ (S.968) are intended to solve a worthy problem, yet the methods recommended by these bills I find to be completely offensive.

I greatly value the protection of intellectual property, yet I place the values of freedom and free speech even higher. We must not enact a bill with power to dismantle these very freedoms.

Please find another way. Punish the offender, not the messenger. I have confidence that you will find an acceptable alternative.

Please undertand very clearly that your vote for these bills in their current forms will be countered with a vote against you in your next election.

Sincerely,

Drew Engelson
Cabin John, MD

Running Django management commands is easy on Heroku. For example, to syncdb you simply execute:

$ heroku run python your_app/manage.py syncdb

Easy enough. But you may find that running a custom management command to be a little trickier. You might run into something like this:

$ heroku run python your_app/manage.py your_custom_command
Running python your_app/manage.py your_custom_command attached to terminal... up, run.2
Unknown command: 'your_custom_command'
Type 'manage.py help' for usage.

Ouch! And you wouldn’t be alone here:

• Django custom commands not showing up on Heroku
• Running Django custom manage.py task on Heroku – Importing Issues
• Custom management command discovery fail

The solution

This really just comes down to Python not finding your app. You just need to adjust your Python path to include your home directory.

On Heroku, your home directory is generally “/app”. You should confirm this by running:

$ heroku run env | grep HOME
HOME=/app

A simple way to adjust your Python path within your Heroku environment (and not mucking with your app) is by setting the PYTHONPATH env variable as follows:

$ heroku config:add PYTHONPATH=/app

To confirm it is set correctly, run:

$ heroku run env | grep PYTHONPATH
PYTHONPATH=/app

Now you can run your custom management command. This also allows you to run these as cron (scheduled) tasks:

$ heroku run python your_app/manage.py your_custom_command
Success!

I hope this post will save you some headbanging. Unless, of course, it’s to the Melvins.

I know why I use Django…

I’m conducting some informal research for a project and I want to hear from you. Why you do use Django? Please leave a comment here or tweet me @handofdoom.

Try to limit your responses to 5 words or less.

{{ "Mary had a little"|add:" lamb." }}

However, in earlier versions of Django [TODO: identify specific version], the add filter only works with numbers. Using it on a string has no effect.

So how to do it?

It looks like we’re left with the stringformat filter. This feels quite dirty to me, but it works.

{{ "Mary had a little"|stringformat:"s lamb." }}

Enjoy!

We had a blast and received many smart questions from the crowd about Solr, Amazon Web Services, Celery and the recent Tupac incident in about that order. Thanks for having us DCPython!

Check out DCPython at http://dcpython.org or follow @DCPython.

AWS East is down

April 21st was quite another story! Early that morning, AWS began experiencing connectivity issues affecting Elastic Block Store (EBS) volumes in the Northern Virginia region (us-east-1), and hence any EC2 or RDS instances that depend on EBS. Oy vey!

The outage was heavily covered in the press:

• Technorati
• Information Week
• MarketWatch [video]
• Reuters

What did this mean for PBS? Well it took out our main portal site (PBS.org) and many core services (Merlin API, COVE video API, TV Schedules API, mobile apps for iPad and iPhone, and more) for a while. Ouch! While we try to leverage multiple availability zones where possible, we are entirely in Amazon’s East Coast data centers.

Recovering from AWS outage

Since the outage affected only EBS-based EC2 and RDS services in the East region, a path to workaround the outage seems obvious:

1. Avoid EBS (at least for now)
2. Go West, young man!

This is exactly what we did this morning. We relaunched some applications from backups on temporarily EBS-less servers. And we migrated some RDS database servers to the West coast region (us-west-1). Once this was accomplished, our public facing systems were back online.

Looking forward

Now what? How can we depend on AWS in the future? Should we migrate our services elsewhere? There is a universal truth that applies here: sh*t happens

In practicality, being in the cloud has provided such an improvement in our overall stability and ability to manage our infrastructure with minimal resources that I can no longer imagine life without it.

So the real question is, “How can we reduce our exposure to this in the future?”

Any ideas?

Links

• AWS Service Health Dashboard

To tell mod_wsgi to pass the Authorization header through to your application, just set the mod_wsgi configuration directive WSGIPassAuthorization to “On”.

# enable authorization headers
WSGIPassAuthorization On

# set up wsgi app
WSGIApplicationGroup myapp-1
WSGIImportScript /projects/myapp/myapp.wsgi application-group=myapp-1 process-group=myapp-1
WSGIScriptAlias /myapp /projects/myapp/myapp.wsgi

Note to self (and others who may have wasted time on such things):
Read The Docs!

Up until recently, we would have been out of luck. But fortunately Amazon released Identity and Access Management (IAM) which makes flexible access control possible. And boto makes it easy in Python.

Take this example

Say your company uses Amazon S3 store your company’s image assets in a variety of S3 buckets. If you needed to grant a third party the ability to upload new images to your S3 account, they will need a set of keys. You wouldn’t want to give them your main keys since not only would they gain access to all of your S3 buckets, but also your EC2 instances, RDS databases, etc. Not a good situation.

Here’s where IAM comes in. With IAM, you can create a user for this specific purpose which would have it’s own unique AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY key pair. Additionally, you can apply an IAM policy to restrict what this user can do… a specific S3 bucket, in this case.

Let’s give it a try

In this example, we’ll create a user called melvins and grant it access to an S3 bucket called houdini.

import boto

# Connect to IAM with boto
iam = boto.connect_iam(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)

# Create user
user_response = iam.create_user('melvins')

# Limit access with IAM policy
policy_json = '''{
    "Statement":[{
        "Sid":"RandomStringIdentifier",
        "Action":"s3:*",
        "Effect":"Allow",
        "Resource":"arn:aws:s3:::houdini/*"
    }]
}'''
iam.put_user_policy('melvins', 'allow_access_houdini', policy_json)

# Generate new access key pair for 'melvins'
key_response = iam.create_access_key('melvins')

We now have a new set of AWS keys with access limited to a single S3 bucket. We could have just as easily further limited it to specific action (i.e. S3:PutObject), or to other services (i.e. EC2:*).

The objects we created with boto provide us with all sorts of nifty info.

>>> import simplejson as json
>>>
>>> user = user_response.create_user_response.create_user_result.user
>>> print json.dumps(user, indent=4)
{
    "path": "/",
    "create_date": "2011-01-22T20:02:27.900Z",
    "user_id": "AIDAXXXXXXXXXXXXXKXBW",
    "arn": "arn:aws:iam::46XXXXXXXX90:user/melvins",
    "user_name": "melvins"
}
>>>
>>> key = key_response.create_access_key_response.create_access_key_result.access_key
>>> print json.dumps(key, indent=4)
{
    "status": "Active",
    "user_name": "melvins",
    "create_date": "2011-01-22T20:16:17.189Z",
    "secret_access_key": "vDskXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX2cob",
    "access_key_id": "AKIAXXXXXXXXXXXXQ5DA"
}
>>>
>>> # Provide new keys to user
>>> print 'AWS_ACCESS_KEY_ID', key.access_key_id
AWS_ACCESS_KEY_ID AKIAXXXXXXXXXXXXQ5DA
>>>
>>> print 'AWS_SECRET_ACCESS_KEY', key.secret_access_key
AWS_SECRET_ACCESS_KEY vDskXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX2cob

Using IAM groups

Suppose your have three trusted employees (Buzzo, Lorax and Dale) who require full access to your AWS account. You could give them the master key pair. But what if Lorax decides to leave the company? You would have to change your master key pair, redistribute to the remaining employees, and hope there was nothing depending on the old ones. I would grow tired of this very quickly.

A better solution is to grant each employee his own key pair (as described above). But rather than managing their policies individually, you could add them to a group and apply a group policy.

import boto

# Connect to IAM with boto
iam = boto.connect_iam(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)

# Create group
iam.create_group('admins')

# Apply full access IAM policy
policy_json = '''{
   "Statement":[{
      "Effect":"Allow",
      "Action":"*",
      "Resource":"*"
      }
   ]
}
'''
iam.put_group_policy('admins', 'allow_all', policy_json)

# Create users
iam.create_user('buzzo')
iam.create_user('lorax')
iam.create_user('dale')

# Generate access keys
iam.create_access_key('buzzo')
iam.create_access_key('lorax')
iam.create_access_key('dale')

# Add users to group
iam.add_user_to_group('admins', 'buzzo')
iam.add_user_to_group('admins', 'lorax')
iam.add_user_to_group('admins', 'dale')

So when Lorax takes off, it’s just a simple matter.

iam.remove_user_from_group('admins', 'lorax')
iam.delete_user('lorax')

The moral of the story is: Quit distributing your master access keys. Use IAM. I doubt you want to learn this lesson the hard way!

Version note: boto added IAM support in 2.0b3.

Links

• AWS Identity and Access Management (IAM)
• boto v2.0 > API Reference > IAM
• Melvins

The goal was quite simple… Use boto to pipe Amazon CloudWatch data into Python Google Chart and hope for some really cool data visualizations.

The results are pretty neat:

Features

pyawschart – v0.2 supports all of the metrics that Amazon makes available for Elastic Compute Cloud (EC2), Relation Database Service (RDS), Elastic Block Store (EBS), and Elastic Load Balancer (ELB).

Example code

Get EC2 cpu utilization chart for the past hour.

import boto
from pyawschart import RANGES
from pyawschart.ec2 import CPUUtilizationChart

# Connect with boto
conn = boto.connect_cloudwatch(AWS_ACCESS_KEY AWS_SECRET_ACCCES_KEY)
chart = CPUUtilizationChart(conn, 'i-6d3efXXX', RANGES['hour'])

# Display chart url
print chart.get_url()

# Download chart image
chart.download('/tmp/i-6d3efXXX-cpu-hour.png')

Get RDS database connections chart for the past day.

import boto
from pyawschart import RANGES
from pyawschart.rds import DatabaseConnectionsChart

# Connect with boto
conn = boto.connect_cloudwatch(AWS_ACCESS_KEY AWS_SECRET_ACCCES_KEY)
chart = DatabaseConnectionsChart(conn, 'my-rds-instance', RANGES['day'])

# Display chart url
print chart.get_url()

# Download chart image
chart.download('/tmp/my-ec2-instance-connections-day.png')

Feedback

While the library works well for most of the use cases I require, I would welcome feedback on your use cases. Use it… Abuse it.. then submit comments on this entry, or post issues/requests on GitHub.

URLs

• Project page: http://tomatohater.com/pyawschart/
• Source code: http://github.com/tomatohater/pyawschart
• Issue tracker: http://github.com/tomatohater/pyawschart/issues

About mid way through the event, the sky over Washington, DC become overcast, putting a damper on the re-emergence phase. But at one point I caught a glimpse just as the moon peeked through a hole in the cloud cover.

As I was tinkering with various exposures just before the eclipse, I accidentally took the following photo which I think came out pretty cool. Check out Orion and Taurus! Note to self: I need to try out more night sky photography.

By the way, as the eclipse reached totality, it was gorgeous through my telescope. Now if only I could figure out how to hook my camera up to that!

But for me, as you might imagine, the most interesting aspect is the technology architecture underlying it all.

A win for Django and Python

There is no doubt that the new PBS.org site probably ranks as one of the most highly trafficked Django sites in existence today. I think there is a lot to be said about how Django may be scaled if an application is properly designed. The tools are all there for the taking. I will write a future post that details the full architecture, but here is a short list of technologies involved in the stack: Python, Django, MySQL, Apache, Memcached, Celery, Piston, Amazon Web Services

And thanks for cloud computing…

You may have noticed Amazon Web Services mentioned above. I slipped it in at the end of the list, but in reality, it is truly the enabling technology that makes all else possible. The new PBS.org site sits on a cluster of Amazon EC2 instances and is powered by a few Relational Database Service (RDS) instances. In my opinion, RDS is already very cool, and getting much cooler.

Putting the applications in the cloud lets us be very agile on the infrastructure side. In fact, just prior to launch, we decided to put the cache on a bigger machine and that took all of 10 minutes. This kind of stuff is just not possible on a traditional infrastructure. Thank you, Amazon.

And big thanks to the entire team of smart and hardworking individuals that made this site launch a success. Hats off to you all!

CloudFront object invalidation is a recent addition… here’s how to do it in Python.

AWS REST authentication

One of the first things you’ll need to do for each request is generate an authentication signature based on your AWS credentials. So let’s define a function for that.

>>> import base64
>>> import datetime
>>> import hmac
>>> import hashlib
>>>
>>>
>>> def get_aws_auth():
>>>     date = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
>>>     auth = 'AWS %s:%s' % (
>>>         AWS_ACCESS_KEY,
>>>         base64.b64encode(
>>>             hmac.new(
>>>                 AWS_SECRET_ACCESS_KEY.encode('utf-8'),
>>>                 date.encode('utf-8'),
>>>                 hashlib.sha1
>>>             ).digest()
>>>         )
>>>     )
>>>     return date, auth
>>>
>>>
>>> print get_aws_auth()
('Thu, 22 Oct 2010 06:08:48 GMT', 'AWS 0PN5J17HBGZHT7JJ3X82:4cP0hCJsdCxTJ1jPXo7+e/YSu0g=')

We can now use get_aws_auth() to sign the following REST requests. Next, we’ll create an invalidation request.

Invalidating CloudFront objects

The invalidation handler expects a POST request with an xml payload containing a list of file paths to be expired.

>>> import urllib2
>>>
>>>
>>> host = 'cloudfront.amazonaws.com'
>>> path = '/2010-08-01/distribution/%s/invalidation' % AWS_CF_DISTRIBUTION_ID
>>>
>>> # create authentication signature
>>> date, auth = get_aws_auth()
>>>
>>> files = ['/file-01.mp4', '/file-02.mp4', '/file-03.mp4']
>>>
>>> # create xml data to post
>>> data = ['<?xml version="1.0" encoding="UTF-8"?>']
>>> data.append('<InvalidationBatch>')
>>> data.append('%s' % ''.join(['<Path>/%s</Path>' % f for f in files]))
>>> data.append('<CallerReference>%s</CallerReference>' % date)
>>> data.append('</InvalidationBatch>')
>>> data = ''.join(data)
>>>
>>> # define required http headers
>>> headers = {
...     'Host': host,
...     'Authorization' : auth,
...     'Date': date,
...     'Content-Type': 'text/xml',
...     'Content-Length': len(data),
... }
>>>
>>> # make REST request, capture 201 (success) response
>>> request_url = 'https://%s%s' % (host, path)
>>> req = urllib2.Request(request_url, data, headers=headers)
>>> try:
>>>     response = urllib2.urlopen(req)
>>> except urllib2.HTTPError, e:
>>>     print e
HTTP Error 201: Created

Of course, you probably also want to check the status of your invalidation request

Retrieving invalidation request status

Invalidation requests can take up to 15 minutes. Hit the GET invalidation API to check on the status of your recent requests.

>>> import urllib2
>>> from xml.dom import minidom
>>>
>>>
>>> host = 'cloudfront.amazonaws.com'
>>> path = '/2010-08-01/distribution/%s/invalidation?MaxItems=2' % AWS_CF_DISTRIBUTION_ID
>>>
>>> # create authentication signature
>>> date, auth = get_aws_auth()
>>>
>>> # define required http headers
>>> headers = {
...     'Host': host,
...     'Authorization' : auth,
...     'Date': date,
... }
>>>
>>> # send REST request and print results
>>> request_url = 'https://%s%s' % (host, path)
>>> req = urllib2.Request(request_url, headers=headers)
>>> response = urllib2.urlopen(req)
>>> xml = response.read()
>>> print minidom.parseString(xml).toprettyxml(indent=' '*4)
<?xml version="1.0" ?>
<InvalidationList xmlns="http://cloudfront.amazonaws.com/doc/2010-08-01/">
    <Marker/>
    <NextMarker>
        I1Z3T4GFYRIAZU
    </NextMarker>
    <MaxItems>
        2
    </MaxItems>
    <IsTruncated>
        true
    </IsTruncated>
    <InvalidationSummary>
        <Id>
            I2HD8ADHMH7R
        </Id>
        <Status>
            InProgress
        </Status>
    </InvalidationSummary>
    <InvalidationSummary>
        <Id>
            IEFHBD78AAQN
        </Id>
        <Status>
            Completed
        </Status>
    </InvalidationSummary>
</InvalidationList>

The easy way: Use boto

As of version 2.0b3, the kick-ass boto library (Python interface to Amazon Web Services) has added support for invalidating CloudFront objects.

>>> import datetime
>>> from boto.cloudfront import CloudFrontConnection
>>>
>>> files = ['/file-01.mp4', '/file-02.mp4', '/file-03.mp4']
>>>
>>> conn = CloudFrontConnection(AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY)
>>> print conn.create_invalidation_request(AWS_CF_DISTRIBUTION_ID, files)
<boto.cloudfront.invalidation.InvalidationBatch object at 0x1012b6390>

So short and sweet.

>>> from xml.dom import minidom
>>>
>>> # from a string
>>> minidom.parseString(xml_string).toprettyxml(indent=' '*4)
>>>
>>> # or from a file
>>> minidom.parse(open(xml_file)).toprettyxml(indent=' '*4)

And for the record…

>>> import simplejson as json
>>>
>>> json.dumps(json_obj, indent=4)

But upon thinking about this for a few moments, it occurred to me that my regular radio already gets NPR, costs less than $199, and it doesn’t even require an internet connection.

Will users flock to Mercurial leaving Git behind? I say, “Hello hg!”

Just launched new site for “Baseball: The Tenth Inning” by Ken Burns. Watch it this week on PBS!

The ease with which I am able spin up a server or 30 still gives me the chills even though I’ve been doing this for years by now. In fact, it is so easy that in addition to launching production applications in the cloud, we’ve been providing developers each their own dedicated EC2 servers for the applications they are building, modifying, or testing. The benefits of doing so are easy to see:

• Development environment is identical in all aspects to production environment.
• Deployment systems (such as Fabric) can be used to push to QA, staging and production servers as easily as to the developers instance.
• Instances can be easily cloned if something goes wrong. Made a mistake on configuring the environment? Who cares? Blow it away and start over.

But is it too easy?

For each active application, we may have a dozen or so developers, each with their own EC2 instance. As more applications and developers come on board, you can foresee how we might end up with multiplicative growth in the number of development servers. Let’s take 10 developers each working on 3 projects… that’s 30 development instances for 3 projects. And that’s not counting QA, staging or production instances. Since a throw-away development instance costs just as much as a mission critical production instance… the cost balance feels out of whack to me.

Is there a way to bring down the cost of development instances while maintaining the above-mentioned benefits? What about hosting development servers in a private AWS compatible cloud, such as those fronted by Eucalyptus? There may be potential here and we are prototyping this out. But I suspect the underlying hardware required to run such a private cloud would be as costly (and less flexible) than using dedicated AWS EC2 instances.

Amazon’s new micro instances should provide some relief. Any other ideas?

As it turns out, the file manager DOES now support the S3 API. Therefore my immediate need for an intermediary FTP interface is gone. While I think it is still a very cool idea, I no longer have an urgent need for it. I will continue to fix some of the major bugs with it, but most of my tinkering will cease. Please let me know if you are interested in using Faetus and I’m happy to help you out if I can.

This project came out of a specific need I have at PBS. I need to put files on Amazon S3 storage, but the 3rd-party file management system doesn’t know how to speak to S3′s APIs. What the system does understand is FTP. So I decided to write an FTP server that utilizes S3 storage as opposed to local filesystems.

The server is based on pyftpdlib, a Python library that implements most, if not the full RFC spec for. Especially nice is the fact that the the library implements the filesystem layer in an abstract fashion. I basically needed to replace all the filesystem calls with S3 API calls. I used boto to handle all the S3 API calls and I quickly had a working prototype.

After playing with my prototype for a few hours, I stumbled upon Chmouel’s ftp-cloudfs… who basically did the exact thing I had just done, but for Rackspace CloudFiles. For consistency’s sake, I rolled my code back, forked ftp-cloudfs, gave the project a questionable new name, and added my S3 code back in. La voila!

• Faetus project page
• Source code on GitHub

It’s basically a bunch of Python scripts that are all imported into one executable file called “google”. All the API calls are full abstracted allowing you do run commands like:

$ google contact list
$ google calendar add "Get up at 5:30am Thursday"

GoogleCL currently supports Blogger, Calendar, Contacts, Docs, Picasa, YouTube… though I’m sure others will be added shortly.

Installing is pretty much a piece of cake. All you really need is a modern version of the gdata library (mine was outdated) and the googlecl package itself.

$ pip install gdata googlecl
$ pip install

I see some serious life automation happening in the near future.

Release notes (v0.1):

# django-openlike

Django template tag for easily implementing and extending OpenLike protocol.
See http://www.openlike.org/ for more info.

FEATURES:
- Templatetags for displaying OpenLike widget

USAGE:
- Basic: {% openlike %}
- Less basic: {% openlike "http://domain.com/" "Page title" %}

TODO:
- Documentation.
- Add support for extended sources.
- Configuration via Django admin

AUTHOR'S NOTE:
This app is most definitely overkill given the simplicy of OpenLike and the
ease of pure JavaScript implementation. But it was a fun project to fully
support a third-party service in Django.

This is my deck from RightScale MeetUp 2010 before it was incorporated into RightScale’s template. The full MeetUp presentation is up on RightScale’s website.

At the MeetUp, I met a ton of really smart people with interesting problems to solve… many with the same problems I’m working on as well.

To date, we have attempted exactly four Drupal sites at PBS. Three of them eventually made it to production. All three experienced significant delivery, quality assurance and/or scalability problems. These apps were built by experienced development shops, even some who had proven track records and delivered exceptional (non-Drupal) applications in the past.

I am sure that it is possible to build a solid, highly scalable Drupal application. There are many examples out in the world. So what’s the deal?

In a recent post, Drupal v. Django, Nick Sergeant provides some excellent talking points on the matter. I agree with every one of his points. Some of my favorites are echoed below:

• Django is a lower level development framework (than Drupal) that encourages rapid development and cleanly written & organized code.
• With Django, we’re able to construct custom functionality at a more rapid pace than Drupal.
• Drupal funnels data into chunks of content referred to as “nodes”, which developers must often force their content into. The content should define the data structure, not the other way around.
• Because Drupal stores so much configuration and definitions of functionality in the database, it becomes incredibly difficult to version control the software.
• Drupal’s default admin interface is riddled with UI and workflow problems (one of the primary focuses of the upcoming Drupal 7 release).
• Django’s default admin interface is designed to encourage quick and easy content management, and allows developers to quickly design custom admin interfaces and workflows.

I don’t mean to add fuel the Django versus Drupal fires, so let’s just rename this post to “Why you shouldn’t introduce Drupal into a Django-centric environment.” I’m just looking to add some sanity to my life.

My experience setting my Django-mingus blog was ridiculously simple. My only issues were related to the pip install script…

1. BitBucket was down tonight…
2. One updated dependency had had trouble finding the version specified …
3. One dependency was broken
4. I had to change it to use git+http:// since my web host blocked traffic on the git:// port

Other than that, I didn’t have to do much customization. I treated my site design just like a new Mingus theme and attempted to make all my changes in CSS. I did have to rearrange some of the HTML to support my layout. I then did a bunch of clean-up in the Django templates to get consistent page titles and page themes.

I’m only using a few of the apps included with Mingus… I’ll be digging into the rest to see whether I’m missing out.