So, I got tired of the way Django-SocialAuth was borked and not working for me, so I forked the project and have put up my own copy at GitHub.

There are three things I noticed about the project right away: First, it forces you to use a broken templating scheme. I haven’t fixed that, but in the meantime I’ve ripped out all of the base.html calls to keep them from conflicting with those of other applications you may have installed. Really, the templates involved have very little meat on them, especially for the login.html page. These are components that would have been better written as templatetags. Second, the project is rife with spelling errors. (The most famous, of course, being that the original checkout was misspelled “Djano”). I am a fan of the notion that a project with spelling problems probably has other problems. I’ll make allowances for someone for whom English is a second language, but I was not filled with confidence. And third, the project violates Facebook’s TOS by storing the user’s first and last name. Along the way I discovered that the Facebook layer was completely non-functional if you, like three million other Facebook users, had clicked “Keep me logged in,” which zeros out the “login expires” field from Facebook. It would never accept you because your expiration date would then always be January 1, 1970, effectively before “now.”

I’ve barely begun hacking on the beast, but already I’ve made some progress. Facebook now works the first time around. I’ve cleaned up much of the spelling and grammar in the documentation, such as it is, and I’ve clipped many of the template naming problems that I saw in my original use of the system. I’ve also revised setup.py so that it runs out of the box, although I’m tempted to go with a different plan, one like django-registration where it is your responsibility to cook up the templates for the provided views. And I’ve ripped out most of the Facebook specific stuff to replace it with calls to PyFacebook, which is now a dependency.

One thing I do want to get to is a middleware layer that interposes the right social authentication layer on those users who come in from the outside world: i.e. if the AuthMeta indicates you’re a facebook user, then request.user will be a lightweight proxy between you and Facebook for those fields that are, by definition, Facebook-only (and a violation of the TOS if you copy them). It might make more sense to have a decorator class, but only if you don’t have a gazillion views.

I haven’t gotten much further than a Facebook layer that satisfies my immediate needs. I haven’t had a need to branch out and explore the Twitter or Oauth components yet. What I needed at the moment was a simple authentication layer that allowed either local users (for testing purposes) or FacebookConnect users, and one that didn’t need to contact Facebook for absolutely every view, whether you wanted it or not, just to check “is this guy still a facebook user?”, which is how the DjangoFacebookConnect toolkit does things. I suppose, if you’re a Facebook app, that’s what you want, but I’m not writing a Facebook app, I’m writing an app that uses FacebookConnect to associate and authenticate my application users’s accounts via their Facebook accounts.

Tags: django, programming, web development

Eddie Sullivan at Chickenwing Software has a fascinating post entitled The Facebook Platform is Dead. I agree with many of his comments. I don’t think there’s anything terrible about the “Facebook Certified Application” program; that’s a business decision, not a software policy decision. But Sullivan says one thing that set me off. He wrote:

The big companies can afford to hire someone full-time to test and re-test their apps against every change to the back-end, but the rest of us cannot.

To which my reaction is: shut up, and don’t be so damned lazy.

Install Celerity, and get headless testing with WATIR in a rapid-response environment. Install Hudson and get fully automated continual integration. Install Git as your repository, and tell Hudson what your master is, and honor it. Install Cucumber so that when it fails, the failure reports are in clear and unambiguous English. Put this all on that archaic hunk of junk PC in your basement, give it a fresh hard drive and install Debian Linux. Give Hudson a mailserver so it can notify you when an automatic test run fails.

None of this, from building your own PC and installing Linux all the way up to installing Ruby, JRuby, Java, and all of the other tools necessary to support your build environment and make it work, ought to be beyond the ken of the average programmer.

Facebook is just a web application.  Treat it as such.  Test against it.  Get a few Facebook Test Accounts, write a few WATIR scripts to automate their Facebook relationships and friend graphs, write more to log in and go to the application, then test the Hell out of your application.

None of this is hard. If you spend one week teaching yourself how to set Hudson and Git up correctly, you’ll benefit forever from Kent Beck’s famous quote, “transmuting fear into boredom.”  Even better, by putting it on Hudson and Git, you get freedom from even the boredom, for the most part.  Instead, you get knowledge that your fixes don’t break anything, and the capability of backing out when they do.

What is hard is being in the habit of testing.  Of writing testing in terms of expectation. I’m fair at it, but I’m getting better.  I aspire to Beck’s mantra: I’m not a great progammer.  I’m a good programmer with great habits.   Test-driven development (and behavior-driven development) are great ideas (although a lot of TDD zealots go overboard, with the predictable backlash), but integrating them with even better, continuous building and continuous testing, should make all web application development better.

Believe me, Facebook apps are in desperate need of two things: automated testing, and better graphic design.  I can at least contribute to one of these.

Tags: django, programming

I use Django signals a lot in my professional work, mostly to create specialized tables that track events in the ecosystem of social networking sites that I build.  For example, if I make a post on a social networking site, that causes an event that creates a signal.  That signal will be heard by, for example: (1) a reward mechanism, which might give me a badge/acheivement/sticker/shiny rock/whatever to acknowledge my place in the social network heirarchy, (2) a news mechanism, to look up who my friends are and tell them what I’m doing, (3) a logging mechanism, which will be of interest to my investors, (4) a social media mechanism, which will analyze my relationships with other social networking sites and ping them, among (5, 6, 7) whatever else you can think of.

These are all unique, filtered views of an action I just took that might serve me as agents of attention, reputation, and illumination.

As I’ve been working in this space, I’ve learned three very important rules for Django:

(1) Any Django application (not project, application) that builds its tables via signals and business logic rulesets must only and ever build its tables via signals and rulesets.  It must not have its own views for doing so.  It’s CUD is signals.  Only the R in CRUD may have views for the signal-built application.

(2) When dumping data for your project, never dump data from the signal-based applications.  When you want to reload this data (after the appropriate mangling/filtering/whatever), the objects in your ecosystem models will send out the appropriate signals to build those tables for you.  (Signal senders in your views that alter data?  Shame on you!)

(3) As a consequence of (2), your signal-built data tables must take their dates from their instances.  Otherwise, the signal-built tables become disordered with respect to the events they’re expected to monitor.

Of course there are exceptions to these rules, but this is a very solid way to think about doing signal-based development.

Tags: django

I needed some rails-like environment settings using Django.  This is the quick and easy way to get that done.   First, after building your “settings.py” file, create an environment under your project root named “env“.   Move your settings.py file to this directory, rename it “development.py"

Now, in your root directory, open up a new file named “settings.py” and put this in:

import os
import os.path

PROJECT_ROOT = os.path.normpath(os.path.dirname(__file__))

local_import = "env/development.py"
if os.getenv("DJANGO_ENV") == 'TEST':
    local_import = "env/test.py"
elif os.getenv("DJANGO_ENV") == 'PRODUCTION':
    local_import = "env/production.py"

import_file = open(os.path.join(PROJECT_ROOT, local_import))
exec(import_file)

Bingo!  Now you have different versions of settings.py depending upon whether or not you’re starting the server in TEST, PRODUCTION, or the default, “development”.  Create production.py and test.py as needed.  You can start the server with:

DJANGO_ENV=TEST ./manage.py runserver

And it will load the correct environment.  Using a shell script to run a test server and then a test harness might not be the most elegant thing in the world, but at least it’s Un*x, and it makes development less stressy.

Sweet!

Tags: django

Ah, the bleeding edge.  It’s a war out there!

This morning, Facebook released fbwatir. I’ve just spent the past few hours knocking it around, and have come to the conclusion that it’s pretty mega-borked but it can be saved.

In fact, I now have it working with Cucumber and Firewatir. There are several major flaws in FBWatir, the biggest of which is that it assumes its own responsibility for the browser object. This is broken, and causes Cucumber to spawn an innumerable number of Firefox windows. Commenting out the browser invocation in FBWatir and putting into cucumber’s own env.rb file is much better.

I also discovered that there’s a bug in Firewatir 1.6.2 that assumes that “window zero” is always broken,  but Javascript indexes the windows starting with zero, so window zero is still a valid window ID.  Annoying as hell, but easily monkeypatched away.

I now have Cucumber working with FacebookConnect…

Given how little I know about Ruby, and given how my Ruby expert says I’m doing “inappropriate things” with the Ruby scope, that’s a freaking miracle.  We’re working together to make it work better.

Tags: django, programming, ruby

Sigh.

I’ve just spent the last few hours wandering around the various “open source” analytics programs trying to find the exact right fit for what I want.  I’m not finding it, which means that (headache ahead) I may have to write something myself.  There’s a django-analytics placeholder in GoogleCode, but it’s empty.  I at least have a model!

Basically, I have a distributed subscriber/producer package, and I want to be able to present individual producers with analysis specific to their work.  Because the work is long-form text, I want to be able to tell the viewer that the reader scrolled every paragraph into view (no, really!) and actually read the work, not just scanned it.  Both of these are more or less beyond the province of Piwik, Google Analytics, or OWA.

Time to put the research aside and concentrate on finishing the product.

Tags: design, django, web development

We frequently write little functions that populate the Django context, and sometimes we want that context to be site-wide, and we want every page and every Ajax handler, basically everything that takes a request and spews a response, in our application to have access to that information.  It might the user’s authentication, or his authorization, or some profile information.  Or it might be environmental: a site might have figured out what time it is on the user’s site, and will adjust backgrounds and themes accordingly.

The context might be a simple variable.  I have an example right here: is the browser you’re using good enough?  (I know, this is considered Bad Form, but it’s what I have to work with) .  The function has the simple name, need_browser_warning.  The context key may as well have the same name.  Using a constant for the context key is the usual pattern; this ensures the Django programmer won’t get it wrong more than once, at least on the view side.  (The template is another issue entirely.  Set your TEMPLATE_STRING_IF_INVALID in settings.py!)

I wanted something more clever in my context processor.  Here’s sickly clever:

import inspect
def need_browser_warning(request):
    return { inspect.currentframe().f_code.co_name:
        not adequate_browser(request.META.get('HTTP_USER_AGENT')) }

Yeah, that’s a little twisted.  It guarantees that the name of the context key is “need_browser_warning“, and the value is True or False depending upon what the function “adequate_browser” returns, which is what we want, so it’s all good.

Obviously, this isn’t good for everything.  Some context processors handle many, many values.  But for a one-key, this is a nifty way of ensuring name consistency.

Tags: django, python

Today’s little snippet: Filtering a loosely coupled many-to-many relationship.  As revealed earlier,  I don’t really “get” the difficulty with many-to-many relationships.  I don’t even get the difficulty with signals; if you define the many-to-many manually, handling signals on it is trivial compared to trying to do it manually in one of the referring classes.

Today, I was working on an issues tracker.  There are two classes at work here, the Profile and the Issue.  One profile may be interested in many Issues, and obviously one Issue may be of interest to many Profiles.

This calls for a ProfileIssue table that stands independent (in my development paradigm) of both Profiles and Issues.   As I was working on a dashboard, I realized that one of the things I wanted was not just a list of the issues the profile was following, but also a list of the issues that the profile was responsible for creating.  As it turned out, adding that query to the ProfileIssueManager is trivial, but requires a little knowledge:

class ProfileIssueManager(models.Manager):
    def from_me(self, *args, **kwargs):
        return self.filter(issue__creator__id = self.core_filters['profile__id'])

The secret here in knowing about the core_filters attribute in the RelatedManager.   It contains the remote relationship key that you can use;  calling from_me from profiles works, but calling it from anywhere else doesn’t.  The IssueRelatedManager won’t have a profile_id and this will blow up.  That’s okay; using it that way is an error, and this is a strong example of Crash Early, Crash Often.

I can here some of you cry, “Now why, why would you need such a thing?” Well, the answer is pretty simple: templates. Having one of these allows me to write:

<p>Issues tracked: {{ profile.issues.count }}</p>
<p>Issues created: {{ profile.issues.from_me.count }}</p>

And everything will work correctly.

Tags: django, python, web development

Repeat after me:

• Registration is not Authentication is not Authorization is not Utilization.
• Registration is not Authentication is not Authorization is not Utilization.
• Registration is not Authentication is not Authorization is not Utilization.

I’ll keep reminding myself of that until I figure out how to disentangle the four from this damned Facebook app.  Registering to use the app is not the same thing as authenticating to use the app, and it’s definitely not authorization to determine your level access.  Nor is any of this related to callbacks to the social application network to give you things like lists of friends and writing on your wall; that’s outside the responsibility of SocialAuth anyway.

Tags: django, python, web development

If you’ve created Django Application A, and then Django Application B, it is acceptable (and even sometimes necessary) for Application B to reference Application A.  The canonical example is Django contrib.auth; everyone references that beast.  It is not acceptable for you to go and edit Application A to reference Application B.  That is officially Doin’ It Wrong.

In a smarter world, you will never use a Django ManyToMany field.  You will create an individual class referencing both objects of the many-to-many relationship.  You will inevitably need more smarts than a mere two-column table, and a separate class, however small and insignificant, will provide both self-documentation and a chance to define the __unicode__() method for administration. Django is smart enough to hook up the relationships under the hood.

Unit testing is goddamned hard when your application is married to FacebookConnect.  A smarter relationship uses the SocialAuth layer, with additional proxies for information and posting handlers.  That way, not only can your application send updates to Facebook walls, but it can also update its activity on Twitter, and allow authentication via Google, and so on.  By using the SocialAuth layer, you can create a pseudo-layer that handles testing.  You’re still beholding to testing the SocialAuth stuff yourself.

If you’re using SocialAuth, push all of your user-related smarts into the UserProfile object, and always refer to it.  Build your UserProfile object to own the proxy to the user’s authenticated relationships with social media.  After login, leave the user alone!  Better yet, use middleware to attach the profile to the request object automagically if and when it’s present, and live with it.

Tags: django, web development