memonic

why I can't stand thread.local (and others)

Save

why I can't stand thread.local (and others)

written by Armin Ronacher, on Monday, July 10, 2006 0:00.

I had a discussion about that thingy on #pythonpaste a few hours ago. Looks like I'm the only person hating thread.local. Maybe I'm ultraconservative but I think I've some good argument agaist the use of this magic type.

preamble: This is my point of view. I know that thread local objects are a nice way to fix the problem with not passing request/session/or any other objects to methods but I don't like them. I'm not fighting against pylons or any other application using thread locals nor do I want the pylons developers to remove thread local objects.

In this particular case I want to talk about the thread local object in pylons. Pylons uses some magic thread local objects named m, g, h, c, cache, session, request, response, params and buffet. You can get those object by importing them from the yourapplication.lib.base package.

Some of them are deprecated and exist just because of backwards compatiblitity.

I was told that pylons uses paste.registry to be able to have more than one application in the same thread without getting threading issues. So it should be technically possible to write something to m without touching the m of another pylons application.

Looks like (beside the fact that the variable names are not that easy to understand when first looking at the code) pylons doesn't have any problems with thread.local since they workaround it with paste.registry

But what about web.py for example. It overwrites sys.stdout to redirect printed data directly to the request of the current thread. Nice idea but using this technique it's only possible to have one running web.py application in the same thread. *grmbl*

django doesn't use thread.local objects (not true, afair it uses thread.local objects somewhere in the core) but lacks a deployable system. You can't mix up django application since they mostly end up in name(space) clashes of either models, templatetags, templates or other magic tags. It's not possible to run two django installations in the same process since they relay on os.environ['DJANGO_SETTINGS_MODULE'] or a directly passed settings module.

I really like django, but never try to write an application you want to distribute.

Same for SQLObject. Imagine MoinMoin and pocoo would both use SQLobject and a pocoo plugin imports the moinmoin package, create a moinmoin wikiconfig and try to parse a wikipage. If you overwrite the global sqlhub the later importer moinmoin would overwrite the connection settings of pocoo. Someone on #python.de also reported stupid threading issues with sqlobject which appeared from time to time. It ended up that he switched to sqlalchemy which doesn't use thread locals any more (sice 0.2). (You can still use the magic technologie, but you have to enable it first)

As and programmer and admin I really like to have applications you can just-deploy (tm).

As just deploy i see something you can put into a debian repository and update using the normal update procedure (apt-get update, aptitute, synaptic...) And I don't want a site-packages for apps like pocoo, moinmoin since they arn't libs but applications. And I want to mix up applications in the same process like I do on ubuntuusers to be able to share user accounts, user settings, the same mailing system etc. between many applications.

And thread.local as such breaks this. paste.registry fixes it again I was told, but one thing is still left:

It's magic, hard to understand for newcommers, breaks my dir() and is slower then passing the given objects to the app

Okay. One of this points is premature optimization ;-)

  • magic / hard for newcommers - immagine that you have a project using paste.registry for a global request and response variable. You also have a globals variable to put request variables in. How do I tell a PHP developer switching to this project (this currently happens at ubuntuusers.de where it was impossible to find any python devs with web experience) that accessing myapp.request, myapp.resposne and myapp.globals is safe whereas accessing any other global namespace, object etc. isn't. It does not make sence at all ^^ Why not pass a request object to the view? Seven chars more to type? or three in case of just writing req.
  • dir() broken - i like dir(), it is the best to debug ^^ Just raise an exception, use a middleware with eval() support and navigate to the frame with the problem. dir() the object and see what's broken. Works as long as you don't use a magic thread local object which breaks dir()
  • slower - it actually is premature optimization but fetching current thread and looking up an object in a dict each time you access it is not very fast.

And here an example of how pocoo defines its views:

from pocoo.pkg.core.models import Forum
from pocoo.application import RequestHandler
from pocoo.template import PagePublisher
from pocoo.http import TemplateResponse


class IndexPage(RequestHandler, PagePublisher):

    def get_page_name(self):
        return 'index'

    def get_relative_url(self):
        return ''

    def get_handler_regexes(self):
        yield u'$'

    def handle_request(self, req):
        categories = req.db.select(Forum, Forum.c.parent_id == None)
        return TemplateResponse('index.html',
            categories=categories
        )

Oki. Maybe the imports suck. But everyone who is reading that source the first time knows where the objects are comming from.

And here a pylons/thread.local version of the above code:

from pocoo.pkg.core.models import Forum
from pocoo.template import PagePublisher
from pocoo.http import *


class IndexController(RegexController, PagePublisher):

    def get_page_name(self):
        return 'index'
        
    def get_relative_url(self):
        return ''

    def get_handler_regexes(self):
        yield u'$'

    def handle_request(self):
        c.categories = d.select(Forum, Forum.c.parent_id == None)
        m.subexec('/index.myt')

This isn't valid "pylons" :-) But it uses the ideas of pylons. Those global c, m etc objects. Of course it's less to write. But don't tell me it's easier to understand