More NikiWiki patches

My pet project, NikiWiki is starting to grow up. Since my last post, I've added a number of features and can cross a few items off the todo list. The most visible change is that NikiWiki can now be used for a basic blog.

I moved neohippie.net over to NikiWiki about a month ago and have been fairly impressed with how easily I can reshape the site with very little effort. Any subdirectory can become a "blog" by adding a .index file inside it with a comma delimited list of page names and unix timestamps. While this certainly isn't the most elegant solution, it works fairly well and doesn't get in the way too much.

NikiWiki now has PAM authentication using Chris Lee's incredibly simple pam.py module. You can implement a number of different authentication methods just by loading and unloading PAM modules. However, if you feel the need to add something a bit more custom, simply implement an authenticate(username, password) method that returns a True or False value indicating whether or not the user has been authenticated. For example, if you wanted to read from an htpasswd file: 

from crypt import crypt

def authenticate(username, password):
     for line in file('/path/to/htpasswd'):
       line = line.split(':', 1)
         if line[0] == username and line[1] == crypt(password):
           return True
         else:
           return False
It's possible that some users won't want to use PAM for authentication but I think it makes sense to push users in that direction anyway. Standard authentication methods are a good thing. If you don't like it, change it.

NikiWiki now includes a module I've been working on called nstore. nstore provides a fairly clean and pythonic method for building complicated data storage systems for key->value pairs. The default NikiWiki build just uses a simple FileStore, but you can change this to give you all sorts of neat configurations. For example, I run neohippie.net on a set of three servers in two physical locations. I keep a central authoritative data store on one server, with local caches on the others. My NikiWiki data store looks like this: 

self.data = nstore.CacheDict(
     nstore.HTTPStore('http://api.neohippie.net:9608/'),
     caches=[
       nstore.FileStore('/web/sites/neohippie/cache/')
     ]
   )
I've added the newlines for clarity. In a nutshell, this configuration performs reads and writes through a CacheDict that uses an HTTPStore as the primary data store and the local filesystem as a cache. The first time an element is read, it is read from the HTTPStore and saved in the local FileStore. On successive reads, the element is read directly out of the local FileStore.

There are a few obvious limitations to this approach, the most notable is that local caches are not updated after the first fetch and they never expire. This is fine for seldom updated sites where I just clear the caches after I change anything but may be a problem for more complicated sites.

I've got some interesting ideas for nstore (DHT anyone?) that should materialize in the near future. Until then, feel free to hack around with it. I've found that writing new UserDicts for nstore can be a lot of fun.

Back to NikiWiki. The other large change introduced recently was a WSGIApp class that completely negates the need for web.py. I was using very little of the functionality within web.py and decided that it would be nice to have one less dependency to worry about. Usage is fairly simple: 

   server = WSGIServer(WSGIApp(urls)).run()
This is assuming that your WSGIServer takes a method reference as it's first argument. NikiWiki uses flup to provide a FastCGI interface that you can plug into your web server but any WSGI server should work.

For those not keeping track, the current list of dependencies looks like this:

All of the dependencies except for Python and nstore can be installed with easy_install.

Known bugs:

Feel free to bug me via email or jabber if you have problems with NikiWiki.

or