Copia

As I've mentioned I added friendlier mutation API to Amara. Deletion didn't come up in the original discussion, but I just got around to addressing that as well. Now checked in are enhancements that support the following use cases:

Use case 10:

Source doc: spameggs
Code: del doc.a.b
Result: doc mutated to eggs

Use case 11:

Source doc: spameggs
Code: del doc.a.b[0]
Result: doc mutated to eggs

Use case 12:

Source doc: spameggs
Code: del doc.a.b[1]
Result: doc mutated to spam

Use case 13:

Source doc: spameggs
Code: del doc.a.b[2]
Result: IndexError

Use case 14:

Source doc: spam
Code: del doc.a.b
Result: doc mutated to spam

Of course there are oddities to go with the new convenience. Check out the following:

>>> from amara import binderytools
>>> doc = binderytools.bind_string("spamspam")
>>> unicode(doc.a.b)
u'spam'
>>> doc.a.b
<amara.bindery.b object at 0x685b2c>
>>> del doc.a.b
>>> unicode(doc.a.b)
u'spam'
>>> #Eh?  Still there, are ye?
...
>>> doc.a.b
<amara.bindery.b object at 0x685b8c>

Perfectly consistent with what the users seem to be saying, I think, but I'll be amazed if this doesn't trip up the odd fellow.

[Uche Ogbuji]

So the title of my last entry was "Holy trésor retrouvé!". Well, that's how you find yet another bug in xmlrpc_metaweblog.py. My first clue something was amiss when I clicked the "Post Entry" button in BloGTK was that I didn't get the usual, annoying pop-up to tell me the entry had been posted. I looked at the console from which I'd launched BloGTK and I saw:

Traceback (most recent call last):   File "/usr/bin/BloGTK", line 664, in prepPost
    post.getPostInfo(self.url, self.user, self.passwd, self.system, blogID, catID, title, body, checkPublish, self.mainGlade, self.useUTF, extended, excerpt, keywords, trackbackURLS, breaks, commentsAllow, pingsAllow)
  File "/usr/lib/blogtk/post.py", line 90, in getPostInfo
    metaweblogPost(url, user, passwd, blogID, title, body, publish, mainGlade, excerpt, extended, keywords)
  File "/usr/lib/blogtk/post.py", line 167, in metaweblogPost
    confirmPost(post, mainGlade)
  File "/usr/lib/blogtk/post.py", line 184, in confirmPost
    postLabel = gtk.Label("Entry Posted. ID = " + post)
TypeError: cannot concatenate 'str' and 'bool' objects

Which usually means xmlrpc_metaweblog.py got confused and reported its confusion with an XML-RPC response of 1. I checked the Copia site and found that the result was an empty file for the entry. I didn't find any useful tracebacks server-side, and I'm out of steam for the night, so no further debugging. I just pasted the entry into vi over ssh and all was well again.

P.S. Same problem with this entry when I tried to post it. WTF? xmlrpc_metaweblog.py should be stateless, so the presumed non-ASCII in title bug shouldn't have affected further posting, should it?

[Uche Ogbuji]

In the previous entry I discussed changes to Amara's mutation API. In the original discussion one of the things that came up was the old element/attribute conundrum. Take the following document:

y

Users like to be able to access both elements and attributes using friendly Python idiom, but here we have a name clash on the resulting a object.Right now Amara exposes the attribute as a.b and the element as a.b_, using name mangling to disambiguate.

The important thing to remember, however, is that such clashes are quite rare in practice, even when you throw in namespaces, so such mangling is rarely necessary, and I personally think Amara's current behavior makes sense. But I may just have a blind spot, so I've been paying attention to suggestions from others.

Jeremy Kloth suggested just always using different idioms. a.[u"b"] for the attribute and a.b for the element. This is not a bad idea, but I feel that given that clashes are rare, that it complicates the common case just to aid the rare case.

Luis Miguel Morillas had an idea I consider almost the opposite. Rather than completely separate element/attribute idioms, Luis suggests embracing how Amara has unified them. Right now Amara rolls up multiple elements of the same name in a convenient way:

yz

Works such that a.b or a.b[0] yields the element with y and a.b[1] yields the element with z. Luis thinks that the following case should just be an extension of this:

yz

And then a.b or a.b[0] would yields the attribute value (u"x"), a.b[1] would yield the element with y, and a.b[2] would yield the element with z. I kinda think of this idea as "so crazy it almost makes perfect sense", but it's way too big a change to introduce before Amara 1.0. I'd be curious to hear what others think of it. Luis actually brings it up in the context of mutation--see his original post (scroll to the bottom)--but I figure that the mutation API will follow naturally from the access API, so I'm focusing my thoughts a bit.

[Uche Ogbuji]

Tom Lazar asked for a friendlier idiom for mutating elements in Amara. I was reluctant at first because the simpler-on-the-surface idioms he wanted would require rather untidy idioms in the code. I relented to the argument that user convenience comes even before clean code. I finally got around to making and committing the changes today. I'd planned to release Amara 1.0b2 as soon as I'd made these changes, and the timing seems perfect since we've just released 4Suite 1.0b1, but the changes are intrusive enough that I think I'll give folks a chance to try things out from CVS and first see whether it craters for anyone. Please give it a go and give me feedback here or on the mailing list. Thanks.

Here are use cases illustrating the new idioms for Amara. I have added them to the test file mutation.py:

Use case 1:

Source doc: spam
Code: doc.a.b = u"eggs"
Result: doc mutated to eggs

Use case 2:

Source doc: spam
Code: doc.a.b[0] = u"eggs"
Result: doc mutated to eggs

Use case 3:

Source doc:
Code: doc.a.b = u"eggs"
Result: doc mutated to

Use case 4:

Source doc: spamspam
Code: doc.a.b = u"eggs"
Result: doc mutated to eggsspam

Use case 5:

Source doc: spamspam
Code: doc.a.b[0] = u"eggs"
Result: doc mutated to eggsspam

Use case 5:

Source doc: spamspam
Code: doc.a.b[1] = u"eggs"
Result: doc mutated to spameggs

Use case 6:

Source doc: spamspam
Code: doc.a.b[2] = u"eggs"
Result: IndexError

Use case 7:

Source doc: spam
Code: doc.a.b = u"eggs"
Result: doc mutated to spam

Note: attributes take precedence over same name elements in binding. See next use case.

Use case 8:

Source doc: spam
Code: doc.a.b_ = u"eggs"
Result: doc mutated to eggs

In a follow-up entry I'll talk about some other suggestions I've received on this matter.

[Uche Ogbuji]

Update: How could I have forgotten --enable-unicode=ucs4 in the Python build instructions?

Just gathering up some details on how to install 4Suite as non-root (i.e. in a user's home directory). This is based on experience installing on Red Hat and Fedora Core, but should work for most POSIX environments.

If you don't have Python installed (or want your own copy):

Grab Python-2.3.x.tgz or Python-2.4.x.tgz and unpack:

tar zxvf ~/dl/Python-2.3.5.tgz
cd Python-2.3.5/
./configure --prefix=$HOME/lib --enable-unicode=ucs4

Pick whatever prefix works for you. --enable-unicode=ucs4 is essential IMO if you're doing XML processing.

make && make install
ln -s $HOME/lib/bin/python $HOME/bin

The last step is to put the Python exe you just built into your $PATH, presumably before any other Python exe in the system.

Now for 4Suite

Grab 4Suite 1.0b1

cd $DOWNLOADS
tar zxvf 4Suite-1.0b1.tar.gz
cd 4Suite-1.0b1
python setup.py config --prefix=$HOME/lib
python setup.py install

Notice the extra "setup.py config" step. This is the key to the whole thing. The "setup.py config" sets the location for all the files installed by 4Suite except for the Python library files, which are installed to the location determined by the Python executable used to invoke the setup script. For more on where 4Suite puts things, see Mike Brown's excellent document "4Suite Installation Locations".

There is also a --home option to setup.py config, but do not use this unless you really know what you're doing. Stick to --prefix.

Finally you may want to make a link for all the 4suite commands to your home's bin directory

ln -s $HOME/lib/bin/4* $HOME/bin

Now you can run the tests.

cd $HOME/lib/lib/4Suite

Remember that this is beta software, and some test failures are to be expected (heck, I'd be amazed if there weren't some test failures with the full 1.0 release).

[Uche Ogbuji]

The announcement

Yaaaaay! This is nominally the feature freeze for the way, way overdue 4Suite 1.0. Let's hope this freeze accelerates progress towards full 1.0. Thanks to all our patient users. The focus of this release is probably performance. Jeremy Kloth, one of the best programming minds I've encountered, threw himself into the challenge of squeezing waste out of Domlette, without losing its great functional benefits. Some of the resulting gains are amazing. There are a lot of other fixes and enhancements, and I think it's a very solid release.

My next step is to release Amara 1.0b2 and kick off a branch to take better use of some of Jeremy's enhancements, including a super-efficient mini-SAX for Domlette.

[Uche Ogbuji]

ReleaseForge

We just put out a 4Suite release and one of my tasks was to update SourceForge. If you've ever maintained a project on SF, you'll know that making a release through the Web forms is a very tedious experience. ReleaseForge is a Python/QT GUI that makes the process easier. Much easier. Normally SF release for 4Suite takes me a half hour or so. With ReleaseForge it was a matter of five minutes.

In installed the ReleaseForge RPM, but it didn't seem to have useful dependency info (a common problem with Python RPMs). I'm on Fedora Core 3, and got it working fine by doing:

apt-get install PyQt sip

After launching ReleaseForge and giving it your SF login info, you start with some basic info for the release.

Then you select all the files to release and use simple controls to set the file types. This is the part that used to really get my goat. Now it's a snap.

At this point ReleaseForge takes over and does all the talking with SF.

ReleaseForge offers an option to post project news as well.

But this seems to use different rules from the SourceForge Web forms. When I posted simple text with line breaks, ReleaseForge humped it all onto one line, as you can see. Posting the same text into the SF Web form had the expected result. But posting news on SF was already easy, so no real complaints about this little quirk.

[Uche Ogbuji]

Generally I use BloGTK via XML-RPC to post here. Sometimes I want to ssh in and hand edit files as well. Occasionally I want to hand create entries and maybe even edit via BloGTK. Either way, permissions become a pain in the ass. Apache runs as user $APACHEU and group $APACHEV (I'm using shell variables rather than revealing the actual names out of bootless paranoia). When I log in, I'm user $UCHE in group $OGBUJIS. Here is the rather brute way I solved part of the problem.

I set the SGID bit on all the directories to which I expect files written through pyblosxom.cgi:

chmod g+s $DATADIR chmod g+s $DATADIR/metadata chmod g+s $DATADIR/comments

I made sure the group of each was $OGBUJIS, of course. So now whenever a new file is created through XML-RPC or some other plug-in, it's created with group $OGBUJIS, and I can happily ssh in to edit as $UCHE.

This doesn't solve the converse problem of using CGI to edit files I created by hand, but this is rare enough that i just log in as root and use chown to sort things out.

Does anyone else have a better way? I thought of Apache suexec, but my past experiences with it have been nightmares of impenetrable detail.

[Uche Ogbuji]

RDF hacking for fun and profit -- Bill de hÓra

"I find 4Suite to be stable software (tho' I'm not sure the RDF stuff is active anymore"

The main limitation with RDF in 4Suite is that it has not not been tracking the latest specs. This sucks, but it reflects the reality of "it works, and grand updates don't scratch anyone's itch". 4Suite's RDF library is actually very stable, and has been accumulating bug fixes, performance fixes and new drivers.

I'd say that 4RDF is fine if you don't need all the nuances of the new specs (which are modest enough). It is heavily used, which is one nice test of its suitability. We do have grand post-1.0 plans, but they are not yet set in stone. My guess is the following:

• We'll abandon our own parser for rdflib. That parser is SAX-based, has been tracking the latest specs, and is very well tested. This is actually something we and the rdflib folks have been discussing near forever. We just haven't got around to the actual work (itch scratching need and all that).
• We'll make the low-level API more Pythonesque. Developments such as iterators and generators have come since the original 4RDF effort, and we want to put them to good use.
• We'll work in a Versa 2.0 (RDF query language). SPARQL is not doing it for me, and for a lot of my colleagues and corresponents. OK. I'll be blunt. I think SPARQL sucks, and I'm likely to support W3C XML Schema before I support it (hint: earthworms will fly of their own locomotion before either event).

"Uche et al have been working on anobind most recently)..."

Well, that's just me, no et alii so far. And Anobind is no more. It has been absorbed into Amara XML Toolkit. I'm developing Amara in order to complement 4Suite, not to supplant it in any way. It's an add-on to 4Suite that gives Pythoneers the super-friendly idioms they like. I still put into 4Suite about as much effort as I do Amara.

One shouldn't make any assumptions on 4Suite development based on Amara.

[Uche Ogbuji]

So this morning I noticed Copia was down. Requests would hang until they timed out. Apache seemed up, and running pyblosxom.cgi directly worked fine. I enabled the CGI debug line:

import cgitb; cgitb.enable()

But this didn't help. It felt like an infinite loop somewhere in the guts of processing. After fitful debugging efforts (got work piling up), I traced the problem to the logstats.py module. My approach in the end was to use py['load_plugins'] in config.py, comment out all plug-ins, and uncomment them until I found the culprit. It was really cool to uncover comments one by one and see different aspects of the site start working step by step. It really reinforces the power of PyBlosxom's "microkernel" approach. My resulting config was:

py['load_plugins'] = [
  "comments",
  "firstdaydiv",
  "keywords",
  "logrequest",
  #"logstats",
  "meta",
  "metatime",
  "pyarchives",
  "pycalendar",
  "pymarkdown",
  "trackback",
  "xmlrpc_blogger",
  "xmlrpc_metaweblog",
  "xmlrpc",
]
#The above list is only for debug purposes.  If *not* debugging,
#Uncomment the following line
#del py['load_plugins']

I'll have to check later into what;s causing logstats.py to kill the server, but other PyBlosxom users might want to take note.

[Uche Ogbuji]