Copia

I've been experimenting with the use of FuXi as an alternative in situations where I had been manipulating application-specific RDF content using Versa within a host language (XSLT). In some cases I've been able to reduce a very complex set of XSLT logic to 1-2 queries on RDF models extended via a handful of very concise rules (expressed as N3). I'm hoping to build some usecases to demonstrate this later.

The result is that I've rearchitected FuXi to work as a blackbox directly with a 4RDF Model instance (it is now query agnostic, so it can be plugged in as an extension library to any other/future RDF querying language bound to a 4RDF model). Prior to this version, it was extracting formulae statements by Versa queries instead of directly through the Model interfaces.

Right now I primarily use it through a single Versa function prospective-query. Below is an excerpt from the README.txt describing it's parameters:

prospective-query

prospective-query( ruleGraph, targetGraph, expr, qScope=None)

Using FuXi, it takes all the facts from the current query context (which may or may not be scoped) , the rules from the <ruleGraph> scope and invokes/executes the Rete reasoner. It adds the inferred statements to the <targetGraph> scope. Then, it performs the query <expr> within the <qScope> (or the entire model if None), removing the inferred statements upon exit

FuXi is is now a proper python package (with a setup.py) and I've moved it (permanently - I hope) to: http://copia.ogbuji.net/files/FuXi

I was a little unclear on Pychinko's specific dependencies with rdflib and cwm in my previous post, but Yarden Katz cleared up the confusion in his comments (thanks).

The installation and use of FuXi should be significantly easier than before with the recent inclusion of the N3 deserializer/parser into 4Suite.

Chimezie Ogbuji

Has fancied for itself the time and energy to do what? Wait. Hold up. There's a Quotīdiē in here.

There were lights and sirens, gunshots firing
Cover your eyes as I describe a scene so violent
Seemed like a bad dream, she, laid in a blood puddle
Blood bubbled in her chest, cold air brushed against open flesh
No room to rest, Pain consumed each breath
Shot twice with her hands up
Police questioned but shot before she answered
One panther lost his life, the other ran for his
Scandalous the police were as they kicked and beat her
Comprehension she was beyond
Trying to hold on
To life. she thought she’d live with no arm
That’s what it felt like, got to the hospital, eyes held tight
They moved her room to room-she could tell by the light
Handcuffed tight to the bed, through her skin it bit
Put guns to her head, every word she got hit
"Who shot the trooper?" they asked her
Putting mace in her eyes, threatened to blast her
Her mind raced till things got still
Opened her eyes, realized she’s next to her best friend who got killed
She got chills. They told her: that’s where she would be next
Hurt mixed wit anger—survival was a reflex
They lied and denied visits from her lawyer
But she was building as they tried to destroy her
If it wasn’t for this German nurse they would have served her worse
I read this sister’s story, knew that it deserved a verse
I wonder what would happen if that woulda been me?
All this shit so we could be free, so dig it, y’all.

[SNIP the verse about further abuse and imprisonment of Assata]

From North Carolina her grandmother would bring
News that she had had a dream
Her dreams always meant what they needed them to mean
What made them real was the action in between
She dreamt that Assata was free in their old house in Queens
The fact that they always came true was the thing
Assata had been convicted of a murder she couldn't have done
Medical evidence shown she couldn't have shot the gun
It’s time for her to see the sun from the other side
Time for her daughter to be by her mother’s side
Time for this beautiful woman to become soft again
Time for her to breathe, and not be told how or when
She untangled the chains and escaped the pain
How she broke out of prison I could never explain
And even to this day they try to get to her
But she’s free with political asylum in Cuba.

—Common—"A Song for Assata", "Like Water For Chocolate"

I'm a bit ashamed that I'm only now hearing that on May 2, 2005 a $1 million bounty was placed on the head of Assata Shakur, and that she was added to the FBI's Domestic Terrorist List, alongside Osama bin Laden (how dare they trivialize in this way the atrocities that bin Laden committed in Nairobi, Dar es Salaam and New York City, among other places)? I suppose I'm more a slave to mainstream media than I'd thought.

As I listened to Common's heart-wrenching song over and over again this morning, I found myself in tears. America has to varying extent venerated Martin Luther King, Malcolm X, despite the fact that they were technically criminals in their time (peace to César Chávez who astonishes me with how much he accomplished acting entirely within the law). We accept the justness of their challenge to lawfulness because of what we have distilled from their struggle—Basic civil rights enjoyed by Black Americans—Basic civil rights for people such as me who can safely come to a country that has through most of its history shown people of my ethnicity nothing but savagery and violence. I'm grateful, and not just to the big names of the struggle, but also to the many who suffered horrifically in the name of freedom, and whose lesser fame has not spared them continuing injustice.

The personal story of Assata Shakur (see her home page and autobiography), like that of most of the Black Panthers, makes for dreadful reading. These people paid a high price for being part of a pincer movement that shamed and threatened America into civility (and no, I do not believe that either the shame from the non-violent movement or the threat from the militant movement would have been as effective alone). I accept a lot of the past horror of what they endured (easy for me, who didn't personally suffer this) as inevitable loss in battle, as much as I view the extensive casualties of the Colonies' minutemen during the Revolutionary war. But for me to witness the recent action of a U.S. regime, in 2005 against one of the hapless combatants in the battle for for civil rights is entirely unbearable. We are in the midst of a loudly proclaimed war on terrorism which we the Americans of African descent are supposed to heartily endorse, and the Justice Department sees fit to address its resources towards settling old white supremacist scores? Unbearable.

There has been plenty of injustice of all variety to go around in the U.S., and abroad in recent years, and in some respects this resumed persecution is just another story to be swallowed up in the ennui of the American public. It would almost be better if people came off their couches to cheer these abominations. At least we would recognize our times as a genuine struggle over the character of civilization rather than the reign of an arrogant cadre that recognizes opportunity in a population gorged and jaded propter panem et circenses.

It is furthermore intolerable to be so completely shamed by the vicious and autocratic regime of Fidel Castro. Assata Shakur ran to Cuba as one of the only places where she could find some escape from U.S. persecution. There was an unfortunate strain of communism that ran through the Black Panther Party, but this is no more remarkable, for example, than the fact that such a Communist strain ran through Jewish intellectuals in the mid 20th century. Intellect frustrated by oppression and alienation often drifts towards communist ideals. Only a vision of utopia sustains hope reared in ghetto soil. As oppression and alienation decrease, it becomes easier to see the evils of Communism. It is important to point this out because people are already conflating the campaign against Shakur with the campaign against her present benefactors. We perjure ourselves when we fling slogans and sanctions at the cause of freedom in Cuba while showing nothing but contempt for the freedom of one of our own in that very country, who has already suffered at our hands so much more than she could possibly deserve.

I urge anyone moved by curiosity of what I've written to learn more about this story. Look at both sides of the story. I certainly did. I'm no knee-jerk anti-establishment polemicist. This matter says as much about America (whether or not you support the actions of the Justice Department) as do the other handful of stories our media has deemed worthy of presenting to you.

[Uche Ogbuji]

For a backgrounder on EXSLT, see my article "EXSLT by example". EXSLT was born of general contribution of the community on the XSL mailing list. Jim Fuller, Dave Pawson and Jeni Tennison started off a private thread to turn the list discussion into action, and I soon joined them. The result was the exslt web site, mailing list and initial set of extension specifications. We all agree that Jeni put the most work into it, and is the leading light of EXSLT, but she has been very busy lately (genius tends to occupy itself in overwhelming volume), and the rest of us have also had a hard time giving EXSLT the effort it desires. Last week, however, Jim Fuller and I decided to take some steps towards getting EXSLT back on course. In part, this is because some people want to get things moving on EXSLT for XSLT 2.0.

Here is a list of the things we're considering taking on in order to get to something we can call "EXSLT 1.0".

Clearing up licensing

I have proposed switching from the intended (but unstated) public domain to a CC attribution license for all EXSLT work products. It seems everyone accepts this, but the main remaining question is assignment of copyright. Some of the suggestions:

• Assign to all the four managers. But do copyright decisions then have to be unanimous amongst us? Is it a problem that we reside in different countries?
• Assign to Jeni Tennison alone. But does she have the bandwidth to dispatch all copyright matters?
• Assign to the EXSLT.org domain. I can't now find who suggested this to me, but my main question is the legality of such a move.

Creating some caretaker foundation for EXSLT is not really an option, because no one (I think) has time for all the work that entails.

Improving the information content of the EXSLT Web site

We need a good overview. We need news that we can keep up to date without too much effort, including references to the many good articles on EXSLT. We need better documentation for almost all modules (the perennial example is that users get confused as to whether or not they have to download stuff from EXSLT.org in order to use the extensions). We need a better way to manage information about implementations. We need a FAQ. We should address outdated URLs from EXSLT specs (e.g. http://lists.fourthought.com/pipermail/exslt/2005-January/002169.html).

We also need to think about the confusion between the namespaces URLs used in EXSLT. The extension URIs are different from the specification URLs. I understand how this distinction came about, but I think it has proven too confusing in practice. I suggest that we should put a RDDL gloss at all the namespace URL end locations, including a summary of the extension module and a pointer to the specifications.

Packaging EXSLT

We need to provide a package we can call "EXSLT 1.0". Something clear, recognizable and ready for download. It should include at least documentation on modules that proved useful over time. We should update the reference stylesheet implementations and examples (see e.g. http://lists.fourthought.com/pipermail/exslt/2005-February/002174.html).

There have also been suggestions and proposals for new modules on the list, including:

• URI parsing and handling ( http://lists.fourthought.com/pipermail/exslt/2005-May/002210.html )
• trig funcs ( http://lists.fourthought.com/pipermail/exslt-manage/2005-January/001224.html )
• environment variables ( http://lists.fourthought.com/pipermail/exslt/2004-December/002158.html )
• error handling ( http://lists.fourthought.com/pipermail/4suite-dev/2002-July/002192.html , thread at http://lists.xml.org/archives/xml-dev/200412/msg00797.html )

My feeling is that we should go to 1.0 with the stuff that's already established on the site. We can add modules for a 1.1. release.

And then we would set up for EXSLT 2.0 extensions, which I understand people are itching to seed (I'll probably watch from the side lines as I work through my overall opinions on XSLT 2.0).

Jim has promised to jump-start all these tasks. I'll keep folks posted.

I welcome comments on this topic here on Copia, but if you really want to help, or want to engage in discussion with the overall community, please do join the mailing list and chip in.

[Uche Ogbuji]

One of the great practical uses of the literary disciplines, of course, is to resist glibness—to slow language down and make it thoughtful. This accounts, particularly, for the influence of verse, in its formal aspect, within the dynamics of the growth of language: verse checks the merely impulsive flow of speech, subjects it to another pulse, to measure, to extralinguistic consideration; by inducing the hesitations of difficulty, it admits into language the influence of the Muse and of musing.

—Wendell Berry—Standing By Words

via David Graham, on the New Poetry mailing list

No sooner have I returned from Amsterdam (more on all that later) than juicy morsel falls straight in my lap for the neglected Quotīdiē. Wendell Berry is not the most interesting poet to me. I find him much like our current Poet Laureate Ted Kooser—Intelligently stated, but with nothing particularly compelling to offer for theme or diction. Not all bucolics have to be as majestic, as, say Vergil's, but I think more of our poets should look to (to give a parallel example) Horace for an example of how to personalize bucolics while still keeping them interesting.

But the quote is not from Berry's poetry, but from his prose, and it compels me to seek out more of Berry's philosophical essays. Many commentators have noted the role of poetry in presenting ideas in a form that requires such care to digest that they become more clearly communicated to the reader. This is so even if, paradoxically, obscurity is one of these tools of clear communication. Obscurity slows things down in the reader's apprehension in order not to lose the nuances. A perfect antonym of poetry from this viewpoint is the sound bite, and I think this comparison is also a good argument as to why poetry is as important today as it has ever been.

Poetry for new media culture

The problem has always been how to make the reader accept the braking effect of poetry on the digestion of information. I don't think it's engaging in too much Luddite hand wringing to say that these days people prefer their information in easily (and indeed trivially) digestible form. This is in part a natural reaction to high volume ("information overload" in the jargon). Most people, even among the trendiest of techies, are quick to praise the resource that presents a topic in both depth and breadth, and in coherent form. They find such treatment a necessary check on the dissociating effects of the contemporary knowledge feed—rapidly evolving blips of high sugar information. They accept a slow-down of perception and carefully read such resources, but only when advised by their peers to do so. They slow down because the "buzz factor" has compelled them to do so.

Poetry serves the same end, and buzz can certainly be important for leading people to poetry, but what really makes it compelling enough for the reader to accept the slow-down in apprehension is concentrated beauty of language. If the musical force of the words is strong enough, the intelligent reader will be obliged to dig more deeply. The reader will have gained a superficial aesthetic reward from the piece, in the sound, and such a reward as they never receive from their more quotidian resources. This reward is very satisfying, even if superficial, and it promises of richer reward, in the matter, once one has taken the time to consider the piece more carefully, most likely through multiple readings, and discussion with peers. And with the best poetry, we learn that the reward in the sound is not really superficial at all, but is the key to better memory of the idea as well as greater enjoyment in its presentation.

This is all well and good, of course, but the question is exactly where will the mastery come from to work new media concerns into compelling poetry? Is any such venture doomed by popular stereotype of poetry, especially its association with the mid-20th century cadre of slovenly, mentally unstable, kvetching pop art beatniks? From what I've heard and read of Paglia's Break, Blow, Burn, she opens up a useful discussion along these lines (yet another book on my really-should-read-soon list). I must also say that the same discussion leads me to question whether she has the critical acumen to help direct the class of potential poets who can serve the world in this time of great need.

[Uche Ogbuji]

Pychinko is a python implementation of the classic Rete algorithm which provides the inferencing capabilities needed by an Expert System. Part of Pychinko works ontop of cwm / afon out of the box. However, it's Interpreter only relies on rdflib to formally represent the terms of an RDF statement.

FuXi only relies on Pychinko itself, the N3 deserializer for persisting N3 rules, and rdflib's Literal and UriRef to formally represent the corresponding terms of a Pychinko Fact. FuXi consists of 3 components (in addition to a 4RDF model for Versa queries):

I. FtRdfReteReasoner

Uses Pychinko and N3RuleExtractor to reason over a scoped 4RDF model.

II. N3RuleExtractor

Extracts Pychinko rules from a scoped model with statements deserialized from an N3 rule document

III. 4RDF N3 Deserializer

see: N3 Deserializer

The rule extractor reverses the reification of statements contained in formulae/contexts as performed by the N3 processor. It uses three Versa queries for this

Using the namespace mappings:

• n3r --> http://www.w3.org/2000/10/swap/reify#
• log --> http://www.w3.org/2000/10/swap/log#

Extract ancendent statements of logical implications

distribute(
  all() |- log:implies -> *,
  '.',
  '. - n3r:statement -> *'
)

Extract implied / consequent statements of logical implications

distribute(
  all() - log:implies -> *,
  '.',
  '. - n3r:statement -> *'
)

Extract the terms of an N3 reified statement

distribute(
  <statement>,
  '. - n3r:subject -> *',
  '. - n3r:predicate -> *',
  '. - n3r:object -> *'
)

The FuXi class provides methods for performing scoped Versa queries on a model extended via inference or on just the inferred statements:

For example, take the following fact document deserialized into a model:

@prefix : <http://foo/bar#> .
:chimezie :is :snoring .

Now consider the following rule:

@prefix ex: <http://foo/bar#> .
{?x ex:is ex:snoring} => {?x a ex:SleepingPerson} .

Below is a snapshot of Fuxi perforing the Versa query “type(ex:SleepingPerson)” on a model extended by inference using the above rule:

Who was FuXi? Author of the predecessor to the King Wen Sequence

Chimezie Ogbuji

Well, I'm heading off to catch the flight to Amsterdam for XTech 2005. I'll blog as much as I can, and I have some FOSS work to do as well, on Amara, especially, to prep the 1.0b2 release.

We've had the spam comment folks doing their thing here, and so far I've been able to keep them mostly in check by deleting them soon after they appear. The trip will probably leave too big a hole for them, so for now I've turned on draft mode for comments. All comments will be held until explicitly approved. I apologize for any inconvenience. I've been tinkering on a more solid spam fighting system, building on the great work others have done on black-listing the punks.

[Uche Ogbuji]

I'm leaving for XTech tomorrow evening, and I'm already dreading a week without any intense exercise (I walk a lot on such trips, but that does nothing for my heart rate, especially not at sea level). Maybe it's because my outdoor amateur soccer team just clinched top spot in our division today (woohoo! but I missed a good chance at a bonus goal), but I'm hoping not to have to go all week without a good game of some sort.

Any other XTech attendees feel the same way? Does anyone want to try to find some recreation while there? I'm up for soccer/football (full, 5- a-side, or whatever), basketball, volleyball or tennis. Leave a comment if so. I wanted to put something up on the XTech Wiki, but I'm not sure where to do so, and you can't edit the categories (I don't think).

Not up for playing, how about watching? I'd like to catch the UEFA Champion's League final live somewhere, presumably at a good sports pub. It's on the first day of the conference (Weds), so it may be tough to peel away, but I plan to try. Again pip up if you're up for it.

[Uche Ogbuji]

"Unicode Secrets"

In his latest Python-XML column, Uche Ogbuji delves broadly and deeply into the world of Unicode, especially with regard to processing XML in Python.

In this one I started out talking about a quick spot check for Unicode compliance in XML tools, then went on to present some tips on Python's Unicode API. The intent was not to be comprehensive. I cherry-picked the particular Unicode facilities I tend to use the most. As one person mentioned in the comments, there are even more means at your disposal than I cover. I'll get to some of them in part 2, in the next column installment.

[Uche Ogbuji]

Motivated by the idea that 4RDF (and 4Suite) could benefit greatly from being able to parse Notation 3 documents (see bottom), I attempted to write an N3 Deserializer for 4RDF that makes use of Sean B. Palmer's n3processor.

Ft.Rdf.Serializers.N3

The module simply needs to be added to 4RDF's Ft/Rdf/Serializers directory. I hesitate to check it in, since 4Suite is now in a feature-frozen beta release cycle. It implements a sink which captures generated triples and adds it to a 4RDF model:

class FtRDFSink:
  """
  A n3proc sink that captures statements produced from
   processing an N3 document
  """
  def __init__(self, scope,model):
     self.stmtTuples = []
     self.scope = scope
     self.model = model
     self.bnodes = {}
     self.resources = {}

  def start(self, root):
     self.root = root

  def statement(self, s, p, o, f):
     #First check if the subject is a bnode (via n3proc convention)
     #If so, use 4RDF's bnode convention instead
     #Use self.bnodes as a map from n3proc bnode uris -> 4RDF bnode urns
     if s[:2] == '_:':
        if s in self.bnodes:
           s = self.bnodes[s]
        else:
           newBNode = self.model.generateBnode()
           self.bnodes[s] = newBNode
           s = newBNode

     #Make the same check for the statement's object
     if o[:2] == '_:':
        if o in self.bnodes:
           o = self.bnodes[o]
        else:
           newBNode = self.model.generateBnode()
           self.bnodes[o] = newBNode
           o = newBNode

     #Mark the statement's subject as a resource (used later for objectType)
     self.resources[s] = None

     if f == self.root:
        #Regular, in scope statement
        stmt=(s,p,o,self.scope)
        self.stmtTuples.append(stmt)
     else:
        #Special case
        #This is where the features of N3 beyond standard RDF can be harvested
        #In particular, a statement with a different scope / context than
        #that of the containing N3 document is a 'hypothetical statement'
        #Such statement(s) are mostly used to specify impliciation via log:implies
        #Such implications rules can be persisted (by flattening the forumae)
        #and later interpreted by a backward-chaining inference process
        #triggered from Versa or from within the 4RDF Model retrieval interfaces
        #Forumulae are assigned a bnode uri by n3proc which needs to be mapped
        #to a 4RDF bnode urn
        if f in self.bnodes:
           f = self.bnodes[f]
        else:
           newBNode = self.model.generateBnode()
           self.bnodes[f] = newBNode
           f = newBNode

        self.resources[f] = None

        self.flatten(s, p, o, f)

  def flatten(self, s, p, o, f):
     """
     Adds a 'Reified' hypothetical statement (associated with the formula f)
     """
     fs = self.model.generateUri()
     self.stmtTuples.append((f,
                             N3R.statement,
                             fs,
                             self.scope))
     self.stmtTuples.append((fs,
                             N3R.subject,
                             s,
                             self.scope))
     self.stmtTuples.append((fs,
                             N3R.predicate,
                             p,
                             self.scope))
     self.stmtTuples.append((fs,
                             N3R.object,
                             o,
                             self.scope))

In addition, I made a patch to the 4RDF command that adds 'n3' as a input format. See my previous blog for an example of using this command to generate diagrams of 4RDF graphs.

For example, this diagram is of rdfs-rules - rendered via the 4rdf command line (patched in able to deserialize n3 documents)

Advantages

First, deserializing N3 will almost always be faster than deserializing from rdf/xml (especially for larger graphs) since it's a text parse vs an XML parse. So, if 4Suite repository XSLT Document Definitions are augmented to be able to deserialize into the model via n3, repository operations on documents with such Document Definition will be significanly faster.

Finally, by allowing the deserialization of SWAP constructs such as log:implies, formulae reification, existential and universal variables, reasoners capable of interpreting N3 rule semantics (such as Sean's pyrple Graph class - see a demonstration of it's inference capabilities) can perform inference externally (without having to build it into 4RDF or Versa) on a 4RDF store containing RDF deserialized from N3 documents with appropriate rules.

One thing to note about this implementation is that the default baseUri of N3 documents is http://nowhere when the specified scope is a urn (since the n3processor is unable to handle urn's). Otherwise, the given scope is used as the baseUri

[Chimezie Ogbuji]

via Copia

I've been reminded today that some folks have taken a very strident tone towards advocacy of XSLT 2.0. Just so there's no mis-connotation of "strident", I'll note that I'm a very enthusiastic booster of Python, and I exhort people to give it a try whenever they I can, but I don't think I go about with the notion that "YOU HAVE TO BE NUTS NOT TO USE PYTHON, DAMMIT". I'd rather show code examples and let them come to such a conclusion. I used to associate strident advocacy with XQuery boosters, reinforced earlier this year when one of them wandered into an XML-DEV thread recently with such a supercilious attitude (in essence: "why would a sane programmer use anything except for XQuery?"). I'm starting to wonder whether the same mechanics are developing with XSLT 2.0. I wonder whether it's not a reaction to the fact that XSLT 2.0 has met with some of the same hostility as XQuery (though as I admit, some people have been changing their minds), combined with the fact that XSLT 2.0 level of implementation seems to be slower in burgeoning than XQuery's. Of course, a counter-argument is that there is plenty of the right kind of advocacy as well, especially in the form of Bob Ducharme's XML.com columns.

I think people will have to understand that XML tools can no longer sue for universality in XML processing just because they come out of the XML oven, especially since the XML oven has been over-cooking its buns for a while now. W3C XML Schema, XQuery, and the SOAP Web services stack are just the egregious examples; all the committees seem to have turned into over-engineering shops. I think I've long ago taken the attitude that the 1.0 series of specs got us as far as we needed to go to tackle XML processing. We can easily get the rest done in "native" environments. This at least seems to be true of the Python and .NET camps. Sure XPath and XSLT 1.0 are limited, but mix them into an expressive enough language or rich enough platform, and I just have trouble seeing the need for the leaps in conceptual load that come with, say XPath 2.0

One thing's, for sure, we're all lucky we're so spoiled for choice.

[Uche Ogbuji]