Rete-inspired N3 Rule Network Finished

See: previous

I called it quits (for now) in trying to retrofit the RETE-based algorithm to handle N3 functions & filters. I've checked in a rewritten Rete module with tests for SHIOF Description Logic axioms.

Tracking dependencies between filter / function variables, became hairy.

An SVG diagram of these rules compiled into a RETE-like network was generated using an included Boost Graph Library utility function.

It's unit test output:

testRules (__main__.TestEvaluateNetwork) ... Time to build production rule (RDFLib): 0.0118989944458 seconds
Time to calculate closure on working memory: 0.478726148605 seconds
ok
----------------------------------------------------------------------
Ran 1 test in 0.751s
OK

I also integrated Python iterator algebra implementation of a relational hash join (for the Beta Nodes).

It workes with RDFLib, so I'd eventually like to integrate the ability to dispatch SPARQL queries over an N3 Closure Graph - generated from the network. The speed in which it is able to render rules graphs rendered the option of serializing a compiled network into persistence a non-issue.

From the README.txt:

Fu Xi (pronounced foo-see) is a forward-chaining inferencing expert system for RDFLib Notation 3 graphs. It is named after the first mythical soveriegn of ancient china who supposedly, 'looked upward and contemplated the images in the heavens, and looked downward and contemplated the occurrences on earth.'.

Originally, it was an idea to express the formalisms of the Yi Jing / I Ching in Description & First Order Logic.

It relies on Charles Forgy's Rete algorithm for the many pattern/many object match problem - which builds a triple pattern reasoning network. It can be used as a reasoner with capabilities for certain expressive Description Logics (via OWL/RDFS axioms in N3) or a general N3 production / expert system. It uses Python hash / set / list idioms to maximize matching efficiency of the network. In particular, it uses an Iterator algebra implementation for the join activation mechanism

An example of its use:

from FuXi.Rete.Util import generateTokenSet
        from FuXi.Rete import *
        from rdflib.Graph import Graph,ReadOnlyGraphAggregate
        from rdflib import plugin
        from FuXi.Rete.RuleStore import N3RuleStore
        store = plugin.get('IOMemory',Store)()
        store.open('')
        ruleGraph = Graph(N3RuleStore())
        ruleGraph.parse(open('some-rule-file.n3'),format='n3')             
        factGraph = Graph(store)
        factGraph.parse(open('fact-file'),format='..')
        closureDeltaGraph = Graph(store)            
        network = ReteNetwork(ruleStore,
                              initialWorkingMemory=generateTokenSet(factGraph),
                              inferredTarget = closureDeltaGraph)            
        for s,p,o in network.closureGraph(factGraph):
            .. do something with triple ..

To check out from cvs:

cvs -d:pserver:anonymous@cvs.4suite.org:/var/local/cvsroot login
cvs -d:pserver:anonymous@cvs.4suite.org:/var/local/cvsroot get Fuxi

Chimezie Ogbuji

via Copia

Business-grade broadband in Superior, Colorado

Looking into upgrading my home broadband to business grade, I've casted about for options. Comcast is my current provider for digital cable plus cable broadband, and I've been happy with them, so I called to ask what they could do for me.

Their quote for (Comcast Workplace Standard) was $110 per month for 6.6Mb down and 768Kb down and 5 static IPs, with a promo waiving installation fee and taking $20 per month off for the first 12 months (requires a 24 month commitment).

Sounded good, and seems it must be. I went hunting around sites such as DSLBroker to find business broadband options for Superior, Colorado, where I live. I found nothing but plans on the order of $200/mo for 1.1Mb symmetric; overall I didn't find anything that even came close to the value Comcast was offering. Does anyone know whether I'm just missing something? Do you have experience with business-grade broadband at home?

[Uche Ogbuji]

via Copia

The madness of Samba setup on Ubuntu

Yeah, Ubuntu has thrown a few medium-sized annoyances at me recently. Tonight I wanted to set up Samba for sharing files with our local Windows machines. I've set up Samba dozens of times, but this is one of the more confusing episodes I've come across recently.

First of all I looked and saw that I had a package named "samba-common", and yet this did not mean I actually had Samba. A little confusing, but I just had to fire up Synaptic and search for "samba". So I installed the package and figured I'd have a serviceable default config. No dice. Rather than hack at /etc/smb.conf I decided to use swat, the Web control panel for Samba, which I've used, and I like. I installed it and tried to surf http://localhost:901/, the usual way to access swat. No response. I read the swat man page which said it runs through inetd. The man page specifically mentions editing /etc/inetd.conf I looked there and found but two lines, one of which was:

#<off>#swat            stream  tcp     nowait.400      root    /usr/sbin/tcpd  /usr/sbin/swat

How quaint! Old school inetd. How odd! Only 2 managed services. Oh well. I uncommented the line and then tried to restart it. Umm. no trace of inetd beside that config file.

At this point I did some googling. I just wanted a simple Ubuntu-specific HOWTO. I wasn't really in much of a mood for pulling out all the hacking stops. Google had almost nothing to offer except others bitching about how hard Samba is to set up on Ubuntu. I went back to work. I checked Synaptic and it turned out I had no inetd package installed. I selected xinetd, which is what I'd been expecting in a modern distro, anyway.

Now I had to configure swat to run behind it, but I was on my own. This is why a package such as swat should depend on a package such as xinetd, so that it can provide a plug-and play config. I'm pretty sure it does on Fedora Core. And certainly there shouldn't be phantom inetd.conf files floating around to trip up users.

I created /etc/xinetd.d/swat, with these contents:

# description: SAMBA SWAT
service swat
{
    disable         = no
    socket_type     = stream
    protocol        = tcp
    user            = root  #should use a more limited user here
    wait            = no
    server          = /usr/sbin/swat
}

And then dpkg-reconfigure xinetd to restart xinetd (and do any other magic needed after updating config). Finally I got swat up. If you want to use swat to administer Samba, you'll need a proper root password. No sudo to bail you out. If you log in as your user, you get a crippled, pee-on swat console for mere mortal users. On one machine I got my Ubuntu root by booting to single user mode and then setting the password. On this machine I used Automatix. Either way, use that root login to get the full plate of admin options. From that point it's fairly smooth sailing. swat isn't perfect, but it's as close as you get to simple Samba administration.

I hope this helps someone else. Maybe it's my imagination, but this seems to be much harder than I remember it ever being on Fedora Core or Mandrake. Oddly enough, I'm getting the impression that Ubuntu rocks for user desktop stuff, but gets really clumsy when it comes to the server management goods that should be bread and butter for Linux.

[Uche Ogbuji]

via Copia

Woohoo! 2 phase commit for PostgreSQL

Recently discussion of distributed transactions came up in the context of the 4Suite repository redesign discussions. We will probably need to move to XA-style two-phase commits (2PC). The problem is that Oracle supports 2PC, but not so many other back ends with 4Suite drivers. Luckily I've found that my favorite DBMS will soon support 2PC. The patch is in for PostgreSQL and became available in the 8.1beta4 release. It should be in production for PG 8.1 (currently in release candidate). Thanks to Tom, Heikki and Alvaro for this wonderful addition. For some interesting discussion of the 2PC patch, see this important thread. One of the well-known issues brought up is that 2PC in almost all implementations is much slower than straight commit. I think we'll have to make our transaction manager smart enough to use direct commit if it detects that all resources are in the same database instance. I've written transaction managers before (specifically in implementing CORBA CosTransactions in Python on top of OmniORB) and I remember such tricks all too well.

Even once PG 8.1 is available we'll still have to deal with non XA-aware back ends. All we can do is fall back to the Last Resource Commit technique in the transaction manager. I found a useful discussion of this technique in "What to do if your database doesn't support XA transactions?", by Dmitri Maximovich. You actually risk losing both atomicity and isolation with this technique, not just atomicity, as Dmitri says, but if you choose minimally competent back ends, such failures should be very unlikely.

[Uche Ogbuji]

via Copia