XPath & XQuery in Qt
September 18, 2007
The Qt snapshots now includes support for XPath 2.0 and XQuery 1.0.
Being part of the XML library, the idea is that Qt 4.4 will ship with a C++ API for running and evaluating such queries. On the side too, is a command line tool called patternist, for quickly testing queries, scripting and old-school web solutions. But who cares, blogs with screenshots is the thing:
Stronger XML support in Qt has been consistently asked for by users over a long time, with XPath being one of the main requests. Hopefully Patternist, with the help of KDE folks, users, and customers expressing what’s missing, will please those needs. Considering the similarities of XQuery and XSL-T, Patternist also serves as a foundation for implementing XSL-T, if so decided.
For KDE folks all this might ring a bell. Patternist was indeed first developed for a long time in the KDE repository, as part of KDOM. We just thought it would make a lot more use as part of Qt.
And I think exactly that makes this exciting. W3C’s XQuery working group has registered an astonishing number of exciting implementations. But for users, reliability is what matter in the end. Whether bugs will be fixed, whether people can answer questions, whether the piece is maintained and documented. Persistency. Trolltech swiftly carries this on its shoulders(assuming I brush my teeth and all that).
Combined with that Qt is open source and the Patternist SDK used for development is as well, this is like eating some nasty chocolate while at the same time singing a little duet with Miss Piggy. I can’t sing, nor can Piggy (although she tries), but you get my point.
Humble modesty aside, it is worth to mention that this still needs work. About 94% of the test suite is passed, the API needs more work, and there is performance issues.
Nailing test cases and trimming code paths are problems that have known solutions (though typically horrible to carry out). Harder is to know what people need and how they need it. It’s hard to guess what kind of APIs or extensions Amarok or KOffice or a GNOME or web application need.
If you got input, feel free to add a comment to the blog, send a report to Trolltech, grab me(FransE) on the Open Projects IRC network, or ask a question or two on the qt-interest mailing list.
The documentation starts over here.
September 18, 2007 at 10:21 am
Heh. Just a moment ago, I was looking at your blog and thinking “what happened to Frans Englich and Patternist, not a blog post since january”.
So I asked in #kde4-devel on FreeNode and they said you were at TT now, and voilà, 4 minutes later there’s a new post on your blog
Great news!
September 18, 2007 at 2:40 pm
Any plans to put XML schema support in the Qt XML framework?
September 18, 2007 at 9:43 pm
Why do you not join forces with the xquilla project ? I think they have already xupdate support.
But anyhow: Great news !
September 19, 2007 at 10:00 am
we had a short discussion a while ago on open-source XML editors (or more precisely, the lack thereof). The fact that Qt 4.4 will include patternist and provide a Xquery/Xpath API is really good news. After using MS Infopath for a while and learning more about XML technologies in general, I am quite depressed when I see how much OSS lags behind in that area
To try and answer your request for comments/suggestions: I have not really been programming using XML, but it seems to me that schema (XML schema as a minimum, possibly also RelaxNG) support would also be a very good thing.
I have not tried patternist so it is possible that it already does what xmlstarlet does… I’ll give it a try one of these days !
I have also been using xmlstarlet quite a lot to manipulate XML files from the command line and I’d suggest you take a look at it if you don’t know it already to get a feel of the kind of API that would be required… Xmlstarlet allows to query (Xpath only, not xquery as far as I can tell), transform, validate, edit (insert/update/suppress nodes and data) XML documents. It is already a pretty good start I guess
Thanks for your excellent work on XML in Qt/KDE and OSS in general,
September 19, 2007 at 11:30 am
Trevor:
W3C XML Schema(WXS) is asked for a lot too. Many, including me, wouldn’t mind having it in Qt. The big problem is that implementing it (properly) requires tons of work. Easily 10 K lines, and the test suite has about 40 K tests. While implementing it one typically becomes a sad, twisted old man unless one finds something divine. Anyone who’ve worked with the WXS, knows this is true.
So, what’s holding back WXS is that there currently are so many other interesting projects, like XSL-T 2.0 which should be relatively easy reachable, and more stuff on the XQuery front.
Though, the day I get to it, my idea is to implement it in such a way that the implementation has one validation automata, that both RelaxNG and WXS can be translated to. If it’s doable. Another idea is to implement WXS by rewriting schemata into XQuery queries. Likely not doable, but I haven’t concluded that yet.
***
rusco:
XQilla is a nice project. If anyone would start an XQuery implementation today, one would consider joining one of the existing ones, naturally.
So, one reason is historical. Others are technical: in what way Qt can make use of XQuery(more on that in the future), and various system requirements such as code foot print and dependencies. While it’s not there yet, one might be able to squeeze Patternist onto mobile phones, for instance.
I think there’s room for XQilla and Patternist, and the usual sharing of experiences with folks like John Snelson through blogs & W3C is very useful. And I don’t mind a little healthy competition
***
Nicolas:
No doubt that XML editors are shallow in the OSS community. I think the main reason to that have been the lack of stacks for core technologies like XQuery(although another reason is that XML tends to currently be a bit more heavy in the non-oss industry). Maybe it is because these stacks are rather heavy to pull up.
But this can change. With Qt being as central on Linux as it is, this is a great opportunity to literally distribute these stacks. In that way others can concentrate on building more user-centric things, with the help of a building block or two.
(XInclude and OASIS XML Catalogs are two other things that might see the light.)
September 19, 2007 at 12:24 pm
Nicolas:
I suppose the huge memory requirement issue has been controlled ? I
don’t know how libxml handles it, but I’ve found no performance issues
with xmlstarlet or xsltproc so far, albeit with fairly small XML
files…
I vaguely recall what this was about, but tons of things have changed since the code was in KDE. Among other things a new representation is used, which is about half the size of QDom. Though, I’m for good reasons not satisfied with it yet, so keep the feedback coming.
September 19, 2007 at 5:36 pm
[...] XPath & XQuery in Qt The Qt snapshots now includes support for XPath 2.0 and XQuery 1.0. Being part of the XML library, the idea is that Qt […] [...]
December 30, 2007 at 3:58 pm
Great! [Strange Q follows] Any plans to implement the Schema-based XML validation so far?
December 30, 2007 at 3:59 pm
to clarify: [Seen the upward comment - just wanted to let know how developers need it as a part of the portable and usable QT products family]
December 30, 2007 at 11:48 pm
Hi Sasha,
As of this writing there’s no roadmap that contains support for W3C XML Schema.
However, very many people ask for it so I will surely bring it up for strong consideration for Qt 4.5.
Until then; schema validation gives deterministic data handling(since the input data has been validated), and XQuery does bring some of the same advantages, although not in the absolute sense that Schema do.
XQuery has all the builtin primitives that W3C XML Schema has, and treat those in a type aware manner. If one writes ones XML code in XQuery, one achieves a fairly high degree of data integrity and code stability, which is the same advantages that Schema gives.