I’ve been thinking lately about what a sleek UI for creating XProc would look like. There’s plenty of big-picture inspiration to go around, from Yahoo Pipes to Mac OSX Automator, but neither of these are as XML-focused as something working with XProc would be.
XML, or to be really specific, XML Namespaces, comes with its own set of challenges. Making an interface that’s usable is no small task, particularly when your target audience includes the 99.9% of people that don’t completely understand namespaces. Take for example a simple step, like p:delete.
In brief, that step takes an XSLTMatchPattern, following the same rules as @match in XSLT, which ends up selecting various nodes from the document, then returns a document without any of those nodes. An XSLTMatchPattern has a few limitations, but it is a very general-purpose selection mechanism. In particular, it could reference an arbitrary number of XML Namespace prefix mappings. Behind a short string like a:b lies a much longer namespace URI mapping to each prefix.
What would an intuitive user interface look like to allow entry of these kinds of expressions? How can a user keep track of unbound prefixes and attach them properly? A data-driven approach could help, say offering a menu of existing element, attribute, or namespace names taken from a pool of existing content. But by itself this falls short in 1) richer selectors, like xhtml:p[@class = “invalid”] and 2) doesn’t help in the general case, when the nodes you’re manipulating might have come from the pipeline, not your original content. (Imagine one step in the pipeline translates your XML to XHTML followed by a delete step that cleans out some unwanted nodes).
So yeah, this seems like a Really Hard Problem, but one that’s worth taking a crack at. If this sounds like the kind of thing you’d enjoy working on, my team is hiring–drop me a note.
-m
One (imperfect) approach to this could be to make effective use of colour-coding. Rather than require a user to bind a namespace URI to a prefix, you instead ask them which colour they want to use for each “family” of elements that you encounter. This setting would persist across sessions and contexts so that, for user x, XHTML elements are always green (say).
Both the prefix and the namespace URI can then be abstracted away completely, they need never be shown to the user (except in “advanced mode”, perhaps). The app simply continues to request colour bindings each time it encounters a new xmlns declaration and use whatever prefix it likes under the hood. Features like auto-complete/intellisense would group by colour of course and allow user-instigated filtration by colour too.
Where this approach falls down completely is for blind and colour-blind users, obviously. There will also be some threshold number of namespaces, beyond which it becomes impractical for the human eye to differentiate colours accurately, without making mistakes when similar shades are in use. Perhaps that could be alleviated somewhat by varying both background and foreground colours if necessary.
Micah,
Creating an intuitive UI for composing XProc pipelines is indeed a challenging task. Especially finding the right balance between providing all (or most) features of XProc and not drowning the end-user in the XProc/XML/XML Namespaces complexity is hard.
Perhaps you are aware that EMC has been working on an XProc Designer tool (you can check out an early development preview at EMC XML Developer network: https://community.emc.com/docs/DOC-4382). The designer is still under heavy development, but it is already usable for simple use cases: you can compose pipelines using drag & drop, save them and open existing ones, and execute them.
If you try the on-line demo, one thing that you will probably notice (next to that it is still rather rough around the edges) is the lack of a UI for authoring XPath expressions and limited support for XML namespaces in general – precisely the same problems that you are describing. This of course makes creating more complex pipelines a rather painful (or sometimes even impossible) task, and the developers are working on some ideas on how to best approach this problem in the next iteration of the designer. While the initial focus in the designer was to implement the foundations for the core XProc-related functionality, now it is the usability aspect.
Obviously, the approach to the XProc UI depends on what kind of XML expertise do you expect from the end-users. In my opinion, XProc will never be for complete XML dummies: just to understand some of the basic concepts behind XProc, you already have to know quite a lot about XML and related technologies. XProc is, after all, a language for processing XML, and not really a tool for creating nice business workflows. With that in mind, the XProc UI probably does not have to shield the end-user from all of the underlying complexities of XML/XProc – but of course, it should make XProc authoring easy (or at least provide enough convenient features) so that end-user will prefer using a graphical environment as opposed to his favorite Emacs :-)
Regards,
Vojtěch
I imagine an XProc gui using a flowcharting metaphor. The namespaces for XProc or those for building ‘flow’ shouldn’t be visible to the user, just the symbols and the different types of lines connecting them. I have no idea how you can do a gui for creating XPath expressions based on any kind of content without the users knowing XPath (and then it is just easier to write the XPath).
Designing/creating an intuitive user interface for XProc sounds intriguing.
Sadly, (being UK based and starting up my own company – and possibly not qualified either) I’m not in a position to consider this myself. However, I would suggest that, whilst a high-level flow-visualisation tool is good to get things rolling, the user will ultimately want to develop, enhance and test the XPath/Patterns in their raw form.
So, to my mind, the success of an XProc tool will be dependent, not on flash graphics (though these will help), but on good use of XPath/Pattern syntax coloring, intelligent auto-complete lists (that don’t just dump all alternatives on the user in one go), XPath/Pattern auto-generators, and how well different parts of an XPath/Pattern can be tested interactively against multiple inputs – even if that input is derived from the pipeline.
Users don’t seem to have issues with namespace/prefix bindings if they can interactively trace through each part of an expression. Also, using a treeview outline of declared namespace prefixes, synchronised with the XML input (with prefixes that may or may not be the same) can be helpful. Auto-generation and auto-completion also provide powerful hints where namespace bindings are missing.
One caveat, my first XPath tracing tool was nominated on the ‘Worst Ever UI’ list on StackOverflow, the main criticism was that it exposed too much to the ‘average’ user (I agreed with some of the comments, but put in my own counter-nomination anyway).
I now see that Vojtech and EMC already have a flowcharting GUI :) Very nice!
I was thinking about this more and I see two perspectives for the application targeting different user types. One would be for developer-types where they configure the application and the other for non-developer users who use the customized application.
For the developer-type:
They create modular, documented pipelines that will be assembled by the manager-type. In their GUI, they set a context in the DB (because the app is connected to the DB) with their first node in the flowchart. In a separate panel, a list of the current context’s results are displayed. They continue to build the flowchart and as they branch off they create new contexts where results are updated in the separate panel. Another panel is available when appropriate or requested that can show the current result, some represtation or state of the entire pipeline’s transformation. When a module is finished or ready to be tested by a non-developer users, the developer adds it to the application that the manager user will be using. In other words, they build unique components and configure a custom application for the non-developer user.
For the manager-type:
They use a a rich, graphical interface to assemble the modules to produce the results for publishing. This UI allows the user to assemble anything from complex projects to simple tasks. This is also hooked up to the DB so the user can see realtime WYSIWYG or graphical representations of current state based on their focus in some pipeline.