Serving Many Masters

After spending a great deal of time involved in discussions about various XML data-models (Infoset, DOM, XPath, XQuery, etc) and a morass of issues related to trying to add edit support to our XPathNavigator API in System.Xml, I found my self pondering XML APIs. Despite the fast that Elliotte Rusty Harold and I have disagreed in the past, a number of the point he brings up in his What’s Wrong with XML APIs (and how to fix them) presentation are very good. I found that rather that doing anything useful, I kept pondering what shape my API would take. One of the first things I worked on here at Microsoft was our XML DOM, and I've spent years helping internal and external customers understand how to get the best out of it. When I started here, I came from a SGML background, and spent a lot of time explaining why mixed content was so important. A lot of people who just pick up XML as a tool, rather than a core part of teir work, are using XML files as simplified databases. To them, mixed content is a very odd notion. Worse, they are often surprised at how whitespace, mixed content, and a few other tricky aspects of XML, reveal themselves in the DOM. So I started pondering how I would design an API that was better balanced for these users.

The problem is that you don't really want two totally different APIs, for related user scenarios. The problem is that a lot of customers don't really know what they want when they first start out using a new technology. They haven’t had enough experience with the technology and their use-cases to know all the ins and outs. XML has demonstrated itself to be unfortunately bad in this. Customers think they only need to worry about X (some simplified subset of XML) until 2 weeks before ship, when someone points out that they have been completely ignoring xml-namespaces / whitespace / mixed-content / processing-instructions / etc... I have seen this time and time again.

So you really want an API that lets people who don't really care about 'markup', just deal with the higher level structure, yet provides convenient access to the 'markup', for those who do need it. Worse, that 'higher level structure' I mentioned is something different to different customers. Remember those multitude of different data-models I mentioned earlier? So how do you service both customers?

Honestly, I don't have an answer. We have the DOM today, and a lot of this can be avoided by using XPath to navigate the DOM (selectSingleNode and selectNodes in both Microsoft's MSXML and System.Xml implementations). XOM definitely looks better than the DOM, but is it enough better to justify replacing the DOM with something like it? I don’t think so. The disadvantage to being Microsoft, is that once you decide to do something like that, it is institutionalized. Unless we can define an API which is so much better that it worth your while to abandon the DOM and port existing code (just for the benefit of those new features you are writing...) to the new API, then it just doesn't seem worth adding a new API, just because it is slightly better.

The problem is that XML, as a standard, and thus any XML API, serves many, many masters. Different customers have different needs, and the XML standard is open enough that they all feel included. To define an API which excludes too many users, implies that it will be destined to be just a niche API. Trying to define an API which allows full markup fidelity (a'la DOM/XmlReader/etc) seems destined to be just as full of traps for the XML neophyte as XML itself is. Trying to define a simplified API that targets the 80% usage (SOAP oriented scenarios for example) conversely rules out some of that very features that the XML 'experts' depend on.

I fear, this is a question to which there is no easy answer.