Every XML node has a set of properties defined by the XDM data model. An element node, for example, has twelve properties, which include the name, relationships to other nodes ([parent], [children], [attributes]), a type, a typed value and a string value. Evaluation of these properties is what drives XPath navigation, guiding it from a set of starting points to a set of destination nodes.

In certain situations, however, the locating of nodes might be greatly simplified if we manage to associate nodes with additional properties which help us to identify the nodes of interest. Such properties could represent metadata, or they could reflect the contained data, for example echoing a value found at a particular data path within the node, or providing a value derived from contained data, like a minimum or maximum. As these additional properties are not XDM properties, their values are not part of the node data. Hence we call them external node properties. Their nature implies an interesting possibility: they can provide information about a node which is accessible without constructing the node. Access to such information might enhance the navigational model of XPath in general, and add support for data-driven navigation in particular. The following sections describe an approach how to achieve this.

An external property has a name and a value. A set of external node properties can be modelled as a map with which the node is associated. The map entries have a key which represents the property name, and a value which represents the property value. Such a map we call a property face of the node, or p-face for short. Striving to keep things simple, we constrain external node properties to have an NCName and a value which is a sequence of atomic items of type xs:untypedAtomic. The value space of p-faces is accordingly constrained.

A p-face is not part of the node data – it is merely associated with them. This relationship can be compared to the general relationship between the keys and values of a map: the keys are not part of the values, but only associated with them in the context of the map. Such an association is not global, but scoped to the map.

The association between a node and its p-face is provided by a node collection which contains the node. We introduce the concept of a p-faced collection (short for property-faced collection), which is a conventional node collection plus a mapping associating each node with a p-face. We do not constrain the individual p-faces to use the same property names: the number and names of external properties can vary from node to node.

The following table shows a simple example of a p-faced collection. In this case, all collection members are associated with the same set of external properties. The collection may therefore be represented by a table in which one column identifies the node and the remaining columns represent external properties.

A p-faced collection can be filtered by supplying a set of conditions to be tested against the p-faces of its nodes. Such a set of conditions we call a p-filter. The result of applying a p-filter to a p-faced collection is a sub collection of the original collection, which we call a p-filtered collection.

A subsequent section will define the syntax and semantics of property filters in detail. For the moment, assume that such a filter may be represented by a string, and further assume that a p-faced collection is identified by a URI. Concatenating the URI and the query string, we can then identify sub collections by a single string, for example:

   http://example.com/pcollections/countries?neighbourCountries=be

A new XPath function may provide the functionality of resolving such strings to the node sequences they identify. If the model were adopted by the XPath standard, such a function might be fn:nodes, and one of its signatures might be:

   fn:nodes($uriQuery as xs:string) as node()*

The following snippet shows a simple XQuery program using this function:

   declare variable $country external;
   declare variable $countries := "http://example.com/pcollections/countries";

   <neighbours cy="{$country}">{
      fn:nodes(concat($countries, '?', $country))
   }</neighbours>

A crucial aspect of p-faced collections is the possibility to separate the evaluation of the p-filter from the construction of the selected nodes. Filter evaluation does not require looking “into” the candidate nodes, in fact it does not require the construction of any nodes. Filter evaluation is an operation on a collection of logical maps, the p-faces. The p-filter engine may treat the nodes as opaque strings, to be delivered as such. Resolving these strings to the nodes they represent is the task of an independent component, the node builder. The operations of p-filter evaluation and node construction are distinct and independent. It follows that the p-filter engine need not be based on XML technology. It is possible, for example, to store the p-faced collection in a relational database table and use as p-filter engine the relational database system. In this case, the p-filter supplied to the fn:nodes function would be translated into a SQL query. It is also possible to store the p-faced collection in a NOSQL object collection. The p-filter engine would then be the NOSQL data base system, and the p-filter would be translated into a NOSQL query. Any p-filter engine can be combined with the same node builder, as the mapping of a string to a node is in no way influenced by the way how the string was obtained.

In summary, p-faced collections promise two distinct benefits. First, they augment the navigational model of XPath with data-based access to information, overcoming the current limitation that document access requires knowledge of the document URI. Second, they offer the possibility to integrate non-XML technologies into the implementation of XML data storage and XPath navigation, which may enhance the scalability of XML based systems significantly. The concept of p-faced collections must now provide

A p-filter defines a condition which can be applied to a p-face and yields a Boolean result. As every node in a p-faced collection is associated with a p-face, a p-filter can be used to specify sub collections. This section proposes a generic model of a p-filter, which includes the data model, syntax and semantics.

   <time>2013-01-01T00:00:00</time>
   <time op="&lt;">2013-01-01T00:00:00</time>
   <time op="&lt;" qua="every">2013-01-01T00:00:00</time>
   <keywords op="~">^semantic.*</keywords>
   <keywords op="~">^semantic.*#i</keywords>
   <keywords op="~">semantic%</keywords>
   <codes>
      <p:item>DUS</p:item>
      <p:item>CGN</p:item>
   </codes

   <p:and>
      <time op="&gt;">2012-12-31T22:30:00</time>
      <time op="&lt;">2013-01-01T00:00:00</time>
      <p:or>
         <travelType op="~">^accommodation.*</travelType>
         <supplier>TUI</supplier>
      </p:or>  
      <p:not>
         <eventDate op="&lt;" qua="every">2013-08-01</eventDate>
      </p:not>
   </p:and>

   property-name operator testvalue
   property-name operator (testvalueItem-1, testvalueItem-2, ...)

   condition and condition
   condition or condition                        
   not condition
   not (condition1, condition2, ...)

   time >= 2012… 
   and time <=2013…
   and (travelType ~ ^accommodation.* or supplier=TUI) 
   and not(eventData $> 2013-01-01)

Given a p-faced collection, p-filters can be used to select a sub collection, retaining only those nodes whose p-face passes the p-filter. This amounts to a logical operation, mapping a collection URI and a p-filter to a sub collection. New XPath functions can make this operation available. Assuming as a tentative function name fn:nodes, three variants may be useful:

   fn:nodes($collectionUri as xs:string, p:filter as element(p:filter)) as node()*
   fn:nodes($collectionUri as xs:string, p:filter as xs:string) as node()*
   fn:nodes($filteredCollectionUri as xs:string) as node()*

The second signature expects the p-filter in query syntax (see Syntax). The third signature expects a string composed of a collection URI and a p-filter in query syntax, separated by a ? character, for example:

   http://www.traveltainment.de/pfc/shoppingCart?booking-date=2013-08-01

Also when concatenated with the collection URI, the p-filter in query syntax is not URI-encoded.

The introduction of these functions should be accompanied by an extension of the dynamic context of XPath evaluation: an additional component available property-faced collections provides a mapping of URIs to property-faced collections. A property-faced collection is formally defined as a collection which is associated with a mapping of p-filter instances to sub collections

A p-faced collection can be exposed as a REST-full web service, using as URI the concatenation of collection URI and (URI-encoded) p-filter described in the previous section. As a web service is expected to return a single document, rather than a sequence of nodes, the selected sub collection may be wrapped by an element and thus integrated into a single document. The proposed convention is to use a <p:collection> element with two attributes providing the collection URI and the (not URI-encoded) p-filter. Example:

   <p:collection uri="http://www.traveltainment.de/pfc/offers" p-filter="destination=PMI"
         xmlns="http://www.traveltainment.de/pfc/offers"
         xmlns:p="http://www.w3.org/2013/xpath-structures"> 
      <offer>...</offer>
      <offer>...</offer>
      <offer>...</offer>
   </p:collection>

An XPath processor may provide access to two different groups of p-faced collections:

A deployed collection is accessed according to configuration data. This might, for example, involve the translation of the p-filter into a SQL query, its execution and the construction of nodes from result set data. See Implementation and configuration for details about the deployment of p-faced collection.

An external collection is accessed by applying the fn:doc function to a URI which is a concatenation of collection URI and (URI-encoded) p-filter in query syntax. The URI is expected to yield a document which represents the selected nodes as child elements of the document element (see Exposure as REST-ful web services)). The fn:nodes function might operate as follows:

A crucial aspect of p-faced collections is that they may be implemented using non-XML technologies, namely relational database tables and NOSQL object collections. This follows from the fact that the node itself is represented by a string and that it is not inspected during the filtering operation. Consequently, the implementation of a p-faced collection is not dealing with any tree structure. The implementation must provide for each node the representation of a logical map in which one entry represents the node itself and the remaining entries represent the names and values of properties. Configuration data include the name of the entry representing the node and specify how the node (if selected) is constructed from the string representing it. The string may be a serialization of the node, or it may consist of locator information, either a regular URI or a locator with implementation-defined semantics.

If the p-faced collection is “homogeneous” in the sense of every node having the same external properties, a relational table is a straightforward choice. If the nodes have different sets of properties, a NOSQL collection may be more appropriate. An example for the latter case are log data, where each log entry has some generic properties including an event type and further properties which are event type-specific. An event of type “file-not-found” may have a property “file-name”, whereas an event “container-delivered” may have properties specifying a container-number and a location.

Configuration data are required in order to enable an XPath processor to access the implementation of a p-faced collection:

The model of a p-filter includes details which may not be well supported by a particular technology which on the whole seems well-suited to implement p-faced collections. For example, the used technology may not support …

Two perspectives must be distinguished:

Defining the integration of p-faced collections into XPath, only the second perspective is relevant, as it impacts the semantics of XPath expressions. The first perspective may affect the names and values of properties, but not the syntax and semantics of XPath expressions. Such limitations are hidden from the collection user by the actual use of property names and values. For example, if the technology excludes multiple property values, the collection simply does not use multiple-valued properties; or if the technology does not support case-sensitive property names or values, the collection arbitrarily defines all property names to be lower case (or upper case).

The concept of collection features is introduced in order to enable a standardized representation of limitations, and to enable a definition of conformance which can be achieved in spite of limited support of the p-filter model. The following table presents a tentative set of collection features.

Scenario: we want to check some country codes, we have access to a dictionary of country codes and names, but this dictionary is represented by a CSV dataset.

Now let us assume there is a standardized XML vocabulary for table data, comprising "table", "row" and "cell" elements. And we further assume the existence of a new fn:doc variant which delivers CSV data as an XML node tree based on the table vocabulary. It enables us to select all invalid country codes by a simple expression:

   doc("foo.xml")//countryCode
      [not(. = doc("countries.csv", "csv")//row/cell[1])]

This example demonstrates a seamless integration of data from XML and non-XML resources: both participate in the info space.

Let us assume that for a given data format F three sets of rules have been defined:

And let us further assume that these rules satisfy two conditions:

where R1(G) denotes the result of applying R1 to a format instance G and R2(N) denotes the result of applying R2 to a node tree N.

C1 and C2 imply a constraint concerning round-tripping which starts with a format instance G: the result is a format instance G' which is not necessarily equal G but which is "FORMAT-deep-equal": R1(G) is deep-equal R1(G'). This amounts to defining the information content of a format instance G: it is the node tree provided by R1. Relying on this definition of information content, instances of F can be safely evaluated or modified by evaluating or modifying the corresponding node tree. This is the foundation of an integration of format F into the info space.

A normative definition of rule sets R1, R2 and R3 enables the standardized integration of a non-XML format into the info space. This requires two extensions of the XML standards. The first extension is an XPath function which parses an instance of the format into a node tree, reflecting R1. The second change extends the XML serialization model [5] by the definition of a serialization to format F, reflecting R2 and R3. The interface of serialization is accordingly extended by an additional value of the serialization parameter model.

A concrete proposal for the required XPath function is the introduction of a second variant of fn:doc which has an additional parameter specifying the source data format:

   fn:doc($uri as xs:string, $mediaType as xs:string) as document-node()

Assuming that rule sets have been standardized for the formats HTML, JSON, CSV, SQL and CSS, access to non-XML resources might look like this:

   fn:doc("http://example.com/foo.htm", "htm")
   fn:doc("http://example.com/foo.jsn", "jsn")                        
   fn:doc("http://example.com/foo.csv", "csv")
   fn:doc("http://example.com/foo.sql", "sql")
   fn:doc("http://example.com/foo.css", "css")

The following sections explore how the rule sets govering parsing and serialization may be defined.

Domain-specific languages (like SQL) and non-recursive generic formats (like CSV) may be represented by a particular XML vocabulary. The formal definition of such a vocabulary might comprise an XSD and a (possibly empty) set of additional constraints expressible as XPath assertions. As a simple example, the vocabulary for representing CSV data might be defined as a schema containing element declarations with names like "table", "row" and "cell" as well as type definitions constraining a table element to have only row children and row elements to have only cell children. An additional constraint might prescribe that every row element has the same number of cell children:

   count(distinct-values(//row/count(cell))) eq 1

As a second example, an SQL vocabulary might contain a specific element for each SQL statement, with a content model designed to capture all information which the SQL syntax may convey.

Wiki dialects are non-XML formats representing a node vocabulary (XHTML). The motivation is of course not integration into the info space, but access to a rendering processor. And here a format was added to an existing node vocabulary, whereas integration adds a node vocabulary to an existing format. Nevertheless, it is interesting to notice the popularity which a non-XML/HTML representation of node trees can attain. Standardization work might take this phenomenon into consideration.

Vocabulary-based integration is natural for formats which do not themselves generate vocabularies as JSON does. Such a generative format might in principle be mapped to a fixed XML vocabulary, by mapping the names supplied by the format instance to XML data, rather than to XML names. However, if the format has a recursive structure – like JSON – the resulting XML would look unnatural and be inconvenient to evaluate. Such a mapping would have very limited practical value, and it would violate the concept of integration as explained in a generic pattern of integration. The concept is based on regarding non-XML formats as alternative representations of node trees, as if the node trees were the original information and the format were a means to capture this information. If the node tree looks unnatural, the image presupposes the desire to convey information which is unnaturally structured in the first place, which is absurd.

If JSON is redefined to be the representation of node trees, these node trees should probably capture JSON names by node names. Several approaches of mapping JSON to XML have done that as far as possible (e.g. [7], [12], [13], [14], [16]) but they could not achieve two goals:

The first problem arises from the fact that XML names are QNames. The second problem reflects a mismatch of data models: whereas JSON uses two distinct content models, one a map and the other a sequence of anonymous items, XML has a single content model which is sequence-and-name based. The XML model can be used to emulate the JSON models, but it cannot express them natively. Those mapping approaches might be classified as vocabulary-assisted: the bulk of items is named after their JSON counterparts, but a small additional vocabulary is used in order to specify relationships between the nodes and their JSON counterparts. These additional items prevent an evaluation of the node tree from ignoring representations and only seeing the node tree itself.

JSON is, like XML, recursively defined and generates vocabularies. The ideal way to integrate it into the info space would be native integration: every JSON instance should be the representation of a node tree which natively expresses the JSON content, so that it can be serialized to JSON based on the node properties only, without requiring the presence of additional nodes guiding the serialization. Native integration of JSON is not possible due to differences of the XML and JSON data models: the XML model is not a superset of the JSON model.

Rennau [17] proposed an extension of the XML data model which claims to enable a native representation of JSON data. The proposal introduces the same boolean choice of content organization as is used in common programming language like Java: a choice between the sequence based model (which is the only model in current XML) and an alternative model which is truly map-based: the child elements are unordered, and every child element has a string-typed key – which is a new node property, [key]. The choice between the two content models is made by each complex element, and therefore it is modeled as a second new node property, [model]. It has two values, "sequence" and "map", and the relationship between the two new properties is constrained as follows: (a) an element with a non-null [key] must have a parent element with [model] equal "map"; (b) an element with a null [key] must not have a parent element with [model] equal "map". In other words: "map" elements have unordered children which must have keys, and "sequence" elements have ordered children which must not have keys.

Concerning the handling of anonymous items, Rennau proposed the introduction of "unspecific standard names", but this is unsatisfactory, as it renounces the goal of a genuinely native representation: the only natural representation of an anonymous item is an anonymous item. Therefore we should modify the XML data model by allowing anonymous elements. A JSON array would thus be an elegant representation of an element node with [model] equal "sequence" and anonymous child elements.

In summary, the following extensions of the XML data model should be considered, as they promise to overcome the limitations which disqualify the model as foundation of the info space: (a) addition of two new node properties, [model] and [key]; (b) modification of the property [node-name], introducing the possibility of a null value.

Introducing new node properties implies the necessity to define the representation of these properties in XML syntax. Rennau proposed the use of "pseudo attributes" which look like attributes but do not represent attribute nodes, as they represent node properties:

where the prefix "udl" stands for "unified document language". As a consequence, however, the XML representation of a JSON instance tends to be cluttered with pseudo-attributes, which is perhaps a less important disadvantage as it may seem, as the processing of JSON-supplied data is as elegant as the processing of XML-supplied data (see below, Extension of the XPath/XQuery language) and the very info space architecture encourages the use of JSON notation whereever it seems more attractive than XML notation (see below, Markup integration) .

However, a refusal to express the new node properties with the same immediateness as the other properties is unsatisfactory. Pseudo-attributes are an adequate representation of a node property when they can be expected to be rare, as for example xml:type attributes. However, knowing in advance that the XML representation of JSON-supplied data will produce XML documents which are studded with pseudo-attributes - rendering them hardly readable - we are motivated to go a step further and truly extend the XML syntax, introducing new syntax constructs. The goal is to ensure readable XML representations also of such node trees as are constructed from JSON instances. The following syntax extensions should therefore be considered:

These rules would allow an acceptable representation of JSON data in XML syntax. A special advantage is that a given information content – e.g. a web service response – might then be served without changes alternatively in JSON or XML. Example:

   <~getWeatherResponse>
         <"temperature min">28</"temperature min">
         <"temperature max">32</"temperature max"> 
   </~getWeatherResponse>
                        
   {
      "temperature min" : 28,
      "temperature max" : 32
   }

To round these considerations off, a further detail of Rennau's proposal which concerns the XML syntax should be retained: for elements which have a parent element with [model] equal "map", the key defaults to the local name. Therefore, a slight modification of the above example would look like this:

   <~getWeatherResponse>
      <temperatureMin>28</temperatureMin>
      <temperatureMax>32</temperatureMax> 
   </~getWeatherResponse>

In this variant the child elements are not any more anonymous but have a local name equal to the key.

Integration of non-XML resources into the info space is driven by the desire to make them accessible to navigation and evaluation. The extension of the data model by a new [key] property must therefore be mirrored by an extension of the XPath semantics. Rennau proposed the introduction of a third node test, the "key test" which functions in analogy to the name test: selecting elements with a given key, rather than elements with a given name. Being a node test, the key test is combined with navigational axes in the same way as a name test. The syntax of a key test is a special character (proposed: "#") followed by the key value within quotes. Examples:

   $response/#"temperature min"
   $response//#"2012"                        
   $item/ancestor-or-self::#"AC#A-2917"

Thanks to the key test, JSON-supplied data can be navigated with the same ease as XML-supplied data, using key tests to select among object members and conventional numerical predicates to select among array items:

   $response//#"weather"/#"temperatures"/*[3]

Whereas the key test provides for navigation, Rennau also proposed an extension of XQuery which enables the construction of nodes using JSON syntax. The extension is purely syntactical: an additional syntax which is a JSON-style shorthand notation for the construction of nodes which, due to their properties, correspond to JSON objects and arrays. For example, the following XQuery constructor:

   <*>
      <"temperature min">{$min}</"temperature min">                        
      <"temperature max">{$max}</"temperature max">
      <"temperatures">{for $v in $values return <*>{$v}</*>}</"temperatures">
   </*>

can equivalently be written using JSON syntax:

   {
      "temperature min" : $min,
      "temperature max" : $max,
      "temperatures" : [ $values ]
   }

The XQuery construction of JSON data using JSON syntax is as convenient as the construction of XML data using XML syntax. Taken together, the proposed extensions of XPath and XQuery ensure a complete integration of JSON into the info space: