Calabash probably supports these advanced xi:include options because it uses an XSLT processor under the hood to load the target document and run the XPath over it.
The Xerces XML parser used by the Jing parser to parse the Docbook 5 XML document indeed does not support the xpath scheme. We have an issue registered for this on our side but I cannot give you a timeline for its implementation.
Xerces processes XML documents in streaming mode, in a way it would need to resolve the xi:include xpath while the target document is being parsed, while the XSLT processor loads the entire target document, creates a node model over it and runs the XPath.
Besides the huge amount of work it would take to build into an XML parser full XPath capabilities, building into the the XML parser XPath query capabilities does not seem like a good architecture as usually it's the other way around, the XSLT processor uses an XML parser underneath to parse the XML content and create a node model from it, over which it runs XPath.
So I'm not sure how we could make this work, probably for automatic validation, each time you would modify the document Oxygen would first need to parse the XML using Xerces with xi:include processing disabled, pass the resulting XML document through an XSLT (or XProc) stylesheet to expand the xi:includes, then give the resulting expanded XML document to the JING parser (which will again use Xerces to parse it) to validate it against the Relax NG schema. And somehow still be able to locate errors reported by the Relax NG schema back to the original location in the XML document (which would be quite hard to accomplish once the XSLT processing has been applied).
For rendering the xi:include in the Author visual editing mode things would probably be more simple at least because there the xi:include content is not computed each time a modification is made in the document.
But validation and the expansion of xi:include elements in the Author visual editing mode need to be consistent.
That's one of the reasons why I prefer DITA to Docbook, DITA has a much better content reuse system set up.
<oXygen/> XML Editor