The official blog for the Gepsio open source project. Gepsio is a .NET-based document object model for XBRL documents. Visit the Gepsio project site at http://gepsio.codeplex.com.

Menu

Gepsio currently supports several different versions of the .NET Framework, but it has long been a goal to support the code on a variety of platforms, including the Universal Windows Platform (UWP) and Xamarin. Microsoft is making strides in the standardization of the .NET platform that should make this goal attainable while allowing the Gepsio codebase to move forward with relative ease into these new platforms.

Microsoft recently introduced .NET Core, and, with it, an API set called .NET Standard. .NET Standard is a set of APIs that all .NET platforms have to implement. As of this writing, the current version of .NET Standard is version 1.6, which supports the following platforms:

.NET Core 1.0

.NET Framework 4.6.2

Mono 4.6

Xamarin.iOS 10.0

Xamarin.Android 7.0

Universal Windows Platform 10.0

Some investigation has been performed on the Gepsio code base with the goal of understanding the impact of moving the Gepsio code base to .NET Standard 1.6. The good news is that the vast majority of the code base moved over without much of an issue, giving hope that the migration of the Gepsio code base away from its sole focus on the .NET Framework and into a broader view of the current platform landscape is achievable.

There is, however, one blocking issue which prevents Gepsio from adopting .NET Standard 1.6 today. Although .NET Standard 1.6 supports the System.Xml namespace, it does not support the System.Xml.Schema namespace. Since XBRL heavily leverages the XML Schema specification, and since Gepsio leverages data from XBRL schemas in its implementation, Gepsio must implement XML Schema support in its codebase in the absence of support from .NET Standard. With System.Xml.Schema unavailable, Gepsio will not be able to leverage the classes in the namespace using .NET Standard 1.6.

There are, at this point, two options for Gepsio:

Provide an independent implementation of XML Schema. Gepsio could, if necessary, provide a “homegrown” implementation of XML Schema support. Since an XML Schema document is, at its core, an XML document, Gepsio could leverage the classes available in System.Xml to read in a schema document and parse it using the requirements in the XML Schema specification. This work would become non-trivial, as the implementation would need to support the specification’s type system and notions of simple and complex types, as well as exposing all of that information to the pieces of Gepsio which require schema information. In this plan, Gepsio would be required to implement the XML Schema implementation “from scratch”.

Wait for XML Schema support in a future version of .NET Standard. Gepsio could also do nothing and wait for the .NET Standard to support the System.Xml.Schema namespace and then move over to that version of the Standard at that time.

The second option seems to be a more practical approach, since it would not require that an entire XML Schema implementation be written from the ground up. The question is: how long will it take for .NET Standard to provide that implementation?

As it turns out, not too long.

The next version of .NET Standard, labeled 2.0, is slated to support the XML Schema namespace. As of this writing, 924 APIs from the System.Xml.Schema namespace will be supported. Work on .NET Standard is already underway and is slated to support the following platforms:

.NET Core vNext

.NET Framework 4.6.1

Mono vNext

Xamarin.iOS vNext

Xamarin.Android vNext

Universal Windows Platform vNext

Given all of this, the plan for Gepsio is as follows:

Wait for .NET Standard 2.0 to be released. Currently, this is slated to release with the next version of Visual Studio.

Load the current Gepsio code into the next version of Visual Studio.

Migrate the Gepsio projects in Visual Studio away from a simple .NET Framework class library and over to .NET Standard.

Ensure that Gepsio’s XML implementation, both from the standpoint of XML instances and schemas, support .NET Standard 2.0

Cut a release and push out to NuGet.

Continue to move forward with the (admittedly long-delayed) continual improvement of Gepsio’s support for the XBRL standard, now in an environment where several platforms can use the same code base.

The last post discussed a future design whereby Gepsio would ship as two assemblies and that, as a result of this work, would no longer be able to support .NET 3.5.

Never mind.

MissingManifestResourceException Thrown In UWP Calls To PCL Resources

Some work has gone into implementing the design discussed in that last post. The design centered around a Portable Class Library (PCL) which held Gepsio’s string resources as well as a small class that returned strings from the PCL to the caller. In the solution structure, a separate PCL called JeffFerguson.Gepsio.Resources which targeted Windows 10 and .NET 4 was created, and all of the Gepsio assemblies referenced that PCL project. The Gepsio RESX string table was added as a resource in the PCL, and a simple class was added to the PCL:

Resource Management Plan Going Forward

The backup plan is to maintain two separate string tables. One string table will be maintained in the Gepsio .NET projects as an RESX file, and a second string table will be maintained in the Gepsio UWP projects as an RESW file. It’s a shame that the string tables will need to be maintained in two places, but, if a single portable location isn’t going to work, then there is not much choice. Hopefully, it won’t be a huge issue.

Retaining Support for .NET 3.5

The last post mentioned that PCLs do not support .NET 3.5 and, as a result of the proposed Gepsio design making use of PCLs, future versions of Gepsio would not be able to support .NET 3.5. Now that the PCL design has proven to be unworkable, Gepsio can and will support .NET 3.5 for the foreseeable future.

Gepsio will no longer be able to support .NET 3.5 and will support only .NET 4.x.

This blog post will examine both of those changes and why they are necessary to be sacrificed to allow Gepsio to support UWP.

A Tale of Two Assemblies

Let’s first take a look at the need for Gepsio to be implemented in two assemblies, rather than just one as in the past.

Gepsio’s Use of String Resources

Gepsio not only parses XBRL, but also validates the contents of an XBRL instance against the XBRL specification. Validation errors are available for inspection after the document instance is loaded, as in the following example:

The messages are built into the Gepsio code base through a resource (RESX) file. This RESX file, which is actually an XML document, contains the strings used to build the validation messages. Building these messages into a separate RESX file, rather than embedding them into the source code directly, allows the strings to be translated into various languages by providing a separate XML file for the new language, without having to change the source code itself. Since XBRL is an international specification, it makes sense to keep strings segregated where they can be easily located and changed when Gepsio needs to support languages other than English.

Microsoft changed the way string resources work for UWP assemblies. Resources are now stored in a RESW file, which has the same underlying XML structure as an RESX file but is incompatible with the tooling for .NET applications. Moreover, the code needed to access strings in an RESW file differs from the code needed to access strings in an RESX file.

Possible Design Approaches

The difference in approach between RESX files and RESW files impacts Gepsio, which strives to have a code base that supports .NET as well as UWP. There are several possible approaches to this problem, and they are described below.

One RESX for .NET, One RESW for UWP

With this approach, two resource files would be maintained: an RESX file for the .NET projects, and a RESW for the UWP project. This would cause a maintenance headache, as a developer working on Gepsio would have to remember that any strings added or modified in the RESX would have to be done again in the RESW file, and forgetting to do so will cause an incompatibility between Gepsio’s .NET implementation and its UWP implementation. If a string was found in one file and not the other, Gepsio may even crash at runtime when trying to find a string resource that doesn’t actually exist. Furthermore, since the code needed to access a string from an RESX file differs from the code needed to access a string from an RESW file, the code would also need to be maintained twice. Generally, making two modifications in two files can be more error-prone than making one modification in one file.

One RESX for .NET, Copied And Renamed for UWP

With this approach, one RESX file would be maintained in Gepsio’s .NET project, and a custom step in the build workflow would copy the RESX file over to the UWP project, rename it with an RESW extension, and allow the UWP project to build with the new RESW file. This approach would have the advantage of only needing to maintain one resource file. However, this would also introduce new complexities into the build process and would still require that two separate codebases be maintained: one to access the RESX file and another to access the RESW file.

Resources in a Portable Class Library

With this approach, the project would maintain a Portable Class Library (PCL) containing Gepsio’s string resources. The PCL would be targeted to support .NET as well as UWP. The PCL project would maintain the string resources as part of the project, as a single RESX file in a single location. The PCL would also contain code to access and return strings and Gepsio, in both its .NET and UWP personalities, would call this portable code when strings are needed.

Approach Going Forward

After considering all of the options (and lamenting the fact that the RESX/RESW discrepancy exists in the first place) it was decided to go down the PCL route. This means that, starting with the next release, Gepsio will ship as two assemblies:

the main Gepsio assembly

the Gepsio string resources assembly

The disadvantage here is that there are now two assemblies to distribute, rather than just the single assembly shipped up this point. The mitigation here is that distribution tools such as NuGet make distribution easier and more automatic, and the chances that a user would ship one Gepsio assembly and forget the other is reduced thanks to these tools.

The other disadvantage to the PCL approach leads us into the next change for the Gepsio platform.

End of Support for .NET 3.5

The “resources in PCL” approach described above signals the end of Gepsio’s support for .NET 3.5 and restricts Gepsio to supporting only .NET 4.0 and above. PCLs do not support .NET 3.5 and their support for .NET goes back only to .NET 4.0. This means that future versions of Gepsio will support the following platforms:

.NET 4.0

.NET 4.5

.NET 4.5.1

.NET 4.5.2

.NET 4.6

.NET 4.6.1

Universal Windows

Though .NET 3.5 will be left off of this list, it is most likely not widely used these days. .NET 3.5 was released in November of 2007 and its replacement, .NET 4.0, was released in April of 2010. It is therefore possible than any installations using .NET 3.5 at one time have been upgraded some time in the last six years to one of the .NET 4.x runtimes.

Every <context> element MUST include the @id attribute. The content of the @id attribute MUST conform to the [XML] rules for attributes with the ID type (http://www.w3.org/TR/REC-xml#NT-TokenizedType). The @id attribute identifies the context (see Section 4.7) so that it may be referenced by item elements.

The specification does not impose any semantics on the contents of the ID attribute.

I am very interested in trying out Gepsio on the windows 10 universal app platform.

I saw that a universal windows project has been added to the repo without some vital xml parts.

Will there be a new patch soon with the remaining parts?

As a matter of fact, yes! That work is underway now.

The email is referencing this changeset, which contains the following somewhat-cryptic description:

Universal Windows project added. XML implementation for Universal Windows is not implemented, and there are no unit tests for Universal Windows, so Gepsio for Universal Windows cannot be used at this time. The .NET builds are, as always, fully supported.

That changeset included a new project in the Gepsio solution – a project which would target a Gepsio assembly tuned for the Windows 10 Universal Windows Platform (or UWP, for short). The vision is to allow Gepsio to be used by developers wishing to build XBRL-enabled applications on Windows 10 UWP.

Thanks to the interface-based separation between Gepsio’s XML service layer and its XBRL semantic layer, described here, the work needed to allow Gepsio to support Windows 10 UWP is not a “do over”. The work involved is, basically, to build new XML interface implementations that use Windows 10 UWP instead of .NET. This work is underway now. In fact, Gepsio already has enough UWP code to open up an XML document and search for <xbrl> nodes in Windows 10. This is exciting, as it means that Gepsio can be used for code that runs on any Windows 10 UWP platform, from the phone, to the tablet, to the desktop, to the Xbox One.

The Windows 10 UWP code is not ready, and, although the project skeleton has been checked in, the XML layer for Windows 10 UWP is not ready. That code is in progress. When it’s ready, there will be a blog post announcing the new code. The code is not a straightforward port from the .NET XML code, for the following reasons:

The .NET code uses the XML classes in the System.Xml namespace. The Windows 10 UWP code is using XML classes in the Windows.Data.Xml.Dom namespace.

The Windows 10 UWP makes use of asynchronous methods; for example, XmlDocument.Load() has become await XmlDocument.LoadFromFileAsync(). Ensuring that Gepsio supports this asynchronous model, while still maintaining the existing needs of the Gepsio XBRL semantic layer, will need some additional work.

Unlike .NET’s System.Xml namespace, the Windows 10 UWP’s Windows.Data.Xml.Dom namespace does not have any built in support for XML Schema. XBRL relies on XML Schema documents for its taxonomies. Gepsio’s XML Schema support will have to be written from the ground up since there is no support for XML Schema in UWP.

Gepsio will retain its .NET builds as well. Gepsio will continue to support .NET 3.5, .NET 4, .NET 4.5, .NET 4.5.1, .NET 4.5.2, and .NET 4.6. If you’re not ready to move to Windows 10 UWP just yet, do not worry – all of the .NET builds will continue to be supported and ship for the foreseeable future.

The Gepsio code base has just undergone a significant restructuring with changeset 77191. As a benefit of the restructuring, the project will, for the first time, be able to provide assemblies for the following platforms:

.NET 3.5

.NET 4

.NET 4.5

.NET 4.5.1

.NET 4.5.2

.NET 4.6

This restructuring also puts Gepsio in the position to provide support for .NET platforms, including the Universal Windows Platform (UWP) available with Windows 10. This is an exciting time for the project, as, up to this point, Gepsio has provided assemblies only for the aging .NET 3.5 platform.

The restructuring relies heavily on the Shared Projects concept available in Visual Studio 2015 to provide support for sharing common code across each of the platform builds. As such, Visual Studio 2015 is now required to open the solution’s source code.

Design

The XLINK specification is now implemented in a specific class, and XBRL linkbase nodes now leverage Gepsio’s implementation of XLINK. The following classes have been refactored to leverage Gepsio’s internal support for the XLINK specification added in this version:

Label

LabelLink

Improved performance of loading an XBRL fragment’s fact collection by setting the capacity of the collection to the maximum number of nodes beneath the XBRL root node before any facts are created and added to the collection, and then trimming the excess after all facts are created and added to the collection. This strategy prevents .NET from having to re-allocate the fact collection each time it needs to expand, which costs time during the fact collection creation process.

The list of facts exposed by an XbrlFragment, formerly of type List<Fact>, is now an object of a class called FactCollection. The FactCollection class allows the fact list to be augmented with some internal dictionaries that can be used to find a fact more quickly than simply iterating through a list and checking each fact.

Gepsio’s XML service layer has been implemented behind an interface since the Nov 2014 CTP. These interfaces, which have been internal in previous CTPs, is now public. See this blog post for more information for the motivation and rationale behind this change.

New Classes

FactCollection: Replaces the simple List<Fact> with additional public methods to quickly find a fact or list of facts by name or ID. The FactCollection class supports public methods such as GetFactById() and GetFactByName() which allows callers to find facts in the collection quickly and efficiently, thanks to internal disctionaries managed by the FactCollection class.

PresentableFactTree: A tree of facts arranged in accordance with the information in a presentation linkbase. See this blog post and this blog post for more information.

PresentableFactTreeNode: A node in a tree of facts arranged in accordance with the information in a presentation linkbase. See this blog post and this blog post for more information.

New Properties Available on Previously Existing Classes

Element

bool IsAbstract: True if the element is an abstract element; false otherwise.

301-10-FootnoteFromOutOfScope.xml [301.10 The instance contains two footnote links. The second one contains an arc with a from value that does not have a corresponding loc in the same extended link.]

301-11-FootnoteToOutOfScope.xml [301.11 The instance contains two footnote links. The second one contains an arc with a to value that does not have a corresponding footnote resource in the same extended link.]

301-12-FootnoteLocOutOfScope.xml [301.12 The instance contains a footnote link. In the footnote link there is a loc element that has an href that points to a fact in another instance document.]

301-13-FootnoteLocInScope.xml [301.13 The instance contains a footnote link. In the footnote link there is a loc element that has an href that points to a fact in the instance document using the instance document file name with a fragment identifier.]

301-14-FootnoteFromResource.xml [301.14 The instance contains a footnote link. The arc in the footnote link has a from value that matches a footnote resource. This is not valid for the fact-footnote arc role on the arc. The from must point to a loc which in turns points to a fact in the same instance document.]

301-15-FootnoteToLoc.xml [301.15 The instance contains a footnote link. The arc in the footnote link has a from value that matches a footnote resource. This is not valid for the fact-footnote arc role on the arc. The from must point to a loc which in turns points to a fact in the same instance document.]

301-16-FootnoteWithoutLang.xml [301.16 The xml:lang attribute is missing on the footnote resource.]

301-17-FootnoteCustomArcRole.xml [301.17 The footnote custom arc role can relate two footnotes to each other instead of just fact to footnote. (Only the standard footnote arc role is restricted to being from item or tuple locators.) Maybe this might be used to indicate how some footnote is “footnoting” another footnote.]

302-10-PeriodDateTimeInvalid.xml [302.10 Invalid duration context with start date later than end date]

302-11-DecimalAttributeOnSegmentInconsistent.xbrl [302.11 Two contexts are S-Equal even though a decimal-valued attribute in their segment elements have different lexical representations. The contexts are S-equal, so a calculation inconsistency MUST be signaled.]

302-12-DecimalAttributeOnScenarioInconsistent.xbrl [302.12 Two contexts are S-Equal even though a decimal-valued attribute in their scenario elements have different lexical representations. The contexts are S-equal, so a calculation inconsistency MUST be signaled.]

303-05-ForeverElementewithInstancePeriodTypeReportedasForever.xbrl [ForeverConcept with Instant Period Type is not allowed]

Unit of Measure Consistency [Section 4.4 The Context Element]

304-01-monetaryItemTypeUnitsRestrictions.xml [304.01 An element with a monetary item type has an ISO currency code for its units (using the standard ISO namespace prefix).]

304-02-monetaryItemTypeUnitsRestrictions.xml [304.02 An element with a monetary item type has an ISO currency code for its units (using a non-standard ISO namespace prefix).]

304-03-monetaryItemTypeUnitsRestrictions.xml [304.03 An element with a type derived by restriction from the monetary item type has an ISO currency code for its units.]

304-04-monetaryItemTypeUnitsRestrictions.xml [304.04 An element with a type derived by restriction from monetary item type has an ISO currency code for its units (using a non-standard ISO namespace prefix).]

304-05-monetaryItemTypeUnitsRestrictions.xml [304.05 An element with a non-monetary item type has an ISO currency code for its units (using the standard ISO namespace prefix).]

304-06-monetaryItemTypeUnitsRestrictions.xml [304.06 An element with a monetary item type does not have an ISO currency code for its units – the namespace is wrong.]

304-07-monetaryItemTypeUnitsRestrictions.xml [304.07 An element with a monetaryItemType does not have an ISO currency code for its units – the local name is wrong.]

304-08-monetaryItemTypeUnitsRestrictions.xml [304.08 An element with a type derived by restriction from monetaryItemType does not have an ISO currency code for its units – the namespace is wrong.]

304-09-monetaryItemTypeUnitsRestrictions.xml [304.09 An element with a type derived by restriction from monetaryItemType does not have an ISO currency code for its units – the local name is wrong.]

304-10-pureItemTypeUnitsRestrictions.xml [304.10 An item with a pureItemType data type MUST have a unit element and the local part of the measure MUST be “pure” with a namespace prefix that resolves to a namespace of “http://www.xbrl.org/2003/instance&#8221;.]

304-11-pureItemTypeUnitsRestrictions.xml [A measure element with a namespace prefix that resolves to the “http://www.xbrl.org/2003/instance&#8221; namespace MUST have a local part of either “pure” or “shares”. The value ‘impure’ is not a valid measure in the XBRL instance namespace.]

304-12-pureItemTypeUnitsRestrictions.xml [Unlike for monetaryItemType and sharesItemType, there is no constraint (in 4.8.2 or elsewhere) requiring an item with a pureItemType data type to have a particular kind of unit.]

304-12a-pureItemTypeUnitsRestrictions.xml [Same as V-12, but the pure measure has no prefix and the default namespace is undefined.]

304-13-sharesItemTypeUnitsRestrictions.xml [304.13 For facts that are of the sharesItemType, units MUST have A single measure element. The local part of the measure MUST be “shares” and the namespace prefix that MUST resolve to http://www.xbrl.org/2003/instance%5D

304-14-sharesItemTypeUnitsRestrictions.xml [304.14 For facts that are DERIVED BY RESTRICTION from the sharesItemType, units MUST have A single measure element. The local part of the measure MUST be “shares” and the namespace prefix that MUST resolve to http://www.xbrl.org/2003/instance%5D

304-15-pureItemTypeUnitsRestrictions.xml [304.15 For facts that are of shares item type, units MUST have A single measure element. The local part of the measure MUST be “shares” and the namespace prefix that MUST resolve to http://www.xbrl.org/2003/instance. In this case the unit has two measure elements, both of which are pure.]

304-15a-sharesItemTypeUnitsRestrictions.xml [Same as V-15 but in this case the unit has has shares but no prefix and the default namespace is undefined.]

304-16-unitsInSimplestForm.xml [304.16 The units must not have numerator and denominator measures that cancel.]

304-17-sameOrderMeasuresValid.xml [304.17 The units equality test which two units have same order measures.]

304-18-sameOrderDivisionMeasuresValid.xml [304.18 The units equality test which two units have same order divisions.]

304-19-differentOrderMeasuresValid.xml [304.19 The units equality test which two units have different order measures.]

304-20-differentOrderDivisionMeasuresValid.xml [304.20 The units equality test which two units have division elements which their order of child measures are different.]

304-21-measuresInvalid.xml [304.21 it tries to essence-alias equality of two elements with different units : where one is pure-feet and the second is pure-pounds. so the alias essence check is invalid and it should throw an error in xbrl validation]

304-22-divisionMeasuresInvalid.xml [304.22 The test tried to essense-alias equality check of two elements with different units : where one is unit between “pure-inch / pound-feet” and other “pure-feet / pound-inch”. The tests is invalid as it should throw an error during xbrl validation.]

304-23-Calculation-item-does-not-match-unit.xml [Variation of 304-15 where the type of the fact value does not match that of the type of the reported element. Shares type versus Monetary unit]

305_07_invalid_instance.xbrl [305.07 a genuine inconsistency due to roll up of child values]

305-08-UnitsSpecifiedOnNilItem.xml [305.08 nil items have no decimals or precision, with unitref, but the type specifies fixed values for decimals and precision.]

Required Arc in Definition Linkbase [Section 5.5.6.4.1.5]

306-01-RequiredInstanceValid.xml [306.01 The instance contains two elements in the same context. The presence of one element forces the presence of the other.]

306-02-RequiredInstanceTupleValid.xml [306.02 The instance contains an item and a tuple. The presence of the tuple forces the presence of the item.]

306-03-RequiredInstanceInvalid.xml [306.03 The instance contains an item and a tuple. The presence of the tuple forces the presence of the item.]

Schema References [Section 5 Taxonomies]

307-01-SchemaRefExample.xml [307.01 A schemaRef element MUST hold the URI of a schema. In this case it does.]

307-02-SchemaRefCounterExample.xml [307.01 A schemaRef element MUST hold the URI of a schema. In this case it does not because the second reference to a schema actually points to an XML document that is a label linkbase. ]

307-03-SchemaRefXMLBase.xml [307.03 schemaRef elements MUST hold the URI of Schemas. In this case the requirement is not satisfied because the schema reference has to be resolved using the XML base attribute that ensures the schemaRef URI resolves to the XML document in the base directory. This document, however, is a label linkbase, not a schema. If the XML base attribute value is not used then the schema in the same directory as the instance is discovered and no issues are noticed.]

308-01-instance.xml [Instance contains two role references to the same URI, INVALID]

308-02-instance.xml [Instance contains two arcrole references to the same URI, INVALID]

LAX validation tests [Test that LAX validation is performed]

314-lax-validation-01.xml [Segment has an element for which there is no definition, so it is allowed; item has an attribute with no definition, so it is allowed. The definitions are imported to the discovered taxonomy.]

314-lax-validation-02.xml [Segment has an element for which there is no definition, so it is allowed; item has an attribute with no definition, so it is allowed. The definitions are found by schemaLocation from the instance document.]

314-lax-validation-03.xml [Same as v-01 but segment has an element defined as integer with string contents]