Details

Description

From Tyler Coffin up on hbase-user@

I am using the Stargate REST interface to HBase for inserting data. When using JSON to transmit the query content, I have found that specific ordering of key/value pairs within the JSON string is required in order for the query to succeed (otherwise a response of 'HTTP/1.1 500 Row key is invalid' error is thrown if "key" and "Cell" are reversed).

As you can see the only difference between these two instances is that the "key" and "Cell" name/value pairs have their order reversed.

In the equivalent XML notation, the ordering is specifically required per the schema. However with JSON Objects (i.e. name/value pairs) order is not required (JSON Arrays are ordered, but not Objects). Some JSON libraries will preserve ordering of Objects but not all which is how I discovered this problem in the first place because I was using the Perl JSON library which does not guarantee order).
I'm unsure if this is a bug in the REST implementation or an inconvenient ambiguity in the JSON specification. Regardless I thought I'd share this discovery with the community for feedback (or at the very least to document this for users' future reference).

Activity

In the equivalent XML notation, the ordering is specifically required per the schema.

... and Jersey adds a marshaller and unmarshaller to the JAXB framework to produce JSON. This is an artifact of jersey-json or something dumb we did when hooking up JAXB. I'll write up some unit tests and look at this soon.

Andrew Purtell
added a comment - 20/Apr/10 23:10 Thanks for the report, very helpful.
In the equivalent XML notation, the ordering is specifically required per the schema.
... and Jersey adds a marshaller and unmarshaller to the JAXB framework to produce JSON. This is an artifact of jersey-json or something dumb we did when hooking up JAXB. I'll write up some unit tests and look at this soon.

Some JSON field ordering issues remain according to reports. These may be an artifact of how JSON is hooked up to JAXB. The weird '$' special field for specifying a value is due to how the bindings work, for example. Reopening for more investigation and at least a documentation update.

Andrew Purtell
added a comment - 03/Jun/12 11:29 Some JSON field ordering issues remain according to reports. These may be an artifact of how JSON is hooked up to JAXB. The weird '$' special field for specifying a value is due to how the bindings work, for example. Reopening for more investigation and at least a documentation update.

If the "$" value field does not come last in a CellModel, Jersey won't deserialize the model correctly. We can

1. Document this clearly and also add a troubleshooting section entry.

2. As an alternative, elsewhere we use Jackson as a JSON serializer/deserializer. Jackson seems widely considered to be better behaved. JacksonJsonProvider in jackson-jaxrs is a drop in replacement for Jersey's JSON binding for JAX-RS. However, the JSON representation of HBase REST may change as a consequence. This would depend on how configurable JacksonJsonProvider is, and if all backwards compatible behaviors can be specified. Digging around Jackson javadoc for a while was inconclusive.

Andrew Purtell
added a comment - 03/Jun/12 22:07 - edited If the "$" value field does not come last in a CellModel, Jersey won't deserialize the model correctly. We can
1. Document this clearly and also add a troubleshooting section entry.
2. As an alternative, elsewhere we use Jackson as a JSON serializer/deserializer. Jackson seems widely considered to be better behaved. JacksonJsonProvider in jackson-jaxrs is a drop in replacement for Jersey's JSON binding for JAX-RS. However, the JSON representation of HBase REST may change as a consequence. This would depend on how configurable JacksonJsonProvider is, and if all backwards compatible behaviors can be specified. Digging around Jackson javadoc for a while was inconclusive.