On Thu, Oct 23, 2008 at 03:21:01PM -0700, Dario Teixeira wrote:
> Hi,
>
> > This protocol definition is fed to the compiler, which
> > generates the OCaml type definitions, as well as the
> > encoders/decoders and pretty-printers (as you can see,
> > the specification uses a mix of OCaml, Haskell and C++
> > syntax, but it's pretty clear IMO)
>
> Basically the XDR approach, but with a syntax inspired
> by more modern, functional languages, right?
Yes, something like XDR (and Google's Protocol Buffers, and Facebook's Thrift,
and and :) with richer data types (algebraic and polymorphic types, etc.) and a
self-describing encoding that allows you to extend the type definitions while
ensuring interoperability.
> > It's not a drop-in solution like sexplib's "with sexp",
> > by design (since it is meant to allow interoperability between
> > different languages), but it's still fairly easy to use.
>
> Personally, I think that a sexplib-like syntax extension is the killer
> feature for serialisation libraries, and the reason why I was immediately
> swayed by sexplib. However, writing a sexplib-like syntax extension for
> your serialisation library would entail solving the reverse problem now
> handled by your compiler. This might not always be possible because some
> features of Ocaml's type system might not map neatly into your format.
> Nevertheless, the sheer convenience of the syntax extension approach makes
> it worth while having, even if on occasion the preprocessor were to produce
> an error message stating that it could not convert a certain structure. For
> reference purposes, you could even have the syntax extension output to an
> external file the inferred structure definition in your language format! (I
> know this would be a very complex project, but it does illustrate the power
> of Camlp4).
In fact, the wire format easily supports all of OCaml's type system (bin-prot
does, after all, and this is essentially a self-describing, extensible
bin-prot). I introduced limitations in the data schema to ensure extensibility
and portability. Any OCaml type can be encoded easily, but not all possible
changes to an OCaml type are safe with regard to protocol compatibility. Using
a separate language makes it easier to prevent altogether (by making them
impossible to express) or catch such errors.
Leaving unsafe protocol modifications aside (which just means that you have to
be careful when you change a type), the approach you suggest (supporting only
a subset of OCaml's type system in a "with protocol"-style syntax extension)
seems very doable. However, sexplib seems to be the safest option for
convenient, more or less future-proof serialization in OCaml, for the time
being.
Cheers,
--
Mauricio Fernandez - http://eigenclass.org