Search This Blog

Dynamic List Formlets in Haskell

The Haskell Formlets library (described here) provides an elegant abstraction that takes the tedium out of HTML form generation, validation, and processing (not to mention make it less error prone). This abstraction allows you to develop HTML form UIs while still thinking in and working with haskell's type system and data structures. It also fits very naturally with Happstack's ideas about using in-memory state to leverage the pure type system.

One important aspect of form abstraction is the handling of lists. Dynamic lists in particular are a bit more of a challenge because they require Javascript to handle the dynamic creation/removal of form fields. Until recently, the Formlets library has had poor support for list formlets. At the recent Hac-Phi Haskell hackathon, Chris Eidhoff and I worked on filling this gap. We made great progress at the hackathon, and it only took a little more work after the hackathon to finish the functionality we set out to solve. I would like to provide some help to potential users via a description some of the issues and design decisions we encountered.

Design Issues

Default Values

One critical aspect of form handling is the ability to specify default values for form fields. I encountered this issue very soon after I started using Formlets. If you want to edit the state of stored objects through an HTML form, you will probably want to be able to create a form where the object's current values for all the fields are already filled in. Then the user just changes the appropriate fields and doesn't have to re-enter all the other data. The Form type by itself is inadequate for this purpose. We added a new type alias to server this purpose:

&gt type Formlet xml m a = Maybe a -&gt Form xml m a

In my experience, working with Formlet instead of Form will make things easier in the long run. There are certainly simple situations where Form is sufficient, but Formlet is an important abstraction to have around.

Error Handling

The second important issue with list formlets in particular is error handling. When a single field has an error, the HTML whole form usually needs to be regenerated with an appropriate error indication. This is especially important in the context of list formlets.

Type Signature

Before continuing, we need to understand the type signature of a list formlet. Ultimately a list formlet will have the type Form xml m [a]. But we don't want to construct these formlets manually. We would like to be able to specify the formlet for a single item and have a function that automatically constructs a formlet representing the list of items. In the old version of formlets, this function was called massInput. We might define massInput as follows:

&gt massInput :: Form xml m a -&gt Form xml m [a]

But this isn't quite right because, as described above, we need the ability to specify default values in a list form. Here's where our Formlet type synonym becomes nice:

&gt massInput :: Formlet xml m a -&gt Formlet xml m [a]

Implementation Possibilities

There are two common approaches to representing a list of items: arrays and linked lists. The same is true of lists of fields in HTML. We could do a linked list implementation by adding a hidden element to each list item and have the value of the hidden element somehow tell us how to determine the name of the field(s) for the next item in the list. A hidden item with an empty value would indicate the end of the list. This is the approach used by the old massInput implementation. Its hidden items contained the prefix to the names of all the fields for the next item.

The other possibility would be to just use a predictable numbering scheme for the field names of list items. Then, instead of traversing next pointers, we would just increment through the form values until we find one that doesn't appear in submitted data.

It wasn't until after I did a significant amount of work with the linked list approach that I realized it has a fatal flaw. When a list item doesn't validate properly, massInput isn't able to recover the value of the hidden "next pointer" for that item. This means that if a user enters a list of 10 items, but makes a mistake in the first item, then the resulting error form will only have one item in the list. An error in any item will require the user to re-enter all subsequent items in the list. If you are using lists of complex items, this can be a lot of work and frustration for the user.

New and Improved

The new version of massInput fixes these problems. It uses the array approach to avoid the problem of vanishing elements when a form has errors, and it allows you to specify a list of default values for the list. This massInput function is in the Text.Formlets package, and provides the core dynamic list functionality.

We also introduced a new package Text.Formlets.MassInput with some higher level convenience functionality. In that package, we include javascript code to drive the dynamic "Add Item" and "Remove Item" buttons. This comes with another massInput function which bundles the core massInput with the add/remove buttons. Many applications will require javascript that is customized to their HTML structure, but the supplied code should be suitable for basic out of the box use as well as provide an example from which to guide customization.

The last code has not been uploaded to Hackage yet, but it is available from the Github repository. If you have any questions, comments, or improvements please let us know.

Popular posts from this blog

On a number of occasions over the years I've found myself wanting to generate realistic looking values for Haskell data structures. Perhaps I'm writing a UI and want to fill it in with example data during development so I can see how the UI behaves with large lists. In this situation you don't want to generate a bunch of completely random unicode characters. You want things that look plausible so you can see how it will likely look to the user with realistic word wrapping, etc. Later, when you build the backend you actually want to populate the database with this data. Passing around DB dumps to other members of the team so they can test is a pain, so you want this stuff to be auto-generated. This saves time for your QA people because if you didn't have it, they'd have to manually create it. Even later you get to performance testing and you find yourself wanting to generate several orders of magnitude more data so you can load test the database, but you stil…

Think of a time you've written tests for (de)serialization code of some kind, say for a data structure called Foo. If you were using the lowest level of sophistication you probably defined a few values by hand, serialized them, deserialized that, and verified that you ended up with the same value you started with. In Haskell nomenclature we'd say that you manually verified that parse . render == id. If you were a little more sophisticated, you might have used the QuickCheck library (or any of the numerous similar packages it inspired in other languages) to verify the parse . render == id property for a bunch of randomly generated values. The first level of sophistication is often referred to as unit testing. The second frequently goes by the term property testing or sometimes fuzz testing.

Both unit testing and property testing have some drawbacks. With unit testing you have to write fairly tedious boilerplate of listing by hand all the values you want to test with. Wit…

As almost everyone with significant experience managing production software systems should know, backwards compatibility is incredibly important for any data that is persisted by an application. If you make a change to a data structure that is not backwards compatible with the existing serialized formats, your app will break as soon as it encounters the existing format. Even if you have 100% test coverage, your tests still might not catch this problem. It’s not a problem with your app at any single point in time, but a problem with how your app evolves over time.

One might think that wire formats which are only used for communication between components and not persisted in any way would not be susceptible to this problem. But these too can cause issues if a message is generated and a new version of the app is deployed before the the message is consumed. The longer the message remains in a queue, redis cache, etc the higher the chances of this occurring.