John Brinkman on form design

Archive for November, 2008

A while ago I was asked to code review a customer form. I enjoy doing this in order to get an understanding of what customers are trying to do with their forms and where their pain points are. In this case, I wanted to share some of the code review on my blog so that others can learn from the experience. This post is a miscellaneous collection of XFA scripting suggestions arising from one customer form.

Repetitive Code

The form had a large block of script that executed before print that looked like:

This pattern shown 3 times above was repeated 31 times in the method. A total of around 155 lines of code. Whew. I enjoy writing code as much as the next person, but this repetitive code is tedious and error prone. Added or removed a new subform? Don’t forget to update this script. Copy and pasted some script code? Hope you updated all the relevant pieces correctly.

The general pattern is that there are a bunch of fields called "MpIsInd" under nested subforms. When the value of this field is zero, hide the parent subform, otherwise make it visible. Here are two alternative approaches:

Solution 1: Use Recursion

We could define a method to recursively search for fields called "MpIsInd" and apply the logic to each one we find:

// Once we’ve found the field, we can exit early. return; } else if (vObject.className == "subform") { // Call this method recursively for each child. for (var i=0; i<vObject.nodes.length; i++) printPrep(vObject.nodes.item(i)); } } // Set visibility based on all the nested MplsInd descendents printPrep(Page1);

The end result is 21 lines of script instead of 155 — and a form that is easier to maintain.

Solution2: Use wildcards in SOM

But recursion is not for everyone. It certainly would not come easily to a novice script writer

The form author could also have used SOM expressions to find all the MplsInd fields. We are used to using the wildcard character (*) inside braces to indicate "all occurrences". e.g. po.item[*].subtotal. I suspect it is not widely known that the wildcard character may also appear in place of an element name to indicate "all elements". Using the wildcard this way, our SOM expression to find all MplsInd grandchildren of Page1 is : Page1.*.MplsInd.

Given that these fields occur as either grandchildren of Page1 or as great-grandchildren, we can use two calls to resolveNodes() to find them:

Resolving SOM expressions involves what we call "scoped matching". In this case, the starting context is the Cell1 button (the field hosting the script). When asked to find Table1, we look for a match by looking at the children of Cell1. If not found, we expand the scope of the search by moving up the hierarchy and checking the parent element: Row1 and the children of Row1. This continues until we encounter Table1. Note that because there were two Row1 elements in the hierarchy, we differentiated them by including Table1 in the SOM expression. Note also that we did not worry about which instance of Row1 was returned. The scoped match will find the direct ancestor before any of its siblings.

The replacement code found the instance manager using its name: _Row1 whereas the original used Row1.instanceManager. Using _Row1 is a better practise, because when there are zero instances of a
subform, the search for Row1 will fail, whereas _Row1 is always available. The naming convention for instance managers is the subform name prefixed with an underscore.

Notice that this script is as efficient as possible when it loops through the children in a list. It declares a variable to hold the size of the list, and uses that variable in the "for loop". The advantage of doing this is that in a "for loop" the condition gets evaluated for each iteration of the loop. It is more efficient to reference a JavaScript variable than it is to evaluate a list.length property. If you have a "for loop" in an area of performance critical code, this is good practise.

Notice another clever part of the script — it makes sure not to lock the current field.

Unfortunately, it was hard to tell how the script was actually used, because it was not referenced anywhere on the form. But a couple of observations:

As of Reader 9.0, the access property is now available on subforms. If your target version is Reader 9, you can code: subform.access = "readOnly" and the result will lock all the descendant fields of the subform. Setting this property on the root subform would lock the entire form — in one JavaScript statement.

Removing a Subform

The form has buttons to remove subform instances with code that looks like this:

For starters, we can simplify the SOM expressions as described above. But the other problem is that this particular Row1 subform is defined to have a minimum of one instance. This means that if the user clicks on the button to remove the last instance of Row1, Reader will generate an error. The way Reader handles JavaScript errors is to write them to the JavaScript console. The end user does not get an indication that there was an error. But in the console you will see:

GeneralError: Operation failed. XFAObject.removeInstance:1:XFA:fields[0]:Page1[0]:RevSec[0]:RevSec2[0]:Table1[0]:Row1[0]:Table2[0]:Row1[0]:Cell2[0]:click The element [min] has violated its allowable number of occurrences.

This version of the script protects against attempting to remove beyond the minimum number of instances:

Here is the last (for now) in the series on how to validate templates in a user-friendly manner. I have attached a new and improved sample (with data).

What is new in this version:

An email submit button that becomes active only when there are no errors on the form

A listbox field that shows all the errors on the field. When you enter into the list field, the field corresponding to the error gets highlighted. Try selecting different elements in the list.

But more importantly, this sample form now holds a toolkit of functions that allow you to take control of the way Reader validates form fields.

Behind the scenes on the script side, there is a fair bit more in the toolbox. My goal is that you can re-used these script objects in your forms and keep your form designs as clean and simple as possible. If you look at the form you will see that the vast majority of script is inside the script objects. The fields and subforms on the form itself have minimal amounts of script.

An added benefit of following this particular form development methodology is that is should be very consistent with future enhancements we add to XFA and Reader.

Here is a summary of the script functions that you can use in your form development:

/** * setStatus(vContainer, vStatus) * Call this method as the last line of your validation script. * This function does two things: * 1) highlights or resets the field according to whether it is valid or not * 2) If invalid, logs the error * * @param vContainer — the container we are validating. * If a subform, process recursively. * @param validStatus — true | false * @return the status for the validation script to use * (for now hardcoded to "true") */

/** * formHasErrors() * Useful for changing the state of form objects depending on if there are * validation errors or not. * Place this call inside a validation or calculation script and make sure * you have called registerListener(this) from your initialization event. * @return true if there are errors. False otherwise. */

/** * registerListener(vListener) * Call this from you initialization event if you want to be able to * call formHasErrors() in your calculation or validation events. * @param vListener — The object that cares about whether there are errors. * We’ll keep track of all the listeners. Every time a field changes valid * state (becomes valid or invalid), we will loop through all listeners and * force their calculate and validate scripts to run. * If a listener is a choiceList object, we’ll synchronize the choicelist * entries with field errors. * When a choicelist has zero entries, we hide it. * @return void */

/* * setFieldMandatory(vObject, vMandatoryState, vForce) * Mark a field or exclusion group as being mandatory. * When calling this from a validation script, * be sure to also call setStatus() * * @param vObject — the field or exclusion group that we’re marking * @param vMandatoryState — true or false * @return boolean — true if the state changed */

/** * clearErrList() * Call this method before removing a subform or re-ordering subforms. * Our tracking is based on storing SOM expressions. * The SOM expression for a field can change if the order of its parent * subform changes. * After clearing the list and removing/moving subforms, * call xfa.form.execValidate() to rebuild the list. * Note that it is not necessary to call this method when appending new subforms. */

Then there are a couple of methods you might choose to modify for your own use — in case you do not like the way the sample highlights invalid fields:

/** * highlight(vObject) * Highlight a field (or exclusion group or subform) to indicate that it * has a validation error * @param vObject — the subform/field/exclusion group to highlight */

/** * unhighlight(object) * reset the form field/subform/exclusion group to the state it was in * the template. * @param object — the subform/field/exclusion group to highlight */

The one thing that does not work back to Reader 7 is the button script that sets focus on an error field. xfa.host.setFocus() came later.

But the rest of script functionality should work fine in Reader 7. The hardest part about making the scripts work in Reader 7 was that the convenience methods for manipulating choice lists are not available there. The script has to manipulate the XML structures directly.

Continuing from the previous post, we are looking at the problem of validating fields without inundating the user with message boxes during their form session in Adobe Reader — and without centralizing all the validation logic.

In the previous post, we established a design pattern for highlighting fields where a script validation fails. In this post we will deal with mandatory (required) fields. Today, when you mark fields as mandatory, then any time your form is validated you will get an error message for each missing field. We can do better.

author the form fields using the standard XFA “required field” settings

At runtime disable the “required field” setting and store our own mandatory flag elsewhere in the field

Use our setStatus() function during validation to monitor whether fields are populated or not.

Changes from the previous sample:

Marked various fields as being mandatory

Additional logic in scValidate.setStatus() to cause empty mandatory fields to show up as invalid

A new script function: scValidate.setFieldMandatory()

Storing the mandatory state in a field involves a technique described in a previous post where we store a variable under a field.desc element.

Once we have logic in the form that stores our mandatory state, we let the XFA processor believe that no fields on the form are required. This way we don’t get any warnings from Acrobat/Reader, but we graphically modify the fields to highlight them for the user.

The only deviation from standard form design is that for mandatory fields you need to add a call to utility.setStatus() in the validation script – even if there is no script validation required. i.e. if you mark a field as mandatory, you need to add this validation script:

Conditionally Mandatory

On the sample form, when the payment type field choice is "Direct Deposit", there are three more fields that need to be filled in. If the payment type is any other choice, these fields do not need to be filled in.

Normally we toggle a field’s mandatory status by setting the property: field.mandatory. But now that our design pattern for mandatory fields has co-opted the XFA mandatory mechanism, we will use one of our new global methods: scValidate.setFieldMandatory(vField, vMandatoryState).

For this to work, the form defines validation scripts on the financialInstitution, branchNumber and accountNumber fields that look like this:

Updated Exclusion Groups

If you are looking at the script code you will notice that I have included the exclusion group script objects that were defined in a previous post. The form has placed the direct deposit subform in an exclusion group with the other payment types. I have modified those scripts so that they work with the validation framework. (You will notice I have also broken them out into separate script objects with a new naming scheme).

When an exclusion subform has not yet reached its minimum number of entries, the subform gets highlighted instead of the individual fields in the subform.

The call to scGroup.setMinAndMax() now includes an optional validation message parameter. This is needed because designer does not yet expose a UI for assigning a validation message to a subform.

Note that this sample now introduces the concept of an invalid subform. If the user has not filled in all the credit card fields, then we mark the subform as invalid — similar to the case where the subform exclusion group did not reach its minimum number of entries. The script to do this looks like:

The validation button

The samples have also updated the logic for the button to validate the form. Now when there are no errors, the button caption will be updated to say: "no errors" (and will be made read-only). This works as long as you do not change the name of the button from : "checkValid". You might prefer a variation on this where the button is hidden when there are no errors.

Backward compatibility

We would like to make it easier you to control validation message handling in a future version of Reader, but for the here and now, you are likely targeting Reader 7/8/9 and need an immediate solution. You can take these script objects, copy them into new forms and use them in forms compatible as far back as Reader 7.

Time to tackle my pet peeve. The aspect of customer written script that concerns me most is how customers validate field data. The number one reason for script bloat is that our users bypass the standard XFA/Reader validation framework.

Of course, there is a reason why script writers bypass validation. The problem is that each validation message appears in an individual dialog box. In some cases when a form is validated, the user may have to dismiss dozens of dialogs. Most users would prefer no dialog boxes at all – just show the invalid fields graphically. This is a problem we would like to solve in a product release, but in the mean time, you need forms that work in previous versions of Reader. Starting today I will tackle this topic in a series of blog entries.

Current Practise

Form authors currently work around the “validation message” issue by moving their validation logic outside the prescribed validation mechanisms. They typically place all the validation logic for their form under the click event of a “validate” button. They end up with a very large chunk of script that look like:

Best Practise

There is a button that checks overall form validation status and if there are errors, places the user in the first invalid field

The solution has 3 parts:

Global Script Objects

Field validation logic

Validation button

Global Scripts

There are six global scripts under the utilities script object. The ones that the form author should be aware of:

setStatus(vField, vStatus)

Called by field validation logic to handle the validation logic for a field.

highlightField(vObject)

Changes the appearance of a field (or exclusion group) to indicate that it is invalid. In this case we change the border color. Anyone who wants to re-use this mechanism can modify this script to get the visual effect they prefer. Note that for this sample, the highlighting works best if fields do not have a raised border. It also works best for dynamic forms. It should be re-specified if used for a static form.

unhighlightField(vObject)

Reverts the field to the appearance defined in the template. If you modified code in highlightField(), you need to make the corresponding change in this function.

There are a three more scripts that are used internally:

registerFieldStatus(vField, vStatus)

Saves a list of invalid fields in a form variable.

getMessage(vObject)

Retrieves the validation message from a field object.

findTemplateField(vField)

Finds the template field that we use to reset visual properties.

Field Definition

Several fields on the sample form have validation logic. The general form of the validation script looks like this:

There are several interesting scripting validation techniques that can be demonstrated in the context of a Canadian postal code field. While the subject domain is fairly specific, the techniques used are applicable to other field validations.

The challenge is to get an accurate and user-friendly data capture experience for a postal code. Some of the considerations:

Make sure the field value conforms to the rules for Canadian postal codes (use regular expressions for validating)

Make sure that error messages are as specific as possible to assist better user input (customize validation error messages)

Make sure the user can optionally key in a space inside their postal code (use an edit picture clause)

Make sure the postal code is entered in upper case (modify keystrokes on the change event)

Make sure the postal code value is consistent with other address fields: province and city (inter-field validation)

Canadian Postal Code Rules

The default way to validate a postal code is to use a validation picture clause: text{A9A 9A9}. However, this picture allows many illegal postal codes. For example, the meta character “A” in a picture clause allows any Unicode letter value – whereas a postal code letter is far more restricted:

The first character of a Canadian postal code corresponds to a geographic region and is limited to the set of characters: ABCEGHJKLMNPRSTVXY

The third and fifth characters may not include the letters D, F, I, O, Q or U (because they are hard for OCR software to deal with).

In order to implement these restrictions, we can use regular expressions. The JavaScript string class has a match() method that searches for a pattern. For example, the code to validate the first character is:

Note that you might choose to store error messages as form variables. That way they will be examined by Designer’s spell checker.

The Optional Space

The user ought to be able to enter their data with or without a space – but we want to store it consistently – without the space. To accomplish this, use an edit picture clause: text{OOO OOO}

Edit picture clauses are used by Reader to:

format the raw value for editing when the user enters the field

parse the edited value to set the raw value when the user exits the field

For example, a raw value: A2A2A2 will be displayed as A2A 2A2 when the user enters the field. When they exit the field, both A2A2A2 and A2A 2A2 will parse into A2A2A2 successfully. Note that I used the meta character “O” (letter or digit). This is so that if the user enters invalid characters, we still parse/format the space correctly.

Naturally, we also use a display picture clause to render with the space: text{A9A 9A9}

Upper Case Only

The easiest way to force the value to upper case is to trap the characters as the user enters them and convert them as they type. We do this with a very simple change event:

Because we know that the input data is in upper case, it makes our validation logic cleaner, as it needs only be concerned with the upper case variations.

Consistency with City and Province

The first character of the postal code determines the geographic region. We can assign the province based on the postal code. In a couple of cases (M and H) we can also assign the value for the city (Toronto and Montréal).

More Considerations

While we now have a better-than-average postal code validation, it does not ensure100% correctness. Since only about 12% of the possible 7.2 million postal code variations are currently allocated, it is still pretty easy to enter a non-existent postal code.

Because this validation script is as specific as possible, it does mean that the script may need to be updated as the postal code rules evolve (The Canadian postal code rules were modified after Nunavut was added in 1999).

A note today on a specific coding practice I have seen in a number of customer forms. The task is to write a loop that examines each subform instance in a repeating subform definition. The pattern I have seen is where users construct SOM expressions inside a loop and call resolveNode() to find the result. I will show some alternate coding patterns that are easier to write and more efficient to execute. I have included a sample that has five buttons – one illustrating each variation. The sample looks at each expense row in an expense report and ensure that the description field is populated.

Variation 2: Use the "all" property

In SOM we have the "[*]" notation. "expense[*]" means "all the instances of the expense subform". Unfortunately, we cannot expose the "[*]" notation in JavaScript. Instead, we introduced the "all" property. Using "expense.all", the script can be written without constructing SOM expressions:

Variation 3: Use resolveNodes()

Both previous solutions assume you have at least one instance of the expense subform to start with. In the case where we might not have any occurrences, we can use resolveNodes().

resolveNode() returns a node object . We use it when we expect our SOM expression to return a single node. Use resolveNodes() (plural) to return a list object when your SOM expression may return multiple objects. In this example, use the expression: "expenses.expense[*].description" to return all the description fields:

Other Issues

Validating outside the field

I should have mentioned this disclaimer earlier: I do not like form design patterns where validation logic lives outside the individual fields. I understand the motivation – the lack of control over validation messaging. But there are better design patterns. However, that is a topic for a whole series of blog entries.

Anchor resolveNode

Note the difference between "this.resolveNode()" and "xfa.resolveNode()". The SOM expression passed to resolveNode() is evaluated relative to the object hosting the call. The original customer version used:

xfa.resolveNode("expenseReport.expenses.expense[0].description");

Whereas if the call is anchored under "this" (the validation button context) the SOM expression can be shortened to:

this.resolveNode("expenses.expense[0].description");

Checking empty

The customer version of the script checks:

if (vDesc == "" || vDesc == null)

Unless you have customized your null handling, this check is sufficient: