Forms and validators

There are four distinct ways to build forms in web2py:

FORM provides a low-level implementation in terms of HTML helpers. A FORM object can be serialized into HTML and is aware of the fields it contains. A FORM object knows how to validate submitted form values.

SQLFORM provides a high-level API for building create, update and delete forms from an existing database table.

SQLFORM.factory is an abstraction layer on top of SQLFORM in order to take advantage of the form generation features even if there is no database present. It generates a form very similar to SQLFORM from the description of a table but without the need to create the database table.

CRUD methods. These are functionally equivalent to SQLFORM and are based on SQLFORM, but provide a more compact notation.

All these forms are self-aware and, if the input does not pass validation, they can modify themselves and add error messages. The forms can be queried for the validated variables and for error messages that have been generated by validation.

Arbitrary HTML code can be inserted into or extracted from the form using helpers.

FORM and SQLFORM are helpers and they can be manipulated in as similar way as the DIV. For example you can set a form style:

form = SQLFORM(..)
form['_style']='border:1px solid black'

FORM

form

accepts

formname

Consider as an example a test application with the following "default.py" controller:

This is a regular HTML form that asks for the user's name. When you fill the form and click the submit button, the form self-submits, and the variable request.vars.name and its value is displayed at the bottom.

You can generate the same form using helpers. This can be done in the view or in the action. Since web2py processed the form in the action, it is OK to define the form in the action.

In the action, we added the requires=IS_NOT_EMPTY() validator for the input field "name".

In the action, we added a call to form.accepts(..)

In the view, we are printing form.vars and form.errors as well as the form and request.vars.

All the work is done by the accepts method of the form object. It filters the request.vars according to the declared requirements (expressed by validators). accepts stores those variables that pass validation into form.vars. If a field value does not meet a requirement, the failing validator returns an error and the error is stored in form.errors. Both form.vars and form.errors are gluon.storage.Storage objects similar to request.vars. The former contains the values that passed validation, for example:

The meaning of the optional parameters is explained in the next sub-sections.

The first argument can be request.vars or request.get_vars or request.post_vars or simply request. The latter is equivalent to accepting as input the request.post_vars.

The accepts function returns True if the form is accepted and False otherwise. A form is not accepted if it has errors or when it has not been submitted (for example, the first time it is shown).

Here is how this page looks the first time it is displayed:

Here is how it looks upon invalid submission:

Here is how it looks upon a valid submission:

The process and validate methods

A shortcut for

form.accepts(request.post_vars,session,...)

is

form.process(...).accepted

the latter does not need the request and session arguments (although you can specify them optionally). it also differs from accepts because it returns the form itself. Internally process calls accepts and passes its arguments to it. The value returned by accepts is stored in form.accepted.

The process function takes some extra argument that accepts does not take:

message_onsuccess

onsuccess: if equal to 'flash' (default) and the form is accepted it will flash the above `message_onsuccess- message_onfailure- onfailure: if equal to 'flash' (default) and the form fails validation, it will flash the above `message_onfailure

next the user to redirect after the form is accepted.

onsuccess and onfailure can be functions like lambda form: do_something(form).

form.validate(...)

is a shortcut for

form.process(...,dbio=False).accepted

Hidden fields

When the above form object is serialized by {{=form}}, and because of the previous call to the accepts method, it now looks like this:

Notice the presence of two hidden fields: "_formkey" and "_formname". Their presence is triggered by the call to accepts and they play two different and important roles:

The hidden field called "_formkey" is a one-time token that web2py uses to prevent double submission of forms. The value of this key is generated when the form is serialized and stored in the session. When the form is submitted this value must match, or else accepts returns False without errors as if the form was not submitted at all. This is because web2py cannot determine whether the form was submitted correctly.

The hidden field called "_formname" is generated by web2py as a name for the form, but the name can be overridden. This field is necessary to allow pages that contain and process multiple forms. web2py distinguishes the different submitted forms by their names.

Optional hidden fields specified as FORM(..,hidden=dict(...)).

The role of these hidden fields and their usage in custom forms and pages with multiple forms is discussed in more detail later in the chapter.

If the form above is submitted with an empty "name" field, the form does not pass validation. When the form is serialized again it appears as:

Notice the presence of a DIV of class "error" in the serialized form. web2py inserts this error message in the form to notify the visitor about the field that did not pass validation. The accepts method, upon submission, determines that the form is submitted, checks whether the field "name" is empty and whether it is required, and eventually inserts the error message from the validator into the form.

The base "layout.html" view is expected to handle DIVs of class "error". The default layout uses jQuery effects to make errors appear and slide down with a red background. See Chapter 11 for more details.

keepvalues

keepvalues

The optional argument keepvalues tells web2py what to do when a form is accepted and there is no redirection, so the same form is displayed again. By default the form is cleared. If keepvalues is set to True, the form is pre-populated with the previously inserted values. This is useful when you have a form that is supposed to be used repeatedly to insert multiple similar records. If the dbio argument is set to False, web2py will not perform any DB insert/update after accepting form. If hideerror is set to True and the form contains errors, these will not be displayed when the form is rendered (it will be up to you to display them from form.errors somehow. The onvalidation argument is explained below.

onvalidation

The onvalidation argument can be None or can be a function that takes the form and returns nothing. Such a function would be called and passed the form, immediately after validation (if validation passes) and before anything else happens. The purpose of this function is multifold. It can be used, for example, to perform additional checks on the form and eventually add errors to the form. It can also be used to compute the values of some fields based on the values of other fields. It can be used to trigger some action (like sending an email) before a record is created/updated.

Detect record change

When filling a form to edit a record there is a small probability that another user may concurrently be editing the same record. So when we save the record we want to check for possible conflicts. This can be done:

Forms and redirection

The most common way to use forms is via self-submission, so that the submitted field variables are processed by the same action that generated the form. Once the form is accepted, it is unusual to display the current page again (something we are doing here only to keep things simple). It is more common to redirect the visitor to a "next" page.

In order to set a flash on the next page instead of the current page you must use session.flash instead of response.flash. web2py moves the former into the latter after redirection. Note that using session.flash requires that you do not session.forget().

Multiple forms per page

The content of this section applies to both FORM and SQLFORM objects. It is possible to have multiple forms per page, but you must allow web2py to distinguish them. If these are derived by SQLFORM from different tables, then web2py gives them different names automatically; otherwise you need to explicitly give them different form names. Here is an example:

When the visitor submits an empty form1, only form1 displays an error; if the visitor submits an empty form2, only form2 displays an error message.

Sharing forms

The content of this section applies to both FORM and SQLFORM objects. What we discuss here is possible but not recommended, since it is always good practice to have forms that self-submit. Sometimes, though, you don't have a choice, because the action that sends the form and the action that receives it belong to different applications.

It is possible to generate a form that submits to a different action. This is done by specifying the URL of the processing action in the attributes of the FORM or SQLFORM object. For example:

Notice that since both "page_one" and "page_two" use the same form, we have defined it only once by placing it outside of all the actions, in order not to repeat ourselves. The common portion of code at the beginning of a controller gets executed every time before giving control to the called action.

Since "page_one" does not call process (nor accepts), the form has no name and no key, so you must pass session=None and set formname=None in process, or the form will not validate when "page_two" receives it.

SQLFORM

We now move to the next level by providing the application with a model file:

The automatically generated form is more complex than the previous low-level form. First of all, it contains a table of rows, and each row has three columns. The first column contains the field labels (as determined from the db.person), the second column contains the input fields (and eventually error messages), and the third column is optional and therefore empty (it can be populated with the fields in the SQLFORM constructor).

All tags in the form have names derived from the table and field name. This allows easy customization of the form using CSS and JavaScript. This capability is discussed in more detail in Chapter 11.

More important is that now the accepts method does a lot more work for you. As in the previous case, it performs validation of the input, but additionally, if the input passes validation, it also performs a database insert of the new record and stores in form.vars.id the unique "id" of the new record.

A SQLFORM object also deals automatically with "upload" fields by saving uploaded files in the "uploads" folder (after having them renamed safely to avoid conflicts and prevent directory traversal attacks) and stores their names (their new names) into the appropriate field in the database. After the form has been processed, the new filename is available in form.vars.fieldname (i.e., it replaces the cgi.FieldStorage object in request.vars.fieldname), so you can easily reference the new name right after upload.

A SQLFORM displays "boolean" values with checkboxes, "text" values with textareas, values required to be in a definite set or a database with drop-boxes, and "upload" fields with links that allow users to download the uploaded files. It hides "blob" fields, since they are supposed to be handled differently, as discussed later.

The SQLFORM constructor allows various customizations, such as displaying only a subset of the fields, changing the labels, adding values to the optional third column, or creating UPDATE and DELETE forms, as opposed to INSERT forms like the current one. SQLFORM is the single biggest time-saver object in web2py.

The class SQLFORM is defined in "gluon/sqlhtml.py". It can be easily extended by overriding its xml method, the method that serializes the objects, to change its output.

The optional second argument turns the INSERT form into an UPDATE form for the specified record (see next subsection).

showid

delete_label

id_label

submit_button

If deletable is set to True, the UPDATE form displays a "Check to delete" checkbox. The value of the label if this field is set via the delete_label argument.

submit_button sets the value of the submit button.

id_label sets the label of the record "id"

The "id" of the record is not shown if showid is set to False.

fields is an optional list of field names that you want to display. If a list is provided, only fields in the list are displayed. For example:

fields = ['name']

labels is a dictionary of field labels. The dictionary key is a field name and the corresponding value is what gets displayed as its label. If a label is not provided, web2py derives the label from the field name (it capitalizes the field name and replaces underscores with spaces). For example:

linkto and upload are optional URLs to user-defined controllers that allow the form to deal with reference fields. This is discussed in more detail later in the section.

readonly. If set to True, displays the form as readonly

comments. If set to False, does not display the col3 comments

ignore_rw. Normally, for a create/update form, only fields marked as writable=True are shown, and for readonly forms, only fields marked as readable=True are shown. Setting ignore_rw=True causes those constraints to be ignored, and all fields are displayed. This is mostly used in the appadmin interface to display all fields for each table, overriding what the model indicates.

formstyle

formstyle determines the style to be used when serializing the form in html. It can be "table3cols" (default), "table2cols" (one row for label and comment, and one row for input), "ul" (makes an unordered list of input fields), "divs" (represents the form using css friendly divs, for arbitrary customization). formstyle can also be a function that takes (record_id, field_label, field_widget, field_comment) as attributes and returns a TR() object.

buttons

is a list of INPUTs or TAG.BUTTONs (though technically could be any combination of helpers) that will be added to a DIV where the submit button would go.

separator

separator sets the string that separates form labels from form input fields.

Optional attributes are arguments starting with underscore that you want to pass to the FORM tag that renders the SQLFORM object. Examples are:

_action = '.'
_method = 'POST'

There is a special hidden attribute. When a dictionary is passed as hidden, its items are translated into "hidden" INPUT fields (see the example for the FORM helper in Chapter 5).

form = SQLFORM(....,hidden=...)

causes the hidden fields to be passed with the submission, no more, no less. form.accepts(...) is not intended to read the received hidden fields and move them into form.vars. The reason is security. hidden fields can be tampered with. So you have to do explicitly move hidden fields from the request to the form:

form.vars.a = request.vars.a
form = SQLFORM(..., hidden=dict(a='b'))

SQLFORM and insert/update/delete

SQLFORM creates a new db record when the form is accepted. Assuming form=SQLFORM(db.test), then the id of the last-created record will be accessible in myform.vars.id.

delete record

If you pass a record as the optional second argument to the SQLFORM constructor, the form becomes an UPDATE form for that record. This means that when the form is submitted the existing record is updated and no new record is inserted. If you set the argument deletable=True, the UPDATE form displays a "check to delete" checkbox. If checked, the record is deleted.

If a form is submitted and the delete checkbox is checked the attribute
form.deleted is set to True.

You can modify the controller of the previous example so that when we pass an additional integer argument in the URL path, as in:

/test/default/display_form/2

and if there is a record with the corresponding id, the SQLFORM generates an UPDATE/DELETE form for the record:

Line 2 finds the record and line 3 makes an UPDATE/DELETE form. Line 4 does all the corresponding form processing.

An update form is very similar to a create form except that it is pre-populated with the current record and it previews images. By default deletable = True which means the update form will display a "delete record" option.

Edit forms also contain a hidden INPUT field with name="id" which is used to identify the record. This id is also stored server-side for additional security and, if the visitor tampers with the value of this field, the UPDATE is not performed and web2py raises a SyntaxError, "user is tampering with form".

When a Field is marked with writable=False, the field is not shown in create forms, and it is shown readonly in update forms. If a field is marked as writable=False and readable=False, then the field is not shown at all, not even in update forms.

Forms created with

form = SQLFORM(...,ignore_rw=True)

ignore the readable and writable attributes and always show all fields. Forms in appadmin ignore them by default.

Forms created with

form = SQLFORM(table,record_id,readonly=True)

always show all fields in readonly mode, and they cannot be accepted.

SQLFORM in HTML

There are times when you want to use SQLFORM to benefit from its form generation and processing, but you need a level of customization of the form in HTML that you cannot achieve with the parameters of the SQLFORM object, so you have to design the form using HTML.

Notice that the action does not return the form because it does not need to pass it to the view. The view contains a form created manually in HTML. The form contains a hidden field "_formname" that must be the same formname specified as an argument of accepts in the action. web2py uses the form name in case there are multiple forms on the same page, to determine which one was submitted. If the page contains a single form, you can set formname=None and omit the hidden field in the view.

form.accepts will look inside response.vars for data that matches fields in the database table db.person. These fields are declared in the HTML in the format

<input name="field_name_goes_here" />

Note that in the example given, the form variables will be passed on the URL as arguments. If this is not desired, the POST protocol will have to be specified. Note furthermore, that if upload fields are specified, the form will have to be set up to allow this. Here, both options are shown:

<form enctype="multipart/form-data" method="post">

SQLFORM and uploads

Fields of type "upload" are special. They are rendered as INPUT fields of type="file". Unless otherwise specified, the uploaded file is streamed in using a buffer, and stored under the "uploads" folder of the application using a new safe name, assigned automatically. The name of this file is then saved into the field of type uploads.

When you insert a new record, the form allows you to browse for a file. Choose, for example, a jpg image. The file is uploaded and stored as:

applications/test/uploads/person.image.XXXXX.jpg

"XXXXXX" is a random identifier for the file assigned by web2py.

content-disposition

Notice that, by default, the original filename of an uploaded file is b16encoded and used to build the new name for the file. This name is retrieved by the default "download" action and used to set the content disposition header to the original filename.

Only its extension is preserved. This is a security requirement since the filename may contain special characters that could allow a visitor to perform directory traversal attacks or other malicious operations.

The new filename is also stored in form.vars.image.

When editing the record using an UPDATE form, it would be nice to display a link to the existing uploaded file, and web2py provides a way to do it.

If you pass a URL to the SQLFORM constructor via the upload argument, web2py uses the action at that URL to download the file. Consider the following actions:

which contains a link to allow downloading of the uploaded file, and a checkbox to remove the file from the database record, thus storing NULL in the "image" field.

Why is this mechanism exposed? Why do you need to write the download function? Because you may want to enforce some authorization mechanism in the download function. See Chapter 9 for an example.

Normally uploaded files are stored into "app/uploads" but you can specify an alternate location:

Field('image', 'upload', uploadfolder='...')

In most operating system, accessig the file system can become slow when there are many files in the same folder. If you plan to upload more than 1000 files you can ask web2py to organize the uploads in subfolders:

Field('image', 'upload', uploadseparate=True)

Storing the original filename

web2py automatically stores the original filename inside the new UUID filename and retrieves it when the file is downloaded. Upon download, the original filename is stored in the content-disposition header of the HTTP response. This is all done transparently without the need for programming.

Occasionally you may want to store the original filename in a database field. In this case, you need to modify the model and add a field to store it in:

Notice that the SQLFORM does not display the "image_filename" field. The "display_form" action moves the filename of the request.vars.image into the form.vars.image_filename, so that it gets processed by accepts and stored in the database. The download function, before serving the file, checks in the database for the original filename and uses it in the content-disposition header.

autodelete

autodelete

The SQLFORM, upon deleting a record, does not delete the physical uploaded file(s) referenced by the record. The reason is that web2py does not know whether the same file is used/linked by other tables or used for other purpose. If you know it is safe to delete the actual file when the corresponding record is deleted, you can do the following:

A person has dogs, and each dog belongs to an owner, which is a person. The dog owner is required to reference a valid db.person.id by '%(name)s'.

Let's use the appadmin interface for this application to add a few persons and their dogs.

When editing an existing person, the appadmin UPDATE form shows a link to a page that lists the dogs that belong to the person. This behavior can be replicated using the linkto argument of the SQLFORM. linkto has to point to the URL of a new action that receives a query string from the SQLFORM and lists the corresponding records. Here is an example:

There is a link called "dog.owner". The name of this link can be changed via the labels argument of the SQLFORM, for example:

labels = {'dog.owner':"This person's dogs"}

If you click on the link you get directed to:

/test/default/list_records/dog?query=dog.owner%3D5

"list_records" is the specified action, with request.args(0) set to the name of the referencing table and request.vars.query set to the SQL query string. The query string in the URL contains the value "dog.owner=5" appropriately url-encoded (web2py decodes this automatically when the URL is parsed).

You can easily implement a very general "list_records" action as follows:

When a set of records is returned by a select and serialized in a view, it is first converted into a SQLTABLE object (not the same as a Table) and then serialized into an HTML table, where each field corresponds to a table column.

Pre-populating the form

It is always possible to pre-populate a form using the syntax:

form.vars.name = 'fieldvalue'

Statements like the one above must be inserted after the form declaration and before the form is accepted, whether or not the field ("name" in the example) is explicitly visualized in the form.

Adding extra form elements to SQLFORM

Sometimes you may wish to add an extra element to your form after it has been created. For example, you may wish to add a checkbox which confirms the user agrees with the terms and conditions of your website:

The variable my_extra_element should be adapted to the formstyle. In this example, the default formstyle='table3cols' has been assumed.

After submission, form.vars.agree will contain the status of the checkbox, which could then be used in an onvalidation function, for instance.

SQLFORM without database IO

There are times when you want to generate a form from a database table using SQLFORM and you want to validate a submitted form accordingly, but you do not want any automatic INSERT/UPDATE/DELETE in the database. This is the case, for example, when one of the fields needs to be computed from the value of other input fields. This is also the case when you need to perform additional validation on the inserted data that cannot be achieved via standard validators.

In the case of a table including an "upload"-type field ("fieldname"), both process(dbio=False) and validate() deal with the storage of the uploaded file as if process(dbio=True), the default behavior.

The name assigned by web2py to the uploaded file can be found in:

form.vars.fieldname

SQLFORM.factory

There are cases when you want to generate forms as if you had a database table but you do not want the database table. You simply want to take advantage of the SQLFORM capability to generate a nice looking CSS-friendly form and perhaps perform file upload and renaming.

This can be done via a form_factory. Here is an example where you generate the form, perform validation, upload a file and store everything in the session :

You need to use an underscore instead of a space for field labels, or explicitly pass a dictionary of labels to form_factory, as you would for a SQLFORM. By default SQLFORM.factory generates the form using html "id" attributes generated as if the form was generated from a table called "no_table". To change this dummy table name, use the table_name attribute for the factory:

form = SQLFORM.factory(...,table_name='other_dummy_name')

Changing the table_name is necessary if you need to place two factory generated forms in the same table and want to avoid CSS conflicts.

One form for multiple tables

It often happens that you have two tables (for example 'client' and 'address' which are linked together by a reference and you want to create a single form that allows to insert info about one client and its default address. Here is how:

Notice the SQLFORM.factory (it makes ONE form using public fields from both tables and inherits their validators too). On form accepts this does two inserts, some data in one table and some data in the other.

This only works when the tables don't have field names in common.

CRUD

CRUD

crud.create

crud.update

crud.select

crud.search

crud.tables

crud.delete

One of the recent additions to web2py is the Create/Read/Update/Delete (CRUD) API on top of SQLFORM. CRUD creates an SQLFORM, but it simplifies the coding because it incorporates the creation of the form, the processing of the form, the notification, and the redirection, all in one single function.

The first thing to notice is that CRUD differs from the other web2py APIs we have used so far because it is not already exposed. It must be imported. It also must be linked to a specific database. For example:

from gluon.tools import Crud
crud = Crud(db)

The crud object defined above provides the following API:

crud.tables

crud.create

crud.read

crud.update

crud.delete

crud.select

.

crud.tables() returns a list of tables defined in the database.

crud.create(db.tablename) returns a create form for table tablename.

crud.read(db.tablename, id) returns a readonly form for tablename and record id.

crud.update(db.tablename, id) returns an update form for tablename and record id.

crud.delete(db.tablename, id) deletes the record.

crud.select(db.tablename, query) returns a list of records selected from the table.

crud.search(db.tablename) returns a tuple (form, records) where form is a search form and records is a list of records based on the submitted search form.

To specify extra functions to be executed after standard validation procedures for crud.update forms:

crud.settings.update_onvalidation = StorageList()

To specify extra functions to be executed after completion of crud.create forms:

crud.settings.create_onaccept = StorageList()

To specify extra functions to be executed after completion of crud.update forms:

crud.settings.update_onaccept = StorageList()

To specify extra functions to be executed after completion of crud.update if record is deleted:

crud.settings.update_ondelete = StorageList()

To specify extra functions to be executed after completion of crud.delete:

crud.settings.delete_onaccept = StorageList()

To determine whether the "update" forms should have a "delete" button:

crud.settings.update_deletable = True

To determine whether the "update" forms should show the id of the edited record:

crud.settings.showid = False

To determine whether forms should keep the previously inserted values or reset to default after successful submission:

crud.settings.keepvalues = False

Crud always detects whether a record being edited has been modified by a third party in the time between the time when the form is displayed and the time when it is submitted. This behavior is equivalent to

form.process(detect_record_change=True)

and it is set in:

crud.settings.detect_record_change = True

and it can be changed/disabled by setting the variable to False.

You can change the form style by

crud.settings.formstyle = 'table3cols' or 'table2cols' or 'divs' or 'ul'

You can set the separator in all crud forms:

crud.settings.label_separator = ':'

You can add captcha to forms, using the same convention explained for auth, with:

Notice the line table.id.represent=... that tells web2py to change the representation of the id field and display a link instead to the page itself and passes the id as request.args(1) which turns the create page into an update page.

Record versioning

Both SQLFORM and CRUD provides a utility to version database records:

If you have a table (db.mytable) that needs full revision history you can just do:

form = SQLFORM(db.mytable, myrecord).process(onsuccess=auth.archive)

form = crud.update(db.mytable, myrecord, onaccept=auth.archive)

auth.archive defines a new table called db.mytable_archive (the name is derived from the name of the table to which it refers) and on updating, it stores a copy of the record (as it was before the update) in the created archive table, including a reference to the current record.

Because the record is actually updated (only its previous state is archived), references are never broken.

This is all done under the hood. Should you wish to access the archive table you should define it in a model:

There is nothing special about these fields and you may give them any name you like. They are filled before the record is archived and are archived with each copy of the record. The archive table name and/or reference field name can be changed like this:

the class of the INPUT tag is the same as the type of the field. This is very important for the jQuery code in "web2py_ajax.html" to work. It makes sure that you can only have numbers in "integer" and "double" fields, and that "time", "date" and "datetime" fields display the popup calendar/datepicker.

the id is the name of the class plus the name of the field, joined by one underscore. This allows you to uniquely refer to the field via, for example, jQuery('#mytable_myfield') and manipulate the stylesheet of the field or bind actions associated to the field events (focus, blur, keyup, etc.).

the name is, as you would expect, the field name.

Hide errors

hideerror

Occasionally, you may want to disable the automatic error placement and display form error messages in some place other than the default. That can be done easily.

In the case of FORM or SQLFORM, pass hideerror=True to the accepts method.

In the case of CRUD, set crud.settings.hideerror=True

You may also want to modify the views to display the error (since they are no longer displayed automatically).

Here is an example where the errors are displayed above the form and not in the form.

Normally validators are called automatically by the function accepts and process of a FORM or other HTML helper object that contains a form. They are called in the order in which they are listed.

One can also call validators explicitly for a field:

db.person.name.validate(value)

which returns a tuple (value,error) and error is None if no the value validates.

Built-in validators have constructors that take an optional argument:

IS_NOT_EMPTY(error_message='cannot be empty')

error_message allows you to override the default error message for any validator.

Here is an example of a validator on a database table:

db.person.name.requires = IS_NOT_EMPTY(error_message='fill this!')

where we have used the translation operator T to allow for internationalization. Notice that default error messages are not translated.

Mind that the only validators that can be used with list: type fields are:

IS_IN_DB(...,multiple=True)

IS_IN_SET(...,multiple=True)

IS_NOT_EMPTY()

IS_LIST_OF(...)

The latter can be used to apply any validator to the individual items in the list.

Validators

IS_ALPHANUMERIC

IS_ALPHANUMERIC

This validator checks that a field value contains only characters in the ranges a-z, A-Z, or 0-9.

requires = IS_ALPHANUMERIC(error_message='must be alphanumeric!')

IS_DATE

IS_DATE

This validator checks that a field value contains a valid date in the specified format. It is good practice to specify the format using the translation operator, in order to support different formats in different locales.

For the full description on % directives look under the IS_DATETIME validator.

IS_DATETIME

IS_DATETIME

This validator checks that a field value contains a valid datetime in the specified format. It is good practice to specify the format using the translation operator, in order to support different formats in different locales.

IS_IN_SET

IS_IN_SET

multiple

Checks that the field values are in a set:

requires = IS_IN_SET(['a', 'b', 'c'],zero=T('choose one'),
error_message='must be a or b or c')

The zero argument is optional and it determines the text of the option selected by default, an option which is not accepted by the IS_IN_SET validator itself. If you do not want a "choose one" option, set zero=None.

The zero option was introduced in revision (1.67.1). It did not break backward compatibility in the sense that it did not break applications but it did change their behavior since, before, there was no zero option.

The elements of the set must always be strings unless this validator is preceded by IS_INT_IN_RANGE (which converts the value to int) or IS_FLOAT_IN_RANGE (which converts the value to float). For example:

IS_IN_SET and Tagging

The IS_IN_SET validator has an optional attribute multiple=False. If set to True, multiple values can be stored in one field. The field should be of type list:integer or list:string. multiple references are handled automatically in create and update forms, but they are transparent to the DAL. We strongly suggest using the jQuery multiselect plugin to render multiple fields.

Note that when multiple=True, IS_IN_SET will accept zero or more values, i.e. it will accept the field when nothing has been selected. multiple can also be a tuple of the form (a,b) where a and b are the minimum and (exclusive) maximum number of items that can be selected respectively.

IS_LENGTH

IS_LENGTH

Checks if length of field's value fits between given boundaries. Works for both text and file inputs.

Its arguments are:

maxsize: the maximum allowed length / size (has default = 255)

minsize: the minimum allowed length / size

Examples: Check if text string is shorter than 33 characters:

INPUT(_type='text', _name='name', requires=IS_LENGTH(32))

Check if password string is longer than 5 characters:

INPUT(_type='password', _name='name', requires=IS_LENGTH(minsize=6))

Check if uploaded file has size between 1KB and 1MB:

INPUT(_type='file', _name='name', requires=IS_LENGTH(1048576, 1024))

For all field types except for files, it checks the length of the value. In the case of files, the value is a cookie.FieldStorage, so it validates the length of the data in the file, which is the behavior one might intuitively expect.

IS_LIST_OF

IS_LIST_OF

This is not properly a validator. Its intended use is to allow validations of fields that return multiple values. It is used in those rare cases when a form contains multiple fields with the same name or a multiple selection box. Its only argument is another validator, and all it does is to apply the other validator to each element of the list. For example, the following expression checks that every item in a list is an integer in the range 0-10:

requires = IS_LIST_OF(IS_INT_IN_RANGE(0, 10))

It never returns an error and does not contain an error message. The inner validator controls the error generation.

IS_LOWER

IS_LOWER

This validator never returns an error. It just converts the value to lower case.

requires = IS_LOWER()

IS_MATCH

IS_MATCH

This validator matches the value against a regular expression and returns an error if it does not match. Here is an example of usage to validate a US zip code:

This function only checks the URL's syntax. It does not check that the URL points to a real document, for example, or that it otherwise makes semantic sense. This function does automatically prepend 'http://' in front of a URL in the case of an abbreviated URL (e.g. 'google.ca').

If the parameter mode='generic' is used, then this function's behavior changes. It then rejects a URL string if any of the following is true:

The list of allowed schemes is customizable with the allowed_schemes parameter. If you exclude None from the list, then abbreviated URLs (lacking a scheme such as 'http') will be rejected.

The default prepended scheme is customizable with the prepend_scheme parameter. If you set prepend_scheme to None, then prepending will be disabled. URLs that require prepending to parse will still be accepted, but the return value will not be modified.

IS_URL is compatible with the Internationalized Domain Name (IDN) standard specified in RFC 3490[RFC3490] ). As a result, URLs can be regular strings or unicode strings. If the URL's domain component (e.g. google.ca) contains non-US-ASCII letters, then the domain will be converted into Punycode (defined in RFC 3492[RFC3492] ). IS_URL goes a bit beyond the standards, and allows non-US-ASCII characters to be present in the path and query components of the URL as well. These non-US-ASCII characters will be encoded. For example, space will be encoded as'%20'. The unicode character with hex code 0x4e86 will become '%4e%86'.

IS_SLUG

If check is set to True it check whether the validated value is a slug (allowing only alphanumeric characters and non-repeated dashes).

If check is set to False (default) it converts the input value to a slug.

IS_STRONG

IS_STRONG

Enforces complexity requirements on a field (usually a password field)

Example:

requires = IS_STRONG(min=10, special=2, upper=2)

where

min is minimum length of the value

special is the minimum number of required special characters special characters are any of the following !@#$%^&*(){}[]-+

upper is the minimum number of upper case characters

IS_IMAGE

IS_IMAGE

This validator checks if a file uploaded through the file input was saved in one of the selected image formats and has dimensions (width and height) within given limits.

It does not check for maximum file size (use IS_LENGTH for that). It returns a validation failure if no data was uploaded. It supports the file formats BMP, GIF, JPEG, PNG, and it does not require the Python Imaging Library.

Note: on displaying an edit form for a table including requires = IS_IMAGE(), a delete checkbox will NOT appear because to delete the file would cause the validation to fail. To display the delete checkbox use this validation:

requires = IS_EMPTY_OR(IS_IMAGE())

IS_UPLOAD_FILENAME

IS_UPLOAD_FILENAME

This validator checks if the name and extension of a file uploaded through the file input matches the given criteria.

It does not ensure the file type in any way. Returns validation failure if no data was uploaded.

Its arguments are:

filename: filename (before dot) regex.

extension: extension (after dot) regex.

lastdot: which dot should be used as a filename / extension separator: True indicates last dot (e.g., "file.tar.gz" will be broken in "file.tar" + "gz") while False means first dot (e.g., "file.tar.gz" will be broken into "file" + "tar.gz").

All three example values are equal, since addresses are converted to integers for inclusion check with following function:

number = 16777216 * IP[0] + 65536 * IP[1] + 256 * IP[2] + IP[3]

Examples:

Check for valid IPv4 address:

requires = IS_IPV4()

Check for valid private network IPv4 address:

requires = IS_IPV4(minip='192.168.0.1', maxip='192.168.255.255')

IS_LOWER

IS_LOWER

This validator never returns an error. It converts the value to lower case.

requires = IS_LOWER()

IS_UPPER

IS_UPPER

This validator never returns an error. It converts the value to upper case.

requires = IS_UPPER()

IS_NULL_OR

IS_NULL_OR

Deprecated, an alias for IS_EMPTY_OR described below.

IS_EMPTY_OR

IS_EMPTY_OR

Sometimes you need to allow empty values on a field along with other requirements. For example a field may be a date but it can also be empty. The IS_EMPTY_OR validator allows this:

requires = IS_EMPTY_OR(IS_DATE())

CLEANUP

CLEANUP

This is a filter. It never fails. It just removes all characters whose decimal ASCII codes are not in the list [10, 13, 32-127].

requires = CLEANUP()

CRYPT

CRYPT

This is also a filter. It performs a secure hash on the input and it is used to prevent passwords from being passed in the clear to the database.

requires = CRYPT()

If a key is not specified, it uses the MD5 algorithm. If a key is specified CRYPT uses the HMAC algorithm. The key may contain a prefix that determines the algorithm to use with HMAC, for example SHA512:

requires = CRYPT(key='sha512:thisisthekey')

This is the recommended syntax. The key has to be a unique string associated to the database used. The key can never be changed. If you lose the key the previously hashed values become useless.

The CRYPT validator hashed the input and this makes it somewhat special. If you need to validate a password field, before it is hash, you can use CRYPT in a list of validators but must make sure it is the last of the list so that it is called last. For example:

requires = [IS_STRONG(),CRYPT(key='sha512:thisisthekey')]

CRYPT also takes a min_length argument which defaults to zero.

Database validators

IS_NOT_IN_DB

It requires that when you insert a new person, his/her name is not already in the database, db, in the field person.name. As with all other validators this requirement is enforced at the form processing level, not at the database level. This means that there is a small probability that, if two visitors try to concurrently insert records with the same person.name, this results in a race condition and both records are accepted. It is therefore safer to also inform the database that this field should have a unique value:

It is enforced at the level of dog INSERT/UPDATE/DELETE forms. It requires that a dog.owner be a valid id in the field person.id in the database db. Because of this validator, the dog.owner field is represented as a dropbox. The third argument of the validator is a string that describes the elements in the dropbox. In the example you want to see the person %(name)s instead of the person %(id)s. %(...)s is replaced by the value of the field in brackets for each record.

The zero option works very much like for the IS_IN_SET validator.

The first argument of the validator can be a database connection or a DAL Set, as in IS_NOT_IN_DB. This can be useful for example when wishing to limit the records in the drop-down box. In this example, we use IS_IN_DB in a controller to limit the records dynamically each time the controller is called:

If you want the field validated, but you do not want a dropbox, you must put the validator in a list.

db.dog.owner.requires = [IS_IN_DB(db, 'person.id', '%(name)s')]

_and

Occasionally you want the drop-box (so you do not want to use the list syntax above) yet you want to use additional validators. For this purpose the IS_IN_DB validator takes an extra argument _and that can point to a list of other validators applied if the validated value passes the IS_IN_DB validation. For example to validate all dog owners in db that are not in a subset:

IS_IN_DB also takes a cache argument that works like the cache argument of select.

IS_IN_DB and Tagging

tags

multiple

The IS_IN_DB validator has an optional attribute multiple=False. If set to True multiple values can be stored in one field. This field should be of type list:reference as discussed in Chapter 6. An explicit example of tagging is discussed there. multiple references are handled automatically in create and update forms, but they are transparent to the DAL. We strongly suggest using the jQuery multiselect plugin to render multiple fields.

i.e., when called to validate a value, a validator returns a tuple (x, y). If y is None, then the value passed validation and x contains a parsed value. For example, if the validator requires the value to be an integer, x is converted to int(value). If the value did not pass validation, then x contains the input value and y contains an error message that explains the failed validation. This error message is used to report the error in forms that do not validate.

The validator may also contain a formatter method. It must perform the opposite conversion to the one the __call__ does. For example, consider the source code for IS_DATE:

On success, the __call__ method reads a date string from the form and converts it into a datetime.date object using the format string specified in the constructor. The formatter object takes a datetime.date object and converts it to a string representation using the same format. The formatter is called automatically in forms, but you can also call it explicitly to convert objects into their proper representation. For example:

The first ten of them are the defaults for the corresponding field types. The "options" widget is used when a field's requires is IS_IN_SET or IS_IN_DB with multiple=False (default behavior). The "multiple" widget is used when a field's requires is IS_IN_SET or IS_IN_DB with multiple=True. The "radio" and "checkboxes" widgets are never used by default, but can be set manually. The autocomplete widget is special and discussed in its own section.

For example, to have a "string" field represented by a textarea:

Field('comment', 'string', widget=SQLFORM.widgets.text.widget)

Widgets can also be assigned to fields a posteriori:

db.mytable.myfield.widget = SQLFORM.widgets.string.widget

Sometimes widgets take additional arguments and one needs to specify their values. In this case one can use lambda

Widgets are helper factories and their first two arguments are always field and value. The other arguments can include normal helper attributes such as _style, _class, etc. Some widgets also take special arguments. In particular SQLFORM.widgets.radio and SQLFORM.widgets.checkboxes take a style argument (not to be confused with _style) which can be set to "table", "ul", or "divs" in order to match the formstyle of the containing form.

You can create new widgets or extend existing widgets.

SQLFORM.widgets[type] is a class and SQLFORM.widgets[type].widget is a static member function of the corresponding class. Each widget function takes two arguments: the field object, and the current value of that field. It returns a representation of the widget. As an example, the string widget could be recoded as follows:

The id and class values must follow the convention described later in this chapter. A widget may contain its own validators, but it is good practice to associate the validators to the "requires" attribute of the field and have the widget get them from there.

Autocomplete widget

autocomplete

There are two possible uses for the autocomplete widget: to autocomplete a field that takes a value from a list or to autocomplete a reference field (where the string to be autocompleted is a representation of the reference which is implemented as an id).

Where limitby instructs the widget to display no more than 10 suggestions at the time, and min_length instructs the widget to perform an Ajax callback to fetch suggestions only after the user has typed at least 2 characters in the search box.

In this case the value of id_field tells the widget that even if the value to be autocompleted is a db.category.name, the value to be stored is the corresponding db.category.id. An optional parameter is orderby that instructs the widget on how to sort the suggestions (alphabetical by default).

This widget works via Ajax. Where is the Ajax callback? Some magic is going on in this widget. The callback is a method of the widget object itself. How is it exposed? In web2py any piece of code can generate a response by raising an HTTP exception. This widget exploits this possibility in the following way: the widget sends the Ajax call to the same URL that generated the widget in the first place and puts a special token in the request.vars. Should the widget get instantiated again, it finds the token and raises an HTTP exception that responds to the request. All of this is done under the hood and hidden to the developer.

SQLFORM.grid and SQLFORM.smartgrid (experimental)

These are two high level gadgets that create complex CRUD controls. They provide pagination, the ability to browser, search, sort, create, update and delete records from a single gadgets.

The first argument of SQLFORM.grid can be a table or a query. The grid gadget will provide access to records matching the query.

Before we dive into the long list of arguments of the grid gadget we need to understand how it works. The gadget looks at request.args in order to decide what to do (browse, search, create, update, delete, etc.). Each button created by the gadget links the same function (manage_users in the above case) but passes different request.args. By default all the URL generated by the grid are digitally signed and verified. This means one cannot perform certain actions (create, update, delete) without being logged-in. These restrictions can be relaxed:

Because of the way grid works one can only have one grid per controller function, unless they are embedded as components via LOAD.

Because the function that contains the grid may itself manipulate the command line arguments, the grid needs to know which args should be handled by the grid and which not. For example here is an example of code that allows one to manage any table:

the args argument of the grid specifies which request.args should be passed along and ignored by the gadget. In our case request.args[:1] is the name of the table we want to manage and it is handled by the manage function itself, not by the gadget.

links is used to display new columns which can be links to other pages. The links argument must be a list of dict(header='name',body=lambda row: A(...)) where header is the header of the new column and body is a function that takes a row and returns a value. In the example, the value is a A(...) helper.

maxtextlength sets the maximum length of text to be displayed for each field value, in the grid view. This value can be overwritten for each field using maxtextlengths, a dictionary of 'tablename.fieldname':length.

onvalidation, oncreate, onupdate and ondelete are callback functions. All but ondelete take a form object as input.

sorter_icons is a list of two strings (or helpers) that will be used to represent the up and down sorting options for each field.

ui can be set equal to 'web2py' and will generate web2py friendly class names, can be set equal to jquery-ui and will generate jquery UI friendly class names, but it can also be its own set of class names for the various grid components:

Notice the extra "children" links. One could create the extra links using a regular grid but they would point to a different action. With a smartgrid they are created automatically and handled by the same gadget.

Also notice that when clicking on the "children" link for a given parent one only gets the list of children for that parent (and that is obvious) but also notice that if one now tried to add a new child, the parent value for the new child is automatically set to the selected parent (displayed in the breadcrumbs associated to the gadget). The value of this field can be overwritten. We can prevent this by making it readonly:

If the linked_tables argument is not specified all referencing tables are automatically linked. Anyway, to avoid accidentally exposing data we recommend explicitly listing tables that should be linked.

The following code creates a very powerful management interface for all tables in the system:

The smartgrid takes the same arguments as a grid and some more with some caveats:

The first argument is a table, not a query

There is a extra argument constraints which is a dictionary of 'tablename':query which can be used to further restrict access to the records displayed in the 'tablename' grid.

There is a extra argument linked_tables which is a list of tablenames of tables that should be accessible via the smartgrid.

All the arguments but the table, args, linked_tables and user_signatures can be dictionaries as explained below.

Consider the previous grid:

grid = SQLFORM.smartgrid(db.parent,linked_tables=['child'])

It allows one to access both a db.parent and a db.child. Apart for navigation controls, for each one table, a smarttable is nothing but a grid. This means that, in this case, one smartgrid can create a grid for parent and one grid for child. We may want to pass different sets of parameters to these grids. For example different sets of searchable parameters.

In this way we made parents searchable but children for each parent not searchable (there should not be that many to need the search widget).

The grid and smartgrid gadgets are here to stay but they are marked experimental because the actual html layout of what they return and the exact set of parameters one can pass to them may be subject to change as new functionalities are added.

grid and smartgrid do not automatically enforce access control like crud does but you can integrate it with auth using explicit permission checking:

The smartgrid is the only gadget in web2py that displays the table name and it need both the singular and the plural. For example one parent can have one "Child" or many "Children". Therefore a table object needs to know its own singular and plural names. web2py normally guesses them but you can set the explicitly: