Standard HMI Controls

Introduction

Containers – for organizing the layout with panels, groups, splitters and tab controls.

Controls – with standard controls for setting parameters and controlling the application state.

File System – for choosing files and directories.

Indicators – for displaying inspection results or status.

Multiple Pages – for creating multi-screen applications.

Password Protection – for limiting access to some parts of an HMI to authorized personnel.

Shape Editors – with controls that allow the end user to define object models or measurement primitives.

Shape Array Editors – with controls that allow the end user to define an array of object models or measurement primitives.

State Management – for loading and saving state of the controls to a file.

Video Box &dash; with several variants of the VideoBox control for high-performance image display.

In a basic machine vision application you will need one VideoBox control for displaying
an image, possibly with some graphical overlays, and a couple of TrackBars, CheckBoxes,
TextBoxes etc. for setting parameters. You will also often use Panels, GroupBoxes, Labels and
ImageBoxes to organize and decorate the window space.

Common Properties

Many properties are the same for different standard controls. Here is a summary of the most important ones:

AutoSize – specifies whether a control will automatically size itself to fit its contents.

BackColor – the background color of the component.

Three sample labels with different background colors.

BackgroundImage – the background image used for the control.

BorderStyle – controls how the border of the control looks like.

Three sample labels with different border styles.

Enabled – can be used to make the control not editable by the end user.

Sample buttons, first enabled, second disabled.

Font – defines the size and style of the font used to display text in the control.

Three sample labels with different fonts.

ForeColor – foreground color of the control, which is used to display text.

Text – a string being displayed in the control.

Visible – can be used to make the control invisible (hide it).

With the available properties it is possible to create an application with virtually
any look and feel. For example, to use an arbitrary design for a button, we can use
a bitmap, while setting FlatAppearance.BorderSize = 0 and FlatStyle = Flat:

Sample custom style for a button and a label.

Note: Most properties of HMI controls can be also set dynamically during program execution,
although only some of them are visible as ports when the control is selected. To expose
additional ports, use the "Edit Port Visibility..." command from the control's context
menu.

HMI Canvas

HMICanvas control represents the entire window of the created application. Other controls
will be placed on it. Properties of HMICanvas define some global settings of the application,
which are effective when it is run with Adaptive Vision Executor (runtime environment):

ConsoleEnabled – specifies whether diagnostic console can be shown in the runtime application.

ToolbarVisible – specifies whether the toolbar of Adaptive Vision Executor will be visible.

WindowMode – specifies whether the application should be run in a fixed-size window, in a variable-size window or full-screen.

WindowTitle – specifies the text displayed as the title of the runtime application.

Elements of the runtime application affected with properties of HMICanvas.

Remarks:

If you are having trouble selecting the HMI Canvas because it is completely
filled with other controls, right click on any control and choose the "Select: HMICanvas" command.

Some parameters of HMI Canvas are automatically inherited by the contained controls. For example,
if one changes the Font property, other controls will automatically have it set to the same value.

Full-screen mode is especially useful, when it is required that the end user does not have
access to elements of the underlying operating system. On Windows systems it is also possible
to set Adaptive Vision Executor as the "system shell", thus removing Desktop, Menu Start etc.
completely.

Controls for Setting Layout of the Window

Although it is possible to put all individual controls directly onto an HMI Canvas,
it is recommended that related items are grouped together for both more convenient editing
as well as for better end user's experience. Available controls in the "Containers" section are:

GroupBox – can be used to group several controls within an enclosing frame with label.

Panel – can be used to group several controls without additional graphical elements.

SplitContainer – can be used to create two panels with movable boundary between them.

TabControl – can be used to create multiple tabs in the user interface.

BackColor was changed from default to white in all containers.

To use any container drag and drop container and then drag and drop HMI controls inside container.
Other method in order to do that is clicking on desired container in "Containers" section and then selecting the area on HMI panel.

Other controls commonly used for making HMI look nice are: ImageBox (for displaying static images
such as company logos) and Label (for any textual items).

However, when you would like to create more sophisticated HMI Panel with multiple screens go to The Multi Panel Control

Controls for Displaying Images

The "Video Box" section of the HMI Controls window contains several controls for high-performance display
of image streams. These are:

VideoBox – the basic variant of the image display control.

SelectingVideoBox – also allows for selecting a box region by the end user.

ZoomingVideoBox – also allows for zooming and panning the image by the end user.

FloatingVideoWindow – a component that allows for opening an additional window for displaying images.

A VideoBox.

To display some additional information over images in a VideoBox it is necessary to modify the images using drawing filters
before passing them to the VideoBox. This gives the user full control over the overlay design, but may expand the program.
To keep the program simple, while still being able to display some overlay information, it is possible use View2DBox filters,
which can be found in he "Indicators" section. They accept images and 2D primitives produced by various filters as input,
however the provided control over the overlay style is limited:

ImageBox from "Controls" section can be used to improve look of HMI for displaying static images such as company logos

Note: Do NOT use ImageBox control for displaying inspection results. It should only be used for static images.

Common properties:

DisplayMode – switches between GDI and DirectX implementations. It depends on the underlying hardware, which one is faster.

SizeMode – specifies how the image should be fitted to the available window space.

Controls for Setting Parameters

When some parameters of the application are to be set by the end user, the following
controls can be used:

TrackBar, Knob, NumericUpDown – for setting numerical values. You set Minimum and Maximum parameters and then the value actually selected by the end user will be available on the outValue output.

CheckBox, ToggleButton, OnOffButton, RadioButton – for turning something on or off. The outValue or outChecked output will provide the state of the setting.

ComboBox, EnumBox – for choosing one of several options.

TextBox – for entering textual data. The entered text will be available on the outText output.

Example: Using TrackBar

TrackBar by default has two connectable inputs and one connectable output. The inValue input
and the outValue output represent the same property – the value indicated
by the current state of the control. The second input, inEnabled, is a boolean value
and controls whether the track-bar is enabled or disabled in the user interface.

A track-bar.

Binding with a Label

Very often it is required that the current value of a TrackBar or a similar control is also visible in the HMI.
This can be done with an additional Label control placed near the TrackBar, with the AutoValueSource
property referring to the selected TrackBar.

Label showing an exact value of the TrackBar.

It is also possible to specify the format of the displayed text. The picture below shows a connected Label with the
AutoValueFormat property set to "Tolerance: % [px]".

TrackBar with a connected Label using an additional text formatting.

Example: Using ComboBox

ComboBox by default has one connectable inputs and two connectable output. To enable more ports, for example inList, right click on HMI control and edit port visibility.
StringArray data type value can be connected To inList input. Other way to enter list is adding more items in ComboBox properties in Data overlap.

Click Add New Item to increase list.

Selecting index in Selection parameter, user defines based selection which is chosen when program is executed. Without this selection outputs will be Nil.

The outSelection output has integer data type and shows index of selected item.
The outText output has String data type and shows text associated with the selection.

Using EnumBox

EnumBox is similar to ComboBox. Main differences between them is predefined lists in EnumBox. User can choose one of the type present in the AdaptiveVisionStudio changing Type in EnumBox's properties.
List allows to chose one option associated with the variable. For example X or Y value can be chosen for Axis type value.

Change Events

Controls that output some value adjustable by the user often also provide outputs
informing about the moments when this value has been changed. For example, the ComboBox
control has an output named outSelection (Integer) for indicating the index of the currently
selected list item, but it also has outSelectionChanged (Bool) which is set to True
for exactly one iteration when the selection has been changed.

Please note, that most of the "outSomethingChanged" outputs are hidden by default and
can be shown with the "Edit Port Visibility..." command in the context menu of the control.

AnalogIndicator

AnalogIndicator and its variant AnalogIndicatorWithScales are
used to display numerical results in a retro style – as analog gauges.
The latter variant can have green-orange-red ranges defined with properties like
GreenColorMinimum, GreenColorMaximum etc.
The ranges are usually disjoint, but there can also be nested ranges: e.g. the red color can span throughout the scale, and
the green color can have a narrower range around the middle of the scale. The
green sector will be displayed on top, producing an effect of a single green
sector, with two red sectors outside.

Analog indicators with and without scales

Event Triggering Controls

Most of the standard controls in Adaptive Vision Studio fall into one of two categories: (1) data presenters
and (2) parameter setters. There are, however, also some controls that can trigger
events – most notably the ImpulseButton control.

The ImpulseButton control is an event triggering control, which has one output, outValue.
The value returned is of the Bool type. As long the button is not clicked,
it will return False. Once the button is clicked, the value becomes True for exactly
one iteration of the program.

The TabControl Control

TabControl is the one of the methods to organize HMI panels within a limited space
of a form. It is a container with several "tabs", where each page contains a different set
of controls.

TabControl.

The MultiPanelControl Control

MultiPanelControl is an easy and convenient way to organize bigger HMI panels. It employs a multi-screen approach
to provide more space and enables adding special purpose windows like e.g. a "Configuration" page.

MultiPanelControl.

To create a multi-screen HMI, first place a MultiPanelControl control in your window and fit it to the window size.
In the upper-right corner of this control you will see a four element Navigation Bar
().
Using the left and right arrows you will be able to switch between consecutive screens. The frame button allows
for selecting the parent control in the same way as the "Select: MultiPanelControl" command in the control's context menu.
And the triple dot button opens a configuration windows as on the picture below:

MultiPanelControl Property Box.

Also when you double-click on the area of a MultiPanelControl, the same configuration window appears.
Using this window you are able to organize the pages. During work with pages there are at least two necessary things you have to remember.
First – all pages names have to be unique globally. Second – one MultiPanelControl contains many MultiPanelPages.
An important thing is the home icon – use it to set the Initial Page of your application.
It will be the first page to appear when the program starts.

A MultiPanelControl control is usually accompanied by some MultiPanelSwitchButton controls that navigate between the pages. Usage of a button of this kind is very simple.
Just place it on your window and set the TargetPage property (also available through double-clicking on the button). Page switching buttons
are usually placed within individual pages, but this is not strictly required – they can also be placed outside of the MultiPanelControl control.

Every HMI Control has built-in input and output ports. In MultiPanelControl, by default, two ports are visible: "inActivePageName" and
"outActivePageName", but there are more. (To make them visible click the right mouse button on the control and select "Edit Ports Visibility...").
The "inActivePageName" input enables setting the active page directly from your program. It is another way to handle page changing.
One is using the dedicated buttons by the operator and here comes another method. It might be useful e.g.
when "something happens" and you would like to change the screen automatically, not by a button click. This can be e.g. information
that a connection with remote host was lost and immediate reconfiguration is needed.
The "outActivePageName" by definition is an output which allows you to check the current active page.

Note: More information is available as "HMI Multipanel Control" in the official examples of Adaptive Vision Studio.

Program Control Buttons

Controlling the program execution process, i.e. starting, iterating and pausing,
by the end user is possible with a set of ProgramControlButton controls, or with
the complete panel named ProgramControlBox. These controls are available
in the "Controls" section of the HMI Controls window.

A Start button with a custom text.

ProgramControlBox, similar to the controls from Adaptive Vision Studio.

Remarks:

The Stop button is not available for the HMI, because the end user
should not be able to stop the entire application at a random point of
execution. If you have a good reason to make it possible, use a separate ImpulseButton
control connected to an Exit filter.

ProgramControlButton and ProgramControlBox controls are useful for the
purpose of fast prototyping. In many applications, however, it might be advisable
to design an application-specific state machine.

File and Directory Picking

There are two controls for selecting paths in the file system, which are
FilePicker and DirectoryPicker. They look similar to a
TextBox, but they have a button next to them, which brings up
a standard system file dialog for browsing a file or directory.

What is worth mentioning about these two controls is the option NilOnEmpty,
which can be useful for conditional execution. As the name suggests, it causes the
control to return the special NIL value instead of an empty string, when
no file or directory is chosen.

Shape Editors

Geometrical primitives, regions and fitting fields, that are used as parameters in image analysis algorithms, are
usually created by the user of Adaptive Vision Studio. In some cases, however, it is also required that the end
user is able to adjust these elements in the runtime environment. For that purpose, in the "Shape Editors" section
available are appropriate controls that bring graphical editors known from Adaptive Vision Studio to the HMI Designer
environment.

Each such editor has a form of a button with four ports visible by default:

inAlignment – for specifying an appropriate coordinate system for the edited shape.

inReferenceImage – for specifying a background images for the editor.

inValue – for setting initial value of the edited shape.

outValue – for getting the value of the shape after it is edited by the end user.

outStoredReferenceImage – a reference image that was displayed in the editor during last editing approved with the OK button. Active when inStoreReferenceImage is set to True.

Sample ShapeRegion editor control in the HMI Designer.

When the end user clicks the button, an appropriate graphical editor is opened:

Sample ShapeRegion editor opened after the button is clicked in the runtime environment.

The following properties specific these controls allow for customizing appearance of an opened editor:

HideSelectors – allows to simplify the editor by removing image and coordinate system selectors.

Message – an additional text that can be displayed in an opened editor as a hint for the user.

WindowTitle – text displayed on the title bar of the opened editor's window.

The reference image is copied to HMI control in each macrofilter iteration. To achieve best performance separate vision algorithm from preparation of algorithm data.

Shape editors can be used to create input values for filter or modify existing values. Every shape editor has assigned proper data type to inValue and outValue.
The following list shows data types assigned to HMI controls and example filters which uses this type of data.

ProfileBox

ProfileBox is an HMI control that displays a profile of evenly sampled measurements. ProfileBox is available
in the "Indicators" section of the HMI Controls window.

ProfileBox may be useful for displaying array of samples of a known length, for example a profile of an image column or a laser line.

Profile of a laser line.

Also it is used for displaying dynamic data from sensors outputs in the function of time.

Temperature chart.

There are several specific properties for this control:

DomainDefaultLength, DomainDefaultSizeMode, DomainScale and DomainStart – set of parameters responsible for appearance and behavior of the X axis.

EnableManualZoomChange – for specifying whether the end user is able to zoom the chart area.

ChartColor, GridColor, BackColor, AxisColor – for specifying the colors of a ProfileBox content.

HighQualityMode – for specifying whether the chart is drawn with high quality or with low quality, but faster.

HorizontalAxisName, VerticalAxisName – for specifying the names of axes.

HorizontalAxisMin, HorizontalAxisMax, VerticalAxisMin, VerticalAxisMax – for setting the limits of the chart axes.

View3DBox

View3DBox is an HMI control that displays 3D primitives. It is available
in the "Indicators" section of the HMI Controls window.

An example of using View3DBox control.

Default settings of this HMI control allow to connect four inData inputs. However, it is possible to use up to eight inputs. To make the all inputs visible, right-click
on the View3DBox, select Edit Ports Visibility... and check the inputs you want to use.

outValue – is textSegmentation data type. This value can be connected to inSegmentationModel input in ExtractText filter.

outValueChanged – is the bool data type value which informs when outValue is changed.

After clicking on textSegmentationEditor button in HMI panel, editor will be opened. In order to do this the program must be executed.

Editor is the same as inSegmentationModel editor known from ExtractText filter.

When proper parameters are set, text segmentation is ready. Now segmented text should be recognized. In order to do this outCharacters[] output from ExtractText filter can be connected to inCharacters[] input in ReadText filter.

Missing ocrModel can be created with OcrModelEditor HMI control.

OcrModelEditor has 5 default ports which are visible:

inReferenceCharacters – is RegionArray data type value. Characters from ExtractText filter can be connected to this input. This input will be used for model training.

inReferenceImage – is an image which will be used as background in model training.

outStoredReferenceImage – is the image set as the editor background.

outValue – is ocrModel data type value. This value can be connected to inOcrModel input in ReadText filter.

outValueChanged – is the bool data type value which informs when outValue is changed.

After clicking on ocrModelEditor button on HMI panel, ocr editor will be opened. In order to do that the program must be executed.

Editor is the same as inOcrModel editor known from ReadText filter.

The simple program which is using TextSegmentationEditor and ocrModelEditor.

KeyboardListener

KeyboardListener is used for getting information about pressed keys on keyboard. Information from this control is given in IntegerArray which includes key numbers saved in ASCI code.

To use KeyboardListener and any other tolls form components overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.

All default output values allow to check which key is pressed, but there is a few differences between them:

outKeysDown – emits array of keys that have been recently pressed.

outKeysPressed – emits array of currently pressed keys. Using this output is recommended is most cases.

outKeysUP – emits array of keys that have been recently released.

To use those values in other place in program the best solution is copying them with CopyObject filter with IntegerArray data type.

VirtualKeyboard

VirtualKeyboard is HMI control used to enter data onto HMI panel.

This functionality can be used in runtime environment when physical keyboard is not available at the workstation.

To use VirtualKeyboard and any other tools form components overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.

The example of using VirtualKeyboard for entering data on HMI panel into TextBox control.

It is possible to display only Numeric Pad in case of entering numeric data types. In order to do this EnableNumPadOnlyKeyboard must be set to True in properties.

The example of using VirtualKeyboard for entering data on HMI panel into NumericUpDown control.

ToolTip

ToolTip HMI control is used to display messages created by the user. Those messages are displayed when user moves the pointer over an associated control.

This functionality can be used when developer wants to give helpful tips to final user.

To use ToolTip and any other tools form components overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.
After previous step it is possible to find and edit specific information for each used HMI control. This message can be edited in each HMI control properties in Misc overlap.

Editable ToolTip message in TrackBar HMI control properties.

ToolTip properties can be adjusted to user's requirements by changing the appropriate values:

Active – determines if the ToolTip is active. A tip will only appear if the ToolTip has been activated.

AutomaticDelay – sets the values of AutoPopDelay, InitialDelay and ReshowDelay to the appropriate values.

AutoPopDelay – determines the length of time the ToolTip window remains visible if the pointer is stationary inside a ToolTip region.

BackColor – the background color of the ToolTip control.

ForeColor – the foreground of the ToolTip control.

InitialDelay – determines the length of time the pointer must remain stationary within a ToolTip region before the ToolTip window appears.

IsBalloon – indicates whether the ToolTip will take on a balloon form.

ReshowDelay – determines the length of time it takes for subsegment ToolTip windows to appear as the pointer moves from one ToolTip region to another.

ToolTipTitle – determines the title of the ToolTip.

UseAnimation – when set to true, animation are used when the ToolTip is shown or hidden.

UseFading – when set to true, a fade effect is used when ToolTip are shown or hidden.

ColorPicker

This function can be used when color changing is required in runtime environment.

The default inValue and outValue ports allow user to get and set color which is saved with pixel data type. outValueChanged informs user when the pixel value is changed.

Color can be set in properties or in executed program clicking on ColorPicker icon on HMI panel. In color picking window user can choose color by clicking on proper color in palette of colors or enter manually values describing the desired color.

Color picking window.

Logic and Automation tools

BoolAggregator

BoolAggregator allows to connect multiple booleans signal sources using logic functions for purpose of controls such as EnabledManager.

This function is useful when one HMI bool state must be controlled by multiple other bool states from HMI.

To use BoolAggregator and any other tolls form Logic and Automation tools overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.

Input boolean values can be assigned by inValue ports. Other way to do that is assign proper bool values to sources in BoolAggregator's Properties. As input value in properties user can choose any bool type value from other used control from HMI panel, for example OnOffButton state.

Using BoolAggregator's properties it is also possible to negate source state choosing True state next to the expected value.

With Mode property it is possible to choose aggregation function which is used to connect multiple data source. It is possible to set one of following aggregation functions:

And

NotAnd

Or

NotOr

Output value from BoolAggregator can be used with EnabledManager HMI tool or used in other way. OutValue in BoolAggregator is not visible with default setting. To enable this output right click on BoolAggregator icon into HMI panel and edit ports visibility.

Bool Aggregator's properties.

EnabledManager

EnabledManager allows to automatically manage controls enable state using other HMI controls as boolean value source.

This function is commonly used with BoolAggregator in order to manage controls enable states in more complex cases.

To use EnabledManager and any other tolls form Logic and Automation tools overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.

After placing EnabledManager tool it is possible to edit enable state of each control placed onto HMI panel. Automation overlap will be present in controls properties. It is possible to choose source data used as an enable signal from list, setting AutoEnabledSource property. It is also possible to Negate source data value changing AutoEnabledNegate from false to true.

In example below TextBox's enable state is controlled with two OnOffButtons through one BoolAggregator and EnabledManager.

Automation overlap in TextBox's properties.

EdgeModelEditor

EdgeModelEditor is used for creating edge models from the HMI panel.

This function is useful when edge model edition is required in runtime environment

Details about creating edge models can be found in this article: CreateEdgeModel.

Default visible input port should be connected to ReferenceImage which is used for model training. outValue output has EdgeModel data type. This output can be connected to inEdgeModel input in LocateObjects_Edges filter.

EdgeModel can be created in EdgeModelEditor window. One way to get to this editor window is clicking field next to the Value property in control's properties. Another way is clicking EdgeModelEditor icon onto HMI panel when program is executed.

First method to edit EdgeModel.

EdgeModelEditor window.

GrayModelEditor

GrayModelEditor is used for creating gray models from the HMI panel.

This function is useful when gray model edition is required in runtime environment

Details about creating gray models can be found in this article: CreateGrayModel.

Default visible input port should be connected to ReferenceImage which is used for model training. outValue output has GrayModel data type. This output can be connected to inGrayModel input in LocateObjects_NCC filter.

GrayModel can be created in GrayModelEditor window. One way to get to this editor window is clicking field next to the Value property in control's properties. Another way is clicking GrayModelEditor icon onto HMI panel when program is executed.

First method to edit GrayModel.

GrayModelEditor window.

GenICamAddressPicker

GenICamAddressPicker is used to choose GenICam GenTL device address in runtime environment. This control can be used with GenICam filter.