Tag: GUI

Now that we have a way for the compute to be described and from that kernel code to be generated and dispatched for execution in proper order, the next step is to provide the capability to the user to be able to change the high-level description easily and in a manner that is understandable and abstract. An obvious option is that the user directly change the high-level description string. But it would be easier and more intuitive for the user to be able to change the compute description string through a more abstract user interface option. So we want to be able to present the user with GUI options that both conveys the current state of the compute’s description, and through which the user can interact and change to the description. Therefore, we first need to be able present a set of GUI controls to the user that are generated from the current HLD, and that convey to the user the current state of the compute. These controls (e.g. selection boxes, button, checkmarks, number fields) not only tell the user what functions, parameters, input/output passes and conditions are currently used in the individual passes of the compute, but also the callback functions of the controls trigger change in the HLD string.

The first basic capability required to acheive this is that to dynamically add GUI controls to a user interface at runtime. We also need to be able to map these GUI controls to specific portions (e.g. passes) of the HLD when the user interacts with them. And for that to be possible, each control needs to be able to internally tell where in the HLD it corresponds to. All modern GUI languages (e.g. C#, objective-C, Swift, Xamarin) provide some form of mechanism to programmatically add controls to a user interface. But how the control’s member variables are used to self-identify (their position in the HLD) is usually up to the implementation. The callback functions of these controls (which get called upon the user interacting with them) need to use these self-identifications to determine how to parse the current HLD description to get to the portion they correspond to, and modify it such that the user’s change to the control becomes reflected in the HLD.

Specifically, upon presenting the user with GUI controls that describe the current description of the compute, each control has callback functions that are invoked when the user interacts and changes the control’s state. The first thing these callback functions should do is use their self-identifying member variables to determine their corresponding position within the current HLD string. Based on that, it then needs to parse the current HLD and get to its corresponding location in the HLD string, where change is to be made. This is necessary because a single HLD for a compute might have multiple passes, each with their own functions, conditions and parameters. Once the HLD description changes are made, it should be saved as the new current HLD for the compute. The user can then try the new compute by initiating kernel generation and execution on input data. If we had not used an intermediate HLDL, the GUI control callback functions would now have to directly manipulate the kernel code – which would be extremely error-prone and difficult to manage.

The condensed code snippet below shows an example of how in Blurate the combobox control for selecting and indicating the primary function in each pass is dynamically presented in the user interface, as conveyed in the HLD description, and how the callback functions are formed to handle changes to these controls. The code for the full GUI presentation of multi-pass compute is much larger and more complicated, but the code for adding other individual controls follows this template. This is the Windows C# version of the code for the studio version of Blurate, but the Mac Objective-C version also follows an equivalent code pattern. In fact, the Mac version was generated by manually translating the C# code line-by-line to objective-C. The only main difference was in-line callback function definitions, available in C# but not in objective-C (at least I never figured out how to use them). In Blurate this is all implemented in what is called the “Kernel Designer”, which can be opened from the main menu of the studio version of the app (Windows or Mac).

// Thread-safe method for modifying the current stored formula description

Orig.SetFullFormula(newFormula);

};

…

}

The drawFormula() function first adds Splitter and Panel controls for each pass of the compute. Then it calls constructPass() function on the created Panel, to which it adds the controls for that pass. The constructPass() function adds a Label to the top of the Panel to indicate the Pass number it corresponds to. Below that Label, it adds a Combobox that has all primary functions listed in it, and its selected option is set to the function specified in the formula. The callback function of this combobox is where the HLD formula is modified when the user changes the option in the combobox.

The first task performed in the combobox callback function is to identify which compute pass it corresponds to. In this C# version of the Blurate code, this is done by parsing the Name member variable of the combobox object. From that, the starting and ending position of the description of the corresponding pass can be determined in the current HLD formula string. The current SelectedItem of the combobox is then replaced as the primary function of that pass in the formula, while leaving the other passes unchanged.

Here the GUI controls are very simple standard ones, and their callback functions directly map to individual passes of the HLD. But that doesn’t need to be the case for other applications. More creative controls can be presented, and they don’t need to each map to individual compute kernels – they can represent more abstract multi-pass functionality and options. As an incremental example, a “Symmetric Convolution” or Sobel function option could be presented to the user in Blurate. Each of these passes would then translate to two kernels, rather than leaving it to the user to specify passes that map to individual compute kernels. This can be extended to more complex compute, requiring even more numerous kernels. The point is that the user does not need to necessarily be presented with every obscure option that directly maps to every angle of variability in the HLDL. It’s up to the creative design of your user interface and what functionality you want to provide the user. The only thing that is a “must” is that those controls that are presented to the user need to be able to find where in the HLD of the compute they correspond to when the user interacts with them and triggers their callback functions.

So far we’ve looked at how to process multi-pass compute descriptions when each pass directly maps to an individually compiled and executed kernel, and how to provide the user with easy GUI options to modify the description of those passes. In the next section we’ll look at optimizing compute performance by fusing multiple passes from the HLD description to one compute kernel, reducing the actual number of kernels executed.