Overview

What is ETFw? (aka PTFw)

The PTP External Tools Framework (a.k.a. Performance Tools Framework) aims to decrease the amount of "Eclipse plumbing" that must be done to integrate existing external tools (Performance Tools and other dynamic tools) in Eclipse, and provide pieces and parts to provide new tools within Eclipse.

The PTFw is aimed at primarily C, C++, and Fortran applications, since these are the targets of PTP users.

The PTFw can be used for Parallel launches, or for regular (e.g. CDT and Photran) launches.

The PTFw provides easy integration of launching existing (e.g. command-line) tools from Eclipse. With just an XML file, an existing app can be launched, with Eclipse UI generated to solicit arguments from the user, launch the tool, and launch any analysis of the results.

Workflow specified in the tool xml file specifies the steps to take. Each tool can have three distinct types of operation which may be duplicated, combined and logically controlled to accomplish fairly complex tasks:

compile - the compile section specifies commands to use to build the executable (e.g. with instrumentation) prior to running the tool. These are commands to prepend the existing compiler. See xml file details and examples.

execute - the execute section includes utilities which prepend the actual executable, and run to gather information.

analyze - the analyze section contains tools which can be launched on data produced after program execution is complete. They run in a predetermined sequence. Analysis tools can be existing (non-Eclipse) apps or command line tools or other visualization tools. Analysis tools can also be created as eclipse plug-ins and are launched to display output within the eclipse workbench. Analysis work-flows can be run on existing files or folders as well as on data generated by program execution in Eclipse

The PTFw can work for any tool that should "wrapper" an executable with an arbitrary number of arguments that can be specified by the user.

Screenshots and brief examples

The following brief example shows how information to be extracted from the user prior to launch is displayed in the eclipse launch dialog,
and how the items appear in the tool.xml file

PTFw implementations

TAU - Wyatt Spear, U.Oregon

TAU was the original application for which the PTFw tools were developed by Wyatt Spear (U. Oregon). Much of the original functionality has been generalized in the PTFw, but extensions are available (and utilized for TAU) for any features not currently available. That is, write your own eclipse plug-in code for any additional features.

TaskFinder - Beth Tibbitts, IBM Research

A tool for instrumenting a binary executable with the Intel PIN tool, and sampling memory references, was developed by IBM. The PTFw was utilized to provide a convenient user interface for the launching of TF with its arguments, and display an eclipse view (analyze section) that displays the results found, and maps the results back to source code lines. Developed by Beth Tibbitts (IBM Research) and release/availability is being determined.

Others are in process and we hope to provide news on them soon. (August '08)

PTFw integration

PTFw integration can be done one of two ways.

Use xml file only to integrate a tool - specify the file via Eclipse Preferences

This requires no programming and users can do it themselves.

Use Eclipse extensions to extend one or more of three extension points for more custom integration

This requires a bit of Eclipse plug-in programming.

PTFw XML file only via Eclipse Preferences

Input an xml file via the Eclipse preference pages. The end-user can do this without any eclipse plug-in programming.

PTFw extension points

Implement eclipse extensions for a more custom integration of your tool. Note, identifiers for extension points recently changed from org.eclipse.ptp.perf to org.eclipse.ptp.etfw.

org.eclipse.ptp.etfw.workflows

Handles specification of the steps to be performed in the perf/profile launch

Three sections: compile, execute, and analyze

Allows integration of external or command line tools, or eclipse plug-in tools

Any of the three steps are optional

allows the workflow XML document to be specified in the plug-in

without this, a workflow XML document can be specified by the end user in the Preferences, without any Eclipse code needed

workflow XML doc can specify one or more steps for each:

compile, execute, and analyze

org.eclipse.ptp.etfw.configurationTabs

Handles additional tabs and UI features in launch dialog

org.eclipse.ptp.etfw.dataManagers

Handles arbitrary post-processing

Availability

Works with Eclipse 3.3/CDT 4.0 or Eclipse 3.4/ CDT 5.0

Also available from the Eclipse CVS Repository

Repository location: dev.eclipse.org

Project: org.eclipse.ptp

In folder: tools/perf

Projects:

org.eclipse.ptp.perf

org.eclipse.ptp.perf.toolopts

Other project include those specific for TAU: org.eclipse.ptp.perf.* projects

Tool XML Reference

The xml format for the external tools system is still being expanded and improved. Please do not hesitate to provide suggestions for additions or modifications.

<toolset.01>: An optional top level tag for enclosing individual tools. Note, if the string "toolset" is used a deprecated parsing scheme will be employed and failure is likely.

name=string: The name for this tool workflow. This is what the user sees when presented with the list of available tools. It should be a unique string. Required.

id=string: This is used to internally identify the tool. Optional. Defaults to the name attribute.

<compile>, <execute>, <analyze>:

Shared Children: <utility> (compile has special syntax), <global>

Shared Attributes:

if="string": If this attribute is defined this step will only be performed if the string specified is set to 'TRUE' in the internal launch configuration used with the tool. Currently this can only be used in conjunction with plug-in extensions to the external tools framework.

<compile>: Represents a compilation step in an analysis workflow. Optional. If specified the project will be rebuilt in a new build configuration based on one specified in the launch configuration, but with modified compiler commands. Otherwise the selected executable will be used in the workflow as usual.

Parents: <tool>

Children: <cc>, <cxx>, <f90>, <allcompilers>, <global>

Attributes:

replace=boolean: If this is "true" then compilers specified here will replace those in the associated eclipse compilation configuration. Otherwise the compiler tools defined here will be prepended to the existing compiler commands. Optional. Defaults to false.

<cc>, <cxx>, <f90>, <allcompilers>: These are just special cases of the utility tag. cc, cxx and f90 replace or prepend the default compilers for those languages. If the individual compiler tags are not defined and a command is specified for allcompilers it will replace or prepend the command for every language. If the individual compilers are defined along with allcompilers the individual compilers will be used and each will be given arguments defined by allcompilers. (This system can probably be simplified...)

<execute>: Represents an execution step in an analysis workflow. Optional. If specified any utilities given will be prepended to the default execution command in order (as with valgrind).

Parents: <tool>

Children: <utility>, <global>

<analyze>: Represents an analysis step in an analysis workflow. Optional. If specified each of the utility commands defined given will be executed in order.

Parents: <tool>

Children: <utility>, <global>

Attributes:

defaultloc=boolean: If set to true and there is no information available on in which directory analysis tools should be run (this usually means the output location of performance data) then the user's home directory will be used. If set to false the user will be prompted for a directory. Optional. Defaults to false.

foralllike=string: Given a file suffix (i.e. ".ppk") this utility will be run once for every file of that type in the working directory (and by default every subdirectory). In value attributes of utilities under this analysis step the string "%%FILE%%" will be replaced with the current file, including the path, from the collected set and the string "%%FILENAME%%" will be replaced with just the main file name, without the path or suffix. Optional.

depth=integer: By default foralllike searches every sub-directory for files matching its criteria. The depth attribute will limit this to a depth equal to its value. Optional. Defaults to -1 (unlimited).

<utility>: Each of these represents a command line invocation of a single application.

Parents: <compile> (compiler specific tags), <execute>, <analyze>

Children: <argument>, <envvar>

Attributes:

command=string: The actual program to be run. The full path is not necessary. Required, except for <allcompilers>.

id=string: A unique tag for this application. Optional. Defaults to the enclosing tool's ID. (Not used)

group=string: All utilities with the same group attribute are assumed to share the same location for their executable files. This simplifies selection of tool locations. Optional.

outtofile=string: If this attribute is defined the standard output for the execution of this utility will be put in a file in the project directory with the name given. Optional.

<argument>: This is a single argument passed to a utility.

Parents: <utility> (and the equivalent compiler tags)

Children:

Attributes:

flag=string: A static value passed to the enclosing utility. Optional.

value=string: A variable (such as a number or file path) value passed to the enclosing utility. The strings "%%REPLACE_WITH_BUILD_DIR%%" and "%%REPLACE_WITH_PROJECT_ROOT_DIR%%" will be replaced with the actual paths specific to the configuration if they are used in this value. %%REPLACE_WITH_PROJECT_ROOT_DIR%% is replaced with the root of the project directory (e.g. ~/workspace/MyProject). "%%REPLACE_WITH_BUILD_DIR%%" is replaced with the directory where the binary built by this project is executed (this can be either the root directory or a different directory depending on the launch system used), which is useful for locating output from the executable. Optional.

separator=string: The string or character put between the flag and value. Optional. Defaults to empty string

confvalue=string: If this is defined the string "%%REPLACE_WITH_CONFIGURATION_VALUE%%" will be replaced with the value of an internal launch-configuration attribute with the given name. Only useful in conjunction with extensions to the workflow system. Optional.

localdir=boolean: If set to true the value attribute is assumed to be a file in the local (project) directory. The absolute path will be prepended to the value attribute. Optional. Defaults to false.

<envvar>: This creates a single environment variable to be specified in the use of a utility. Note that environment variables are presently only implemented for the execution step and their association with any given utility is arbitrary because there is only one point of entry for environment variables in the Eclipse launch environment. Note that the envvar and argument tags are closely related and the functionality of the latter is generally applicable to the former. However as of this writing only use of the flag and value attributes has been tested.

Parents:<utility> (and the equivalent compiler tags), <global>

Children:

Attributes:

flag=string: The name of the environment variable

value=string: The value assigned to the environment variable.

<global>: Some environment variables do not logically fit with the invocation of any tool. In that case envvars and toolpanes that define environment variables can be placed in a global construct rather than a utility. Presently the global tag is only used by the execute tag.

Parents: <compile>, <analyze>, <execute>

Children: <envvar>, <toolpane>

Attributes:

command=string: The unique string identifier for the command or function with which these environment variables are associated.

<parametric>: A parametric workflow performs multiple tests on the same application using variable sets specified in this top-level tag. Note that the behavior of envvar and argument tags within a parametric tag differs from their other uses.

Parents: <tool>

Children: <envvar>, <argument>

Under parametric, envvar and argument have the same attributes and no sub-tags:

name=string: The flag of the parametrized option

values=string: A comma separated list of each value to be used in the parametrized tests for the associated option

weakscaling=boolean: If true then the values attribute must have a number of entries equal to the number of entries in the mpi processes list. This entry's nth parameter will be selected in conjuction with the nth process count (weak scaling). Defaults to false.

Attributes:

weakscaling=boolean: If set to true the parametric test will be weak-scaling. Each set of parameters must have the same number of entries and one test will be performed for each matching index. If it is false every possible combination will be tested. Optional. Defaults to false.

numprocs=string:A comma separated list of integers. For PTP-based parametric tests each integer will be used as the number of mpi processes in combination with other parameters.

compileropt=string: A comma separated list of integers from 0 to 3. Currently only gnu-compiler optimization levels are supported for parametric testing. If this attribute is used parametric tests will be run for a binary created with each given optimization level. Optional.

<toolpane>: Each toolpane entry represents a UI pane containing a set of widgets to generated an argument string or set of environment variables. The utility command containing a toolpane is passed a single string generated from the list of non envvar widgets.

Parents: <utility> (and the equivalent compiler tags), <global>

Children: <togoption>

Attributes:

title=string: The title associated with this tool pane. It should be unique for this tool unless the pane is virtual. This value will show up on the tab displaying the pane's content. Required.

virtual=string: If this attribute is set no ui elements must be specified! Instead set the title attribute equal to that of a previously defined tool pane and this virtual tool pane will return the same value. This is useful for workflows that use the same input set in multiple places. It can also be used to combine custom coded UI element output with standard toolpane output. (Todo: Use the name of a valid pane for input instead of a boolean) Optional.

prependwith=string: If specified this string will come before any of the options taken from the UI widgets. Optional.

enclosewith=string: If specified the entire set of arguments will be prepended and appended with this string. (Todo: subdivide into prepend and append for different start and end characters, such as parentheses). Optional.

separatewith=string: This value will be placed between each individual argument generated by the UI. Optional. Defaults to newline.

enclosevalues=string: This string will be appended and prepended to each individual argument generated by the UI. Optional. Defaults to ".

separatevalues=string: This string will be placed between individual arguments flags and values if any. Optional. Defaults to =. (Todo: Make this option available at the value tag level instead)

displayoptions=boolean: If true the assembled argument string will be displayed and updated in a static text pane in the UI. Optional. Defaults to true.

<togoption>: The base widget used by toolpanes. Each togoption creates a single argument and value based on user input. If no value sub-tag is given then it will produce a check-box which supplies an argument if checked.

Parents: <toolpane>

Children: <optvalue>

Attributes:

envvar=boolean: If set to true this widget will set an environment variable named with its name attribute and given the value derived from its value component. Otherwise it will combine the two into a string which will be included in the option string sent to the associated application. Optional. Defaults to false.

label=string: The label for the option to be used in the UI. Required.

name=string: The actual name or flag of the option passed to the application. Optional. Defaults to the value supplied for label.

tooltip=string: This text will show up as a tool-tip for this widget. Optional.

defstate=boolean: The check-box associated with this option will default to true or false in accordance with the provided boolean value. Optional. Defaults to true.

required=boolean: If this is set to true then there is no check-box provided and the option may not be unselected. Optional. Defaults to false.

visible=boolean: If this is selected then the option is set to true permanently and is not displayed in the UI.

<optvalue>: This tag describes the value component of a togoption.

Parents: <togoption>

Children:

Attributes:

type=string: The following types are currently available. Some other attributes are only used by specific types. Optional. Defaults to the standard checkbox.

text: A text field will be added to the widget and its contents appended to the options string.

dir: A text field will will be added to the widget along with a browse button to browse for directories.

file: A text field will be added to the widget along with a browse button to browse for files.

number: Not yet supported. Will produce a numeric field with optional value restrictions.

combo: Not yet supported. Will produce a combo box based on a list of possible options.

toggle: Instead of adding or removing the string, the check-box for this widget changes the string used. For example, checked might result in "-foo=yes" and unchecked might result in "-foo=no". Presently toggle is only set for environment variables.

default=string: This applies to any type with a value widget. The contents of the value widget will be set to this string, rather than empty. Optional.

filelike=string: This applies to type="file". Given a file suffix the browse button will only list files with that suffix. Optional.

seton=string: This applies to type="toggle". This value will be used when the check-box is checked. Optional. Defaults to the empty string.

setoff=string: This applies to type="toggle". This value will be used when the check-box is unchecked. Optional. Defaults to null, which results in no value being set at all in the unchecked case.

fieldrequired=boolean: If set to true then this option will only be set if there is some value in the value field. Optional. Defaults to false.

Annotated Tool XML file

Annotated tool.xml file

This example contains 5 tools, each of which can contain one or more sections of compile, execute, and/or analyze.

<?xmlversion="1.0"encoding="UTF-8"?><!-- Use the unqualified 'toolset' only for the old xml format. The current version is toolset.01 --><toolset.01><!-- There may be multiple tools per tool file. Each tool should have a unique name tag, which will be used to select it in Eclipse--><toolname="VampirTrace"><!-- There are three distinct sections for each tool; compile, execute and analyze. Note that each of these steps is optional --><compile><!-- By default the compiler commands set here prepend whatever compiler is already in use in Eclipse. If you set the tag replace="true" for the compile element the compilers will be replaced entirely with the command specified here. Each compiler type, c, c++ and fortran, is defined as shown below. --><!-- Every command referencing a file on the system should include a group tag. The group tag indicates that the relevant binary files or scripts are located in the same place for each command sharing that tag --><CCcommand="vtcc"group="vampirtrace"><!-- Arguments to be passed to a command may be specified with the argument tag as shown here. --><argumentvalue="-vt:cc"/></CC><CXXcommand="vtcxx"group="vampirtrace"><argumentvalue="-vt:cxx"/></CXX><F90command="vtf90"group="vampirtrace"><argumentvalue="-vt:f90"/></F90></compile><execute><!-- The execute section is comprised of utilities which prepend the actual executable. --><utilitycommand="mpirun"group="mpi"><argumentvalue="-np 4"/></utility></execute><analyze><!-- The analyze section contains tools which are launched on data produced after program execution is complete. The tools listed here will be run in sequence, allowing the definition of a multi-step processing procedure. The type tag can be set to view for tools that show data or process for tools that process data and (generally) output another file. Use caution when specifying arguments. Some tools produce dynamically named output which is not yet supported by this plugin. --><utilitytype="view"command="vampir"group="vampir"><argumentvalue="a.otf"/></utility></analyze></tool><toolname="Kojak"><compile><!-- The allcompilers compiler definition allows the same command to be used for c, c++ and fortran. If used in conjunction with the specific compiler definitions their commands will override the allcompilers command, but arguments specified for allcompilers will be added to each individual compiler --><allcompilerscommand="kinst-pomp"group="kojak"/></compile><analyze><utilitycommand="expert"group="kojak"><argumentvalue="a.elg"/></utility><utilitycommand="paraprof"group="tau"><argumentvalue="a.cube"/></utility></analyze></tool><toolname="Perfsuite"><execute><utilitycommand="mpirun"group="mpi"><argumentvalue="-np 4"/></utility><utilitycommand="psrun"group="perfsuite"></utility></execute><analyze><utilitycommand="psprocess"type="process"group="perfsuite"><argumentvalue="-o xmlprocess.txt"/></utility><utilitycommand="paraprof"type="view"group="tau"><argumentvalue="-f psrun *.xml"/></utility></analyze></tool><toolname="Valgrind"><execute><!-- Use this line only if valgrind is a shell script. The plugin can not execute scripts natively --><utilitycommand="bash"group="inbin"/><utilitycommand="valgrind"group="valgrind"/></execute></tool><toolname="CAT"><execute><!-- Use this line only if valgrind is a shell script. The plugin can not execute scripts natively --><utilitycommand="cat"group="cat"/></execute></tool></toolset.01>

Example: TaskFinder

The following example shows the integration of an existing command-line dynamic tool, TaskFinder, into Eclipse via the PTFw.

TaskFinder does dynamic instrumentation of binaries and samples the execution to provide suggestions of region that could be parallelized.

The following image shows the launch UI dialog in which the user enters arguments used by the launching of the TaskFinder tool. This is specified in the execute workflow stage of the tf.xml.
The use begins the profile launch from the profile icon in the eclipse toolbar, and is presented with the launch dialog. Create a "Performance Analysis" launch configuration and fill in as follows:

1. User indicates executable to launch on the Main tab

2. User indicates application args, if any, on the Arguments tab

3. User indicates perf tool, on the Performance Analysis tab, on the Tool Selection tab within that

4. User enters args, on the Performance Analysis tab, on the TF Options tab within that. Note that the "TF Options" tab is entirely configured from the tf.xml. There can be multiple pages here if desired. Note the arguments field is automatically generated (large Text area detailing how args will be passed to the tool). Need to document how to show/hide this. (note: enabled by default. displayoptions="false" turns it off.)

the console log (view at the bottom with red text) as the executation takes place, and

the eclipse view that is generated from the parsed output of the TaskFinder (command-line) tool after launch and execution (just above the console view). This is specified in the analyze workflow stage of the tf.xml. This shows how the PTFw can integrate with standard eclipse plug-in code to display results. Double-clicking on an item in the TaskFinder view navigates to the associated source code line in the eclipse editor.

The plugin.xml (if used to imbed the tool xml; it can also be specified by the end user in a preference page)
used by TaskFinder includes the following two extensions:

workflows - this specifies the location of tf.xml (shown below) which specifies how the tool is launched and how to present the UI to the user.

dataManagers - this specifies the post-processing to be performed: in this case, a java class within the TF eclipse plugin which parses the output file, determining its location from the PTFw, and showing the results in an eclipse view.

Widgets from XML

argument display - enabled by default. displayoptions="false" in the optionpane tag turns it off

See the TaskFinder examples above for examples of these

No widget - an argument is added to the launch arguments, but nothing appears in the UI due to visible="false"

<togoptionoptname="-anaalgo"visible="false">

Checkbox - this is the default widget. In this example, the label is "Multithreaded", the option added to the executable arg is "-mt", and the default state="true" indicates the checkbox is initially checked. If the state is false, the argument is not included.

From the PTP update site (See release notes from main PTP wiki page for details), or from the archived update site, or from a local update site e.g. on a CD (e.g. from a PTP tutorial session), install:

The PTP Performance Analysis Framework

The rest of PTP should not be necessary for just PTFw usage on a CDT app

The core of PTP is required for the parallel performance PTFw component

Photran (www.eclipse.org/photran) is required for the Fortran PTFw component

FAQ

Will more SWT widgets be added to those currently available?

Currently the UI information from the tool.xml file only handles checkboxes, text entry boxes, and File specifications via file dialogs.

We agree more widgets would be useful.

Wyatt has done some more extensive UI work by writing Java code via an eclipse extension to the org.eclipse.ptp.configurationTabs extension point.

So if you need more widgets immediately, you can use this extension point to implement them for your applicaition

Will the profile launcher handle remote launchers?

We have not tested this. PTP parallel launch can launch remotely, but we have not tried it.

We agree this is a useful feature.

The framework can handle two types of launch: CDT launch and Parallel Launch.

Parallel Launch (via PTP) is now available. Again, we have not tested this with the Perf Framework

CDT remote launch is not yet available but should be as part of RDT (Remote Development Tools) being developed by IBM Toronto, which should be available in the near future. At that time we can evaluate adding a remote CDT launch to Perf Framework.

File upload/download feature is needed

The PTP Remote Tools, used by PTP remote launch/monitor/debug have the tools to be able to do this already, but this is not built in to PTP launch currently. PTP 2.1 will have this capability.

Perf FW does not do this currently.

If you just need to access remote information from a class that is called in your plug-in extension (like in my Taskfinder Post-processing class) then that should be do-able now by using RemoteTools directly.

Are instrumentation tools available?

TAU utilizes several eclipse features for instrumentation - need to describe here.

Instrumentation of source code may be added to PTFw in the future. Beth has a rough framework for instrumenting java code and hopes to extend that to PTP-compatible sources (C, etc) in the future.

What does PTFw require?

PTFw does not require PTP core or PLDT, unless you want to use the Parallel application launch. It can be installed stand-alone