There are a number of files and folders on computers in an HPC cluster that are important for system security. For example, job submission and job activation filters are important because they run with the permissions and privileges of the System account, which means that someone who can change and run a filter can gain control of the system. This topic lists files and folders that are important, describes why they are important, and tells how their location is determined.

To help prevent an unauthorized user from changing important files, track and protect the computers and folders where these files are stored, and as appropriate, limit permissions on the files themselves. For example, if HPC cluster administrators export node templates to XML files, track and protect the computers and folders where these XML files are stored, and as appropriate, limit permissions on the XML files.

Files that contain saved or exported information in XML format

The following XML file types are important in an HPC cluster because they store information about nodes, jobs, tasks, or diagnostics. These files are often used to “export” information, that is, to save information that can be imported later for use in a different environment (such as for testing or production).

Important

With Windows HPC Server 2008 R2, you (an HPC administrator) can create all of the following types of XML files, and users can create job or task XML files. When you create such files, you can choose the locations where they are stored, and you can move or copy them to other locations, increasing their usefulness but also increasing their vulnerability. Therefore, for the reasons described in the following list, it is important to review and track the way you store these XML files.

Node templates (exported as XML files)

This section describes node templates that are exported as XML files.

Note

Node templates that are exported as XML files are different from “node XML files” (described in the next section).

Security considerations: Node templates work with associated files such as images, driver files, and custom disk-partitioning files (Diskpart.txt files), to control the installation and maintenance of compute nodes. When you create node templates by using HPC Cluster Manager, they are stored in one of the SQL Server databases that support the cluster, but node templates can also be exported into an XML file format. Protecting node-template XML files, and associated files such as images, drivers, and custom Diskpart.txt files, helps ensure that the nodes will deploy as expected. It also helps protect against installation of malicious software (malware) or other unintended software on compute nodes.

How node templates are exported or imported as XML: In HPC Cluster Manager, two of the actions that are available when you select a node template from a list are Export and Import. You can also perform these actions by using cmdlets in HPC PowerShell.

Node XML files

This section describes node XML files.

Note

Node XML files are different from node templates that are exported as XML files (described in the previous section).

Security considerations: Node XML files provide a flexible way to specify certain characteristics for nodes that otherwise would either be generated automatically or would have to be configured manually. For example, node XML files can specify custom computer names that are not automatically generated as a series. Protecting these files helps ensure that nodes have the characteristics that you intend, and helps prevent an attacker from connecting to and controlling compute nodes.

How node XML files are exported and imported: You can use HPC Cluster Manager to generate (export) a base node XML file, and then edit the file. Then you can use HPC Cluster Manager to add compute nodes by importing the edited node XML file. You can also perform these actions by using cmdlets in HPC PowerShell.

Job and task XML files

This section describes job and task XML files.

Note

“Job and task XML files” are different from job templates that are exported as XML files (which are described in the next section).

Security considerations: When a job owner or HPC cluster administrator saves or exports settings for jobs and tasks, the resulting file is a job or task XML file. If someone tampers with a job or task XML file and the file is later used, it can cause the job or task to run in an unexpected way. If the file was deliberately damaged, the job or task might not run.

Note

While protecting job and task XML files, you (as an HPC administrator) can also limit or influence how jobs are run, by using job templates or job submission and activation filters. Job templates and filters are described later in this topic.

How job and task XML files are created and run: A job owner or HPC cluster administrator can use HPC Cluster Manager or HPC Job Manager to save (or export) job or task XML files. To do this, under Job Management, in the New Job properties sheet, click the Save Job XML File button (or on the Edit Tasks tab, the Save Task File button). For a defined job or task that is selected in a list, the Actions pane includes Export Job or Export Task. You can also perform these actions by using cmdlets in HPC PowerShell.

After a job or task XML file is created, there are several interfaces that can be used for running a job with that file. The job can be run through HPC Cluster Manager or HPC Job Manager by selecting New Job from XML File and selecting the file. The job can also be run by using the New-HPCJob cmdlet with the -JobFile <string> parameter. (It can also be run by using job commands, although this is not mentioned in the following “Additional information” list).

Job templates (exported as XML files)

This section describes job templates that are exported as XML files.

Note

Job templates that are exported as XML files are different from “job XML files” (which are described in the previous section).

Security considerations: A job template allows you to associate a set of default values and value constraints for job properties with a particular set of users. When you create job templates through HPC Cluster Manager, they are stored in one of the SQL Server databases that support the cluster, but they can also be exported into an XML file format. Protecting job templates (in any form) helps protect the cluster from denial-of-service (DoS) attacks or other attacks where jobs that use the templates are run in an unexpected way. It also helps control which users can run a particular job or set of jobs.

For information about setting job template permissions, see the following “Additional information” list.

How job templates are exported or imported as XML: In HPC Cluster Manager, two of the actions that are available when you select a job template from a list are Export and Import. You can also perform these actions by using cmdlets in HPC PowerShell. When exporting a job template as an XML file, you specify the location and file name.

Custom diagnostic tests (saved as XML files)

The following list describes custom diagnostic tests that are saved as XML files.

Security considerations: In Windows HPC Server 2008 R2, you (as an HPC administrator) can add custom diagnostic tests, either tests you create or tests you obtain from a solution provider. A diagnostic test can potentially run a powerful script, so it is important to control the tests that can be run on your HPC cluster.

How custom diagnostic tests are added: You can add custom diagnostics tests by using the Add-HpcTest cmdlet.

Files with a location that is determined during an installation or configuration process

For most of the files in the following list, the location is determined during an installation or configuration process, and no additional steps are needed to ensure that the HPC cluster can find the files. The exceptions are Excel add-ins (XLL files), which may include user-defined functions (UDFs). Because XLL files use a path variable, if you add XLL files to a custom location (for example, a shared folder on the network), you can then adjust the path variable so that the HPC cluster can locate the XLL files.

Diagnostic test result files

The following list describes diagnostic test result files.

Security considerations: Depending on the tests that have been run, diagnostic test result files might contain information about the cluster that could help an attacker determine where the cluster might be vulnerable.

You can also use a cmdlet in HPC PowerShell to export the results of diagnostic tests, that is, to create a set of files that can be viewed later in HPC Cluster Manager. With exported test results, use the same level of security precautions as you would with any other stored test results.

SOA service registration files and the DLL files that they point to

The following list describes service-oriented architecture (SOA) service registration files and the DLL files that they point to.

Security considerations: The SOA service registration files are XML files that register SOA services on the cluster. The files also contain settings that configure service behavior and that point to the location of the DLL (assembly) for that service. Protecting the service registration files, and the associated DLL files, helps ensure that the services run as intended. Protecting these files can also help protect against the installation of malicious DLLs.

Path: The service registration files can be installed centrally (on the head node) or locally (on multiple nodes). For more information, see “Deploy and Edit the Service Configuration File” in the following “Additional information” list.

Excel add-ins and any associated user-defined functions

The following list describes Excel add-ins (XLL files), which may include user-defined functions (UDFs).

Security considerations: Excel add-ins (XLL files), and any UDFs that are included in them, can perform actions on compute nodes or workstation nodes. Protecting the folders where XLL files are located can help prevent malicious software (malware) from being run in your HPC cluster.

Path: The path used by XLL files is specified by the XllPath variable, which is described in “Advanced Service Configuration for HPC Services for Excel” in the following “Additional information” list.

Files in temporary folders on workstation nodes

If your HPC cluster includes workstation nodes, your security planning should include a review of the location of job input and output folders on each workstation node, and how those folders (and the files they contain) are protected. The location of these folders can vary, depending on the application. For example, an application may create a folder called c:\Temp\Output and store files in it. Review the following considerations for these folders:

Limit permissions on these folders, so that unintended users cannot access them. This can help keep such users from tampering with files in those folders (which could have an effect on how jobs run) or viewing sensitive information in those files.

Consider placing quotas on these folders to limit folder size. This can help prevent a denial-of-service attack that uses jobs to write large quantities of data to these folders, filling up the disks on the workstations and causing them to become unusable.

Also, it can be important to review the software or scripts that run jobs on workstation nodes, to ensure that data is not left behind in input or output folders after the script or software runs. This can help prevent unintended access to that data, and help prevent the disk from filling up with data mistakenly left behind by past jobs.

Caution

We strongly recommend that you do not add workstation computers to your HPC cluster on which users have local administrative privileges. For more information, see "Considerations for user accounts used on the workstation computers" in Requirements for Adding Workstation Nodes in Windows HPC Server 2008 R2 (http://go.microsoft.com/fwlink/?LinkId=202684).

An HPC cluster includes a small number of specialized files for HPC session configuration, power control, and unattended installation. These are powerful files, and can be customized. If you customize these files, you should track and handle them carefully, and protect the folders where you store them. For example, you should limit access to a temporary folder where you are modifying a copy of the unattended installation file called Unattend.xml. If an attacker makes changes to the file without your awareness, you might later deploy that version of the file, which would result in unintended changes to the way that the compute nodes are installed.

These specialized files are described in the following list. They are all installed on the head node in the \Bin subfolder. For example, if you are using the default installation path, the files are available in the following folder:

C:\Program Files\Microsoft HPC Pack 2008 R2\Bin

The files are as follows:

HpcSession.exe.config is a configuration file that you can use to customize your HPC configuration, for example, to provide the head node with the name of Windows Communication Foundation (WCF) broker node that is running on a failover cluster instead of running as a stand-alone server.

CcpPower.cmd is the centralized power control script for starting and stopping compute nodes. A default version of the script is installed with HPC Pack 2008 R2. The default version calls standard operating system commands to stop and restart compute nodes. However, you can modify CcpPower.cmd so that it calls custom power control scripts that control the way that compute nodes are started, stopped, or restarted. An example of a custom power control script is an Intelligent Platform Management Interface (IPMI) script.

Note

It is important to track and protect not only CcpPower.cmd, but also the custom scripts that it calls. This helps ensure that compute nodes will start, stop, and restart as expected. It also helps protect against installation of malicious software (malware) or other unintended software on the compute nodes.

Startnet.cmd and Unattend.xml are files that are associated with the Pre-Boot Execution Environment (PXE) and unattended installation. You can use these files to customize the process of unattended installation for your compute nodes.

Log files that contain detailed information such as trace information from SOA sessions

In Windows HPC Server 2008 R2, you can collect detailed information in log files and use various methods to copy these log files to other locations. For example, you can collect diagnostic traces for SOA sessions and save the traces in WCF log files. WCF log files can then be copied (by using HPC Cluster Manager or a cmdlet in HPC PowerShell) from the compute nodes to a single location so that you can view the log files with Service Trace Viewer.

Because detailed log files (WCF log files or other detailed log files) can contain important information such as information about the HPC cluster topology and nodes, carefully track and protect the locations where you save them. Also, if you collect traces or other detailed logs, we recommend that you remove these files after you are finished reviewing them. For additional information about log files, see the following topics: