Tag Archives: Citrix XenApp 6.0

Unable to remove a XenAPP 6.x server from the farm. The server still shows up in the delivery console. The elegant way of removing it gracefully was not possible because it was a Citrix Provisioned server.

Help!

Right now I am researching a Citrix published applications and content redirection issue that happened once in the last three years in XenApp 6.0. I simply unassociated the files association, applied, then re-associated. No issues…. now it is happening and opening PDF reader instead of Office 2007 (don’t ask… I am afraid of introducing the new ribbon interface to the user community without training).

Uncheck “show all available file types for this application” (if necessary), click apply, and then recheck the option. This should refresh the file extensions added to the registry in previous steps. This is exactly what I did a couple years ago, but it isn’t working after a day or less. Also this time around it is opening PDF reader instead of office 2007?????

Server to Client content redirection; this makes it possible to start URLs locally on the client PC. Embedded URLs are intercepted on the server running Presentation Server and sent to the client. The user’s locally installed browser is used to display the website.

Client to Server content redirection; when a user double-clicks a file the corresponding application is started on the Citrix server.

To display a list of file extensions and their associations, type assoc at a command prompt, and then press ENTER.

To display the association for a specific file extension, type assoc .<xxx> at a command prompt, and then press ENTER, where <xxx> is the file extension whose association you want to view.

To change the association for a specific file extension, type assoc .<xxx>=<file type> at a command prompt, and then press ENTER, where <xxx> is the file extension whose association you want to change, and <file type> is the program, dynamic data exchange (DDE), or OLE object you want to associate with the file extension.

To display the open command to use when launching a certain file type, type ftype <file type> at a command prompt, and the press ENTER, where <file type> is the program, dynamic data exchange (DDE), or OLE object you want to associate with the file extension.

To change the program association for a specific file type, type ftype <file type>=<program path> at a command prompt, and the press ENTER, where <file type> is the program, dynamic data exchange (DDE), or OLE object you want to associate with the file extension and <program path> is the path to the executable used to open the application.

If the file type for the extension you are wanting to add already exists (for example Excel.Sheet.12), all you would have to do is associate the new extension with that file type. This would allow the new extension to open with the program associated with that file type.

If the file type for the extension you are wanting to add does not exist or you do not know what its file type is, you would have to add both the association and the file type. The example below associates the extension .tstx with the file type test.document. It the associates the file type test.document to open with the program test.exe. This would allow any documents with the extension .tstx to open with test.exe.

Once the association has been added to the registry, complete the following steps in the Citrix Management Console to view the new file associations:

Within the console, browse to Citrix server from which you are running the console (this should be the same server on which you added the file extensions)

I was working on a new Citrix PVS image for one of my current projects and the test before running the P2PVS.exe were good, all my apps worked like they should so I prepped the server for the imaging (preparing antivirus, last PVS optimizations and a full pre-scan of my VM) and I ran P2PVS.

After a successful P2PVS I ran into a couple of problems: The RES Workspace Manager Service didn’t start and one of the key components of this image (an outlook plugin) gave a weird error that didn’t show up at first. I did some troubleshooting and found that the RES Software WM Service couldn’t start because the executable wasn’t found, I could browse to the directory and the res.exe was there. Luckily Barry Schiffer came up with the idea to check the shortnames which are used to start the RES WM Service and a lot of software still use the shortnames for the Office folder as well so apparently we had shortname issues with Citrix XenApp and Citrix PVS.

Note: Even in 2012 some applications still rely on 8.3 names. Scanning the for commonly used director names (i.e. C:PROGRA~1) can help revealing affected programs.

After changing this value back to “0” we were able to run a successful P2PVS and the errors in RES Workspace Manager and the key application were gone. Here’s somebackground info on why to disable shortnames and how to check this from the command line.

To disable/enable 8.3 naming convention, users can issue the set command as follows:

Kees Baggerman

Kees Baggerman is a senior performance and solution engineer for desktop and application virtualization solutions at Nutanix. Kees has driven numerous Microsoft and Citrix, and RES infrastructures functional/technical designs, migrations, implementations engagements over the years.

Citrix XenApp commands provide an alternative method to using the console for maintaining and configuring servers and farms. Citrix XenApp commands must be run from a command prompt on a server running Citrix XenApp.

Security Restrictions

Information

This article contains examples of PowerShell script designed to launch XenApp commands and being able to perform data manipulation between commands in one script.

Requirements

Microsoft Windows Server 2008 R2

Citrix XenApp 6.0 or 6.5

Microsoft PowerShell

Background

In some cases administrators have the need to capture information from PowerShell and also manipulate it for other wrong commands. Many PowerShell commands can be combined together using pipes. For example: Get-XAApplication | Clear-XAApplicationLoadEvaluator

This command removes the load evaluator from each one of the applications.

In this one line command example, the administrator has no opportunity to see the data captured by the Get-XAApplication before it is used under the Clear-XAApplicationLoadEvaluator, this article contains examples of other ways to launch this command so there is space in between for some data manipulation or data reporting.

PowerShell Script Designed to Launch XenApp Commands

Following are two examples to understand PowerShell script designed to launch XenApp commands:

Example 1

Usage

The following PowerShell command reports each application name and removes its load evaluator.

This script works the same as the preceding one line command. The data between the steps can be worked on in this approach. This method provides flexibility to report or validate with certain condition.

Example 2

Usage

The following PowerShell command reports each application name, removes its load evaluator, except on applications from the Microsoft Office Suite.

Adding extra steps to the script can verify each application name and validate if it contains the word Microsoft, if it exists load evaluator is not removed, if it does not exist the load evaluator for the application is cleared.

In certain areas of the script, code is added to write messages to the console, this is useful for farms that have a lot of applications. Using this message an administrator gets an idea that the script is running and the progress so far. Also this script contains a count result to report how many applications were not modified in total.

Conclusion

This article implies that the XenApp PowerShell commands are quite flexible, similar to the native commands existing in Windows PowerShell. Many tasks can be completed using the same logic. This can be used for monitoring, reporting, configurations, and so on. The one line commands as mentioned at the beginning of the article can be used, but to have all or most of your routines simplified or even automated this is a good approach. This helps reducing resources in routine and replacing it with higher priority tasks.

Applicable Products

While managing your Citrix XenApp 6 farm from the comfort of the Citrix Delivery Services Console GUI tool is a very powerful thing, you should know that XenApp 6 also has a subset of command-line tools that are at your disposal and can help with advanced farm management and troubleshooting:

1. Altaddr: If you’re a Citrix old timer, you probably remember using this tool before secure gateway was available, in order to extend the published applications to users outside the secure network. In essence, what Altaddr allows you to do is give your XenApp server an alternate IP address. Traditionally this was an outside facing public IP address or some kind of a NAT. Of course, the down side is if you have 100 XenApp servers, you would need 100 public IP addresses. Altaddr is not in use or popular anymore, but it is still there if you have the distinct situation where it makes sense.

2. App: Typically used for scripting or customizing an application’s behavior. You can use App in your scripts to control the application’s environment or to satisfy its prerequisites before the application runs.

3. Auditlog: A handy utility that, for the most part, dumps the security event log from Windows and allows you to pipe that log into a text file. You can use the output to audit who logged in or out of the server and when. This could come in handy if you are troubleshooting, for example, if your print service crashes; you can see who logged in during that time and if they are mapping a bad driver, etc.

5. Ctxkeytool: You can use this utility to generate an encryption key that can be used when enabling IMA encrypted communication in the Citrix farm.

6. Ctxxmlss: This handy little utility allows you to modify the XML port on the XenApp servers should you need to change that port for any reason.

7. Dscheck: I use this tool frequently to check for consistency on the Citrix IMA data store. It is particularly handy in conjunction with the /clean switch, which you would use to clear out any inconsistencies in the database.

8. Dsmaint: All interaction with your data store can be manipulated using this tool. For example, if you want to change data store database servers, you can use Dsmaint to do so. You can also use it to verify the local host cache on the XenApp server. The Local Host Cache or LHC is a subset of the data store database that runs on each XenApp server. Sometimes, as a troubleshooting step, you may want to verify the accuracy of that data or even refresh it altogether.

9. Enablelb: This utility restores XenApp servers into the load balancing mix after they have failed Citrix health monitoring tests.

10. Icaport: Here’s a too that allows you to change the default port that the ICA protocol typically runs on.

11. Imaport: allows you to change the default port that the IMA protocol runs on

12. Query: Arguably the most useful command-line tool, I use the query command for troubleshooting, for verifying load on the server, for just about anything administration related. If I need to know, I always start with the query command. It has a subset of values:

Query Farm or QFARM returns information on the XenApp servers in the farm and which one is a data collector

Query Session returns information about the sessions that are running on the XenApp server

Query Process will return the all the running processes on the server

Query User or QUSER returns information about all the users on the server

All these tools are available to you with the default installation of XenApp 6. However, as you can see from the list above, the number of Citrix XenApp 6 command-line tools has been shrinking with each new version and that is not because of insufficient development. Instead, it’s because Citrix is kind of following the Microsoft lead and porting most of its command-line tools to PowerShell. Event so, XenApp 6 has a very rich PowerShell footprint, wihich I plan to explore more in future blogs.

Many rich graphical applications such as CAD applications may benefit from allocating more than 1 vCPU per VM (1 vCPU is the default) via XenCenter or the command line. This will be of interest to many of those evaluating XenDesktop and XenApp HDX 3D Pro GPU pass-through and GPU sharing technologiesand can lead to noticeable performance enhancements. The performance gains possible though are highly application specific and users should evaluate and benchmark for their particular application and user workflows.

Overprovisioning

Virtualisation allows users to define more virtual vCPUs than there are physical CPUs (pCPUs) available. A key driver for server consolidation is to virtualise workloads which spend much of their time idle. When virtualising, the expectation is often that when one of the VMs requires increased processing resources, several other of the VMs on the same host will be idle. This type of statistical multiplexing enables the total number of vCPUs across all VMs on a host to be potentially much greater than the number of physical cores.

In the context of virtualising graphical workloads (such as those that benefit from GPU-accelerations) this means you really should look at the demographics and usage patterns of your user base – I blogged about this a few months ago. If you are streaming video or similar constantly to all users there may be little scope to overprovision but if your users use applications in ways that use the CPUs in bursts there is often significant scope to consolidate resources.

Reservation of Capacity for Dom0

Before proceeding, it is worth noting that the XenServer scheduler ensures that dom0 cannot be starved of processing resources, given that it is crucial for normal operation of the platform. The scheduler ensures that dom0′s CPUs are allocated at least as much real CPU cycles as they would receive if they had a dedicated physical core. In other words, today, we ensure that dom0 uses the entirety of 4 physical cores, if it needs them. The maximum amount of CPU resource that dom0 can use is the equivalent of half the number of physical cores on the system.

This means that on a heavily loaded host, capacity for VMs will be the number of physical cores, minus 4.

vCPU Consolidation Ratio

As a rule of thumb, the ideal consolidation ratio will mean that ~80% of physical CPU utilisation is achieved under normal operation, to leave a 20% margin for bursts of activity. Therefore, the consolidation ratio that can be achieved is highly variable. It is conceivable that a 10:1 consolidation ratio is not _theoretically_ unreasonable.

Having said this, very high consolidation ratios can result in unexpected performance impacts. For example, when attempting to use 150 vCPUs on a 54 physical core box, 4 cores will go to dom0. This then means that we have a consolidation ratio of 3:1 (quite low). However, if all the CPUs are reasonably heavily loaded, this means that each vCPU will receive a 30 millisecond time slice, then have to wait for 90 milliseconds before its next time slice. The higher the consolidation ratio, the longer the interval between the time slices.

This waiting can impact aspects such as TCP network throughput, because packets appear to take far longer than expected to arrive at the VM. In the worst case, with very high consolidation ratios, if a VM receives a time slice extremely infrequently (once every few seconds) it will appear to freeze.

There is no absolute limit/number for the vCPU consolidation ratio. Clearly the greater the ratio, the greater the expectation is that the VMs are going to spend a greater fraction of their lives idle. The way to understand whether a consolidation ratio is appropriate on a running system is to examine the overall physical utilisation of the host.

cores-per-socket

Windows client operating systems support a maximum of two CPU sockets e.g. Windows 7 can only use 2 sockets. XenServer’s default behaviour is to present each vCPU with 1 core per socket; i.e. 4 vCPUs would appear as a 4 socket system. Within XenServer you can set the number of cores to present in a socket. This setting is applied per guest VM. With this setting you can tell XenServer to present the vCPUs as a 4 core socket.

Users will need to assert whether any additional constraints are imposed by the licensing conditions of their graphical software and operating systems.

Maximum number of vCPUs per VM

The underlying Xen Hypervisor ultimately limits the number of vCPUs per guest VM, see here. Currently the Citrix XenServer limit is lower and defined by what is auto and regression tested, this limit is currently 16 vCPUs, you should reference the XenServer Configuration Guide for the version of XenServer you are interested in, e.g. for XS6.2, see here. It is therefore likely that higher numbers of vCPUs will work which maybe of interested to unsupported users, however for those with support we advise you stick to the supported limits. If there was sufficient demand raising the supported limit is something we would evaluate.

The XenCenter GUI currently allows users to set up to 8 vCPUs, for higher numbers users must use the command line CLI.

Anecdotal information

I’ve collected some of the feedback I’ve had from those involved with vGPU deployments and although it cannot be considered official Citrix endorsed best practice, I felt it was useful to share:

Most of the major CAE analysis apps will probably benefit from >2 cores although in some cases it depends on licensing. So Ansys, Abaqus, SolidWorks analysis module etc. all usually take as many cores as they find. Some applications will limit you to 2 cores unless you license an HPC pack or similar though.

Similarly ray tracing apps even when they are GPU accelerated will often use multiple cores (in addition to the CPU) although in current versions they may not take all cores since they want to leave some for interactivity.

We’ve had some feedback from customers doing analysis they expect applications use all cores so it runs faster, and they have noticed in a virtualized environments when CPU cores get over subscribed (please be aware of the need to understand overprovisioning above).

Our general rule is that to do good 3D interactive graphics you need a minimum of 4 vCPU cores, and for workstation cases 8 vCPU cores not because 8 will be used during graphics but because most high-end workstations are 8 cores and users and apps expect it.

Likewise during graphics all 4 vCPU cores may not be 100% busy but it’s likely that 2 probably are between the various needs of application code, graphics driver executions, operating system, virus checkers etc. So if you only have 2 vCPU cores we frequently observe a noticeable impact on performance.

How to investigate vCPU usage and server consolidation

One useful tool for investigating the performance of a system is xentop,details of xentop and other useful tools are available here. This allows you to understand the overall load on the system, and hence whether you are below the 80% load recommended above.

Over the last few versions of XenServer we’ve been increasing the number of metrics available and also improving the documentation and alerting mechanisms around these metrics. These new metrics complement those comprehensively documented in Chapter 9 of the XenServer 6.2 Administrators Guide.If you are interested in measuring the vcpu overprovisioning from the point of view of the host, you can use the host’s cpu_avg metric to see if it’s too close to 1.0 (rather than 0.8, i.e. 80%): If you are interested in measuring the vcpu overprovisioning from the point of view of a specific VM, you can use the VM’s runstate_* metrics, especially the ones measuring runnable, which should be less than 0.01 or so. These metrics can be investigated via the command line or XenCenter.

Investigating cores per socket and thread per core

The Xen hypervisor utility xl can be of some use for debug and diagnosis in XenServer environments, particularly the inquiry options such as info [-n, --numa], by which you can query hardware information such as cores_per_socket and threads_per_core and similar data that you might want to log or keep in benchmarking assessments. Further details and links to this utility are detailed on this page, alongside other useful tools for use in XenServer environments.

Fine tuning

This type of fine tuning is indeed fiddly, and we are open to user feedback on how this could be improved, including the documentation of this type of information. In fact our documentation team are specifically watching this post to see what comments it solicits. A large number of you have already written blogs and articles of your experiences of best practice in your own “real life” XenServer deployments and they are of interest to us and many customers, see:

For many of these applications a pinning strategy that considers the NUMA architecture of the server can also lead to performance enhancements. It is highly application and server specific and you really do need to investigate this with your particular applications, user demographic profile and usage patterns. There is some information linked to at the bottom of this page, but really it’s one best addressed in a future blog….. in the meantime I’d be interested to hear user anecdotes if anyone has tried this though.

One Comment

Indeed it should also be noted that for many thin clients, one of the VCPUs may be used pretty much entirely just for reformatting and dealing with the video output, so we — in fact — always assign two VCPUs for all our VMs at a minimum to provide extra CPU cycles where they are needed. With as many as 80 or more VMs running on a 32-VCPU server youd think this would be inefficient, but with the environment we have, the dom0 load on a server is typically just 20-40%. Even with 20 or so pretty active users, it rarely climbs above 60%. Note that we allocate eight dom0 instances, each with 4 GB of memory, to be able to accommodate heavy use or boot storms on the XenDesktop environment and stress tests have shown that four dom0 instances at times were just not quite enough. With 256 GB of memory, the extra 16 GB were worth allocating for that purpose.

The use of a GPU/vGPU can cut VCPU load on a VM by 30 to 50%. We have not had the chance to see how well this scales with many users, but the impact is clearly not negligible. Current testing underway to see if allocating a portion of a GPU to a VM is worse or better with many users than, say, creating a XenApp VM running on a XenServer and leveraging GPU passthrough for the entire XenApp server and having the VMs leverage the GPU that way. And of course, you can do both! The recent articles on PVH are, of course, tantalizing.

We have not tried CPU pinning and in an environment with this many VMs and a parade of users coming and going, it probably wouldnt make that much sense. The CPU multiplexing certainly doesnt seem to incur a big penalty in this environment, at least from qualitative observations.

We have also experimented with various XenServer parameters, such as modifying the txqueuelen and some of the network and TCP parameters with apparently positive effects. Linux has such a plethora of tuning options, which of course should be modified with caution.

How Does Web Interface Work

In a Microsoft Windows environment, Web Interface works with Internet Information Services (IIS) to provide users with access to published resources. Users will use a standards based Internet browser or the Citrix Receiver to access their resources.

A Web Interface (WI) server will have one or more XenApp Web sites or XenApp Services sites configured. Each site will be configured for one or more XenApp farms. Each XenApp farm will have one or more XML Brokers listed to handle user authentication and resource enumeration. Once a user has been authenticated and selects a published resource, the Zone Data Collector (DC) is contacted. The DC determine s if the user has an existing session on the server hosting the published resource and if a session exists, that session is reused (called Session Sharing). If the user does not have an existing session, a session is created and the published resource is started.

The XML Broker will also request a session ticket from the Secure Ticket Authority (STA). The STA is responsible for issuing session tickets in response to the request to connect to the published resources. These session tickets form the basis of authentication and authorization for access to the published resources.

A Web Interface server is normally placed in a DMZ; however, it may be placed inside the corporate network. Web Interface requires no XenApp components to be installed. A Web Interface server is not typically a member of a XenApp farm, nor is it typically a member of an Active Directory domain. However, in the smallest of networks, it is possible and common for Web Interface to be deployed on a XenApp farm member and/or on a member of an Active Directory domain.

First, let’s stop, take a step back and review some basics.

What is a XenApp farm? A XenApp farm is a group of XenApp servers that can be managed as a unit, enabling the administrator to configure features and settings for the entire XenApp farm rather than being required to configure each server individually. All the servers in a farm share a single data store.What is a data store? The data store provides a repository of persistent information about the farm that each server can reference, including the following:

Farm configuration information,

Published resource configurations,

Server configurations,

XenApp administrator accounts,

Printers,

Printer drivers,

Policies,

Load Evaluators, and

Folders.

What is a Zone? A Zone is a logical grouping of XenApp servers that share a common zone data collector. Zones allow the efficient collection of dynamic farm information. Each zone in a farm has exactly one data collector. All of the member servers in a particular zone communicate their dynamic information to the data collector for their zone.

What is a zone data collector? A zone data collector is a server that stores and manages dynamic information about the XenApp servers in a zone, including:

Published resource usage,

Server load,

User sessions,

Online servers,

Connected sessions,

Disconnected sessions, and

Load balancing information.

The data collector shares this information with all other data collectors in the XenApp farm.

All XenApp servers in the farm use the Independent Management Architecture (IMA) service and protocol in server-to-server communication. IMA also is used by the Access Management Console or the Delivery Services Console or AppCenter (depending on the version of XenApp used) to allow XenApp farm administrators to manage and configure various XenApp farm and server settings.

What is an XML Broker? The Citrix XML Broker functions as an intermediary between the XenApp servers in the XenApp farm and the Web Interface. When a user authenticates to the Web Interface, the XML Broker:

Receives the user’s credentials from the Web Interface and queries the XenApp farm for a list of published resources that the user has permission to access. The XML Broker retrieves this application set from the IMA system and returns it to the Web Interface.

Upon receiving the user’s request to launch a resource, the DC locates the servers in the farm that host this application and identifies which of these is the optimal server to service this connection based on several factors. The DC returns the address of this server to the Web Interface.

The XML Broker is a function of the Citrix XML Service. By default, the XML Service is installed on every server during the XenApp installation process. Multiple XenApp servers can have their XML Service specified in Web Interface to allow those servers to function as a XML Broker. The XML Service on the other farm servers still runs but is not used for servicing end-user connections.

The Secure Ticket Authority is also installed on every XenApp server.

For most small to medium sized XenApp farms, one XenApp server is dedicated to be the Zone Data Collector, XML Broker and STA server. In some large XenApp farms, it may be necessary to dedicate a XenApp server for each of the three roles.

Dedicating a XenApp server for each role is easy to do. You would have three XenApp servers with no end-user applications installed. In the Zone settings for the farm, you would configure one of the servers as the Most Preferred data collector and the other two as Preferred data collectors. The server to be dedicated as the XML Broker would only be used when an XML Broker needs to be entered. The server to be dedicated as the STA server would only be used when an STA server needs to be entered.

Figure 1 illustrates the interaction between Web Interface and other servers in a XenApp farm.

Figure 1

Figure 2 shows some of the steps involved in the Web Interface process.

Figure 2

Step

Action

Graphic

1

A user connects to a Web Interface server from any device that has Citrix client software installed.

2

The user enters their credentials on the login page.

3

The web server reads the user’s credentials and forwards the credentials to the Citrix XML Service on the servers listed in the server farms.

4

If the user’s credentials are not valid, return to Step 2. If the user’s credentials are valid, the Citrix XML Service retrieves a list of resources from the XenApp servers the user has permission to access. This list of resources is called the user’s resource set. The Citrix XML Services returns the resource list back to the Web Interface server.

5

The Web Interface server builds a custom HTML web page consisting of the resources the user has permissions to run.

6

The user clicks one of the published resource icons.

7

The Citrix XML Service locates a server in the required farm that has an existing session for the user and the settings for the resource being launched match the settings for the resources running in the existing session. If those conditions match, the Citrix XML Service requests a session ticket and returns the server’s IP address and session ticket to the Web Interface server. If those conditions are not met, the Citrix XML Service requests a session ticket from the least-busy server and returns the server’s IP address and session ticket to the Web Interface server.

8

Web Interface creates a custom launch.ica file and sends the file to the user’s Citrix client.

9

The Citrix client software receives the file and initiates a session with the server specified in the file.

10

The published resource runs on the XenApp server and is displayed on the end-user device.