Σχόλια 0

Το κείμενο του εγγράφου

1

1.z/OS Infrastructure

WLM considerations:

Setting the WLM goals properly can have a very significant effecton application throughput. TheWebSphere for z/OS system address spaces should be given afairly high priority. As workcomesinto the system, the work classification of the enclaves should be based on your businessgoals.

Use STC classification rules to classify work for daemons, controller and servant regions.

Classify location service daemons

as

SYSSTC

or high velocity.

Classify controller regions as high velocity. These

regions do some processing to receive work into thesystem, manage the HTTP transport handler, classify the work and do other housekeeping tasks.

Classify servants regions withreasonably highvelocity goals.

Java garbagecollection(GC)runs under this classification. Java GC is a CPU and storageintensive process, so if you set the velocity goal too high GC could consume more of thesystem resources than desired. On the other hand, if your Java heap is correctly tuned, GC

for each server region should run no more than 5% of the time. Also, providing properpriority to GC processing is necessary since other work in the server region is stoppedduring much of the time GC is running.

JSP compilesalsorununder this

classification. If your system is configured to do thesecompiles at runtime, setting the velocity goal too low could result in longer delays waitingfor JSP compiles to complete.

Application Environment for work running under servants. This is the actual execution of theapplication. This work is classified under the CB classification as described below:



Classifications for application environment work is classified underCB.



Classification

basedupon server name, server instance name, User ID, or

transactionclass



Percentage response time goal is recommended.

These goals should be achievable.

Keep in mind, even administration console work is classified under CB.

A

goal that 80% of the work will complete in .25 seconds is a typical goal.

Velocity goals for

application work are not meaningful and should be avoided.



A reasonabledefault service classshould be definedfor CB, the default is SYSOTHER

which is a discretionary goal and on a busy system will provide very poor service.

Your goals can be multi-period. This might be useful if you have distinctly short and long runningtransactions in the same service class. On the other hand, it is usually better to filter this work into adifferent service class if you can. Being in a different service class will

place the work in a differentservant which allows WLM much more latitude in managing the goals.

The goals that are definedshould be attainable not optimistic.

2

CurrentFinding:

The first two steps of the controller

regionstart-up process invoke

theBPXBATCH shell script, anddoes not inherit the service classification of the started task (according to STC rules.) This cangreatly elongate the startup of a controller

region because

it will be classified on the OMVS rulesand if not handled appropriately on a busy system will cause multiple minutes delay beforeBBOCTL gets control.

The purpose of the additional step is to check the service level delivered, applied, and pending forWebSphere, and log the results.

best performance, ensure that LE and C++ modules are loaded into LPA. The SCEELPAdataset can be

added to LPA as well as the following sample PROGxx members in SCEESAMP:

CEEWLPA

(LE Runtime)

EDCWLPA

(C++ Runtime)

CurrentFinding:

2. WebSphere Infrastructure

JVM considerations:

The Java heap parameters also influence the behavior of garbage collection. Increasing the heapsize supports more object creation. Because a large heap takes longer to fill, the application runslonger before a garbage collection occurs. However, with a larger heap garbage collection takeslonger. Initial and maximum Java heap sizes can be specified using the-Xms and-Xmx options.The general recommendation is that these values should be the same.

Note that Java Heap information is contained in SMF records and can be viewed dynamically usingthe console command:

Ratio between the length of a single garbage collection call and the average timebetween calls.

This can be viewed as the percentage of time spent doing GC processing.



System paging activity (from RMF or another system monitor).

CurrentFinding:

If the heap free space settles at 85% or more and the percentage of time in GC processing is low,consider decreasing the initial and maximum heap size values because the application server andthe application are under-utilizing the memory allocated for heap. If system real storage isconstrained, overall performance may be improved by reducing the Java heap size, even if thepercentage of time in GC is higher than desired.

It is good for the Initial Heap Size to equal the Maximum Heap Size because it allows the allocatedstorage to be completely filled before GC kicks in.

Otherwise, GC will run more frequently than

4

necessary, potentially impacting performance.

Make sure the region is large enough to hold thespecified JVM heap.

Beware of making the Initial Heap Size too large.

While it initially improvesperformance by delaying garbage collection, it ultimately affects response time when garbagecollection eventually kicks in (because it runs for a longer time).

Paging activity on your systemmust also be considered when you set your JVM heap size.

If your system is already pagingheavily, increasing the JVM heap size might make performance worse rather than better.

Todetermine if you are being affected by garbage collection, you can enable Verbose GarbageCollection on the JVM Configuration tab.

The default is not enabled.

This will write a report to theoutput stream each time the garbage collector runs.

This report should give you an idea of what isgoing on with Java GC.

Here is the URL for GC techdoc help:

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD101216

Ensure

that the JITC

is enabled, especially in the production regions.

J2RE 1.4.2 IBM z/OS Persistent Reusable VM build cm142-20050929 (SR3)

(JIT enabled: jitc)Java(TM) 2 Runtime Environment, Standard Edition

Keep up with the most current version oftheJVM, almost every release improves performance.

The informational APAR II13519 can be used to identify the available levels of the JVM.

CurrentFinding:

Currency of WebSphere is also important. If you let your environment age significantly, upgradingWebSphere to fix a problem you’re encountering can become sizable.For example, if you need toapply a PTF that became available last month

and your current level is a year old you will need toapply

11 months worth of maintenance.

It might be desirable to use more controller regions thus reducing the number of servants under acontroller. This would make restarts a faster process and alsolessen the impact if a controllerregion is lost.

There

are numerous timeouts

and settings. It is assumed that the client has varioustimers/timeouts that should be reviewed to insure that they are in sync with the WebSpheretimeouts such as those listed

below:



Security cache

Ususally the defaults for size

and timeout values work ok



LTPA



HTTP



KeepAlive

The WebSphere Application Server class loading function uses asynchronous scope alarms instead ofJava timers. If you put the alarm manager into quiet mode, class reload only runs when the servercompletes the processing of a request. If the server is not processing any requests, the class reloadfunction does not run. To enable, set the following variable in the servant JVM Custom Properties

It might be good to review whether to use JDBC or SQLJ. The RebookDB2 for z/OS andOS/390: Ready for Java

(SG24-6435)may be helpful:

Ensure

DB2tracingis turned off. To do so, verify

the db2.jcc.propertiesFile jvm propertyhas been defined to specify a DB2 jcc properties file to the WebSphere Application Serverfor z/OS,ensure that the following tracestatementsin the

file are commented out if theyare specified:

jcc.override.traceFile=<file name>

db2.jcc.t2zosTraceFile=<file name>

CurrentFinding:

You will have to define more connections called threads in DB2.

WebSphere for z/OS usesa lot of threads. Sometimes this is the source of throughput bottlenecks since the server willwait at the create thread until one is available.

The Connection Pool statistics shoiuld be monitored as the workload increases.

CurrentFinding:

Make sure you are current with JDBC maintenance. Many performance improvementshave been made to JDBC. To determine the JDBC maintenance level, enter the followingfrom:

OMVS

java com.ibm.db2.jcc.DB2Jcc–version

It is

recommended

that you enable dynamic statement caching in DB2. To do this, modifyyour ZPARMS to sayCACHEDYN(YES) MAXKEEPD(16K). Depending on the application,this can make a very significant improvement in DB2 performance. Specifically, it can helpJDBC and LDAP queries.

Typical tuning of the DB2 environment should also be performed.

6

MQ Series

MQ

connections are being made using BINDINGS mode and

shared queues to CICSsystemswithin the parallel sysplex.

Ensure MQtracing

is turned off. To do so, modify the following MQ ZPARMS parameter asfollows:

TRACSTR=NO

Similarly,it is recommendedto turn off tracing in the channel initiator. Unlike base MQtracing, this parameter cannot be enabled dynamically. To turn tracing back on for debugpurposes, you will need to reassemble your MQ XPARMS. Enabling channel initiatortracing can degrade your system by 5-10%. Thefollowing parameter controls this tracing:

TRAXSTR=NO,

START TRACE AUTOMATICALLY YES|NO

Ensure that your logger configuration is optimal by using SMF 88 records. See the tuningsection ofz/OS MVS Setting Up a Sysplex

or the chapter on System Logger Accounting inz/OS MVS System Management Facilities (SMF)

for details. In any case, you shouldmonitor the logger to ensure that there is a sufficient size in the CF and that offloading isnot impacting the overall throughput. The transaction logs are one of the only shared I/Ointensive resources in the mainline and can affect throughput dramatically if they aremistuned.

Typical tuning of the MQ environment should also be performed.

Performance data should be collected and sent per the email I sent you for analysis.

Resource Recovery Services-

RRS

If you have no need for the

archive log, you may consider not defining it as it will continueto grow unless it redefined.

Application Reload Interval

Disable class reloading

(Check the box "Reload Enabled" under

applications-> enterpriseapplications

-> <application name> and set the "Reload Interval" to zero).

Or, if reload is needed, set the "Reload Interval" to a high value.

7

Distributing HTTPRequests over multiple Servants

WebSphereuses a "hot server" strategy to route HTTP requests.

By this, we mean that WebSphere routes

to servant regions which had recentlydispatched work with threads available

(i.e."hot servers"). These “hot servers”have pages in memory, application methods and cache full of data.

HTTPrequests with session affinity are routed to the servant region where the

sessionobject(s) reside.

However, this can cause imbalances in some situations:

"Hot" servant regions can get over-loaded with work.GC and loss of a servantregion can impact many sessions.

Ensure that component tracing is disabled for WebSphere. To determine if tracing isdisabled, the following MVS command can be entered:

D TRACE,COMP=ALL

To change a component to its minimum tracing level issue:

TRACE CT,OFF,COMP=<component identified>

CurrentFinding:

Ensure JRAS tracing is disabled by setting in the admin console:

Troubleshooting → Logs and trace → server_name →diagnostic trace

Set the following trace specs: *=all=disable

CurrentFinding:

Monitoring

RMF reports should be run regularly to track the growth of the application during the rollout.This should to provide the necessary information to properly create realistic projections for

8

the capacity requirements of the application for future rollouts to the indicated maximum of27500 terminals.

The RMF reports request should include CF Activity Reports to monitor the CF utilizationplus and the sizing of the structures, especially those for RRS, DB2, and MQ.

It has beenrecommended thatLuis Tosta De Sa

should be engaged to assist with thiseffort.

Application Issues

Application Logging

It appears that the application is generating large amounts of log information being directedto SYSPRINT. The application should limit the amount of logging in the productionenvironment to the absolute minimum. Logging toa HFS is also not recommended, in thissituation an

unshared zFS is recommended.

Pass By Reference

Pass by reference specifies how the ORB passes parameters.

If enabled, the ORB passesparameters by reference instead of by value, which avoids making an object copy.

If youdo not enable pass by reference, the parameters are copied to the stack before everyremote method call is made, which can be expensive.

You can use this option only when the Enterprise JavaBeans (EJB) client and the EJB areon the same classloader.

This requirement means that theEJB client and the EJB must bedeployed in the same EAR file.

If the EJB client and the EJB server are installed in the same WebSphere ApplicationServer instance, and the client and server use remote interfaces, enablingpass byreference can improve performance up to 50%

Enable this property with caution, because unexpected behavior can occur. If an objectreference is modified by the remote method, the caller might change.

If this variable isn’tcoded, pass by value is used.To enable code the following variable under”

Servers

>

Application Servers

>

serverName

>

ORB Services

com.ibm.CORBA.iiop.noLocalCopies=true

CurrentFinding:

System Authorization Facility

DisableSAF calls for successful HFS accesses

via the BPX.SAFFASTPATH Facility classif audits of successful HFS accesses are not needed and you are not using theSAFcallable services router installation exit

IRRSXT00.

9

Security Domain Identifier

If multiple

WebSphere cells are going to be configured utilizing a common securitydatabase, you may want to consider utilizing a security domain identifier. A security

domain identifier will provide unique EJBROLE checks that are performed by theadministration console to control what administration functions the user can perform. If noidentifier has been defined, a security administrator in one WebSphere cell has access thesame access to all cells that are not using an identifier.

For example:

If a security domain identifier has been defined as PROD, then all EJBROLEaccesses will be checked against PROD.<ejbrole>. If the request would normallyaccess administrator as defined by the administration console

applications’sdeployment descriptor, WebSphere will change the EJBROLE from administrator

toPROD.administrator thus providing a unique EJBROLE name for the cell withoutany application changes.

Access to applications that utilize EJBROLEs are also controlled in the same way as theadministration console.

The same security domain identifier is also used as the APPL name, the APPL class isactive on the system.

In this example, a check would be made against PROD

for allaccesses to this cell.

3. Network Infrastructure

TCP/IP considerations:

First, ensure that you have defined enough sockets to your system and that the default sockettime-out of 180 seconds is not too high. To allow enough

sockets, update the BPXPRMxxparmlibmember:

Set MAXSOCKETS for the AF_INET filesystem high enough

NETWORK DOMAINNAME(AF_INET) DOMAINNUMBER(2) MAXSOCKETS(30000)

CurrentFinding:

Set MAXFILEPROC high enough.

MAXFILEPROC(nnnnn)

CurrentFinding:

The general rule ofthumb is

setting MAXSOCKETS and MAXFILEPROC to atleast 5000 for low-throughput,10000 for medium-throughput, and 35000 for high-throughput WebSphere transactionenvironments.

MAXSOCKETS should at least be equal to MAXFILEPROC. A socket open in UnixSystems Services is considered an open file.Setting high values for these parameters should notcause excessive use ofresources unless the sockets or files are actually allocated.

10

The maximum value for MAXSOCKETS is16777215

and for MAXFILEPROC is131071.

Check the TCPIP profile dataset to ensure that NODELAYACKS

is specified. Changing this couldimprove throughput by as much as 50% (this is particularly useful when dealing with trivialworkloads). This setting is important for good performance when running SSL.NODELAYACKScan be specified at on the TCPCONFIG,PORT, PORTRANGE, BEGINROUTES, and GATEWAY

statements. DELAYACKS is the default.

CurrentFinding:

You should ensure that your DNS configuration is optimized so that lookups for frequently-used

servers and clients are being cached.

Caching is sometimes related to the name server's TimeToLive (TTL) value.

On the one hand, setting the TTL highwill ensure good cache hits.

However,setting it high also means that, if the Daemon goes down, it will take a while foreveryone in thenetwork to be aware of it.

A good way to verify that your DNS configuration is optimized is to issue theoping

anddnslookup

USS commands. Make sure they respond in a reasonable amount of time. Often aDNS or DNSserver name will cause

delays of 10 seconds or more.

Increase the size of the TCPIP send and receive buffers from the default of 16K to at least 64K.The size of the buffers

including control information beyond what is present in the data that you aresending

in your application.To do this specify the following:

TCPCONFIG

TCPSENDBFRSIZE 65535

TCPRCVBUFRSIZE 65535

Note:It would not be unreasonable, in some cases, to specify 256K buffers

CurrentFinding:

Increase the default listen backlog. This is used to buffer spikesin new connections which comewith a protocol like HTTP. The default listen backlog is 10 requests. We recommend that youincrease this

value to something larger. Forexample:

protocol_http_backlog=100

protocol_https_backlog=100

protocol_iiop_backlog=100

protocol_ssl_backlog=100

CurrentFinding:

11

In the most demanding benchmarks you may find that even

defining 65K sockets and filedescriptors does not give you enough ’free’ sockets to run

100%. When a socket is closedabnormally (forexample, no longer needed) it is not made available immediately. Instead it isplaced into a state called finwait2 (this is what shows up in the netstat-s command). It waits therefor a period of time before it is made available in the free pool. The default for this is 600 seconds.

Note:

Unless you have trouble using up sockets, we recommend that you leave this set to thedefault value.

If you are using z/OS V1.2 or above, you can control the amount of time the socket stays in finwait2state by specifying the followingTCPCONFIG.When this timer expires, it is reset to 75 secondsand when it expires a second time, the connection is dropped.