HP Diagnostics 9.00, 9.01

Readme

Software version: 9.01 / September 2010

This file provides the following information about HP Diagnostics:

Documentation

Installation Overview

What's New 9.01

What's New 9.00

System Requirements

Notes and Limitations

Support

Legal Notices

Documentation

The first page of this readme document contains the following identifying information:

  • Version number, which indicates the software version.
  • Publish date, which changes each time the document is updated.

In addition to this readme, please see the Upgrade_Patch_Install_Instructions.pdf (in each download
package) for important upgrade instructions.

The full Diagnostics documentation set was updated and is included with the product. The manuals
are also posted on the manuals download site for the 9.01 release. To check for recent updates or
to verify that you are using the most recent edition, visit the following URL:

http://h20230.www2.hp.com/selfsolve/manuals

This site requires that you register for an HP Passport and sign-in. To register for an HP Passport
ID, go to:

http://h20229.www2.hp.com/passport-registration.html

Or click the New users - please register link on the HP Passport login page.

You will also receive updated or new editions if you subscribe to the appropriate product support service.
Contact your HP sales representative for details.

To view files in PDF format (*.pdf), Adobe Acrobat Reader must be installed on your system. To download
Adobe Acrobat Reader, go to the following URL: www.adobe.com

Installation Overview

Diagnostics patch releases, such as 9.01, contain a full replacement of the Diagnostics components; therefore
you need to follow the same instructions as for an upgrade.

Upgrade and patch installation instructions are provided in the HP Diagnostics Installation and
Configuration Guide in Appendix G.

For patch releases, installation instructions are also provided in a document available with each package
download (Upgrade_Patch_Install_Instructions.pdf). The patch install instructions are a duplicate of
Appendix G, provided in each download package as a convenience.

Important Upgrade Note:

If you are integrating Diagnostics with Business Service Management (BSM) and you are upgrading BSM
9.00 to 9.01 you need to do the following in order to ensure the CIs from Diagnostics are correct. (defect
48318) This information is also in the BSM readme file.

  • The following note is relevant if you are upgrading from Business Service Management 9.00 to 9.01.
    After you finish the BSM upgrade process and start Business Service Management, perform the below
    steps on the Data Processing Server (DPS). If you have more than one DPS, perform the steps on the
    DPS that runs the KPI ENRICHMENT service.
    • If you have more than one DPS, determine which DPS is running the KPI ENRICHMENT
      service:
      • Access the JMX console on any DPS via http://<HP Business Service Management DPS
        machine>:8080/jmx-console and enter the user name and password.
      • Click service= hac-manager in the Topaz section.
      • Locate the first java.lang.String listAllAssignments() operation (the one that requires no
        parameters) and click Invoke.
      • Locate the line showing the KPI_ENRICHMENT service for the relevant customer.
      • Check the Process column for the name of the server that runs the KPI ENRICHMENT
        service.
    • Stop the KPI Enrichment Service (KES):
      • Return to the JMX Console entry page (http://<HP Business Service Management DPS
        machine>:8080/jmx-console).
      • Click service=KES Service in the Topaz section.
      • Locate void stop()
      • Click Invoke.
      • Wait for the KPI ENRICHMENT service to stop (this should take less than a minute). To
        determine whether the service is stopped, follow the procedure described above to determine
        which DPS is running the KPI ENRICHMENT service, and after invoking the
        java.lang.String listAllAssignments() operation and locating the KPI_ENRICHMENT
        service line, check the State column for the service status.
    • Refresh the out-of-the-box repositories:
      • Access the JMX console on the DPS via http://<HP Business Service Management DPS
        machine>:8080/jmx-console and enter the user name and password.
      • Click service=repositories_manager in the Topaz section.
      • Locate the java.lang.String cleanDefaultRepositoryTypeDBAndCache() operation.
      • Click Invoke.
    • Start the KPI Enrichment service (KES):
      • Return to the JMX Console entry page (http://<HP Business Service Management DPS
        machine>:8080/jmx-console).
      • Click service=KES Service in the Topaz section.
      • On the KES Server JMX MBEAN View page, locate void start().
      • Click Invoke.

What's New in 9.01

Diagnostics 9.01 has a number of defects fixes and a number of new features shown below.

New Features

  • .NET Agent - Support multiple application domains with the same name and distinguish them so that
    the data from them does not get aggregated together. (42733)
  • Background: Change the format for the ASP.NET AppDomains which were previously not qualified
    with the IIS Path.

    Description:

    Old Probe_config.xml Configuration

    <appdomain name="CallChain">

    <points file="CallChain.points" />

    </appdomain>

    The configuration format does not provide a way to distinguish the 3 distinctly configured CallChain
    applications in IIS. The result of this was that you had to enable all(CallChain appdomains) or
    disable all. This is not a desirable level of control to enable\disable Web Applications. More so, the
    probes instantiated by all the CallChain AppDomains were named exactly the same and so the server
    requests from these applications got attributed to the single consolidated probe on the server. The probe
    metrics were also averaged over 3 completely independent instances which would be incorrect data
    presented to the user.

    This feature changes the format for ASP.NET(IIS) AppDomain configuration format in the
    probe_config.xml file to enable distinguishing the AppDomains by qualifying it with the IIS
    configuration path.

    The new configuration for the 3 CallChain applications would be as follows

    NEW Probe_config.xml Configuration

    <appdomain enabled="false" name="1/ROOT/CallChain/CallChain" website="Default Web Site">

    <points file="Default Web Site-CallChain-CallChain.points" />

    </appdomain>

    <appdomain enabled="false" name="1/ROOT/CallChain" website="Default Web Site">

    <points file="Default Web Site-CallChain.points" />

    </appdomain>

    <appdomain enabled="false" name="2/ROOT/CallChain" website="WebSite2">

    <points file="WebSite2-CallChain.points

    Benefits: Functionality

  • Integrations and Enterprise UI - Enhance Diagnostics from within BSM to include the Help links
    (48575)
  • Background: The help menu including "Help on the View" is disabled when bringing up Diagnostics UI
    from within BSM. This means users can only access Diagnostics help from the BSM menu and they lose
    the context sensitive help links from the Diagnostics UI.

    Description: Enabled the help menu links in the Diagnostics UI when bringing up Diagnostics from
    within BSM.

    Benefits: Usability.

Defect Fixes

Server

  • 47433 - IllegalStateException during OMi registration (BSM/Diagnostics Registration) in log file
  • Problem: This error gets logged when doing registration of BSM/Diagnostics but it does not indicate a
    real problem. A bad code path was doing a send with an event channel integration error (along with a
    successful BAC registration), but then continuing to try to ALSO "sendSuccess". Doing two responses to
    one request was the problem.

    Resolution: The unnecessary 2nd send was the one that was failing. Now it is avoided.

  • 48066 - The discovery that adds probes/server requests/hosts to an Application when a Business
    Transaction is added to the Application only works for probes (SRs/hosts) that are not directly connected
    to the commander.
  • Problem: The discovery that adds probes/server requests/hosts to an Application when a Business
    Transaction is added to the Application only works for probes (SRs/hosts) that are not directly connected
    to the commander.

    Resolution: The new feature of automatically adding entities to apps will now work on probes directly
    connected to the commander.

  • 47650 - Newly created Business Transactions briefly show a latency of zero with big numbers for
    min/max.
  • Problem: Immediately after a Business Transaction is created via the Diagnostics UI or BSM it shows up
    in Diagnostics with latency of 0. The latency looks strange because it shows zero
    values and very high/low numbers for min/max. After a few seconds the latency numbers go away (until
    real numbers come in).

    Resolution: The Business Transaction will only show latency now when an actual value is present and
    ready to show.

Java Agent

  • 48656 - Getting Mismatch Serialization UID
  • Problem: When adding an instrumentation with regular expression including all classes and methods,
    client application was failing with mismatch serialization error. Specifying a specific class and method
    avoided the problem.

    Resolution: To avoid application crashes we now do not add method wrappers for serializable classes. At
    the same time the user gets precise control over where the wrappers will be added.

  • 48962 - Probe must not override good information with "Unknown"
  • Problem: Customers sometimes see "Unknown" instead of the actual web service fields (for example they
    may see the web service operation as "Unknown").

    Resolution: The probe now reports the best information it can. If non-Unknown information was
    captured at some point in the chain, then we will use it.

  • 49374 - For a single outbound call, the Outbound calls screen can show multiple entries
  • Problem: For a single outbound call, the Outbound calls screen can show multiple entries. This was in
    the case of nested outbound calls, i.e. when an outbound instrumentation point was hit multiple times
    consecutively.

    Resolution: Fixed the code so that only one entry is shown.

  • 30602 - Wrong root method name is shown for the call profile of a cross-VM callee side server request
  • Problem: Wrong root method name "Outbound Call" is shown for the call profile of a cross-VM callee side
    server request.

    Resolution: The right root method name will now be shown.

  • 49460 - Service topology does not connect the SOAP over JMS web services.
  • Problem: SOAP over JMS Web Services were not connected in the Service topology screen.

    Resolution: The code problem has been fixed and now they are connected.

.NET Agent

  • 33682 - CLR Issue with instrumentation (fixed for CLR 4.0 only) - Hp.Profiler.dll is loaded into
    applications not configured for monitoring.
  • Problem: The .NET agent inspects all .NET applications which come up for instrumentation by verifying
    scope defined in the probe_config.xml. This is how the .NET profiler works; it invokes the agent for every
    .NET application instantiated. Some applications (read executables) are fleeting, for example the Visual
    Basic compiler and C# compiler which get triggered in certain deployment scenarios in IIS. We are
    unable to handle the fleeting nature of such applications as they exit even before we have a chance to
    verify scope. Also there are applications like the Visual Basic compiler (mainly Microsoft binaries) which
    do not make sense to try and instrument or verify whether they are in scope. Verifying scope adds
    considerable start-up overhead. The .NET agent should maintain a list of such applications (preferably
    configurable) so that we recognize the futility of trying to instrument these processes right at the start
    and ignore them.

    Resolution: Changed the return value for Initialize for CLR\.NET V4 to
    CORPROF_E_PROFILER_CANCEL_ACTIVATION for Applications which are NOT configured to be
    monitored. This will release the HP.Profiler.dll from the process memory. This is not available with
    earlier version of the profiler API hence cannot be solved for CLR\.NET V2

Collector

  • 47933 - Diagnostics Collector - creates/updates node CI with wrong (unqualified) primary DNS name
    attribute.
  • Problem: The DB Collectors were sometimes reporting the unqualified DB host names to the Diagnostics
    Server (and hence the node CI was created with the unqualified name)

    Resolution: Added comments to the config files of both DB Collectors and the HP Diagnostics Installation
    and Configuration Guide to instruct the user to use the fully qualified host name when configuring the
    DB Collectors.

Enterprise UI

  • 49580 - Service Health 360 View Drill to Probe Group Summary opens with incorrect probe group
    context
  • Problem: Diagnostics Probe Group Summary opens in context of probe group that was not selected in the
    360 view.

    Resolution: Diagnostics Probe Group Summary opens in context of the probe group selected in the 360
    view.

  • 48539 - Graph is blank on the new "Server - Basic Metrics" view when first navigated to as a top level
    view
  • Problem: Graph is blank on the new "Server - Basic Metrics" view when first navigated to and when
    navigated to as a top level view.

    Resolution: The default graph is not blank and the graph shows the throughput data.

  • 30577 - Call profile wedding cake for JMS onMessage() does not show the time for the onMessage() itself
  • Problem: The call profile is displayed for queue://queue://FX. However, the top level bar/layer in the
    wedding cake does not show the time for the onMessage(). Instead, it shows the times for the methods
    called by the onMessage().

    Resolution: Re-architecting the probe made this change possible.

  • 49424 - Business Transaction view should not show CPU column
  • Problem: Don't show CPU column because it isn't used for business transactions.

    Resolution: Removed column from screen definition.

Integrations

  • 48259 - HIs missing "sec" units.
  • Problem: Health Indicators from Diagnostics missing "sec" units.

    Resolution: Added "sec" to Synthetic Business Transaction HIs.

  • 48318 - There are 2 indicators in HPBSM\AppServer\webapps\site.war\bam\conf\repositories\external\diagnostics\indicators\def.xm
    l that have same name on the same CI type
  • Problem: Duplicate Health Indicator definition.

    Resolution: Removed the duplicate TransactionManager definition on diagnosticsprobe.

  • 49972 - Diagnostics Server Linux 64-bit Agent Registration Fails with BSM 9.0x
  • Problem: The Agent Registration with BSM 9.0x fails with the Diagnostics Commanding Server Linux
    64-bit version.

    Resolution: The <install dir>/bin/swith_ovo_agent.sh script could not find the path of the Agent 64-
    bit libraries. The script was using /opt/OV/lib when it should have been using /opt/OV/lib64. Set lib64
    directory.

Documentation

  • 49236 - Setting up SiteMinder module
  • Problem - Problem with the configuration guidelines found in the Diagnostics Installation guide for
    setting up SiteMinder:

    Set up Apache Server 2.2 with reverse proxy (reverse proxy redirection works fine) to Diagnostics

    Enable the SiteMinder Agent in the Apache Server

    Configured the SiteMinder Module in the jaas.configuration

When entering into the SiteMinder site and provide credentials, the Browser shows "Internal Server Error".

Resolution: Updated the 9.01 Installation Guide, Appendix B, section on "Using Reverse Proxy with
SiteMinder JAAS LoginModule" in the example of Apache reverse proxy setup on HP-UX, updated the
procedure as follows:
Edit the Apache configuration file httpd.conf and add the following properties:

ProxyPass /siteminderagent !

ProxyPass / Error! Hyperlink reference not valid. of Diagnostics Server>:2006/
(2006 is the default Diagnostics Server port, use the port configured for your Diagnostics Server)

ProxyPassReverse / Error! Hyperlink reference not valid. of Diagnostics Server>:2006/
(2006 is the default Diagnostics Server port, use the port configured for your Diagnostics Server)

What's New in 9.00

Diagnostics 9.00 has a large number of defects fixed and a number of new features shown below.

New Features

  • Business Transactions
    • Diagnostics can now automatically detect and report on "Business Transactions", identifying
      cross VM dependencies and reporting the time taken along the different paths of the
      transaction and showing the topology in the UI.
    • The configuration of these transactions can be done in either the Diagnostics UI or the
      BSM UI.
    • The Business Transactions can be viewed in either the BSM UI or the Diagnostics UI
      as a topology or table view.
    • In addition, users will be able to see server request topologies in Diagnostics (not
      previously available).
  • Business Service Management integration
    • We have greatly improved integration with the BSM framework including the population
      and synchronization of many Configuration Items (CIs) within the ODB.
      In addition, we are populating Key Performance Indicators (KPIs) and Health
      Indicators (HIs) inside of the BSM Dashboard.
    • Business Transactions can be viewed in the BSM Transaction Management UI and can be used to track service levels with SLM.
  • UI Enhancements
    • Two new View Groups
      • Aggregate View Group - entities aggregated by probe group (useful in cluster environments)
      • Transactions View Group - Business Transactions (All Users) and Synthetic Transactions
        (BPM)
    • New Java Probes and .NET Probes views under Standard views
    • SQL Statements by Probe view added to Standard views
    • Outlook Style tabbed pane now organizes the view groups
    • Virtual Hosts - VM Guest value is displayed in Host Topology view
    • Open in Custom View - User can create a custom view that can be opened in context for a
      selected entity. (For example > Save view for a probe and open it in context of any probe,
      this is similar to opening a probe in the Probe Summary view.)
    • Chart Groups - In any Standard entity view, the user can group the chart trends
      by metric or by entity.
    • View context sensitive help for each UI view through use of new Help menu item
      "Help on View"
    • User can export data from a Diagnostics view into a CSV file
      • Export Selected chart data to CSV
      • Export the data zoomed to in a chart in Standard views
      • Exports the trend for all charts in summary views
    • The Mercury System user can use Diagnostics in a large deployment to manage System
      Health
      • Show number of probes per probe group and number of probes per mediator (Capacity View)
      • User can see only the probes of interest in the Diagnostics UI screens (System Health)
    • UI configurations (ui.properties) for large deployments
      • Set the Top N probe entities query parameter
      • Throttle server query frequencies by query type
    • CAM Discovery Policy informs UI to Include probe's entities in top level views
      • Default setting for all applications
      • Informs the UI to not filter by application below the probe level
  • Diagnostics will automatically discover Business Transactions CIs based on server requests-
    Rule based definition of Business Transactions
  • .NET agent enhancements
    • Scalability improvements - The .NET agent deployments can scale more with the
      Diagnostics server with the use of the .NET Probe Aggregator Service on the agent system.
      This reduces significantly network I/O on both the agent and server systems.
    • Monitoring of asynchronous .NET applications; Sending and receiving messages
      using MSMQ
    • WCF features match those of other protocols
    • Diagnostics .NET agent will provide collection data to the server (LWMD)
    • .NET agent collects more metrics OTB and the user can easily add new metrics on demand.
    • CLR 4.0 .NET environment support
    • .NET agent will support parameter capture
    • .NET agent captures parameters of the first method that sees an exception thrown
    • .NET agent has enhanced correlation capability to outbound http calls made to Java
      components
  • Java Agent enhancements
    • New Collection Leak Pinpointing feature automatically detects memory leaks
    • Java agent correlation across multiple threads - In complex Enterprise Service Bus
      solutions like Tibco Active Matrix, WebLogic's AquaLogic and others - a single
      "server request" can span multiple threads within the ESB. Until now, Diagnostics
      could not correlate the server request across the threads, now it can show the total
      latency across the threads. Initially, we support WebLogic AquaLogic and will add
      others (like Tibco) as customer demands dictate.
    • Cross VM correlation can be configured through instrumentation to support any protocol, including home grown generic TCP communication.
  • User can see Server Request topology and Business Transaction flow from a .NET probe to a
    Java probe based on an HTTP call (and back).
  • Other misc enhancements:
    • CORBA Cross MV Tracing
    • User can select probe metrics to be included in the LR, PC offline analysis file.

System Requirements

The HP Diagnostics Installation and Configuration Guide contains detailed information about system
requirements. This guide is available in a PDF version (Diagnostics_Install_Guide.pdf) located on the
installation disk or in the Diagnostics Server download package. General system requirements are
provided below for the Diagnostics Server.

Requirements for the Diagnostics Server

Diagnostics uses the Java 1.6 JVM for the Server (and the Collector).

Java 1.6 JVM is supported on the following operating systems and requires the patches listed below.

For the most recent information on supported environments refer to the Diagnostics Product Availability
Matrix at http://support.openview.hp.com/sc/support_matrices.jsp.

The following table lists the desired system requirements for the host of a Diagnostics Server with Java
Probes.

Platform

Item

Up to 50 Java probes

Up to 100 Java probes

Up to 200 Java probes

Windows

CPU

2x 2.4 GHz

2x 2.8 GHz

2x 3.4 GHz

Windows

Memory

4 GB

4 GB

4 GB

Solaris

CPU

2x Ultrasparc 3

2x Ultra Sparc 4

2x Ultra Sparc 4

Solaris

Memory

4 GB

4 GB

4 GB

Linux

CPU

2x 2.0 GHz

2x 2.4 GHz

2x 2.8 GHz

Linux

Memory

2 GB

4 GB

4 GB

HP-UX

CPU

PA-RISC 2x 650 MHz

PA-RISC 2x 699 MHz

PA-RISC 2x 750 MHz

HP-UX

Memory

2 GB

4 GB

4 GB

All

Java Heap

512 M

750 M

1280 M

All

Disk

4 GB per probe

Notes regarding the test environment:

Call profile (depth of method calls) for each Server Request: 5

Number of unique Server Requests per probe: 23

The following table lists the desired system requirements for the host of a Diagnostics Server with .NET
Probes.

Platform

Item

Up to 10 .NET probes

Up to 20 .NET probes

Up to 50 .NET probes

Windows

CPU

1x 1.0 GHz

1x 2.0 GHz

2x 2.4 GHz

Windows

Memory

768 MB

1 GB

3 GB

Solaris

CPU

1x Ultrasparc 2

2x Ultra Sparc 2

2x Ultra Sparc 3

Solaris

Memory

1 GB

1.5 GB

3 GB

Linux

CPU

1x 1.0 GHz

1x 2.0 GHz

2x 2.4 GHz

Linux

Memory

768 MB

1 GB

3 GB

HP-UX

CPU

1x 1.0 GHz

1x 2.0 GHz

2x 2.4 GHz

HP-UX

Memory

768 MB

1 GB

3 GB

All

Heap Size

350 M

700 M

1400 M

All

Disk

3 GB per probe

Information on System Impacts When Probed (.NET Probe)

Item

Minimum

Latency

+3% while CPU < 50%-once the CPU starts saturating, latency overhead
increases

Memory

60MB additional RAM

CPU

+5%

Disk Space

200MB free disk space is required for the probe install

Network I/O

+3% to +5% (assuming a normal ASP.NET application with a remote database
Backend)

Disk I/O

Impact should be statistically insignificant

Throughput

< 1% reduction while CPU < 50% - should not exceed 15% reduction even
under peak load

Be careful with other metrics once throughput is impacted - as the application
is doing less work, results are not directly comparable

RAM utilization

Baseline footprint increase of up to 20MB per worker process

Note: This is quite difficult to measure accurately.

Information on System Impacts When Probed (Java Probe)

Item

Minimum

-Xmx

Add 40MB

Disk Space

200MB free disk space is required for the intial probe install. Note, more space
may be required during runtime due to the creation of logfiles and classmap.
For large applications, it is recommended to have an additional 200MB
available per probe for logfiles and classmap data.

Latency

+3% while CPU < 50%-once the CPU starts saturating latency overhead
increases

CPU

+5% to +10%

Network I/O

+3% to +5% (assuming a normal Java EE application with a JDBC, RMI, or
Web-services Backend)

Disk I/O

Impact should be statistically insignificant

Throughput

< 1% reduction while CPU < 50% - should not exceed 15% reduction even
under peak load

Be careful with other metrics once throughput is impacted - as the application
is doing less work, results are not directly comparable

RAM utilization

Baseline footprint increase of about 40MB

The requirements and general impact on the system depends on the monitored application and on the
probe configuration. The values listed above are for the default probe configuration only.

Notes and Limitations

Notes and limitations are provided for the following component areas:

  • Java Agent
  • NET_Agent
  • Integrations (BSM, PC, LR, TV, SaaS)
  • Collectors (Oracle, SAP, MQ, SQL Server)
  • Java Profiler
  • .NET Profiler
  • Diagnostics Server
  • User Interface
  • Documentation
  • Environment

Java Agent

  • When the dynamic property enable.stack.trace.sampling is set to "auto" (the default value), it activates
    stack trace sampling as described below: (38822) (43937)
    • On any Java 1.5 implementation with the following limitations:
      • For HotSpot JVM, version 1.5.0_04 or later is required
      • For JRockit JVM, JRockit version R27.4 or later is required
    • On any Java 1.6 implementation with the following limitations:
      • For PA-RISC HP-UX, JRE version 1.6.0.03 or later is required
  • Special configuration needed to support SAP JVM NetWeaver 7.1 and later (43388)
  • Because NetWeaver 7.1 runs in a clustered environment, you will need to do one of the following in order
    to distinguish the probes in the cluster:

    • Find a way to set the probe.id for each individual server process in your cluster (an expert SAP
      administrator might be able to help).
    • If the above is not possible, then you need to specify the keyword %0 in the probe name. Doing
      this will name each probe in the cluster uniquely. For example, if you specify "-
      Dprobe.id=myProbe%0" then the first probe that comes up will be called "myProbe0", the second
      one "myProbe1" and so on.
  • There is a potential issue on HP-UX 11.23 with IBM WebSphere: When multiple probes are installed on
    the same system, it can happen that the probe's communication port (35000) will be overwritten with
    each startup of the probe. In this case, simply specify a different port range for each of the probes in
    etc/webserver.properties, for example:
    • jetty.port=36000
    • jetty.max.port=36100
    • Or specify the above parameters on the app server's startup configuration (via -D). (44062)

  • Using the Heapwalker feature with JRockit JVM version 27.3 may cause the application to crash.
    Versions 27.2 and 27.4 and higher work as expected, so the problem is only with version 27.3. Please
    upgrade to a higher version of JRockit to fix this problem. (43993)
  • When instrumenting an IBM 1.6 JVM on z/OS only, using the output of the JRE Instrumenter, the JVM
    will fail to start with a java/lang/NoClassDefFoundError exception. (50969)
  • To correct this issue, modify the -
    Xbootclasspath parameter by appending ":<probe_install_dir>/classes/boot, substituting for
    <probe_install_dir> appropriately. In particular note the required colon after instr.jre.

    For example, if the output of the JRE Instrumenter was:

    "-Xbootclasspath/p:/opt/JavaAgent/DiagnosticsAgent/classes/IBM/1.6.0/instr.jre" "-
    javaagent:/opt/JavaAgent/DiagnosticsAgent/lib/probeagent.jar" -Xshareclasses:none

    Change it to:

    "-Xbootclasspath/p:/opt/JavaAgent/DiagnosticsAgent/classes/IBM/1.6.0/instr.jre:
    /opt/JavaAgent/DiagnosticsAgent/classes/boot" "-
    javaagent:/opt/JavaAgent/DiagnosticsAgent/lib/probeagent.jar" -Xshareclasses:none

  • CPU times can be larger than latency times on Windows 2003.
  • On Windows 2003, the CPU timer has only 15.625 ms accuracy. In other words, the CPU timer is
    incremented only 64 times per second (64 * 15.625 ms = 1 second). So every method invocation will
    "consume" CPU time with a multiplicity of 15.625 ms, zero included, but no other values are possible.

    For larger latencies, for long time intervals, and for frequent server requests the average CPU remains
    reasonable. The problem is visible for short, infrequent server requests only, where `luck' causes the
    server request to execute when the CPU timer jumps by 15.625 ms.

    JAVA Probes will demonstrate this problem. This is a problem with Windows 2003 and cannot
    be fixed by Diagnostics. (42371)

  • When the Diagnostics/TransactionVision Agent is run in "dual" mode, there is a limitation on JDBC
    calls. "Dual" mode means that both Diagnostics Java probe and TransactionVision Java sensor are
    enabled on the system. In this configuration, Diagnostics has the following limitations:
  • 1) No data will show up from nodes in "dual" mode in the "SQL Statements" view.

    2) When drilling down to a Call Profile, JDBC calls will not show the SQL statement in the Arguments of
    the call.

    This problem will be fixed in a future release of Diagnostics. (42989)

  • GC Time Spent in Collector metric may be inaccurate (JRockit JREs only)
  • The problem is caused by a bug in JRockit JRE which reports the time spent in GC in nanoseconds
    rather than in milliseconds, as specified.
    Workaround: either upgrade JRockit to version 1.5.0_10, build
    R27.2.0 (or later) or modify the Rate setting as follows:

    In <probe_install_dir>/etc/metrics.config, change "[0.1]" to "[0.0000001]" in the line which defines "GC
    Time Spent in Collections" metric. For example:

    Java\ Platform/java.lang\:type\=GarbageCollector,*.CollectionTime = RATE[0.0000001](GC Time
    Spent in Collections|percent|GC) (42344)

  • If the following WARN message is found in the probe.log file on the probe system:
  • 2006-11-28 07:07:27,171 WARN com.mercury.opal.capture [ExecuteThread: `8' for queue:
    `weblogic.kernel.Default'] Maximum number of SQL queries cached (4096). The values of some prepared
    SQL queries will be lost. See sql.cache.size in capture.properties.

    Then you will need to increase the sql.cache.size in the capture.properties file on the probe system.
    (40639)

  • Do not run jreinstrumenter from the Java Agent Setup Module if you are installing the probe only in
    order to get the standalone system metrics agent. (42069)
  • By default, Diagnostics does not monitor server requests that always execute in under 51 milliseconds.
    These requests are trimmed-no information is captured about them. SQL statement executions made
    from these trimmed requests are not recorded, even if those same SQL statements are timed and
    recorded in the context of executing other server requests. In addition, the probe-level layer breakdown
    does not include time taken by trimmed server requests.
  • An exception to this is in a case where, on at least one occasion, a server request takes longer than 51ms
    to execute. Future executions of that server request are recorded, even if the subsequent requests are
    faster than 51ms. The reason for this behavior is to report accurate averages (and not mislead the user
    into thinking that this request only ran once or twice when in actual fact it is constantly running, just
    very quickly). This "Always Record" flag lasts for 1 hour from the last time an execution over 51ms was
    seen.

    You can redefine the 51ms trimming threshold. For the Java Probe
    integrated with LoadRunner or Performance Center, you configure this setting in the
    <diagnostics_server_install_dir>\etc\trimming.properties file. For the Java Probe in all modes, you
    configure this setting in the minimum.fragment.latency property in the
    <probe_install_dir>\etc\dispatcher.properties file. (40631)

  • A defect in the JVM on Linux (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6330858) results in
    the thread CPU times being the same for all threads in BEA WebLogic 9.0. This bug was fixed in JDK
    1.5.0_07. (40391, 39387)
  • The Java Probe causes core dump at startup of WebLogic 9.1 when used with JRockit VM version
    150_04. The JRockit bug is fixed on jdk1.5.0_06. (38545)
  • By default, for performance reasons, Diagnostics does not capture target information for Database calls.
    Therefore, these calls do not show up in the Outbound Calls view. You can enable it as follows:
    • For version 7.0 and higher of the Java Probe, set create.database.fragmentArcs=true in the
      probe's dispatcher.properties file.
    • Once capturing of Outbound database calls has been enabled, these calls will be displayed in the
      Outbound Calls view. (40722)

  • VM Heap Breakdown is based on an experimental api (JVMPI) in the JVM and is not
    expected to work in the following JVMs:
    • Sun 1.4.2_01: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4861809
    • Sun 1.4.2: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4899339 {open}
    • IBM JREs in general do not have a stable API implementation for this API
  • Running the profiler directly on z/OS is not supported. The script "profiler.sh" has been removed in the
    7.50 version of the z/OS probe, it will exist in previous versions but cannot be successfully run on z/OS.
    To run the profiler for a probe on a z/OS system, do so from a platform other than z/OS. (42369)

.NET Probe

  • CPU times can be larger than latency times on Windows 2003.
  • On Windows 2003, the CPU timer has only 15.625 ms accuracy. In other words, the CPU timer is
    incremented only 64 times per second (64 * 15.625 ms = 1 second). So every method invocation will
    "consume" CPU time with a multiplicity of 15.625 ms, zero included, but no other values are possible.

    For larger latencies, for long time intervals, and for frequent server requests the average CPU remains
    reasonable. The problem is visible for short, infrequent server requests only, where `luck' causes the
    server request to execute when the CPU timer jumps by 15.625 ms.

    NET Probes will demonstrate this problem. This is a problem with Windows 2003 and cannot be fixed
    by Diagnostics. (42371)

  • By default, Diagnostics does not monitor server requests that always execute in under 51 milliseconds.
    These requests are trimmed-no information is captured about them. SQL statement executions
    made from these trimmed requests are not recorded, even if those same SQL statements are timed
    and recorded in the context of executing other server requests. In addition, the probe-level layer
    breakdown does not include time taken by trimmed server requests.
  • An exception to this is in a case where, on at least one occasion, a server request takes longer than 51ms
    to execute. Future executions of that server request are recorded, even if the subsequent requests
    are faster than 51ms. The reason for this behavior is to report accurate averages (and not mislead the
    user into thinking that this request only ran once or twice when in actual fact it is constantly running,
    just very quickly). This "Always Record" flag lasts for 1 hour from the last time an execution over 51ms
    was seen.

    You can redefine the 51ms trimming threshold. For the .NET Probe in all modes you configure this
    setting in the <diagnostics_server_install_dir>\etc\trimming.properties file. All .NET Probe trimming
    configuration is done in the <.NET_ probe_install_dir>\etc\probe_config.xml file. (40631)

  • By default, for performance reasons, Diagnostics does not capture target information for Database calls.
    Therefore, these calls do not show up in the Outbound Calls view. You can enable it as follows:
    • For .NET Probes, set create.database.fragmentArcs=true in the server.properties file for the
      Diagnostics Server in Mediator mode. You should also use this workaround if
      enable.probe.aggregation in the Java probe has been changed from its default value of true to
      false.
    • Once capturing of Outbound database calls has been enabled, these calls will be displayed in the
      Outbound Calls view. (40722)

  • On platforms where Windows Process Activation (WPAS) is installed, the .NET Diagnostics agent
    uninstall has an implicit runtime dependency on up to four "listener adapter" services. During the
    uninstall, MSI pops up a "The following processes need to be closed before continuing..." dialog if any
    of the listener adapter services are started. Stopping them (temporarily) allows the uninstall to
    continue. However, the services do NOT have to be stopped in order for removal to end successfully.
    The user can select the "Ignore" button on this dialog and continue uninstallation. (38151)

Integration (BSM, LoadRunner, Performance Center, TransactionVision, SaaS)

  • Diagnostics Server does not work with IIS Basic Authentication with Business Availability Center and
    BSM Reverse Proxy Server. (43987)
  • Offline vs. Online. The following features and functionality in LoadRunner Offline Analysis are different
    from the Diagnostics Online.
    • Data in the profiler that is not sent into the Diagnostics Server will not be in the offline analysis
      after Performance Center/LoadRunner runs. This includes LWMD, Heap Breakdown, Allocation
      Analysis, Exceptions, SOAP Faults and SOAP payloads.
    • Oracle 10g data will not show up in offline analysis.
    • Instance trees will not be available in the offline analysis only aggregate trees after the needed
      drill down.
  • When configuring Real User Monitoring in Business Availability Center to use a Page definition
    including parameters, Diagnostics will not be able to make a match unless parameter capturing has
    been enabled. To enable this option on the Java Probe, list all desired parameter names in reverse
    alphabetical order in the args_by_class property in auto_detect.points under the "[HttpCorrelation]"
    section. (40496)
  • HP Performance Center offline files are kept by default. To manage offline files, you need to configure
    the Diagnostics Servers in Mediator mode so that they delete these files. You do this by setting the
    property distributor.offlinedelivery.preserveFiles to true in the
    <diagnostics_server_install_dir>/etc/server.properties file. When set to true, this property causes the
    run-specific "offline" files stored in the server's data directory to be retained for the amount of time
    specified in the facade.run_delete_delay property in the server's webserver.properties file (default period
    is 5 days).  During this retention period, the run can be successfully collated. Sometime after the
    retention period has ended, the associated offline files will be deleted from the system. (40739)
  • Performance Center users that drill to Diagnostics get full admin privileges to the Diagnostics Server
    and to the Probes that are connected to it. All restricted actions (for example, changing thresholds,
    alerts, and Custom Attributes) are accessible by all Performance Center users. Furthermore, every
    custom screen created by a Performance Center user is shared by all Performance Center users. (40133)
  • Starting in Diagnostics 7.0, many instrumented methods have been flagged when-root-rename in
    auto_detect.points to prevent many spurious and uninteresting server requests from being created and
    displayed in the UI.  Should these methods execute outside the context of another server request, they
    will still be recorded, but into a pseudo server request with the name "Background - <Layer>", where
    <Layer> is the layer name for the method.
  • For example, the Background JDBC connection testing that WebLogic runs used to be reported into a
    few separate server requests, such as netJDBCPreparedStatement.executeQuery(), but will now be
    recorded as invocations of a "Background - Database" pseudo server request.

    In addition, when using LoadRunner, these new pseudo server requests will not appear in LoadRunner
    Offline Analysis. 

    Should you wish to keep the old behavior for a particular instrumentation point and see these entries in
    LoadRunner Offline Analysis, carefully remove the when-root-rename detail parameter from the
    auto_detect.points. (42127)

  • When Diagnostics is integrated with HP Software-as-a-Service (SaaS), you must log on through Business
    Availability Center when viewing data for customers other than "Default Client". Failing to do so will
    cause several subtle issues with how data is reported. You can still use the standalone login to view the
    "Default Client" data. (42178)
  • LoadRunner Collate fails when the Diagnostics Server was restarted during Performance Center
    run/load test. (34682)

Collectors (Oracle, SAP, MQ, SQL Server)

  • To maintain accurate MS SQL Server metrics, the database option AUTO_CLOSE must be set to OFF.
    If AUTO_CLOSE is set to ON, MS SQL Server metrics will show up with incorrect and negative values
    after certain database operations (bcp, backup, shrink, etc) (43075).
  • In order to properly populate MS SQL Server and Oracle configuration items in the ODB, users should
    use the Fully Qualified Domain Name in the collector XML in order for Diagnostics to properly
    populate the primary DNS name attribute in the CI (47933).

JAVA Profiler

  • Starting with Diagnostics 9.0, the JAVA Profiler requires JVM 1.6 to run. However, on HP-UX, the
    default JVM that is installed with the Java Profiler is 1.5. Therefore in order to run standalone
    Java profiler, the following must be performed. NOTE - The Java Profiler runs fine on
    HP-UX when launched from the enterprise UI.
    To run the Java profiler in standalone mode,
    do the following:
  • Set JAVA_HOME to your java 1.6 installation location
  • export JAVA_HOME=<java 1.6 home directory path>
  • example:  export JAVA_HOME=/opt/java6
  • Run the java profiler: $PROBE_HOME/bin/profiler.sh &

.NET Profiler

  • When running the .NET Profiler under locales that use an Asian character set, the default font size may
    be too small. In such cases, change the text size used by the Web browser. (43070)

Diagnostics Servers

  • When enabling HTTPS between Diagnostics components the user should monitor the
    <install_dir>/MercuryDiagnostics/server/log/jetty.log for warnings similar to:
  • 2008-11-17 15:33:07,528: WARNING : WARN!! [RangeSocketListener-
    69]org.mortbay.http.SocketListener.isOutOfResources(SocketListener.java:358)22> OUT OF THREADS:
    RangeSocketListener@0.0.0.0:8443

    2008-11-17 15:33:35,582: INFO : EVENT [RangeSocketListener-
    14]org.mortbay.http.SocketListener.isLowOnResources(SocketListener.java:325)04> LOW ON
    THREADS ((200-198+7)<10) on RangeSocketListener@0.0.0.0:8443

    Workaround. If these messages appear, increase the value of jetty.threads.max by increments of 100 in
    the <install_dir>/MercuryDiagnostics/server/etc/webserver.properties file.

    Continue to monitor and increase if necessary. (44007)

  • After configuring the Diagnostics Server in Mediator mode, if it appears missing or inactive in the
    Diagnostics Server in Commander mode view, then it is possible that "commander.url" in
    server.properties has an ending "/" on it. For example, if you open server.properties on the Diagnostics
    Server in Mediator mode and set to commander.url=http://amkisty01:2006/ instead of
    commander.url=http://amkisty01:2006 (Note that the difference is the appending /), then the Diagnostics
    Server in Mediator mode will appear missing or inactive. (40527)
  • Broadly scoped allocation instrumentation may fill up perm-gen and cause the VM to crash. Users
    should avoid widely scoped instrumentation and instead make instrumentation as narrow as useful to
    minimize system impact. (39339)
  • When average latencies are less than 50ms, users may see CPU time slightly higher than latencies
    because of resolution differences. (39690)
  • Entity purging results in contribution/breakdown not summing up to the total because the deleted
    entities data was already rolled up and is not recalculated at this point. (39344)
  • Running the Diagnostics Server from a network drive, or configuring it to store the archive on a network
    drive, is not supported.
  • When you configure the Diagnostics Server to run only in SSL mode, it only affects port 2006. The
    Diagnostics Server also has an embedded Diagnostics Probe (for internal troubleshooting purposes) that
    listens, as usual, on port 35000.  If you are trying to lock your environment to SSL only for security
    purposes, you will also need to re-configure that embedded probe to listen only over SSL.

User Interface

  • Prior to exporting an Application Explorer view to html you should examine all of the tabs to activate the
    population to the desired charts, otherwise the exported chart could be empty. (28801)
  • Navigation menu items on the popup (context) menu are different when right-clicked alone vs. selected
    and then right-clicked. This occurs because only limited information is available when the first right-
    click on an entity occurs, and the context menu is populated with what is known at that time. However,
    the right-click operation immediately kicks off a query to the Diagnostics Server to find out more about
    this entity, and so subsequent right-click operations use the returned data and the popup menu includes
    more options. (29253)
  • In a custom 2-way view, both of the Navigation controls are updated with the navigations for the
    selected entity. (29097)
  • When an application is selected (Application Explorer view) or a Probe Group is selected ( Server
    Summary view) you may see two "View Probe" navigations. This will happen when the Application or
    Probe Group contains both .Net and Java Probes. (26471)
  • The navigation control only tracks the global time control. (29268)
  • Drilling from some alerts to "View Threshold Violation" results in a view that is empty except for the
    message "Either no data is available for the selected time range, or the data at this level of detail has
    been purged from the database."
  • This generally happens when the violating entity is 2 levels below the alert entity. For example, a probe
    group alert from a server request threshold or a probe alert on a portal component--life cycle method
    alert. 

    Workaround:  Notice the breadcrumb of the empty view.  This indicates the view that shows the data
    causing threshold violation.  Add the alert to a snapshot.  From the snapshot, use the Navigations pane
    to drill-down.  In most cases, you would navigate to Probes first and the destination view below that.
    For Life Cycle Methods, navigate to Portal Components (under Probes) first.  For better navigation from
    alerts to threshold violations, keep alert rules closer to the entities being monitored. (27612)

  • For server requests under 10 ms, Minimum Time will always be reported as zero. (34125)

Documentation

  • When the Diagnostics Server is installed on 64-bit Linux systems (only) the docs directory is not created
    and so the manuals are not accessible from the <diagnostics_server_install_dir>/docs directory. Copies of
    the pdf files can be found on the installation media and in the
    <diagnostics_server_install_dir>/html/opal/docs/eng/help/pdfs directory. The manuals are also accessible
    from the Diagnostics Online help.

Environment

  • For an IPV6 environment, the following restrictions exist:
    • All Diagnostic Server, Collector, and Java Probe configuration must be based on host names (not
      IP addresses).
    • .NET Probes are not supported.
    • Collectors must be running on IPV4 tunneled networks.
    • Linux and Solaris operating systems must be explicitly configured to use the IPV6
      network host/IP resolution. On dual hosts, update /etcnsswitch.conf. On Solaris, update ipnodes

Support

You can visit the HP Software Support Web site at:

http://support.openview.hp.com/

This web site provides contact information and details about the products, services, and support that HP
Software offers.

HP Software online support provides an efficient way to access interactive technical support tools. As a
valued support customer, you can benefit by using the support site to:

  • Search for knowledge documents of interest
  • Submit and track support cases and enhancement requests
  • Download software patches
  • Manage support contracts
  • Look up HP support contacts
  • Review information about available services
  • Enter into discussions with other software customers
  • Research and register for software training

Most of the support areas require that you register as an HP Passport user and sign in. Many also require a
support contract. To find more information about access levels, go to:
http://h20230.www2.hp.com/new_access_levels.jsp

To register for an HP Passport ID, go to:
http://h20229.www2.hp.com/passport-registration.html

Legal Notices

© Copyright 2004 - 2010 Hewlett-Packard Development Company, L.P.

Confidential computer software. Valid license from HP required for possession, use or copying. Consistent
with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and
Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard
commercial license.

The only warranties for HP products and services are set forth in the express warranty statements
accompanying such products and services. Nothing herein should be construed as constituting an additional
warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

The information contained herein is subject to change without notice.

For information about third-party license agreements, see the Documentation directory on the product
installation media.

Java™ is a US trademark of Sun Microsystems, Inc.

Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation.

Oracle® is a registered US trademark of Oracle Corporation, Redwood City, California.

UNIX® is a registered trademark of The Open Group.

If you have any comments or suggestions regarding this document, please send them by e-mail to
SW-Doc@hp.com.



Copyright 2010 Hewlett-Packard Development Company, L.P.