Online Help for Administrators > Administration Console Screens > Data Processing

Data Processing

You can use the Data Processing page to monitor the workflow processes that move data to the data store. You can monitor the status of the job streams of each installed Content Pack component. You can also use this page to view historical execution trends and variations of workload or focus on operational data about the job streams.

The Data Processing page includes:

Stream Details tab

The Stream Details tab includes:

Field Description
Content Pack Component name

Name of the Content Pack component.

Number of Streams Total number of job streams. This value indicates the number of streams that are currently running, or the number of streams that are loaded for execution, or the number of streams that completed in the last run.
Stream Status Details

Status of the job stream.

  • Status include:
    • OK—The number of job streams that completed successfully.
    • Warning—The number of job streams that completed successfully. However, certain job steps in the stream completed with warnings. These warnings, however, do not hinder the execution of the remaining job steps in the stream.
    • Error—The number of job streams that failed to complete. This indicates that a job step in the stream failed its execution process and generated an error. This causes the entire stream stop running until the error state is resolved.
  • Total: Total number of streams that are running.

Stream Detail for Content Pack

Field Description
Stream Name Name of the job stream.
Step Status (Completed/Total)

Status of the job stem in the stream. Completed indicates the number of steps in the job stream that have completed irrespective of whether it was successful or generated a warning or error. Total indicates the total number of job steps in the stream.

Step Status

Status of the job step execution:

indicates that the job stream has not started running but has been loaded for execution.

indicates that all job steps in the stream completed successfully.

indicates that some job steps completed with warnings.

indicates that a job step in the stream failed to complete and generated an error.

Start Time Local time when the execution of the job step started.

Step Detail for Job Step

Field Description
Step name The name of the job step.
Process Id The unique identifier for the job step.
State

The state of the job step. The options include:

  • FINISHED—indicates that the job step finished running.
  • WAITING—indicates that the job step is waiting to be run.
  • RUNNING—indicates that the job step is currently running.
Status

Status of the job step. The options are:

  • SUCCESS—indicates that the job step execution was successful.
  • WARNING—indicates that the job step completed but generated a warning.
  • ERROR—indicates that the job step failed to complete and generated an error.
  • MAX_EXEC_TIME_EXCEEDED—indicates that the job step failed to complete within the defined execution timeframe.
  • WAITING—indicates that the job step in currently in the waiting state because the preceding job step is yet to complete.
Detail Message Displays the execution log of the job step with detailed information on how the warning or error was generated during the execution of the job step. An experienced user can also obtain all the necessary details related to the stream execution status by looking at the relevant dictionary and run-time tables.
Log File The name of the log file where information about the job step process is logged.
Command The command that was used to run the job step.
Max Retries Maximum number of retries for the job step. This field only appears for job steps with error or warning states.
Remaining Retries The number of remaining retries. This field only appears for job steps with error or warning states.
Max Execution Time (Mins) Maximum execution time, in minutes, for the job step. This field only appears for job steps with error or warning states.
Start Time Time when the execution of the job step started.
End Time Time when the execution of the job step ended.
Audit Details A summarized view of the data audit information for the job step. The information can include the number of files or rows that were processed, passed, and failed and the time taken for the audit step.

Historical Stream Overview tab

The Historical Stream Overview tab includes:

Field Description
Stream Detail

Option to display the list of job stream that ran during a specified period. The options include:

  • Last 1 day
  • Last 7 days
  • Last 30 days

Note: The workflow framework stores information about the job steps in run-time tables for a maximum period of 30 days, after which the data is automatically purged by the PMDB Platform IM Service.

Content Pack Component name Name of the Content Pack component.
Number of Streams Total number of job streams that ran during the specified period of time.
Stream Status Details

Status of the job stream.

  • Status include:
    • OK—The number of job streams that completed successfully.
    • Warning—The number of job streams that completed successfully. However, certain job steps in the stream completed with warnings.
    • Error—The number of job streams that failed to complete. This indicates that a job step in the stream failed its execution process and generated an error. This causes the entire stream stop running until the error state is resolved.
  • Total: Total number of streams that are running.

Stream Detail for Content Pack Component

Field Description
Stream name Name of the job stream.
Stream Status Details

Status details of the stream.

  • Status include:
    • OK—The number of the job steps that have completed successfully.
    • Warning—The number of the job steps that have completed successfully but with warnings.
    • Error—The number of the job steps that failed to complete.
  • Total: Total number of job steps that ran.

Historical Stream Details tab

Field Description
CP Name of the Content Pack component the stream belongs to.
Stream Name of the job stream.
Severity

The severity option to generate the graph for. The options include:

  • WARNING—displays the number of warning states encountered by the job stream during its execution.
  • ERROR—displays the number of error states encountered by the job stream during its execution.
State

Execution state of the job stream. The options include:

  • ALL—all states of the job stream.
  • FINISH—state where the job stream completed successfully irrespective of the warning or error encountered during its execution.
  • ABORT—state where the execution of the job stream was aborted by the user because of a job step error.
Graph A graphical representation of the number of errors or warnings encountered by a job stream over a period of time.

Use the Data Processing page to:


© 2010 - 2013 Hewlett-Packard Development Company, L.P.