You can use the Data Processing page to monitor the workflow processes that move data to the data store. You can monitor the status of the job streams of each installed Content Pack component. You can also use this page to view historical execution trends and variations of workload or focus on operational data about the job streams.
The Data Processing page includes:
The Stream Details tab includes:
Field | Description |
---|---|
Content Pack Component name |
Name of the Content Pack component. |
Number of Streams | Total number of job streams. This value indicates the number of streams that are currently running, or the number of streams that are loaded for execution, or the number of streams that completed in the last run. |
Stream Status Details |
Status of the job stream.
|
Field | Description |
---|---|
Stream Name | Name of the job stream. |
Step Status (Completed/Total) |
Status of the job stem in the stream. Completed indicates the number of steps in the job stream that have completed irrespective of whether it was successful or generated a warning or error. Total indicates the total number of job steps in the stream. |
Step Status |
Status of the job step execution:
|
Start Time | Local time when the execution of the job step started. |
Field | Description |
---|---|
Step name | The name of the job step. |
Process Id | The unique identifier for the job step. |
State |
The state of the job step. The options include:
|
Status |
Status of the job step. The options are:
|
Detail Message | Displays the execution log of the job step with detailed information on how the warning or error was generated during the execution of the job step. An experienced user can also obtain all the necessary details related to the stream execution status by looking at the relevant dictionary and run-time tables. |
Log File | The name of the log file where information about the job step process is logged. |
Command | The command that was used to run the job step. |
Max Retries | Maximum number of retries for the job step. This field only appears for job steps with error or warning states. |
Remaining Retries | The number of remaining retries. This field only appears for job steps with error or warning states. |
Max Execution Time (Mins) | Maximum execution time, in minutes, for the job step. This field only appears for job steps with error or warning states. |
Start Time | Time when the execution of the job step started. |
End Time | Time when the execution of the job step ended. |
Audit Details | A summarized view of the data audit information for the job step. The information can include the number of files or rows that were processed, passed, and failed and the time taken for the audit step. |
The Historical Stream Overview tab includes:
Field | Description |
---|---|
Stream Detail |
Option to display the list of job stream that ran during a specified period. The options include:
Note: The workflow framework stores information about the job steps in run-time tables for a maximum period of 30 days, after which the data is automatically purged by the PMDB Platform IM Service. |
Content Pack Component name | Name of the Content Pack component. |
Number of Streams | Total number of job streams that ran during the specified period of time. |
Stream Status Details |
Status of the job stream.
|
Field | Description |
---|---|
Stream name | Name of the job stream. |
Stream Status Details |
Status details of the stream.
|
Field | Description |
---|---|
CP | Name of the Content Pack component the stream belongs to. |
Stream | Name of the job stream. |
Severity |
The severity option to generate the graph for. The options include:
|
State |
Execution state of the job stream. The options include:
|
Graph | A graphical representation of the number of errors or warnings encountered by a job stream over a period of time. |
Use the Data Processing page to: