Summary
Error
QC log shows the following error when the issue occured:
Messages:
Failed to start run. Failed to connect to PC service; Could not send Message.; ; Failed to connect to PC service; Could not send Message.;
The Performance Center environment becomes extremely slow at this point and stops responding.
Cause
Issue can occur due to several reasons:
1) IIS entered a deadlock state due to several reasons.
2) Anti-virus installed on the PC Server machine might be one of the reason.
3) If there are lot of host machines that are in non-operational state this might also slow down the system.( as PC Server constantly checks them to see if they are back to operational)
Fix
1) Disable anti-virus ( in case of McAfee, add exception for PC Service in HIPS)
2) Recycle PCQCWSAppPool at regular intervals ( in addition to fixed time everyday)
On the PC Server -> Open IIS Manager -> Application Pools -> Right click on PCQCWSAppPool -> Recycling -> Regular time intervals -> Enter 240 minutes
3) Make the unused non-operational machines to 'un-available' state.