Retrieving YARN Logs for a Failed/Completed Application

  • Document ID:KM03646603
  • Creation Date:02-Jun-2020
  • Modified Date:16-Dec-2020
    • Micro Focus Products:
      interset ueba 5.9
      interset ueba 5.7

Summary

To retrive YARN logs use yarn cli tool: yarn log --applicationId

Question

Very often when examining analytics' logs it's neccessary to get a full log of a  failed or completed application.

How to retrive YARN Logs for a Failed/Completed Application?

 

Answer

To retrive YARN Logs for a Failed/Completed Application

  1. Log in to a MASTER node as the interset user.
  2. Run:
    sudo -u spark yarn logs --applicationId <APPLICATION_ID_GOES_HERE> > /tmp/yarn.log

for an unsecured cluster

or

sudo -u i_spark yarn logs --applicationId <APPLICATION_ID_GOES_HERE> > /tmp/yarn.log

for a secured (kerberized) cluster

  1. Review the output log file located at /tmp/yarn.log

 

You can find the latest analytics job failed and that the failed application ID is at the end of analytics logs.(/opt/interset/log dir).