Very often when examining analytics' logs it's neccessary to get a full log of a failed or completed application.
How to retrive YARN Logs for a Failed/Completed Application?
To retrive YARN Logs for a Failed/Completed Application
- Log in to a MASTER node as the interset user.
sudo -u spark yarn logs --applicationId <APPLICATION_ID_GOES_HERE> > /tmp/yarn.log
for an unsecured cluster
sudo -u i_spark yarn logs --applicationId <APPLICATION_ID_GOES_HERE> > /tmp/yarn.log
for a secured (kerberized) cluster
- Review the output log file located at /tmp/yarn.log
You can find the latest analytics job failed and that the failed application ID is at the end of analytics logs.(/opt/interset/log dir).