Resolution
When running a major version upgrade of Access Governance Suite such as from Access Governance Suite v5.0x to Access Governance Suite 5.1x, you will want to take a moment out to review the iiq console memory settings available prior to actually running the iiq upgrade script. This state is important, as it will help insure that you will not run into any potential errors you might encounter such as:
java.lang.OutOfMemoryError: Java heap space
that can occur if the allocated heap memory is too small to deal with the amount of accumulated Identity History Data within the system. This can be quite large if there are Archived objects as they may be stored as CLOBs (a Character Large Object, which can be up to 4GB in size). If your memory heap can not handle the CLOBs, you will experience the issue shown above, as you will exceed the heap space available in memory, and the system will begin to swap and things will progress slowly until the system heap crashes.
When checking for the current available memory within your JVM, you should be able to run a simple command to determine what is available on a Linux platform:
ps aux | grep 'java'
This will return something similar to the following:
chuck@nowhere:~$ ps aux | grep 'java'
chuck 6880 0.9 19.5 4767712 1182872 pts/1 Sl 10:54 2:05 /usr/bin/java -Djava.util.logging.config.file=/home/chuck/Tomcat/conf/logging.properties -server -Xms3072m -Xmx3072m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:MaxPermSize=1024m -XX:CodeCacheMinimumFreeSpace=2M -XX:ReservedCodeCacheSize=64M -Dsun.lang.ClassLoader.allowArraySyntax=true -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/home/chuck/Tomcat/endorsed -classpath /home/chuck/Tomcat/bin/bootstrap.jar -Dcatalina.base=/home/chuck/Tomcat -Dcatalina.home=/home/chuck/Tomcat -Djava.io.tmpdir=/home/chuck/Tomcat/temp org.apache.catalina.startup.Bootstrap start
chuck 9856 0.0 0.0 13128 1060 pts/1 S+ 14:37 0:00 grep --color=auto java
which is what shows when my Access Governance Suite is running locally. When I shut down Access Governance Suite and startup the iiq console though, the result of the command shows a much different picture:
chuck@nowhere:~/Tomcat/bin$ ps aux | grep 'java'
chuck 10023 100 4.3 595516 264744 pts/0 Sl+ 14:41 0:23 java -Xms128m -Xmx256m -Dsun.lang.ClassLoader.allowArraySyntax=true sailpoint.launch.Launcher console
chuck 10055 0.0 0.0 13124 1060 pts/1 S+ 14:41 0:00 grep --color=auto java
As you can see, the console available heap size is very small at this point.
The iiq console Memory heap is set within the iiq script itself, specifically line 16 of the iiq shell script:
JAVA_OPTS="-Xms128m -Xmx256m -Dsun.lang.ClassLoader.allowArraySyntax=true"
In order to give the system the full power and ability of your system, we recommend that you use the minimum recommend ed heap size for the console for 64-bit environments of 4GB for both the Xms and Xmx values, provided that you have sufficient RAM onboard and a 64-bit operating system. You need to retain enough free RAM (not dedicated to the heap) for the Operating System overhead and other needs (2GB should be sufficient during the upgrade) so that the system can process the upgrade efficiently. It may not go any faster or slower with more memory available, but it will not exceed the available memory during the CLOB processing, and having more memory will allow it to operate more efficiently and in a less-constrained manner.
IMPORTANT:Once the upgrade has completed, do not forget to reset the console back to your original settings for the memory as previously found. You can choose to add more memory to the console settings or balance the two (both Xms and Xmx the same, preferred), but in no condition should you leave the system with less than 2 GB of free, non-heap dedicated RAM or attempt to exceed the available RAM by trying to claim the maximum for both the console and the GUI in the JVM as this could lead to unpredictable and unpleasant application behavior. If this were done, a person logging into the console might take all available resources from the system and render it unusable.
For the upgrade period though, it is a worthwhile effort to make your upgrade experience smoother and less likely to experience heap errors.