Failed to connect to Broker. Caused by: User name [system] or password is invalid.

After upgrading Sentinel to 7.4.1 I am seeing the following severe error on the server0.0.logs. 

Wed Jun 01 10:35:41 CEST 2016|SEVERE|Container Startup Thread|com.esecurity.common.communication.strategy.jmsstrategy.TopicConnection.doConnect

                Failed to connect to Broker.

                Caused by: User name [system] or password is invalid. (java.lang.SecurityException); Root cause: Password does not match (javax.security.auth.login.FailedLoginException)

                javax.jms.JMSSecurityException: 1-1|org

                                at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:52)

                                at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1392)

                                at org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1495)

                                at org.apache.activemq.ActiveMQConnection.setClientID(ActiveMQConnection.java:413)

                                at com.esecurity.common.communication.strategy.jmsstrategy.TopicConnection.doConnect(TopicConnection.java:89)

                                at com.esecurity.common.communication.strategy.jmsstrategy.activemq.ActiveMQTopicConnection.doConnect(ActiveMQTopicConnection.java:122)

                                at com.esecurity.common.communication.strategy.jmsstrategy.activemq.ActiveMQPublisherConnection.doConnect(ActiveMQPublisherConnection.java:95)

                                at com.esecurity.common.communication.strategy.jmsstrategy.JMSConnection.connect(JMSConnection.java:188)

                                at com.esecurity.common.communication.strategy.jmsstrategy.activemq.ActiveMQStrategy.initConnections(ActiveMQStrategy.java:221)

                                at com.esecurity.common.communication.strategy.jmsstrategy.activemq.ActiveMQStrategy.initialize(ActiveMQStrategy.java:196)

                                at com.esecurity.common.communication.Communicator.loadStrategy(Communicator.java:340)

                                at com.esecurity.common.communication.Communicator.loadConfiguration(Communicator.java:601)

                                at com.esecurity.common.communication.Communicator.loadConfigurationWithRetry(Communicator.java:393)

                                at com.esecurity.common.communication.Communicator.initialize(Communicator.java:751)

                                at com.esecurity.common.communication.Communicator.createCommunicator(Communicator.java:157)

                                at com.esecurity.common.communication.Communicator.getInstance(Communicator.java:143)

                                at com.esecurity.common.communication.Communicator.getInstance(Communicator.java:123)

                                at com.esecurity.common.communication.Communicator.getInstance(Communicator.java:85)

                                at esecurity.base.subscription.SubscriptionManager.registerWithCommunicator(SubscriptionManager.java:258)

                                at esecurity.base.subscription.SubscriptionManager.subscribe(SubscriptionManager.java:172)

                                at esecurity.base.subscription.SubscriptionManager.subscribe(SubscriptionManager.java:108)

                                at com.esecurity.workflow.server.WorkflowServerComponent.componentActivated(WorkflowServerComponent.java:75)

                                at esecurity.base.ccs.BasicComponent.activate(BasicComponent.java:240)

                                at esecurity.base.ccs.proxy.ComponentElementProxy.activate(ComponentElementProxy.java:141)

                                at esecurity.base.ccs.proxy.ComponentElementProxy.changeChildrenStatus(ComponentElementProxy.java:213)

                                at esecurity.base.ccs.proxy.ComponentElementProxy.activate(ComponentElementProxy.java:134)

                                at esecurity.base.ccs.proxy.ComponentElementProxy.componentStateChange(ComponentElementProxy.java:121)

                                at esecurity.base.ccs.services.ComponentListenerSupport.fireListenerChange(ComponentListenerSupport.java:74)

                                at esecurity.base.ccs.services.ComponentListenerSupport.fireListenerChange(ComponentListenerSupport.java:55)

                                at esecurity.base.ccs.services.ComponentServices.activateAllComponents(ComponentServices.java:73)

                                at esecurity.base.ccs.proxy.ContainerProxy.activateContainer(ContainerProxy.java:61)

                                at esecurity.util.service.ContainerService$1.run(ContainerService.java:82)

                                at java.lang.Thread.run(Thread.java:745)

                Caused by: java.lang.SecurityException: User name [system] or password is invalid.

                                at org.apache.activemq.security.JaasAuthenticationBroker.authenticate(JaasAuthenticationBroker.java:89)

                                at org.apache.activemq.security.JaasAuthenticationBroker.addConnection(JaasAuthenticationBroker.java:68)

                                at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:98)

                                at org.apache.activemq.broker.MutableBrokerFilter.addConnection(MutableBrokerFilter.java:103)

                                at org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:818)

                                at org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)

                                at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:339)

                                at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188)

                                at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)

                                at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)

                                at org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:271)

                                at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)

                                at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)

                                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

                                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

                                ... 1 more

                Caused by: javax.security.auth.login.FailedLoginException: Password does not match

                                at org.apache.activemq.jaas.PropertiesLoginModule.login(PropertiesLoginModule.java:92)

                                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

                                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                                at java.lang.reflect.Method.invoke(Method.java:497)

                                at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)

                                at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)

                                at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)

                                at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)

                                at java.security.AccessController.doPrivileged(Native Method)

                                at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)

                                at javax.security.auth.login.LoginContext.login(LoginContext.java:587)

                                at org.apache.activemq.security.JaasAuthenticationBroker.authenticate(JaasAuthenticationBroker.java:84)

1. Open the /etc/opt/novell/sentinel/config/activemqusers.properties file.  

Example of un-encrypted passwords. 

system=d8c94c3771b77066c80df5840ea32eaf 

collectormanager=12b84d85 

correlationengine=968a47fa 

Example of encrypted passwords.

system=46bO31AToMMLVbVRI0XKse74zRf6DRHry9hhxixb9RmZ+LviZgpKMSfmUrLlCQzE
collectormanager=GUDr/6zWqCW6MTwQrMQCEw==
correlationengine=gF8igacxRq3hE4dSmJR/Jw==

Note: If the passwords are indeed un-encrypted move to step 2

2. Encrypt the currently un-encrypted passwords

• go to /opt/novell/sentinel/bin/
• su novell 
• Run the encryptpwd script to encrypt the passwords from step 2 

Example:

encryptpwd -e d8c94c3771b77066c80df5840ea32eaf

encryptpwd -e 12b84d85

encryptpwd -e 968a47fa

3. Update the results in the activemqusers.properties file

Example:

system=46bO31AToMMLVbVRI0XKse74zRf6DRHry9hhxixb9RmZ+LviZgpKMSfmUrLlCQzE

collectormanager=GUDr/6zWqCW6MTwQrMQCEw==

correlationengine=gF8igacxRq3hE4dSmJR/Jw==

Note: If after updating the activemqusers.properties file Sentinel will not start, the auth.login file likely needs to be updated as well. 

4. Open the /etc/opt/novell/sentinel/config/auth.login file

5. If this file did not get properly updated the section below will still have the PropertiesLoginModule required entry.

Example:

activemq-domain { 

    org.apache.activemq.jaas.PropertiesLoginModule required 

        org.apache.activemq.jaas.properties.user="activemqusers.properties" 

        org.apache.activemq.jaas.properties.group="activemqgroups.properties"; 

}; 

Replace the incorrect entry with EncryptedPropertiesLoginModule required. 

Example: 

};

activemq-domain {

    esecurity.ccs.auth.jaas.EncryptedPropertiesLoginModule required

        org.apache.activemq.jaas.properties.user="activemqusers.properties"

        org.apache.activemq.jaas.properties.group="activemqgroups.properties";

};

7. Start or restart Sentinel for the changes to take affect.

The error is occurring because the passwords in the /etc/opt/novell/sentinel/config/activemqusers.properties file did not get encrypted during the upgrade. It also possible that the /etc/opt/novell/sentinel/config/auth.login file did not get properly updated.