Summary
Error
After a restart of the agent, when doing POST calls to Operations Connector it was taking 15 ms to process each call, after some WS calls the time for each POST increased to almost 600000 ms.
Using streamweaver can see the execution time of the WS Call:
HttpResponse(200 OK,List(Cache-Control: no-cache, Date: <Date>, senderid: <ID>, server: BBC 12.06.013; opcgeni 01.00.000),HttpEntity.Strict(application/octetstream,ByteString()),HttpProtocol(HTTP/1.1)) (596494ms)
HttpResponse(200 OK,List(Cache-Control: no-cache, Date: <Date>, senderid: <ID>, server: BBC 12.06.013; opcgeni 01.00.000),HttpEntity.Strict(application/octetstream,ByteString()),HttpProtocol(HTTP/1.1)) (596610ms)
Cause
Large environment sending a lot of POST calls, agent with default socket set i.e. 60 and served by 6 threads(1:10), even if 59 connections are closed whole socket set is active in the memory.
Fix
In large environments with more connections, keeping the socket set smaller is more efficient. In such scenarios if the number of threads is 3 and the socket set size is 10(1:3), the requests will be processed faster and results in faster release of the socket set. Due to this reason in large environments where there are many connections and high traffic, keeping the value of SOCKETS_PER_SOCKETSET to 10 and MAX_SOCKETSET_THREADS to 3 is recommended.
How to configure the variables:
/opt/OV/bin/ovconfchg -ns bbc.http -set MAX_SOCKETSET_THREADS 3
/opt/OV/bin/ovconfchg -ns xpl.net -set SOCKETS_PER_SOCKETSET 10
/opt/OV/bin/ovc -kill
/opt/OV/bin/ovc -start
/opt/OV/bin/ovconfchg -ns xpl.net -set SocketPoll TRUE
Variables:
MAX_SOCKETSET_THREADS (6 default) / SOCKETS_PER_SOCKETSET (60 default): To achieve concurrency, ovbbccb keeps all the connections in a pool, called socket_set. Per socket set, there are 60 connections by default and 6 threads serving them by default. On an average 10 connections are served by 1 thread.
MAX_CONNECTIONS :After these many connections, ovbbccb tries to close the unused connections. Say if the total connections reaches > 2000, it will close the unused set of connections from the existing 2000 connections.