In our production environment, entity manager creation, the line below, sometimes take upto few ten's of seconds to create. Is there any reason that is known which may be causing this ? Any suggestions on how this can be addressed ?
EntityManager entityManager = factory.createEntityManager(properties);
More info is needed. If you share the relevant profiling data (e.g. using VisualVM) it may highlight the cause.
JVM memory profile:
JVM snapshot during the time of issue:
From our logs the issue seems to have occurred between 3:45 am to 4:15 am on Nov 22nd.
We need to see which threads are active, what they are trying to do and why they are so slow, when this happens. For this CPU and thread profiling, e.g. from VisualVM, will be needed.
This issue is sporadic and hence collecting the data that you are asking for is difficult. If we manage to get it we'll share it, but meanwhile are there any reasons that you think may be leading to this situation? Are there any timeouts that can be configured to prevent such long wait times?
There are no known issues that may be related.
How do you know that the issue happens on createEntityManager (your first post)?
Loggers were added above and below exactly around that line (createEntityManager) and it took close to 35 seconds to return.
Is it client-server mode or embedded mode? If client-server mode, are the client and server on the same machine/VM?
Do you see anything relevant in the ObjectDB log files (check both client and server log files if client-server is used).
If you try another createEntityManager immediately when it happens, will it also be slow? Maybe it could be useful to check then what happens.
Yes, its client-server mode. The logs do not show any thing unusual.
Check the maximum connections in the configuration file (on the server side), maybe your application occasionally hits the maximum? Try using 0 for unlimitted.
Are the client and server on the same machine? createEntityManager requires a new TCP/IP connection. Maybe the delay is due to a firewall or other external network settings?
If you can take a thread dump on the client side when this happens we can check where the client is waiting. You can try using Thread methods (dumpStack, getStackTrace, getAllStackTraces) automatically from another thread that will monitor the threads that invoke createEntityManager.
Note also that ObjectDB 2.7.6 that you use is old. If it is an issue with ObjectDB rather than something external, it is quite possible that a new ObjectDB version will not have this issue.