385 words

High memory consumption even after close() of EntityManagerFactory

#1
2015-09-18 09:55

Hello, we still have memory usage problems. When debugging, we have now observed a case in which the execution of multiple queries increasingly byte array were generated (about 1.5 GB) which from GC could not be released. Not even after the EntityManagerFactory closed.

We created a small test with which you can observe similar behavior.
The huge memory usage only occurs if we execute the query (line 56 to 61). If we remove this query, all is well.


The memory is only released after we re-run a query (on a different database).

Do you have an explanation?

btc_es
btc_es's picture
Joined on 2014-10-20
User Post #75
#2
2015-09-18 14:54

The memory is held by ObjectDB in a database page object pool, with the purpose of reducing large object creation. It is unclear if this optimization is still useful in more modern JVM implementations.

Build 2.6.3_06  introduces an option to disable this pool by setting a system property:

    System.setProperty("objectdb.temp.avoid-page-recycle", "true");

This can also be set as a JVM argument:

> java ... -Dobjectdb.temp.avoid-page-recycle=true

In addition, this memory will automatically be released (if the pool is not disabled) on EntityManagerFactory close.

 

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #2,312
#3
2015-09-21 06:53

The optimization of memory usage was very effective.

  • The test requires now only ~600MB Memory (previously ~1100MB)
  • The not been released memory has been reduced from 400 to 100 MB.

But, the 100 MB byte array are still not released. The option objectdb.temp.avoid-page-recycle makes no change.

The memory can only be released from the GC after the query to the second database connection is performed (line 92).

btc_es
btc_es's picture
Joined on 2014-10-20
User Post #77
#4
2015-09-21 07:30

Addendum:
With the configuration <temp path = "$ temp" threshold = "1mb" /> is the byte array only 3MB.
I think that's OK.

But at threshold = "64mb" the memory is not it released as expected.

btc_es
btc_es's picture
Joined on 2014-10-20
User Post #78
#5
2015-09-21 09:58

Different types of memory buffers are used by ObjectDB:

  • Pool of database pages for reuse - this can be disabled now with objectdb.temp.avoid-page-recycle
  • Other lists, including query results, flush updates, etc. are not affected by the new option, but can be controlled by the temp threshold configuration (per list, not globally).

The pool of database pages is now released (if active) on EntityManagerFactory close. However, there are other structures that should be released too, and this will be addressed in next versions.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #2,313
#6
2015-09-21 12:38

You may try build 2.6.3_07 that should improve query resources release.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #2,315
#7
2015-09-21 13:11

Thanks, now it works (Or rather, the memory consumption is as expected).

But we observed no effect by the new property 'objectdb.temp.avoid-page-recycle'. Now it works well with and without this property!

btc_es
btc_es's picture
Joined on 2014-10-20
User Post #79

Post Reply

Please read carefully the posting instructions - before posting to the ObjectDB website.

  • You may have to disable pop up blocking in order to use the toolbar (e.g. in Chrome).
  • Use ctrl + right click to open the browser context menu in the editing area (e.g. for using a browser spell checker).
  • To insert formatted lines (e.g. Java code, stack trace) - select a style in the toolbar and then insert the text in the new created block.
  • Avoid overflow of published source code examples by breaking long lines.
  • You may mark in paragraph code words (e.g. class names) with the code style (can be applied by ctrl + D).
  • Long stack traces (> 50 lines) and complex source examples (> 100 lines) should be posted as attachments.
Attachments:
Maximum file size: 32 MB
Cancel