Issue #1651: GC Memory

Hi there,

I was hoping to get some support on ObjectDB regarding memory usage.  I received the following error.  Context for the error is found below.

[2015-04-27 13:16:18 #20 server]

[ObjectDB 2.6.1] Unexpected exception (Error 990)
  Generated by Java HotSpot(TM) 64-Bit Server VM 1.7.0_76 (on Windows 7 6.1).
Please report this error on http://www.objectdb.com/database/issue/new
com.objectdb.o.InternalException: java.lang.OutOfMemoryError: GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.objectdb.o.PRU.aa(PRU.java:210)
at com.objectdb.o.VUT.k(VUT.java:689)
at com.objectdb.o.VUT.j(VUT.java:364)
at com.objectdb.o.VUT.j(VUT.java:350)
at com.objectdb.o.FCL.h(FCL.java:110)
at com.objectdb.o.PGT.q(PGT.java:142)
at com.objectdb.o.UPT.C(UPT.java:121)
at com.objectdb.o.UPT.l(UPT.java:110)
at com.objectdb.o.TSK.i(TSK.java:145)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.UPT.s(UPT.java:157)
at com.objectdb.o.PGT.q(PGT.java:109)
at com.objectdb.o.UPT.C(UPT.java:121)
at com.objectdb.o.UPT.l(UPT.java:110)
at com.objectdb.o.TSK.i(TSK.java:145)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.UPT.s(UPT.java:157)
at com.objectdb.o.PGT.q(PGT.java:109)
at com.objectdb.o.UPT.C(UPT.java:121)
at com.objectdb.o.URT.l(URT.java:171)
at com.objectdb.o.TSK.i(TSK.java:145)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.TSM.e(TSM.java:86)
at com.objectdb.o.UTT.A(UTT.java:365)
at com.objectdb.o.UTT.l(UTT.java:203)
at com.objectdb.o.TSK.i(TSK.java:145)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.MST.Vg(MST.java:1323)
at com.objectdb.o.PRG.ak(PRG.java:950)
at com.objectdb.o.PRG.ag(PRG.java:659)
at com.objectdb.o.PRG.af(PRG.java:553)
at com.objectdb.o.QRM.Vb(QRM.java:286)

I am doing a bulk load for customer data.  The customer data is sharded over 16 segments.  Each segment has its own load file and an odb file and all odb files are managed by a single object database server instance.  A large customer file (around 4.5 Gb) is broken out into 16 segments of around 260 MB each.  The process then loads each segment file into memory and then calls persist on each object.  About 1000 entities are persisted each transaction.  This loading continues until the entire segment file is loaded into ObjectDb.  After the segment is loaded into objectdb, the EntityManager is flushed and released.  This process is repeated until all 16 segments are loaded.

Around the 10th segment file, ObjectDb throws the above error.  Please advise as it is crucial to resolve this issue.

Regards.

A stack trace may be helpful in understanding many issues, but unfortunately it is usually useless for OutOfMemoryError issues. We will need a heap dump in order to understand which objects fill up the heap.

Common causes of OutOfMemoryError:

• Persisting too many large objects in one transaction. Note that only commit can release memory, not flush. So bulk load of data into the database should be done in smaller transaction (with begin and commit for each, and possibly also clear after commit)
• Using classes that are not enhanced. In reflection mode ObjectDB has to keep some strong references to objects, which are weak references when entity classes are enhanced.