1173 words

Cascading makeTransient

#1
2015-07-03 10:50

Hi!

We have a memory leak in our app because of ObjectDBs __odbTracker fields.

The situation is that we read the objects from the DB (many and large objects of class A) and then keep in the application cache only some embedded parts of those objects (let's call them class B). This application cache should work independently on the DB as if it contained transient objects.

However those instances of B contain the field __odbTracker of type com.objectdb.o.EMT, which then holds a reference to the object A and its __odbTracker of type com.objectdb.o.ENT.

As a result the instances of A can not be garbage-collected even after the PersistenceManager is closed and they are no longer needed from the application point-of-view. So as long as I keep B in application cache, also A is present in the memory and it represents additional unwanted memory consumption.

I tried to solve that via calling pm.makeTransient() on A but it removed only A.__odbTracker, however A.B.__odbTracker remained unchanged. I supposed that makeTransient() should work somehow cascading even for the fields of the Entity, but it does not.

When I call makeTransient() on B then I get an exception saying "Type B is not defined as an entity (@Entity is missing)" because B is defined as embedded-only.

I tried also calling makeTransient(A, true) with a FetchPlan containing FetchGroup with category ALL, but it had only the same effect as a simple makeTransient(A). 

So my question is how to solve this situation ? 

jakab
jakab's picture
Joined on 2011-05-12
User Post #24
#2
2015-07-03 15:48

As you analyzed correctly, this situation conflicts with current ObjectDB enhancement. The problem is the direct reference from com.objectdb.o.EMT to A that prevents garbage collecting of A instances (the reference com.objectdb.o.ENT is less important since after making an instance transient the reference to that instance from its com.objectdb.o.ENT should become week or soft). The direct reference was added in version 2.6.2 in order to prevent garbage collecting of the owner object as long as the embedded object is in use, as a fix to issue #248 (so this reported issue is probably new).

Cascading makeTransient may solve the issue, but may cause other problems (slower makeTransient action, difficulties in tracking changes if the entity is made persistent again) so should be considered carefully.

As a workaround you can nullify the direct reference from B's com.objectdb.o.EMT to A when needed using reflection.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #2,268
#3
2015-07-06 13:21

The suggested workaround is no big help in our case as we have objects with many levels of embedding, so the only possible way would be to traverse the whole graph of object nodes via reflection which is neither a nice nor an effective solution when we need to use it almost all the time when reading the DB.

Our target is to simply get an instance of an object from the database which is no longer connected in any way to the database. Is there any other way to achieve this ?
Perhaps some copying or the above mentioned method PersistenceManager.makeTransient(object, true) with a correct FetchPlan?
The documentation of this method states that: 

If the useFetchPlan parameter is true, then the current FetchPlan is applied to the pc parameter, as if detachCopy(Object) had been called. After the graph of instances is loaded, the instances reachable via loaded fields is made transient. The state of fields in the affected instances is as specified by the FetchPlan.

So I assumed it should make transient even the embedded instances if they are loaded.

 

jakab
jakab's picture
Joined on 2011-05-12
User Post #25
#4
2015-07-06 13:23

Object state is relevant only to instances of entity classes (persistence capable classes) and not for embedded objects, which are considered to be part of the containing objects. Note that when A becomes transient it already includes B, so everything is already transient, just that ObjectDB sets a permanent strong reference from B to A.

Will it help if we add an option to use ObjectDB in a mode of pre 2.6.2 version, i.e. without that reference? It should work if you do not apply JPA operations on embedded objects after the containing objects are garbage collected.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #2,270
#5
2015-07-07 07:01

Yes, that would help us as we use only JDO.

jakab
jakab's picture
Joined on 2011-05-12
User Post #26
#6
2015-07-13 07:33

Build 2.6.3 adds an option to treat embedded objects on detachment (eventually it may be better than our suggestion at #4 above, to avoid issue #248 again).

To enable it you will have to set a system property before using ObjectDB:

    System.setProperty("objectdb.temp.detach-embeddable", "true");

Then embedded objects are expected to be disconnected when owner entity objects are detached (made transient).

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #2,273
#7
2015-07-15 15:25

The situation has been improved with the build 2.6.3_01, but it's still not solved.

When I use this build with the above mentioned property, then on

pm.makeTransient(A)

both fields A.__odbTracker and A.B.__odbTracker are nulled. So this is the improvement.

However the whole object A is still not transient completely, because I also have some fields on A and B of types ArrayList and HashMap which keep their objectDB specific types (objectdb.java.util.ArrayList, objectdb.java.util.HashMap) even after the makeTransient() call. Those collections also keep their __odbTracker fields which have the same value as A.__odbTracker was.

Therefore when I call makeTransient(A), pm.close() and then I try to change those collections I got the following error:

 

Exception in thread "main" [ObjectDB 2.6.3_01] Unexpected exception (Error 990)
  Generated by Java HotSpot(TM) 64-Bit Server VM 1.8.0_40 (on Mac OS X 10.10.2).
Please report this error on http://www.objectdb.com/database/issue/new
com.objectdb.o.InternalException: null
com.objectdb.o.InternalException
at com.objectdb.o.InternalException.f(InternalException.java:236)
at com.objectdb.o.STA.K(STA.java:339)
at com.objectdb.o.ENT.F(ENT.java:543)
at com.objectdb.o.ENT.ai(ENT.java:1404)
at com.objectdb.o.ENT.ah(ENT.java:1257)
at com.objectdb.o.ENT.beforeModifySCO(ENT.java:1187)
at objectdb.java.util.HashMap.__odbBeforeModify(Unknown Source)
at objectdb.java.util.HashMap.put(Unknown Source)

 

So it looks to me that those collections are still managed by the ODB Tracker which they kept.

Would it be possible to nullify also the trackers of those embedded collections ?

jakab
jakab's picture
Joined on 2011-05-12
User Post #27
#8
2015-07-20 13:00

Please try build 2.6.3_02 that should disconnect collections as well.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #2,282
#9
2015-07-20 13:38

The new build resulted in a failing makeTransient().

Here is the mistake I get:

com.objectdb.o._JdoUserException: Failed to get reference value of field field eu.extech.quant.dbfix.NotFinalFieldsClass.fieldInt using enhanced method
NestedThrowables:
java.lang.ClassCastException: java.lang.String cannot be cast to com.objectdb.spi.TrackableUserType
at com.objectdb.o.JDE.g(JDE.java:126)
at com.objectdb.o.ERR.f(ERR.java:56)
at com.objectdb.o.JDE.f(JDE.java:52)
at com.objectdb.o.OBC.onObjectDBError(OBC.java:1503)
at com.objectdb.jdo.PMImpl.makeTransient(PMImpl.java:1556)
at com.objectdb.jdo.PMImpl.makeTransient(PMImpl.java:1597)
at eu.extech.serverImpl.jdo.JDOConnection.makeTransient(JDOConnection.java:263)
... 2 more
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to com.objectdb.spi.TrackableUserType
at com.objectdb.o.UMR.H(UMR.java:746)
at com.objectdb.o.UMR.F(UMR.java:711)
at com.objectdb.o.MMM.ag(MMM.java:1085)
at com.objectdb.o.MMM.ag(MMM.java:1111)
at com.objectdb.o.ENT.D(ENT.java:356)
at com.objectdb.o.ENT.af(ENT.java:975)
at com.objectdb.jdo.PMImpl.makeTransient0(PMImpl.java:1586)
at com.objectdb.jdo.PMImpl.makeTransient(PMImpl.java:1553)
... 4 more

 

The source code of the entity classes follows:

public class NotFinalFieldsClass {
 
private int fieldInt;
private String fieldString;
private HashMap<String, Object> fieldMap;
private ArrayList<String> fieldList;
private StringWrapper fieldWrapper;
 
private NotFinalFieldsClass() {}
 
public NotFinalFieldsClass(int fieldInt){
  this.fieldInt = fieldInt;
  this.fieldString = ""+fieldInt;
  this.fieldMap = new HashMap<String, Object>();
  this.fieldMap.put(fieldString, this.fieldInt);
  this.fieldList = new ArrayList<String>();
  this.fieldList.add("FirstString");
  this.fieldList.add("SecondString");
  this.fieldWrapper = new StringWrapper("12345", "67890");
 
  StringWrapper thirdObject = new StringWrapper("third", "");
  this.fieldMap.put("thirdObject", thirdObject);
}
 
public int getFieldInt() {
  return fieldInt;
}
public void setFieldInt(int fieldInt) {
  this.fieldInt = fieldInt;
}
public String getFieldString() {
  return fieldString;
}
public void setFieldString(String fieldString) {
  this.fieldString = fieldString;
}
public HashMap<String, Object> getFieldMap() {
  return fieldMap;
}
public void setFieldMap(HashMap<String, Object> fieldMap) {
  this.fieldMap = fieldMap;
}
 
@Override
public String toString() {
  return getClass().getSimpleName() + "[fieldInt: " + fieldInt + ";fieldString: " + fieldString + ";fieldMap: " + fieldMap.toString() + "]";
}
 
public StringWrapper getFieldWrapper() {
  return fieldWrapper;
}
 
public void setFieldWrapper(StringWrapper fieldWrapper) {
  this.fieldWrapper = fieldWrapper;
}
}

 

public class StringWrapper {
 
private String string;
private ArrayList<String> list;
 
public StringWrapper(String string, String string2) {
  this.string = string + string2;
  this.list = new ArrayList<String>();
  this.list.add(string);
  this.list.add(string2);
}
 
public String getString() {
  return string;
}
 
public void setString(String string) {
  this.string = string;
}
 
@Override
public String toString() {
  return "StringWrapper : "+ getString().length();
}
 
public ArrayList<String> getList() {
  return list;
}
 
public void setList(ArrayList<String> list) {
  this.list = list;
}
}

 

 

 

 

jakab
jakab's picture
Joined on 2011-05-12
User Post #28
#10
2015-07-20 15:31

You are right. Could you please try build 2.6.3_03?

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #2,283
#11
2015-07-20 15:38

Great!
With this build the simple testcase finished smoothly and the object is looking completely transient now.

I will check our memory-leak issue in the profiler with our real app later with this build.

Thank you very much! 

jakab
jakab's picture
Joined on 2011-05-12
User Post #29

Post Reply

Please read carefully the posting instructions - before posting to the ObjectDB website.

  • You may have to disable pop up blocking in order to use the toolbar (e.g. in Chrome).
  • Use ctrl + right click to open the browser context menu in the editing area (e.g. for using a browser spell checker).
  • To insert formatted lines (e.g. Java code, stack trace) - select a style in the toolbar and then insert the text in the new created block.
  • Avoid overflow of published source code examples by breaking long lines.
  • You may mark in paragraph code words (e.g. class names) with the code style (can be applied by ctrl + D).
  • Long stack traces (> 50 lines) and complex source examples (> 100 lines) should be posted as attachments.
Attachments:
Maximum file size: 32 MB
Cancel