Cascading makeTransient

Hi!

We have a memory leak in our app because of ObjectDBs __odbTracker fields.

The situation is that we read the objects from the DB (many and large objects of class A) and then keep in the application cache only some embedded parts of those objects (let's call them class B). This application cache should work independently on the DB as if it contained transient objects.

However those instances of B contain the field __odbTracker of type com.objectdb.o.EMT, which then holds a reference to the object A and its __odbTracker of type com.objectdb.o.ENT.

As a result the instances of A can not be garbage-collected even after the PersistenceManager is closed and they are no longer needed from the application point-of-view. So as long as I keep B in application cache, also A is present in the memory and it represents additional unwanted memory consumption.

I tried to solve that via calling pm.makeTransient() on A but it removed only A.__odbTracker, however A.B.__odbTracker remained unchanged. I supposed that makeTransient() should work somehow cascading even for the fields of the Entity, but it does not.

When I call makeTransient() on B then I get an exception saying "Type B is not defined as an entity (@Entity is missing)" because B is defined as embedded-only.

I tried also calling makeTransient(A, true) with a FetchPlan containing FetchGroup with category ALL, but it had only the same effect as a simple makeTransient(A). 

So my question is how to solve this situation ? 

As you analyzed correctly, this situation conflicts with current ObjectDB enhancement. The problem is the direct reference from com.objectdb.o.EMT to A that prevents garbage collecting of A instances (the reference com.objectdb.o.ENT is less important since after making an instance transient the reference to that instance from its com.objectdb.o.ENT should become week or soft). The direct reference was added in version 2.6.2 in order to prevent garbage collecting of the owner object as long as the embedded object is in use, as a fix to issue #1620 (so this reported issue is probably new).

Cascading makeTransient may solve the issue, but may cause other problems (slower makeTransient action, difficulties in tracking changes if the entity is made persistent again) so should be considered carefully.

As a workaround you can nullify the direct reference from B's com.objectdb.o.EMT to A when needed using reflection.

The suggested workaround is no big help in our case as we have objects with many levels of embedding, so the only possible way would be to traverse the whole graph of object nodes via reflection which is neither a nice nor an effective solution when we need to use it almost all the time when reading the DB.

Our target is to simply get an instance of an object from the database which is no longer connected in any way to the database. Is there any other way to achieve this ?
Perhaps some copying or the above mentioned method PersistenceManager.makeTransient(object, true) with a correct FetchPlan?
The documentation of this method states that: 

If the useFetchPlan parameter is true, then the current FetchPlan is applied to the pc parameter, as if detachCopy(Object) had been called. After the graph of instances is loaded, the instances reachable via loaded fields is made transient. The state of fields in the affected instances is as specified by the FetchPlan.

So I assumed it should make transient even the embedded instances if they are loaded.

Object state is relevant only to instances of entity classes (persistence capable classes) and not for embedded objects, which are considered to be part of the containing objects. Note that when A becomes transient it already includes B, so everything is already transient, just that ObjectDB sets a permanent strong reference from B to A.

Will it help if we add an option to use ObjectDB in a mode of pre 2.6.2 version, i.e. without that reference? It should work if you do not apply JPA operations on embedded objects after the containing objects are garbage collected.

Yes, that would help us as we use only JDO.

Build 2.6.3 adds an option to treat embedded objects on detachment (eventually it may be better than our suggestion at #4 above, to avoid issue #1620 again).

To enable it you will have to set a system property before using ObjectDB:

    System.setProperty("objectdb.temp.detach-embeddable", "true");

Then embedded objects are expected to be disconnected when owner entity objects are detached (made transient).

The situation has been improved with the build 2.6.3_01, but it's still not solved.

When I use this build with the above mentioned property, then on

pm.makeTransient(A)

both fields A.__odbTracker and A.B.__odbTracker are nulled. So this is the improvement.

However the whole object A is still not transient completely, because I also have some fields on A and B of types ArrayList and HashMap which keep their objectDB specific types (objectdb.java.util.ArrayList, objectdb.java.util.HashMap) even after the makeTransient() call. Those collections also keep their __odbTracker fields which have the same value as A.__odbTracker was.

Therefore when I call makeTransient(A), pm.close() and then I try to change those collections I got the following error:

Exception in thread "main" [ObjectDB 2.6.3_01] Unexpected exception (Error 990)
  Generated by Java HotSpot(TM) 64-Bit Server VM 1.8.0_40 (on Mac OS X 10.10.2).
Please report this error on http://www.objectdb.com/database/issue/new
com.objectdb.o.InternalException: null
com.objectdb.o.InternalException
at com.objectdb.o.InternalException.f(InternalException.java:236)
at com.objectdb.o.STA.K(STA.java:339)
at com.objectdb.o.ENT.F(ENT.java:543)
at com.objectdb.o.ENT.ai(ENT.java:1404)
at com.objectdb.o.ENT.ah(ENT.java:1257)
at com.objectdb.o.ENT.beforeModifySCO(ENT.java:1187)
at objectdb.java.util.HashMap.__odbBeforeModify(Unknown Source)
at objectdb.java.util.HashMap.put(Unknown Source)

So it looks to me that those collections are still managed by the ODB Tracker which they kept.

Would it be possible to nullify also the trackers of those embedded collections ?

Please try build 2.6.3_02 that should disconnect collections as well.

The new build resulted in a failing makeTransient().

Here is the mistake I get:

com.objectdb.o._JdoUserException: Failed to get reference value of field field eu.extech.quant.dbfix.NotFinalFieldsClass.fieldInt using enhanced method
NestedThrowables:
java.lang.ClassCastException: java.lang.String cannot be cast to com.objectdb.spi.TrackableUserType
at com.objectdb.o.JDE.g(JDE.java:126)
at com.objectdb.o.ERR.f(ERR.java:56)
at com.objectdb.o.JDE.f(JDE.java:52)
at com.objectdb.o.OBC.onObjectDBError(OBC.java:1503)
at com.objectdb.jdo.PMImpl.makeTransient(PMImpl.java:1556)
at com.objectdb.jdo.PMImpl.makeTransient(PMImpl.java:1597)
at eu.extech.serverImpl.jdo.JDOConnection.makeTransient(JDOConnection.java:263)
... 2 more
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to com.objectdb.spi.TrackableUserType
at com.objectdb.o.UMR.H(UMR.java:746)
at com.objectdb.o.UMR.F(UMR.java:711)
at com.objectdb.o.MMM.ag(MMM.java:1085)
at com.objectdb.o.MMM.ag(MMM.java:1111)
at com.objectdb.o.ENT.D(ENT.java:356)
at com.objectdb.o.ENT.af(ENT.java:975)
at com.objectdb.jdo.PMImpl.makeTransient0(PMImpl.java:1586)
at com.objectdb.jdo.PMImpl.makeTransient(PMImpl.java:1553)
... 4 more

The source code of the entity classes follows:

public class NotFinalFieldsClass {

private int fieldInt;
private String fieldString;
private HashMap<String, Object> fieldMap;
private ArrayList<String> fieldList;
private StringWrapper fieldWrapper;

private NotFinalFieldsClass() {}

public NotFinalFieldsClass(int fieldInt){
  this.fieldInt = fieldInt;
  this.fieldString = ""+fieldInt;
  this.fieldMap = new HashMap<String, Object>();
  this.fieldMap.put(fieldString, this.fieldInt);
  this.fieldList = new ArrayList<String>();
  this.fieldList.add("FirstString");
  this.fieldList.add("SecondString");
  this.fieldWrapper = new StringWrapper("12345", "67890");
 
  StringWrapper thirdObject = new StringWrapper("third", "");
  this.fieldMap.put("thirdObject", thirdObject);
}

public int getFieldInt() {
  return fieldInt;
}
public void setFieldInt(int fieldInt) {
  this.fieldInt = fieldInt;
}
public String getFieldString() {
  return fieldString;
}
public void setFieldString(String fieldString) {
  this.fieldString = fieldString;
}
public HashMap<String, Object> getFieldMap() {
  return fieldMap;
}
public void setFieldMap(HashMap<String, Object> fieldMap) {
  this.fieldMap = fieldMap;
}

@Override
public String toString() {
  return getClass().getSimpleName() + "[fieldInt: " + fieldInt + ";fieldString: " + fieldString + ";fieldMap: " + fieldMap.toString() + "]";
}

public StringWrapper getFieldWrapper() {
  return fieldWrapper;
}

public void setFieldWrapper(StringWrapper fieldWrapper) {
  this.fieldWrapper = fieldWrapper;
}
}

public class StringWrapper {

private String string;
private ArrayList<String> list;

public StringWrapper(String string, String string2) {
  this.string = string + string2;
  this.list = new ArrayList<String>();
  this.list.add(string);
  this.list.add(string2);
}

public String getString() {
  return string;
}

public void setString(String string) {
  this.string = string;
}

@Override
public String toString() {
  return "StringWrapper : "+ getString().length();
}

public ArrayList<String> getList() {
  return list;
}

public void setList(ArrayList<String> list) {
  this.list = list;
}
}

You are right. Could you please try build 2.6.3_03?

Great!
With this build the simple testcase finished smoothly and the object is looking completely transient now.

I will check our memory-leak issue in the profiler with our real app later with this build.

Thank you very much!