evictAll() behavior

I'm writing a multi-user drawing tool, using objectdb as the central store.  When a user press F5, I want to pick up all the changes from the database.  So, I call evictAll() in the following ways:

pmf.getDataStoreCache().evictAll();  // the factory
pm.evictAll();                       // the persistence manager

A couple of questions about this:

1. the first evictAll() seems to be clearing the L2 cache -- is this correct

2. the evicts only work when they are done inside a transaction -- is this specified in the JDO spec?

Cheers,
Andrew

Refreshing managed objects in JDO is a bit tricky.

• pmf.getDataStoreCache().evictAll() clears the L2 cache.
• pm.evictAll() clears the persistence manager (L1) cache.
• But evictAll does not affect objects that have been changed by the current PersistenceManager - it is useful only to see changes that have been committed by other persistence managers.
• To undo changes of the current PersistenceManager you will have to invoke pm.refreshAll()
• You have to invoke refreshAll inside a transaction, because according to the JDO specification - "When called with a transaction active, the refreshAll method with no parameters causes all transactional instances to be refreshed. If a transaction is not in progress, then this call has no effect."

"But evictAll does not affect objects that have been changed by the current PersistenceManager"

Cool, that is exactly the effect I am looking for.

What is the difference / characteristics of the L1 versus L2 cache?  I presume L1 is only maintained for a transaction whereas L2 is more like a session cache?

The L1 cache is per EntityManager / PersistenceManager and the L2 cache is per EntityManagerFactory / PersistenceManagerFactory, i.e. shared by multiple EntityManager / PersistenceManager instances.

There are also other differences in the configuration possibilities.

In addition, currently the L2 cache is disabled by default.

Hi

 I have further question according evictAll method and cache. I'm adding a lot of object in one transaction and i want to reclaim memory after commit or even better after flush but  I can imagine that after flush method it could be problematic so i did sth like this

System.out.println("Cache size before commit= " + pm.getManagedObjects().size());
pm.currentTransaction().commit();
pm.currentTransaction().begin();
System.out.println("Cache size after commit= " + pm.getManagedObjects().size());
pm.evictAll();
System.out.println("Cache size after pm.evict = " + pm.getManagedObjects().size());

and it seems that objects are not evicted, the number of managed objects after evict is the same like before evictAll() mth and i in the result i get OutOfMemory

How I can check size od the L2 cache?

br

Tomasz

This behavior might indicate that your persistence capable (entity) classes are not enhanced.

ObjectDB supports working with no enhancement as an extension to JDO - but it is less efficient and has some limitations. For example, strong references to your objects are always used (in order to detect future changes) and objects cannot be evicted.

If your classes are not enhanced - enhancing them could solve the problem without any additional action (such as evict).

hi I've checked in debuger that every data model class has fileds like: __odbTracker ENT, jdoFlags, jdoStateManager so as i understand they are enhanced

OK. So please check the state of the objects before evictAll.

evictAll affects only objects in PERSISTENT CLEAN and NON TRANSACTIONAL CLEAN.

i've checked the same code with objectdb and versantdb (only jdo.properties were changed) and there is a result

OBJECTDB OUTPUT:

main--testDbProvisioning
Cache size before COMMIT/BEGIN= 1
transient
transient
Cache size after COMMIT/BEGIN= 2630
transient
transient
Cache size after pm.evict = 2630

VERSANT OUTPUT:

main--testDbProvisioning
Cache size before COMMIT/BEGIN = 280
persistent-new
hollow/persistent-nontransactional
Cache size after COMMIT/BEGIN = 0
hollow/persistent-nontransactional
hollow/persistent-nontransactional

i've checked state of root object, objects are persisted correctly but there is something with their state and cache, if You need more info please tell me what to check

br

Hi i've changed enhacment from the load time to compile time and objects states are correct now, but evict still does not work

main--testDbProvisioning
Cache size before COMMIT/BEGIN= 1
persistent-new
hollow/persistent-nontransactional
Cache size after COMMIT/BEGIN= 2630
hollow/persistent-nontransactional
hollow/persistent-nontransactional
Cache size after pm.evict = 2630

This is a step forward. By the way - could you please provide more details about the load time enhancement that didn't work - did you use javaagent? which JVM are you using (JDK / JRE, Sun/JRockit/J9)?

About the cache - evictAll makes objects hollow but doesn't remove them from the persistence context cache immediately (unlike makeTransient / makeTransientAll).  ObjectDB doesn't hold strong references to hollow objects, so if your application doesn't hold strong references either - they can be garbage collected.

evictAll - i could use makeTransient but if the L2 cache is using weak references  I should not get OutOfMemory exception,  Im adding objects in a loop and i do not have any other references from the app

Enhancer - I'm using jre in version  (build 1.6.0_26-b03), I've checked classes if they are implementing interfaces PersistenceCapable and Detachable after postprocesing and they do not. I spoted that I did not give target interface JPA or JDO Do You have separate anhancers for each API or I must give additional option?

To Enhance classes in load time i did sth like this com.objectdb.Enhancer.enhance("graphicselements.*");

br

Tomasz

after enhancement classes implements interface com.objectdb.spi.TrackableUserType but they do not implement PersistenceCapable and Detachable as Isaid earlier

The ObjectDB 2 Enhancer doesn't use the PersistenceCapable interface, which was mandatory in JDO 1 but not in JDO 2. Anyway,  it seems that there might be an enhancement issue here. ObjectDB should not hold strong references to your objects unless they contain new data that has to be stored in the database - if the classes are enhanced correctly.

If you can post a program that demonstrates the problem it could help.

Hi

 I've pasted provisioning method, and atached data objects classes, i hope that it helps 

public void testDbProvisioning() {
  long time = System.currentTimeMillis();
  System.out.println(Thread.currentThread().getName() + "--testDbProvisioning");
  long picturesToAddNumber = Integer.parseInt(System.getProperty("picturesNumber"));
  int shapesPerNumber = Integer.parseInt(System.getProperty("shapesNumber"));

  PersistenceManager pm = pmf.getPersistenceManager();
  // pm.addInstanceLifecycleListener(new JDOLifecycleListener(), null);
  pm.currentTransaction().begin();

  List<Color> colors = new ArrayList<Color>();
  for (int i = 0; i < 20; i++) {
   colors.add(new Color(i, i + 2, i - 3));
  }

  for (int p = 0; p < picturesToAddNumber; p++) {
   Picture picture = new Picture();
   Square canvas = new Square(picture, 400);
   canvas.setBackgroundColour(colors.get(0));
   picture.setCanvas(canvas);

   for (int i = 0; i < shapesPerNumber; i++) {
    Triangle triangle = new Triangle(picture, i % 100);
    triangle.setBackgroundColour(colors.get(i % colors.size()));
    triangle.setColorForLines(colors);
    picture.getShapes().add(triangle);

    Square square = new Square(picture, i % 100);
    square.setBackgroundColour(colors.get(i % colors.size()));
    square.setColorForLines(colors);
    picture.getShapes().add(square);

    Polygon polygon = new Polygon(picture, i % 10, 1 % 100);
    polygon.setBackgroundColour(colors.get(i % colors.size()));
    polygon.setColorForLines(colors);
    picture.getShapes().add(polygon);
   }
   pm.makePersistent(picture);
   if ((p % 100) == 0) {
    System.out.println("Cache size before flush = " + pm.getManagedObjects().size());
    System.out.println(JDOHelper.getObjectState(picture));
    pm.currentTransaction().commit();
    System.out.println(JDOHelper.getObjectState(picture));
    pm.currentTransaction().begin();
    System.out.println("Cache size after flush = " + pm.getManagedObjects().size());
    System.out.println(JDOHelper.getObjectState(picture));
    pm.evictAll();
    System.out.println(JDOHelper.getObjectState(picture));
    System.out.println("Cache size after pm.evict = " + pm.getManagedObjects().size());
    pmf.getDataStoreCache().evictAll();
    System.out.println(Thread.currentThread().getName() + " --" + "flush " + p);
   }
  }

  if (rollbackOnly) {
   pm.currentTransaction().rollback();
   System.out.println(Thread.currentThread().getName() + " --" + "provisioning rollback");
  } else {
   pm.currentTransaction().commit();
   System.out.println(Thread.currentThread().getName() + " --" + "provisioning commit");
  }

  System.out.println("Elapsed time : " + (System.currentTimeMillis() - time));
  long numOfPicturesAfter = countObjects(pm, Picture.class);
  System.out.println(Thread.currentThread().getName() + " --" + "Total Pictures: " + numOfPicturesAfter);
  pm.close();
}

Please provide a complete runnable program as explained in these instructions (i.e. one Java file, minimal entity classes as static classes, main method, instantiation of a pmf, etc.).

This problem seems also related to a failure in using enhancement (since in reflection mode ObjectDB has to keep strong references to objects to detect changes on commit by snapshot comparison).

OK it works fine with your enhancer, I've used RI enhancer because in spec there is said that every provider should support that enhancer, our project is composed with many small subprojects and we do not to be dependent in that projects on provider specific enhancer but with ri enhancer it works almost fine. Thanks for your support

Actually supporting the RI enhancer (or a standard Enhancer) was a requirement of JDO 1 that became optional in JDO 2 (see the JDO 2 spec). ObjectDB 1.x used the standard Enhancer. ObjectDB 2.x uses it own Enhancer (which is not compatible with the RI enhancer) to provide better performance (e.g. objects are written and read using the new Enhancer in one piece instead of as field by field - which is more appropriate in relational databases).