ObjectDB ObjectDB

Performance and memory usage of queries

#1

Hello,

we can't understand why the performance of queries is not the best and the execution requires so much memory.

In the implementation of our UnitTests we have used "DELETE FROM Object" to delete the database. That did not work because we often get an "out of memory" exception.

What's all done in the execution of queries? Why is so much memory is required, although from the application point of view, no object is loaded.

An Example: [To get the error they please limit the memory to 512mb  (VM-argument:  -Xmx512m)]

import java.util.ArrayList;
import java.util.List;

import javax.persistence.CascadeType;
import javax.persistence.ElementCollection;
import javax.persistence.Entity;
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.FlushModeType;
import javax.persistence.OneToMany;
import javax.persistence.Persistence;
import javax.persistence.Query;

public class MemoryUsageOfQuerys {

static int steps = 500000;

public static void main(String[] args) {

  EntityManagerFactory emf = Persistence
    .createEntityManagerFactory("objectdb:$objectdb/db/test.tmp;drop");

  EntityManager em;

  for (int i = 0; i < 100; i++) {
   em = emf.createEntityManager();
   em.setFlushMode(FlushModeType.AUTO);
   em.getTransaction().begin();

   MyEntity myEntity = new MyEntity();
   myEntity = em.merge(myEntity);

   for (int m = 0; m < steps; m++) {
    MyEntityElement myEntityElement = new MyEntityElement();
    myEntity.getList().add(myEntityElement);
   }
   em.getTransaction().commit();
   em.close();
   System.out.print(".");
  }

  em = emf.createEntityManager();
  em.setFlushMode(FlushModeType.AUTO);
  em.getTransaction().begin();

  Query q = em.createQuery("DELETE FROM Object");
  q.executeUpdate(); // <- OUT OF MEMORY   with VM-argument:  -Xmx512m

  em.getTransaction().commit();
  em.close();

  emf.close();
}


@Entity
public static class MyEntity {
  @OneToMany(cascade = CascadeType.ALL)
  private List<MyEntityElement> list = new ArrayList<>();

  public List<MyEntityElement> getList() {
   return list;
  }
}


@Entity
public static class MyEntityElement {
  @ElementCollection
  private float value = 0;
}
}
edit
delete
#2

In addition, the error message we get by the example:

[ObjectDB 2.6.1_01] Unexpected exception (Error 990)
  Generated by Java HotSpot(TM) 64-Bit Server VM 1.7.0_71 (on Windows 7 6.1).
Please report this error on http://www.objectdb.com/database/issue/new
com.objectdb.o.InternalException: java.lang.OutOfMemoryError: GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.objectdb.o.VUT.k(VUT.java:476)
at com.objectdb.o.VUT.j(VUT.java:364)
at com.objectdb.o.VUT.j(VUT.java:350)
at com.objectdb.o.SEV.Z(SEV.java:194)
at com.objectdb.o.VUT.k(VUT.java:679)
at com.objectdb.o.VUT.j(VUT.java:364)
at com.objectdb.o.VUT.j(VUT.java:350)
at com.objectdb.o.PRU.aa(PRU.java:212)
at com.objectdb.o.VUT.k(VUT.java:689)
at com.objectdb.o.VUT.j(VUT.java:364)
at com.objectdb.o.VUT.j(VUT.java:350)
at com.objectdb.o.FCL.h(FCL.java:110)
at com.objectdb.o.VLT.j(VLT.java:128)
at com.objectdb.o.PGW.aj(PGW.java:292)
at com.objectdb.o.PGW.ai(PGW.java:222)
at com.objectdb.o.UPT.C(UPT.java:134)
at com.objectdb.o.UPT.l(UPT.java:110)
at com.objectdb.o.TSK.i(TSK.java:145)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.UPT.s(UPT.java:157)
at com.objectdb.o.PGT.q(PGT.java:109)
at com.objectdb.o.UPT.C(UPT.java:121)
at com.objectdb.o.UPT.l(UPT.java:110)
at com.objectdb.o.TSK.i(TSK.java:145)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.UPT.s(UPT.java:157)
at com.objectdb.o.PGT.q(PGT.java:109)
at com.objectdb.o.UPT.C(UPT.java:121)
at com.objectdb.o.UPT.l(UPT.java:110)
at com.objectdb.o.TSK.i(TSK.java:145)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.UPT.s(UPT.java:157)
edit
delete
#3

It might be worth mentioning my similar experience.

I remember I had an issue several times with deleting many object at once. I didn't have time to report it, but I experienced it several times. In my case the operation wouldn't complete (or maybe it was garbage collecting so much due to the apparent memory issue). It was going and going and going...never ending. (or maybe I didn't wait enough, but it was simply extremely long for a simple delete operation)

edit
delete
#4

This DELETE query is very simple, but indeed requires a lot of memory:

    em.getTransaction().begin();
    Query q = em.createQuery("DELETE FROM Object");
    q.executeUpdate(); // <- OUT OF MEMORY   with VM-argument:  -Xmx512m
    em.getTransaction().commit();

On update ObjectDB prepares all the revised pages in memory before writing them to the database file. In this DELETE operation the entire database is affected, so ObjectDB needs RAM in the size of the database itself. Obviously deleting the entire database file would be more efficient than keeping the database file and deleting all the objects.

ObjectDB uses temporary files for many operations (e.g. query processing) and should in future versions use temporary files also to enable very large transactions. Currently the size of the transaction (i.e. the total size of database pages that have to be replaced) is limited by the JVM heap size.

ObjectDB Support
edit
delete

Reply

To post on this website please sign in.