Memory error during commit, if @ElementCollection field contains many elements

#1

Hello,

we have following entity and we work with enhanced entities.

@Entity
public class TCStep {
    @Index
    private long stepNumber;
    @ElementCollection
    private Map<Integer, String> values = new HashMap<>();

The element collection contains more than 10.000 elements.
There are more than 100.000 TCSteps persisted in the database.

In one of our use case we load all available TCSteps and modify each stepNumber.
But this one runs into memory error and happens only, if we have many elements in the element collection although the element collection is actually not load because it is a lazy field.
Why we run into a memory error?

#2

When an entity object (such as a TCStep instance) is retrieved its full content, including embedded collections, is loaded from the database. Some content is loaded as byte[] to save space, and extracted into objects only when needed, but that loaded content still requires memory space. Lazy setting in JPA is considered as a hint to the implementation, but the implementation can still load content eagerly. ObjectDB follows the lazy setting hint for relationships between entity objects but not for content in the entity itself, as usually it increases performance to load the entire content in one operation.

Possible directions to solve the memory issue:

  1. Split the TCStep entity class, and move the large embedded collection into a new entity classes. Every TCStep will reference a new entity object that will contain the collection and will be loaded lazily.
  2. If changing the schema is difficult, try to process the TCStep entity objects in smaller batches.

 

ObjectDB Support

Reply