Missing (null) elements in eager loaded references

I have the following problem:

Setup:

• Two hierachical Entities (Element, DataElement), which contain an ElementCollection of Embeddables (Costs).
• The Embeddable from the ElementCollection has a OneToMany relationship to DataElement.
• Every relationship is eagerly fetched

 

Problem:

Sometimes! only on our server! I get a NullPointer exception, because an element is null which is! not null.

 

Example1:

Element.getChildren.get(0).getCosts().get(0).getDataElements().get(0).getLabel()

The getLabel() throws an NullPointer because the DataElement at index 0 is not yet present and thus null.

But if I run the exact same code with the exact same data again everything is fine.

The problem occures only on our server and not on our local machines (faster) even with the same db and application.

 

Sometimes it also happens while filtering Elements to only get those with a specific cost amount:

Element.getCosts().get(0).getAmount() > X

 

Could it be that despite being fetched eager the elements are yet not loaded from the db and thus are null?

Otherwise a cannot explain why sometimes alle data is present and sometimes some of the deeper ones are still missing.

Or maybe there is a problem with my data structure?

 

Super classes:

@MappedSuperclass
@SequenceGenerator(name = "seq", initialValue = 1, allocationSize = 100)
public abstract class BaseElement {
  @Id
  @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "seq")
  private Long id;

  etc...
}

@MappedSuperclass
public abstract class BaseCostElement extends BaseElement {
  @ElementCollection(fetch = FetchType.EAGER)
  private List<Cost> costs;

  etc...
}

Hierarchical entities:

@Entity
public class Element extends BaseCostElement {
  @OneToMany(mappedBy = "parent", fetch = FetchType.EAGER)
  private List<Element> children;

  @ManyToOne
  private Element parent;

  etc...
}

and:

@Entity
public class DataElement extends BaseCostElement {
  @OneToMany(mappedBy = "parent", fetch = FetchType.EAGER)
  private List<DataElement> children;

  @ManyToOne
  private DataElement parent;

  @Basic
  private String label;

  etc...
}

Embedable:

@Embeddable
public class Cost {
  @OneToMany(fetch = FetchType.EAGER)
  private List<DataElement> dataElements;

  @Basic
  private long amount;

  etc...
}

 

Questions:

1. Are your classes enhanced (all of them)?
2. Does it happen when the EntityManager is open or closed?

Note that if you are working with detached objects (e.g. after closing the EntityManager) then anything that has not been already loaded would be missing.

1. No, none are enhanced, neither serverside nor locally
2. They are always closed. My @Transactional is on service level, it is opend to load all needed data and closed after return. After everything is loaded I start my calculation/filter etc.

So you say that despite of the eager fetching, it is possible that some deeper references are not yet resolved before the elements are returned and the EntityManager is closed?

> So you say that despite of the eager fetching, it is possible that some deeper references are not yet resolved before the elements are returned and the EntityManager is closed?

Possibly. Loading (including eagerly) is synchronized, but maybe another thread closes the EntityManager too early?

You should also try enhanced classes. First, it is usually much more efficient. Second, it uses a different mechanism, which maybe could solve the issue in this case.

It happend yesterday after we installed an update and I was the only logged in user.

So it beeing another Thread closing the EntityManager is not possible.

It happend when I made an export, which happens all in one Thread.

 

Is there an event, notification which indicates that all references have been resolved?

So I could wait for that before returning the elements?

> Is there an event, notification which indicates that all references have been resolved?

Loading is not asynchronous, so there is no need for any event, and it is not the application responsibility to wait, unless the application is multithreaded.

Try to get more details if this happens again. You may also try accessing the fields explicitly before closing the EntityManager, to see if it helps at least as a workaround. It is also highly recommended to use enhanced classes, which may work better.

Okay, was a little bit confused because of the "possibly"...

So normally everything should be included inside the element if it is returned before the EntityManager is closed.

 

Not knowing why unfinished elements are returned or what could cause that, my only option is to try enhancing the classes to speed it up.

 

Thanks for the help so far, I will get back in contact if I have new/more information.

 

Okay, I created a little test project which reproduces the error.

Inside the ErrorTest.java are two test which both fail because of missing embedded elements.

 

Thank you for this report, however, we would appreciate if you could reduce this to a minimal self contained test case, as explained in the posting instructions, with no external dependencies, only the minimum required classes, in each class only the minimum required code, in a one file format (i.e. not a large Gradle project). This project has 21 Java files. Clearly reducing the project will require some extra work from you, sorry about that, but unfortunately we cannot explore issues in projects of this size.

Okay, here is a only main class project with no gradle or maven taken from the posting instructions example.

Thanks for submitting this one file test case.

Apparently the test passes if you change the following line:

    this.getChildren().parallelStream().forEach(planElement -> result.addAll(planElement.getAllCostObjects()));

to

    this.getChildren().stream().forEach(planElement -> result.addAll(planElement.getAllCostObjects()));

The exception that is thrown when using parallelStream doesn't seem to be related to ObjectDB but to concurrent access to an unsynchronized ArrayList.

It is unclear if this is the original issue from post #1.

So the problem is not the db or the db with the parallel stream but the steam api.

If I change it to a normal stream or for each it works, as welll if I use this:

result.addAll(this.getChildren().parallelStream().flatMap(planElement -> planElement.getAllCostObjects().stream()).collect(Collectors.toList()));

It regards to post #1 because the recursive method that collects the elements is the one with the parallelStream.

Thank you for everything, this issue can be closed.

Regards

Kevin