Issue #2575: NullPointerException using with "refresh" method

Occasionally I get this NPE on a refresh. This makes everything quite unreliable!

Caused by: com.objectdb.o.InternalException: Unexpected internal exception
    at com.objectdb.o.JPE.e(JPE.java:170)
    at com.objectdb.o.ERR.h(ERR.java:66)
    at com.objectdb.o.OBC.onObjectDBError(OBC.java:1588)
    at com.objectdb.jpa.EMImpl.refresh(EMImpl.java:749)
    at com.objectdb.jpa.EMImpl.refresh(EMImpl.java:687)
    at com.objectdb.jdo.PMImpl.refresh(PMImpl.java:976)
    at technology.tavla.misc.data.DefaultDataManager$DefaultEntityManager.refresh(DefaultDataManager.java:128)
    at technology.tavla.product.heat.os.runtime.interaction.heat.script.ConfigurationScript.getConfiguration(ConfigurationScript.java:263)
    at technology.tavla.product.heat.os.runtime.interaction.heat.script.ConfigurationScript.isInitialized(ConfigurationScript.java:49)
    ... 15 common frames omitted
Caused by: java.lang.NullPointerException: null
    at com.objectdb.o.MPT.readArray(MPT.java:404)
    at com.objectdb.o.MPT.t(MPT.java:384)
    at com.objectdb.o.MFT.a(MFT.java:97)
    at com.objectdb.o.LDR.c(LDR.java:1168)
    at com.objectdb.o.LDR.t(LDR.java:1322)
    at com.objectdb.o.LDR.YL(LDR.java:1257)
    at com.objectdb.o.FTP.c(FTP.java:141)
    at com.objectdb.o.PFT.a(PFT.java:118)
    at com.objectdb.o.LDR.c(LDR.java:1201)
    at com.objectdb.o.LDR.m(LDR.java:885)
    at com.objectdb.o.LDR.d(LDR.java:840)
    at com.objectdb.o.LDR.Zy(LDR.java:1071)
    at com.objectdb.o.MST.ay(MST.java:571)
    at com.objectdb.o.MST.au(MST.java:491)
    at com.objectdb.o.MST.Zc(MST.java:462)
    at com.objectdb.o.WRA.Zc(WRA.java:268)
    at com.objectdb.o.LDR.u(LDR.java:558)
    at com.objectdb.o.LDR.ZR(LDR.java:475)
    at com.objectdb.o.OBC.ZR(OBC.java:1128)
    at com.objectdb.o.ERF.c(ERF.java:105)
    at com.objectdb.o.TVS.b(TVS.java:105)
    at com.objectdb.o.TVS.b(TVS.java:94)
    at com.objectdb.jpa.EMImpl.refresh(EMImpl.java:746)
    ... 20 common frames omitted

I also moved from ext3 to ext4 (using Debian Buster) and had some trouble with zero length files using atomic moves, and after a power loss. Maybe you should also check if ObjectDB is safe here. Just in case, because both issues came up at the same time.

Related sources:
- https://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/
- https://en.wikipedia.org/wiki/Ext4#Delayed_allocation_and_potential_data_loss

Please check the recovery setting in your ObjectDB configuration.

If it set to sync="false" (default, for faster operation) you may want to try sync="true".

Check the database in the Doctor, if it is already corrupted then first fix it using the Doctor.

Can data in ObjectDB be lost in case of power loss or arbitrary system failure?

With ext4 and journal this is not possible.

Why does this NPE come up? Is my entity null or is something in ObjectDB going wrong?

> Can data in ObjectDB be lost in case of power loss or arbitrary system failure?
> With ext4 and journal this is not possible.

In some systems you may have to follow #3 to make sure that no data is lost.

> Why does this NPE come up? Is my entity null or is something in ObjectDB going wrong?

Unexpected data. It has nothing to do with a null value in your entity. As suggested in #3 above, please check the database with the ObjectDB Doctor to see if the database is corrupted.

I still get this NPE, also with sync="true".

Is this a problem with the most recent version? Shall I switch back to 2.8.1_01?

This one is really really bad!

Have you checked the database with the Doctor?

It would be extremely difficult to help you without getting the required information.

If the database is not corrupted then it is not related to sync, and we have to look for something else.

The ultimate way to find and fix issues is to provide a test case. See instructions.

I can't do any doctor now. I need to know now, if I can use an older version to avoid this problem. There is no time for debugging.

Unfortunately we cannot tell you if an old ObjectDB version will solve the issue.

Probably not, but we cannot be sure.

Note, however, that if the problem is the sync setting and the move to ext4, changing the setting without fixing the database with the Doctor cannot help, as explained in #3 above.

After another restart - by power loss the NPE does not occur. So sometimes it comes up, sometimes not.

As you don't have open source, would you be able to check your implementation (NPE should be checked) or explain what is going on, in case it could be related to our setup.

We are very sure now, it is not related to ext4. We also ran ext4 before in other cases. Our ext4 issue with atomic moves was related to another part of our software. I just wanted to let you know in order to check.

The exception is thrown during an attempt to refresh a map field in an entity object.

The NullPointerException indicates an unexpected state (the map seems to be detached, which is unexpected as the owner entity object is probably not detached).

It is unlikely that the exception is thrown because of a database corruption, but it is important to exclude it first, by a Doctor test. If the database is not corrupted then the exception is probably not related to the power failure.

We need more information: What can you tell about the refreshed object? Is it always the same object? the same type? What is the status of that object when it is refreshed (clean, dirty, detached, etc.)? Which maps does it have? Do you do something special with the object maps, e.g. sharing a map by two different entity objects, which is not allowed?