Issue #2915: ObjectDB Server not able to handle large collections

We've tried to do a migration in large collection which failed with the following exception in client-server mode.

Using objectdb version 2.8.8.b05

When switched to embedded mode, the migration succeded. 

java.util.NoSuchElementException: java.lang.ArrayIndexOutOfBoundsException: arraycopy: last destination index 2203995484 out of bounds for byte[2147483639]
        at java.base/java.util.AbstractList$Itr.next(AbstractList.java:379)
        at com.objectdb.o.RSL$a.next(RSL.java:237)
        at eu.extech.update.PersistentCollectionMigrator.process(PersistentCollectionMigrator.java:86)
        at eu.extech.update.PersistentVersionedBDOCollectionMigrator$1.process(PersistentVersionedBDOCollectionMigrator.java:42)
        at eu.extech.update.common.migration.OneTimeGridMigrationWithStandaloneTemplates.performTask(OneTimeGridMigrationWithStandaloneTemplates.java:76)
        at eu.extech.update.common.migration.OneTimeSmartComponentDataMigration.performTask(OneTimeSmartComponentDataMigration.java:50)
        at eu.extech.update.OneTimeMigration.performTask(OneTimeMigration.java:28)
        at eu.extech.update.UpdateTask.performTaskWithSynchronizationImpl(UpdateTask.java:270)
        at eu.extech.update.UpdateTask.lambda$performTaskWithSynchronization$0(UpdateTask.java:259)
        at eu.extech.communication.server.RequestContext.forkAndRun(RequestContext.java:82)
        at eu.extech.update.UpdateTask.performTaskWithSynchronization(UpdateTask.java:257)
        at eu.extech.serverImpl.MiddlewareServerLauncher.runOneTimeMigrations(MiddlewareServerLauncher.java:529)
        at eu.extech.serverImpl.MiddlewareServerLauncher.runMigrationsBeforeLoadingCaches(MiddlewareServerLauncher.java:495)
        at quant.phdsc_new.sentry.SentryTool.runChildTransaction(SentryTool.java:358)
        at eu.extech.serverImpl.MiddlewareServerLauncher.launchServer(MiddlewareServerLauncher.java:343)
        at eu.extech.server.BasicJacORBServerLauncher.launchServer(BasicJacORBServerLauncher.java:60)
        at eu.extech.server.BasicServerLauncher.launch(BasicServerLauncher.java:74)
        at eu.extech.server.BasicJacORBServerLauncher.runProductionServer(BasicJacORBServerLauncher.java:26)
        at eu.extech.serverImpl.MiddlewareServerLauncher.main(MiddlewareServerLauncher.java:232)
Caused by: java.lang.ArrayIndexOutOfBoundsException: arraycopy: last destination index 2203995484 out of bounds for byte[2147483639]
        at java.base/java.lang.System.arraycopy(Native Method)
        at com.objectdb.o.BYW.h(BYW.java:708)
        at com.objectdb.o.BYR.h(BYR.java:684)
        at com.objectdb.o.RCO.ZW(RCO.java:166)
        at com.objectdb.o.RTT.r(RTT.java:263)
        at com.objectdb.o.RTT.q(RTT.java:234)
        at com.objectdb.o.RST.l(RST.java:183)
        at com.objectdb.o.RTT.g(RTT.java:143)
        at com.objectdb.o.RTT.o(RTT.java:163)
        at com.objectdb.o.RST.n(RST.java:123)
        at com.objectdb.o.PGT.j(PGT.java:155)
        at com.objectdb.o.RST.k(RST.java:95)
        at com.objectdb.o.RTT.g(RTT.java:141)
        at com.objectdb.o.RTT.o(RTT.java:163)
        at com.objectdb.o.RST.n(RST.java:123)
        at com.objectdb.o.PGT.j(PGT.java:155)
        at com.objectdb.o.RST.k(RST.java:95)
        at com.objectdb.o.RTT.g(RTT.java:141)
        at com.objectdb.o.TSK.b(TSK.java:146)
        at com.objectdb.o.TSK.c(TSK.java:96)
        at com.objectdb.o.TSM.a(TSM.java:88)
        at com.objectdb.o.RTT.o(RTT.java:161)
        at com.objectdb.o.RST.n(RST.java:123)
        at com.objectdb.o.PGT.j(PGT.java:155)
        at com.objectdb.o.RST.k(RST.java:95)
        at com.objectdb.o.RTT.g(RTT.java:141)
        at com.objectdb.o.TSK.b(TSK.java:146)
        at com.objectdb.o.TSK.c(TSK.java:96)
        at com.objectdb.o.TSM.a(TSM.java:88)
        at com.objectdb.o.RTT.o(RTT.java:161)
        at com.objectdb.o.RST.n(RST.java:123)
        at com.objectdb.o.PGT.j(PGT.java:155)
        at com.objectdb.o.RST.k(RST.java:95)
        at com.objectdb.o.RTT.g(RTT.java:141)
        at com.objectdb.o.TSK.b(TSK.java:146)
        at com.objectdb.o.TSK.c(TSK.java:96)
        at com.objectdb.o.MST.ax(MST.java:635)
        at com.objectdb.o.MST.Zp(MST.java:591)
        at com.objectdb.o.WRA.Zp(WRA.java:281)
        at com.objectdb.o.WRA.Zp(WRA.java:281)
        at com.objectdb.o.WSN.Zp(WSN.java:348)
        at com.objectdb.o.STC.p(STC.java:424)
        at com.objectdb.o.SHN.U(SHN.java:462)
        at com.objectdb.o.SHN.x(SHN.java:151)
        at com.objectdb.o.HND.run(HND.java:134)
        at java.base/java.lang.Thread.run(Thread.java:1583)

Due to the size of that collection the containing entity object exceeds the maximum supported size for an object in client-server mode, which is about 2GB. Usually an "Attempt to persist an object that is too big" exception is thrown earlier, on the attempt to persist the object. Since you used embedded mode to store the object no exception was thrown and you get this unexpected error now when you switched to client-server mode. There are currently no plans to support objects larger than 2GB in client-server mode (using such objects is usually not very efficient anyway, especially in client-server mode), but following your report we may improve the exception that is thrown in this case.

In the collection which we got from the DB query and failed to iterate in the client-server mode we had 650 smaller objects. None of them exceeded 2GB in size. So I don't get which "containing entity object" would exceed that limit. Do you mean that the ODB Server has the 2 GB limit for the whole collection of independent entity instances?

Yes, any communication between the client and the server is limited to about 2GB per operation. It was unclear from the original post that this is a query result collection rather than an entity object with a large embedded collection, but the limit of 2GB data per client-server operation is the same.

Use query result paging to retrieve results in smaller chunks of less than 2GB.