Issue #2257: log entry and massive performance issues

we are facing massive problems with our system using the latest version of objectdb:

- around 2 times per day we have a simple object remove (entityManager.remove) that takes more than 5 minutes to complete

- it seams that there is a memory leak (we did not change our application at all but just use the latest version 2.7.4_02, with versions of 2.7.3 we never had these problems)

- we have this log entry in our log:

...

RetrievalTask.reportPrimaryResult0: page#13887921, reader.getPos() = 1229

...

another point:

- our application/objectdb can be killed doing the following

-- we are using objectdb in our application in embedded mode

-- open explorer and connect via embedded server port 3333

-- set up a non optimized query (simple select)

>>> our application step by step blows up (as more and more starts executing a simple select query are hanging)

Thank you for this report. In order to investigate it further more information is needed:

1. The "RetrievalTask.reportPrimaryResult" message should be followed by an exception, do you have a stack trace of such an exception in the log? Did you get that log error once or is it repeating? Does it happen when the slow removal happens?
2. It may help to identify the exact build in which this issue is started. 2.7.3 works well and 2.7.4_02 doesn't. Can you check version 2.7.4?
3. Could you please provide more information about the removed object? Is it a single object or a large graph of objects? What is special with that specific object?
4. You mentioned a memory leak, can you take a heap dump and report more details?
5. The last point regarding killing an application by using embedded-server mode, is it related to the move to version 2.7.4_02? If it is a separate issue please report it in a separate thread.

Thanks.

ad 1: the entry that I posted is without any further exception, it was created when a query that normally takes around 5 seconds was terminated by the timeout that we have set (200 seconds)

ad 2: unfortunately we cannot step back and we are not able to step back at this customer where we have this issue

ad 3: it is a single object, there is nothing special (we'll put the database to our extranet and inform about), the ObjectNode.nodepath is like '/(EL)ANYNAME/(EX)F', you can find a number of these objects linked to objects with nodePath '/(EL)...'

ad 4: we will try to collect more data

ad 5: yes this is a new issue, sorry, it happens with all versions

you can find the database on hummingbird-systems.com, login with user/password your db name, there is a main menu with your database name, download coresystemdb.rar

Do you know how to reproduce the slow removal problem using this database?

You wrote:

> the ObjectNode.nodepath is like '/(EL)ANYNAME/(EX)F', you can find a number of these objects linked to objects with nodePath '/(EL)...'

Can you provide classes/primary keys of such problematic objects?

Is it possible to see the problem by removing an object in the Explorer?

unfortunately it cannot be reproduced as it just happens spuriously several times a day.

the class is: ObjectNode

an example entity is: ObjectNode.uUid = '706c3b84-bdd9-4b06-b593-2d9db5ff212e'

isn't it possible that the database is somehow corrupt (we did the last repair last ween with 2.7.4_02 Doctor)?

In addition to this slow behaviour we have several times a day the situation that a normal select query is blocking everything

A quick check of the database didn't find critical issues.

Additional information is needed such as a heap dump of the memory leak and thread dumps of the blocking queries and slow delete operation, or any other relevant information.

After examining all the changes between versions 2.7.3 and 2.7.4_02 nothing that can explain the reported issues was found. You may want to try repairing the database with the Doctor to see if it solves the new issues.

Do you still have this issue?

Maybe it is related to issue #2297. Please report if configuration changes that were discussed in issue #2297 affects this issue as well.