Hi,
Is it possible to reduce Doctor memory usage? Right now for recovering 67MB database, I need to set heap space to at least 450MB. Even then whole process takes ages. Optimal setting for this database is about 650MB. It's about ten times of size of database.
You may try decreasing the temporary file threshold.
The Doctor will use less memory and more temporary files, but it might become even slower.
Not working - still need about 450MB of heap space to only scan database file.
Since new computers today have at least 2GB RAM, I don't think that running the Doctor with 512MB max heap size should be a problem. The Doctor tool has been tested successfully with databases of up to 10GB, so I think that the 1:10 ratio is only for relatively small databases.
It is true, however, that it is slow. But since it is a background tool (integrity check can be done on a backup copy generated by the online backup) - I think it is not an issue of high priority.
However, if you have a test database that you can upload - I will take a look at this issue - maybe in some cases it is slower.
I did some more tests:
-rw-r--r-- 1 odb odb 67895296 Mar 20 18:10 HomplexTool.odb_nonclosed
and results:
odb@devodb:~/objectdb-2.1.1_04/db-files$ java -Xmx650m -cp ../bin/objectdb.jar com.objectdb.Doctor HomplexTool.odb_nonclosed HomplexTool.odb ObjectDB Doctor [version 2.1.1_04] Copyright (c) 2011, ObjectDB Software. All rights reserved. Scanning the database file..............................java.lang.OutOfMemoryError: Java heap space at com.objectdb.o.VLV.p(VLV.java:270) at com.objectdb.o.DRH.e(DRH.java:127) at com.objectdb.o.DMR.B(DMR.java:259) at com.objectdb.o.DMR.z(DMR.java:154) at com.objectdb.Doctor.main(Doctor.java:22) [ObjectDB 2.1.1_04] Java heap space
and second try:
odb@devodb:~/objectdb-2.1.1_04/db-files$ java -Xmx950m -cp ../bin/objectdb.jar com.objectdb.Doctor HomplexTool.odb_nonclosed HomplexTool.odb ObjectDB Doctor [version 2.1.1_04] Copyright (c) 2011, ObjectDB Software. All rights reserved. Scanning the database file................................... 64MB (total) Analyzing database structure... 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Global Value Errors ------------------- [1] Unexpected total page count: 33024 (expected 33152) Creating a database copy...............[ObjectDB 2.1.1_04] Unexpected exception (Error 990) Generated by Java HotSpot(TM) 64-Bit Server VM 1.6.0_22 (on Linux 2.6.32-5-amd64). Please report this error on http://www.objectdb.com/database/issue/new com.objectdb.o.InternalException: java.lang.OutOfMemoryError: GC overhead limit exceeded java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.objectdb.o.VUT.j(VUT.java:359) at com.objectdb.o.VUT.j(VUT.java:332) at com.objectdb.o.PGW.at(PGW.java:413) at com.objectdb.o.PGW.am(PGW.java:230) at com.objectdb.o.UPT.z(UPT.java:130) at com.objectdb.o.UPT.l(UPT.java:106) at com.objectdb.o.TSK.i(TSK.java:146) at com.objectdb.o.TSK.f(TSK.java:95) at com.objectdb.o.UPT.q(UPT.java:153) at com.objectdb.o.PGT.p(PGT.java:104) at com.objectdb.o.UPT.z(UPT.java:117) at com.objectdb.o.UPT.l(UPT.java:106) at com.objectdb.o.TSK.i(TSK.java:146) at com.objectdb.o.TSK.f(TSK.java:95) at com.objectdb.o.UPT.q(UPT.java:153) at com.objectdb.o.PGT.p(PGT.java:104) at com.objectdb.o.UPT.z(UPT.java:117) at com.objectdb.o.URT.l(URT.java:171) at com.objectdb.o.TSK.i(TSK.java:146) at com.objectdb.o.TSK.f(TSK.java:95) at com.objectdb.o.TSM.e(TSM.java:87) at com.objectdb.o.UTT.z(UTT.java:362) at com.objectdb.o.UTT.l(UTT.java:218) at com.objectdb.o.TSK.i(TSK.java:146) at com.objectdb.o.TSK.f(TSK.java:95) at com.objectdb.o.TSM.e(TSM.java:87) at com.objectdb.o.MST.UX(MST.java:1141) at com.objectdb.o.DMR.Q(DMR.java:827) at com.objectdb.o.DMR.N(DMR.java:766) at com.objectdb.o.DMR.N(DMR.java:703) at com.objectdb.o.DMR.B(DMR.java:275) at com.objectdb.o.DMR.z(DMR.java:154) [ObjectDB 2.1.1_04] Unexpected internal exception
I will create support ticket with link to this database.
I just checked the 64MB sample database.
With 512MB max heap size (-Xmx512m) it works well with the default configuration - it took ~10 seconds to check the database and additional 20 seconds to rebuild a new database.
With 256MB max heap size I got the same OutOfMemoryError exception - as you got.
However, if I change the temp threshold from 64MB to 8MB:
<temp path="$temp/ObjectDB" threshold="8mb" />
then it runs very well also with -Xmx256m.
Well, I must say, that I don't know why, but I set wrong threshold and didn't notice that - sorry. After double checking, every thing works exactly as you say.
Thanks for explanation.