Issue #894: Indexes broken after schema update

I don't want to keep creating these issues but the Exception tells me to :), I started a thread to see if anyone else has encountered this at http://www.objectdb.com/database/forum/484

Essentially what has happened if i've added a bunch of new fields to an existing entity, I restarted the DB as per the previous issue I logged, I haven't added or removed any entities but I try and query an existing one and I get 0 results when trying to query by an existing index(that still exists) and the below errors in the logs. I can query the object fine by non-indexed fields and by the @Id.

[2012-08-19 19:42:25 #74 store]
*** Attempt to remove a non existing index entry:

[2012-08-19 19:42:25 #75 store]
BTree ID: -104

[2012-08-19 19:42:25 #76 store]
BTree Name: profileIndex:com.netarray.userservice.entities.User[profile]

[2012-08-19 19:42:25 #77 store]
BTree Page: 7

[2012-08-19 19:42:25 #78 store]
Request code: 4

[2012-08-19 19:42:25 #79 store]
Request Key: 'chrisiona-decc'

[2012-08-19 19:42:25 #80 store]
Request Key Code: -25

[2012-08-19 19:42:25 #81 store]
Request Real Key: 'chrisiona-decc'

[2012-08-19 19:42:25 #82 store]
Request Real Key Code: -25

[2012-08-19 19:42:25 #83 store]
Request Small Key: 'chrisiona-decc'

[2012-08-19 19:42:25 #84 store]
Request Small Key Code: -25

[2012-08-19 19:42:25 #85 store]
Request Value: 1:'374571ee-21db-4e5b-9ef7-580fa9df04a7'

[2012-08-19 19:42:25 #86 server]
[ObjectDB 2.4.2_02] Unexpected exception (Error 990)
  Generated by Java HotSpot(TM) Client VM 1.6.0_25 (on Linux 2.6.32.12).
Please report this error on http://www.objectdb.com/database/issue/new
com.objectdb.o.InternalException: null
com.objectdb.o.InternalException
at com.objectdb.o.InternalException.f(InternalException.java:236)
at com.objectdb.o.SPW.aq(SPW.java:109)
at com.objectdb.o.PGW.am(PGW.java:210)
at com.objectdb.o.UPT.C(UPT.java:134)
at com.objectdb.o.URT.l(URT.java:171)
at com.objectdb.o.TSK.i(TSK.java:146)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.TSM.e(TSM.java:87)
at com.objectdb.o.UTT.z(UTT.java:365)
at com.objectdb.o.UTT.l(UTT.java:218)
at com.objectdb.o.TSK.i(TSK.java:146)
at com.objectdb.o.TSK.f(TSK.java:95)
at com.objectdb.o.TSM.e(TSM.java:87)
at com.objectdb.o.MST.Vb(MST.java:1305)
at com.objectdb.o.WRA.Vb(WRA.java:369)
at com.objectdb.o.WSM.Vb(WSM.java:174)
at com.objectdb.o.WRA.Vb(WRA.java:369)
at com.objectdb.o.WSN.Vb(WSN.java:658)
at com.objectdb.o.STC.v(STC.java:511)
at com.objectdb.o.SHN.an(SHN.java:563)
at com.objectdb.o.SHN.K(SHN.java:170)
at com.objectdb.o.HND.run(HND.java:133)
at java.lang.Thread.run(Thread.java:662)

Could you please check the database with the Doctor?

> java -cp objectdb.jar com.objectdb.Doctor ../../db/user.odb
ObjectDB Doctor [version 2.4.2_02]
Copyright (c) 2012, ObjectDB Software. All rights reserved.

Scanning the database file...
0MB (total)

Analyzing database structure...
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

-------------------------
No errors have been found
-------------------------

I still have the same problem when relaunching it, can't get any records by index.

To help the troubleshooting I have attached the database example as it is very small with only 1 User in it. To replicate the issue you can do the following:

1. SELECT u from User u - We get a single User back.
2. SELECT u from User u WHERE u.profile = "chrisiona-decc" - notice no results, profile is indexed.
3. SELECT u from User u WHERE u.dateCreated = 1344761076 - notice single result, dateCreated is not indexed.

Please check the attached file. It cannot be opened as an ObjectDB database file.

Hmmm weird, I wonder if the upload mangled the attachment. I have zipped up the same file and tested that I can open this with the explorer locally.

Please try again.

I can see in the new database that the indexes are broken.

Fixing the database with the Doctor solves the problem (by rebuilding the indexes).

However, the cause of the problem is unclear, so it could help if you can describe how exactly you created that database (when the entity was created, when new fields were added, whether or not the entity was updated, etc.).

I'll dig up the change revision for the entity so I can work out what fields were added. Also I have the full DB record history if that would help?

Is it common for there to be no errors found in the diagnosis yet the repair still fixes a problem? As I saw no problems when running the Doctor diagnosis hence I didn't run a repair, did you see the same?

The database record history may help.

You are right about running the Doctor diagnosis - it checks that the data is fine and that the indexes are fine but currently it doesn't check that they are synchronized. Such a check would be very time consumption, but it probably should be added at least as an option.