2095 words

Error using query with MAX() function

#1
2011-04-24 04:51

I have a class that is persisting a Entity that holds remarks and a time.

Then when I query for the latest remark I get a internal error when the remark is larger than 1966 chars.

Here is a test class and entity that reproduces the problem on my system.

@Entity
@Table(name = "TestEntity", schema = "mySchema")
public class TestEntity
{
  @Id
  private int    id;
  @Basic
  @Lob
  private String  remark;
  @Basic
  @Temporal(value = TemporalType.TIMESTAMP)
  private Date  changeDate;
 
  public int getId()
  {
    return id;
  }
 
  public void setId(int id)
  {
    this.id = id;
  }
 
  public String getRemark()
  {
    return remark;
  }
 
  public void setRemark(String remark)
  {
    this.remark = remark;
  }
 
  public Date getChangeDate()
  {
    return changeDate;
  }
 
  public void setChangeDate(Date changeDate)
  {
    this.changeDate = changeDate;
  }
 
}
 
public class Test
{
  private final EntityManagerFactory emf;
  private final EntityManager      em;
 
  public Test()
  {
    emf = Persistence.createEntityManagerFactory("$objectdb/db/test.odb");
    em = emf.createEntityManager();
  }
 
  public static void main(String[] args)
  {
    Test t = new Test();
    t.doit();
  }
 
  public void doit()
  {
    for(int i=1;i<32766;i++)
    {
      em.getTransaction().begin();
      TestEntity v = em.find(TestEntity.class, i);
      if(v==null)
      {
        v = new TestEntity();
        v.setId(i);
        em.persist(v);
      }
      StringBuffer sb = new StringBuffer();
      for(int j=0;j<=i;j++)
      {
        sb.append("X");
      }
      String sbs = sb.toString();
      v.setRemark(sbs);
      v.setChangeDate(new Date(System.currentTimeMillis()));
      em.getTransaction().commit();
 
      em.getTransaction().begin();
      CriteriaBuilder cb = em.getCriteriaBuilder();
      CriteriaQuery<Object> q = cb.createQuery();
      Root r = q.from(TestEntity.class);
 
      Path c = r.get("changeDate");
 
      Expression me = cb.max(c);
      CriteriaQuery<Object> s = q.select(me);
 
      Predicate predicate = cb.isNotNull(c);
      q.where(predicate);
 
      TypedQuery<Object> tq = em.createQuery(s);
      Date mDate = (Date)tq.getSingleResult();
      em.getTransaction().commit();
      System.out.println("MaxDate = "+mDate+" i="+i);
    }
  }
}

 

The Error output is

MaxDate = Sun Apr 24 00:49:47 EDT 2011 i=1962
MaxDate = Sun Apr 24 00:49:47 EDT 2011 i=1963
MaxDate = Sun Apr 24 00:49:47 EDT 2011 i=1964
MaxDate = Sun Apr 24 00:49:47 EDT 2011 i=1965
MaxDate = Sun Apr 24 00:49:47 EDT 2011 i=1966
MaxDate = Sun Apr 24 00:49:47 EDT 2011 i=1967
Exception in thread "main" [ObjectDB 2.2.2] Unexpected exception (Error 990)
  Generated by Java HotSpot(TM) 64-Bit Server VM 1.6.0_24 (on Linux 2.6.32-71.24.1.el6.x86_64).
Please report this error on http://www.objectdb.com/database/issue/new
com.objectdb.o.InternalException: null
com.objectdb.o.InternalException
  at com.objectdb.o.BYR.s(BYR.java:113)
  at com.objectdb.o.BYR.J(BYR.java:353)
  at com.objectdb.o.VUT.k(VUT.java:545)
  at com.objectdb.o.REG.D(REG.java:339)
  at com.objectdb.o.EXR$d.c(EXR.java:131)
  at com.objectdb.o.VAR.an(VAR.java:769)
  at com.objectdb.o.VAR.am(VAR.java:712)
  at com.objectdb.o.BCN.y(BCN.java:282)
  at com.objectdb.o.BCN.i(BCN.java:251)
  at com.objectdb.o.PBI.A(PBI.java:141)
  at com.objectdb.o.PBI.p(PBI.java:111)
  at com.objectdb.o.OBI.U5(OBI.java:235)
  at com.objectdb.o.BQI.Ve(BQI.java:134)
  at com.objectdb.o.GQI.Us(GQI.java:130)
  at com.objectdb.o.PRG.aa(PRG.java:584)
  at com.objectdb.o.QRM.UR(QRM.java:256)
  at com.objectdb.o.MST.UR(MST.java:878)
  at com.objectdb.o.WRA.UR(WRA.java:286)
  at com.objectdb.o.WSM.UR(WSM.java:113)
  at com.objectdb.o.WRA.UR(WRA.java:286)
  at com.objectdb.o.WSN.UR(WSN.java:422)
  at com.objectdb.o.QRR.g(QRR.java:216)
  at com.objectdb.o.QRR.b(QRR.java:139)
  at com.objectdb.jpa.JpaQuery.getSingleResult(JpaQuery.java:607)
  at com.ibm.check.cmvc2git.Test.doit(Test.java:69)
  at com.ibm.check.cmvc2git.Test.main(Test.java:30)

This brigns up the question, what is the size limit for a persisted @Lob String in objectDB?

 

awhawks
awhawks's picture
Joined on 2011-04-06
User Post #3
#2
2011-04-24 11:13

ObjectDB supports strings in any length that is supported by Java (i.e. up to 2GB).

Your test case demonstrates an ObjectDB bug in processing large objects in queries.

Please try build 2.2.2_02 that should fix this issue.

Thank you very much for the bug report and for the useful test case.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #212
#3
2011-04-24 19:25

It works better but not completly.

When my remarks data has null remarks interspersed in the data I get the another error.

 

Add the following to the above test code and you can see the problem.

      String sbs = sb.toString();
      if( i % 20 == 0)
      {
        sbs = null;
      }
      v.setRemark(sbs);

The error output I get is

MaxDate = Sun Apr 24 15:23:41 EDT 2011 i=1975
MaxDate = Sun Apr 24 15:23:41 EDT 2011 i=1976
MaxDate = Sun Apr 24 15:23:41 EDT 2011 i=1977
MaxDate = Sun Apr 24 15:23:41 EDT 2011 i=1978
MaxDate = Sun Apr 24 15:23:41 EDT 2011 i=1979
Exception in thread "main" [ObjectDB 2.2.2_02] Unexpected exception (Error 990)
  Generated by Java HotSpot(TM) 64-Bit Server VM 1.6.0_24 (on Linux 2.6.32-71.24.1.el6.x86_64).
Please report this error on http://www.objectdb.com/database/issue/new
com.objectdb.o.InternalException: java.lang.NullPointerException: null
java.lang.NullPointerException
  at com.objectdb.o.OBI.s(OBI.java:373)
  at com.objectdb.o.OBI.q(OBI.java:263)
  at com.objectdb.o.OBI.U5(OBI.java:231)
  at com.objectdb.o.BQI.Ve(BQI.java:134)
  at com.objectdb.o.GQI.Us(GQI.java:130)
  at com.objectdb.o.PRG.aa(PRG.java:584)
  at com.objectdb.o.QRM.UR(QRM.java:256)
  at com.objectdb.o.MST.UR(MST.java:878)
  at com.objectdb.o.WRA.UR(WRA.java:286)
  at com.objectdb.o.WSM.UR(WSM.java:113)
  at com.objectdb.o.WRA.UR(WRA.java:286)
  at com.objectdb.o.WSN.UR(WSN.java:422)
  at com.objectdb.o.QRR.g(QRR.java:216)
  at com.objectdb.o.QRR.b(QRR.java:139)
  at com.objectdb.jpa.JpaQuery.getSingleResult(JpaQuery.java:607)
  at com.ibm.check.cmvc2git.Test.doit(Test.java:73)
  at com.ibm.check.cmvc2git.Test.main(Test.java:30)

 

awhawks
awhawks's picture
Joined on 2011-04-06
User Post #4
#4
2011-04-24 23:16

Apparently the previous fix was incomplete. Please try the new build (2.2.2_03).

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #213
#5
2011-04-25 00:33

2.2.2_03 seems to fix the problem with my data.

awhawks
awhawks's picture
Joined on 2011-04-06
User Post #5
#6
2011-04-25 17:35

Thank you for the update and thanks again for the bug report.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #214
#7
2011-05-03 05:12

I'm getting an error with as stack trace again on my DB and data.

The test code above passes but my data and Entity fails.

At the spot where my code fails there are no Notes entities yet.

After the failure the database fails Doctor check

ObjectDB Doctor [version 2.2.3_03]
Copyright (c) 2011, ObjectDB Software. All rights reserved.
 
Scanning the database file...
.................................................. 100MB
.................................................. 200MB
............................... 262MB (total)
 
Analyzing database structure...
10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
 
Global Value Errors
-------------------
[1] Unexpected total object count: 1812225 (expected 1812197)
 
BTree Value Errors
------------------
[1] com.ibm.check.cmvc2git.entities.Notes
  - Unexpected object count: 59999 (actual 59971)
 
Page Relation Errors
--------------------
 
...
...
...

I've downloaded the latest 2.2.3_03 with no change.

I don't know how to narrow this one down.

How do you debug the cause of internal errors other than setting log level to debug and getting the stack trace?

Here is my log output at the time of the error with log level set to debug.


[2011-05-02 21:10:36 #150 query.manager]
<queryRequest query="SELECT MAX($1.addDate) FROM Notes $1 WHERE $1.addDate IS NOT NULL" args="null" transactionId="-1" />

[2011-05-02 21:10:36 #151 query.tokenizer]
<tokenList>
    <token name="SELECT" code="132" postion="0:6" />
    <token name="MAX" code="118" postion="7:10" />
    <token name="(" code="14" postion="10:11" />
    <token name="$1" code="1" postion="11:13" />
    <token name="." code="6" postion="13:14" />
    <token name="addDate" code="1" postion="14:21" />
    <token name=")" code="15" postion="21:22" />
    <token name="FROM" code="103" postion="23:27" />
    <token name="Notes" code="1" postion="28:33" />
    <token name="$1" code="1" postion="34:36" />
    <token name="WHERE" code="150" postion="37:42" />
    <token name="$1" code="1" postion="43:45" />
    <token name="." code="6" postion="45:46" />
    <token name="addDate" code="1" postion="46:53" />
    <token name="IS NOT NULL" code="62" postion="54:65" />
</tokenList>

[2011-05-02 21:10:36 #152 query.tokenizer]
<token name="QUERY" code="67" postion="0:0">
    <token name="SELECT" code="132" postion="0:6">
        <token name="MAX" code="118" postion="7:10">
            <token name="(" code="14" postion="10:11">
                <token name="." code="6" postion="13:14">
                    <token name="$1" code="1" postion="11:13" />
                    <token name="addDate" code="1" postion="14:21" />
                </token>
                <token name=")" code="15" postion="21:22" />
            </token>
        </token>
    </token>
    <token name="FROM" code="103" postion="23:27">
        <token name="$1" code="1" postion="34:36">
            <token name="Notes" code="1" postion="28:33" />
        </token>
    </token>
    <token name="WHERE" code="150" postion="37:42">
        <token name="IS NOT NULL" code="62" postion="54:65">
            <token name="." code="6" postion="45:46">
                <token name="$1" code="1" postion="43:45" />
                <token name="addDate" code="1" postion="46:53" />
            </token>
        </token>
    </token>
</token>

[2011-05-02 21:10:36 #153 query.compiler]
<queryCompilation query="SELECT MAX($1.addDate) FROM Notes $1 WHERE $1.addDate IS NOT NULL">
    <symbolManager>
        <parameters />
        <variables>
            <$1 varId="0" type="com.ibm.check.cmvc2git.entities.Notes" subtypes="include" typeId="5" binding="null" nav="1">
                <keys />
                <navigations>
                    <addDate member="addDate" memberType="java.util.Date" id="0" />
                </navigations>
            </$1>
        </variables>
    </symbolManager>
    <queryCompData>
        <query exp="query(where(($1.addDate!=null)),select(max($1.addDate)))" var="1" directVar="0" ext="1">
            <where exp="where(($1.addDate!=null))" var="1" directVar="0" ext="1">
                <notEquals exp="($1.addDate!=null)" type="boolean" var="1" directVar="0" ext="1">
                    <member exp="$1.addDate" type="java.util.Date" var="1" directVar="0" ext="1" memberName="addDate" />
                    <literal exp="null" type="java.lang.Object" var="0" directVar="0" ext="0" />
                </notEquals>
            </where>
            <select exp="select(max($1.addDate))" var="1" directVar="0" ext="1">
                <max exp="max($1.addDate)" type="java.util.Date" var="1" directVar="0" ext="1">
                    <member exp="$1.addDate" type="java.util.Date" var="1" directVar="0" ext="1" memberName="addDate" />
                </max>
            </select>
        </query>
        <globals globalVariables="0" globalNavigations="1" />
    </queryCompData>
</queryCompilation>

[2011-05-02 21:10:36 #154 query.compiler]
<query-planning filter="($1.addDate!=null)">
    <step step="1" var="$1" filter="($1.addDate!=null)">
        <allPlans>
            <filterPlan plan="filter(extract($1,type(Notes[all])),($1.addDate!=null))" eval="8.948" var="1" prm="1" reg="1" filter="($1.addDate!=null)" />
        </allPlans>
        <selectedPlans>
            <filterPlan plan="filter(extract($1,type(Notes[all])),($1.addDate!=null))" eval="8.948" var="1" prm="1" reg="1" filter="($1.addDate!=null)" />
        </selectedPlans>
    </step>
</query-planning>

[2011-05-02 21:10:36 #155 query.compiler]
<finalPlans>
    <groupPlan plan="group(filter(extract($1,type(Notes[all])),($1.addDate!=null)))" eval="9.2033" var="1" prm="1" reg="1" group="null" eval="[max($1.addDate)]">
        <filterPlan plan="filter(extract($1,type(Notes[all])),($1.addDate!=null))" eval="8.948" var="1" prm="1" reg="1" filter="($1.addDate!=null)">
            <extractPlan plan="extract($1,type(Notes[all]))" eval="8.9268" var="1" prm="1" reg="1" variable="$1">
                <btreePlan plan="type(Notes[all])" eval="8.9268" var="1" prm="1" reg="0" variable="$1" />
            </extractPlan>
        </filterPlan>
    </groupPlan>
</finalPlans>

[2011-05-02 21:10:36 #156 query.manager]
<newQueryProgram>
    <groupPlan plan="group(filter(extract($1,type(Notes[all])),($1.addDate!=null)))" eval="9.2033" var="1" prm="1" reg="1" group="null" eval="[max($1.addDate)]">
        <filterPlan plan="filter(extract($1,type(Notes[all])),($1.addDate!=null))" eval="8.948" var="1" prm="1" reg="1" filter="($1.addDate!=null)">
            <extractPlan plan="extract($1,type(Notes[all]))" eval="8.9268" var="1" prm="1" reg="1" variable="$1">
                <btreePlan plan="type(Notes[all])" eval="8.9268" var="1" prm="1" reg="0" variable="$1" />
            </extractPlan>
        </filterPlan>
    </groupPlan>
</newQueryProgram>

[2011-05-02 21:10:36 #157 *]
[ObjectDB 2.2.3_03] Unexpected exception (Error 990)
  Generated by Java HotSpot(TM) 64-Bit Server VM 1.6.0_24 (on Linux 2.6.32-71.24.1.el6.x86_64).
Please report this error on http://www.objectdb.com/database/issue/new
com.objectdb.o.InternalException: null
com.objectdb.o.InternalException
at com.objectdb.o.BYR.s(BYR.java:113)
at com.objectdb.o.BYR.z(BYR.java:194)
at com.objectdb.o.OBI.r(OBI.java:272)
at com.objectdb.o.OBI.U5(OBI.java:232)
at com.objectdb.o.BQI.Ve(BQI.java:134)
at com.objectdb.o.GQI.Us(GQI.java:130)
at com.objectdb.o.PRG.aa(PRG.java:596)
at com.objectdb.o.QRM.UR(QRM.java:256)
at com.objectdb.o.MST.UR(MST.java:878)
at com.objectdb.o.WRA.UR(WRA.java:286)
at com.objectdb.o.WSM.UR(WSM.java:113)
at com.objectdb.o.WRA.UR(WRA.java:286)
at com.objectdb.o.WSN.UR(WSN.java:422)
at com.objectdb.o.QRR.g(QRR.java:220)
at com.objectdb.o.QRR.b(QRR.java:143)
at com.objectdb.jpa.JpaQuery.getSingleResult(JpaQuery.java:610)
at com.ibm.check.cmvc2git.database.Database.getNotesMaxaddDate(Database.java:315)
at com.ibm.check.cmvc2git.database.NotesSync.updateDatabase(NotesSync.java:51)
at com.ibm.check.cmvc2git.database.CMVCdbSync.syncDB(CMVCdbSync.java:99)
at com.ibm.check.cmvc2git.CMVCSync.syncDB(CMVCSync.java:200)
at com.ibm.check.cmvc2git.CMVCSync.runCommand(CMVCSync.java:115)
at com.ibm.check.cmvc2git.CMVCSync.main(CMVCSync.java:61)

 

awhawks
awhawks's picture
Joined on 2011-04-06
User Post #6
#8
2011-05-03 05:54

There seems to be a link between how often I commit and the results of success or failure.

If I commit after every update it seems to work.

If I commit every 1000 updates it fails.

awhawks
awhawks's picture
Joined on 2011-04-06
User Post #7
#9
2011-05-03 05:41

Are you sure that there are no Notes instances in the database yet when the exception is thrown?

Please open a support ticket to discuss options to solve this problem. If you can submit the entire Doctor report (in the support ticket) it might help in understanding the problem.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #233
#10
2011-05-03 19:07

If you are using EntityManager's flush you should try build 2.2.3_04 that fixes a critical bug.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #234
#11
2011-05-04 03:27

Every time I remove the database and rebuild it I get an error but each time it moves.

It's always in the same method in my code.

I have a if to reduce memory in my loop in this function

   if( (count % config.getUpdateFreq()) == 0)
   {
 
        em.getTransaction().commit();
 
        em.getTransaction().begin();
 
  }

After much trying to reliably recreate this problem.

I've decided to give up on the above snippet of code and replace it with the following I found in the manual.

   if( (count % config.getUpdateFreq()) == 0)
   {
 
            em.flush();
 
            em.clear();
 
   }

 

awhawks
awhawks's picture
Joined on 2011-04-06
User Post #8
#12
2011-05-04 03:36

Thank you for the update. I am more worried about the Doctor errors than the query exception. Please keep running the Doctor frequently to verify that the problem was solved.

Edit

This report is similar to the report in issue #288  that was fixed now in version 2.7.1.

The new version fixes a critical bug that in some cases (rare but possible) may cause database corruption, so it is highly recommended to use the new build.

ObjectDB Support
ObjectDB - Fast Object Database for Java (JPA/JDO)
support
support's picture
Joined on 2010-05-03
User Post #243

Post Reply

Please read carefully the posting instructions - before posting to the ObjectDB website.

  • You may have to disable pop up blocking in order to use the toolbar (e.g. in Chrome).
  • Use ctrl + right click to open the browser context menu in the editing area (e.g. for using a browser spell checker).
  • To insert formatted lines (e.g. Java code, stack trace) - select a style in the toolbar and then insert the text in the new created block.
  • Avoid overflow of published source code examples by breaking long lines.
  • You may mark in paragraph code words (e.g. class names) with the code style (can be applied by ctrl + D).
  • Long stack traces (> 50 lines) and complex source examples (> 100 lines) should be posted as attachments.
Attachments:
Maximum file size: 32 MB
Cancel