Several parallel threads to speed up query on one DB

#1

Hi,

I have an application: producer (from a DB) to a consumer.

The current bottleneck is the producer from DB (based on a query) and I would like to speed it up.

I have tried using several entity managers (same DB instance) to benefit from the parallelisation - but it looks like I don't get much benefit from doing that. 

I profiled the application - and it looks like there are some benefits - i.e. each query in a thread is running simultaneously - but it looks sometimes the "query threads" are waiting from something (appears in "monitor" in netbeans). I am not sure what it is - but my guess it is coming from some ObjectDB low level I/O settings?

Any help on how to speed up query by doing parallelization would be appreciated - or best practice.

Thanks

 

 

 

#2

Another thing I just noticed: I used to do a clear after running each query -

The lock contention disappears when I removed the "em.clear()" - Obviously I need to clear the entity managers - it looks like the em.clear() of one query blocks the other query. 

Hope it helps.

 

#3

More information is needed. Please post thread dumps.

ObjectDB Support
#4

Please find attached a reproducible case.

Step1: run persist() method - to create the Db and store all the entities

Step2: run test_ONE_query() - in profiling mode in netbeans - select the Threads view (timeline) and the lock contention view.

You can see that Thread R1 is running smoothly with no lock

Step3 : run test_TWO_queries() - you can see there are many interruptions - and they seem to get to lock each other many times for small amount time.

Thanks

#5

Thank you for the test case. Some locking and waiting is normal.

Please describe your findings that you think are abnormal (including specific stack traces, if possible).

ObjectDB Support
#6

Thanks - I was just surprised about the locking and waiting - so it is normal. So there is nothing I can do to reduce the locking/waiting?

Thanks

#7

Although some locking ans waiting is normal, we have to see the thread dumps and additional information in order to get better understanding of the specific situation in this case.

It may be possible to improve performance by tuning the configuration (e.g. the maximum concurrent threads).

ObjectDB Support

Reply