in-memory and on-disk support?

I've had a quick read through the features/FAQ and haven't been able to find a clear answer.

I'm looking at replacing a setup of H2 where I have 40GB in-memory backed by 800GB on-disk, the 40GB in memory represents the first 40GB of the 800GB on-disk for super-fast recent entry searching, the records at the end continuously purge as the new ones come in(duplicate write, one to in-memory and one to on-disk).

Does ObjectDB offer a similar in-memory/on-disk option? I noticed the embedded and TCP/IP option but I would like both DB's accessible in-program without the need for TCP/IP as it's all on the one server.

Thanks in advance.

Ben

Sure, you can manage two separate ObjectDB databases:

• The large database (800GB) will be managed on an ordinary disk.
• The small database (40GB) will be managed on a RAM disk / RAM drive.

Both databases can be managed in embedded (in process) mode with no TCP/IP.

Alternatively you can try the more conventional setting of one ordinary disk database, with a large memory cache. In addition,  the operating system usually uses free physical memory as cache, so you may find that you get the performance that you need also with one database.

Thanks for the reply.

RAM Disk isn't really ideal as a filesystem is going to make inefficient use of my memory, however your alternative option of larger memory cache may be acceptable.

Given I have a fixed DB size(800GB) with the need to purge oldest records as new ones come in would it be more optimal to set the <size> element as follows:

<size initial="800000mb" resize="0kb" page="2kb" />

Or would that result in excessive query times in reading in 1 large file? traditionally I would use partitioning to split this but i'm not to sure what is the best method here.

Also is there any feature that will allow the purge oldest as new comes in 1 for 1? or is this something I will have to do myself?

Thanks again.

Ben

Actually, a RAM drive provides excellent performance and because ObjectDB uses a single file (you can disable logging, recovery, etc. for the memory database) - file system overhead is minimal.

See for example comparison of RAM drives, including against a regular hard disk.

Notice also that ObjectDB uses JVM heap for the cache - a very large cache (40GB) has not been tested yet.

Starting with a 800GB database file seems as a good idea in this case. You should use a separate hard drive or at least run defragmentation before allocating this space. Getting a continuous unfragmented disk space is the main advantage. Eliminating the need for space allocation later is a small additional advantage. It will only help query performance.

Purging old data will have to be done by your application.

For scenarios where available RAM (e.g. 32GB+) would be sufficient to store data, but far exceeds the JVM heap (e.g. 3GB), are there any plans to offload into off-heap memory instead of just the filesystem?

Use of off-heap memory seems to be in vogue lately (e.g. hazelcast, terracotta, coherence).

https://www.kdgregory.com/index.php?page=java.byteBuffer

Yes, it is very likely that this will be implemented in future versions.

You may fill a feature request, and if you need it urgently for a specific application  - please contact ObjectDB support via a private support ticket.

Update: Support of Memory Mode was added to ObjectDB in version 2.6.4.