Explorer in 2.3

First, the new Explorer in ObjectDB is fantastic! This is exactly what we've needed - an easy way to do ad-hoc JPQL queries and navigate through the results and even make small changes to primitive data and Strings.

After playing with it for a while, here is a list of additional features I'd love to have:

1. Export of selected data:

a. Simple - Select a range of cells or a column header and copy text to the clipboard or save to a file (csv, html, etc..). (There appear to be copy menu options, but I cannot get them to work.)

b. XML - Select some objects and serialize them to XML (using ids for circular references)

c. Binary - Select some objects, serialize the data to a binary file. Then allow import of that file into another database.

2. Lazy loading of data, especially in table view - With large amounts of data, it takes a long time for the Swing Table or Tree views to display. Any movement of the scrollbar also causes large pauses (> 30 seconds in some cases). I am using remote servers, it may be faster for local files.

3. On the Query page, please add a checkbox to request the Results either as a table or tree.

4. It would be terrific if the Database tab had lots more information about the database health

a. Show actual running values for all parameters in objectdb.conf (size, threads, query-cache, memory usage, etc...) to use for tuning

b. Show all database connections, allow to close individual or all.

c. Buttons to restart database, hot backup (remote server to local HD), etc...

5. Evaluation of expressions when editing fields - It would be very useful to be able to enter a Java expression into a field. For example, suppose you have a User entity with a password field. I would like to enter SHA1.hash("mypassword");  You might use beanshell for the expression evaluation.

For me, #1, #2 and #3 would be the most important.

Also, I've found some small bugs:

1. Auto-login of last opened (remote) database puts the full URL into the Database Path field, when it should be only the database path. This causes an error on starting Explorer.exe. Same when opening 'recent connections'.

2. When editing a field in an @Embedded object, I get an error message: "Attempt to lock a non entity object". The change stays on the screen, but is not persisted to the database.

Again, many thanks for a tremendously useful new addition to ObjectDB.

Carl

Thank you very much for this valuable feedback. These are very good ideas that will certainly be taken into consideration in future development of the Explorer.

Some initial thoughts:

> 2. Lazy loading of data

Data is already loaded lazily. There is a problem when opening a table or tree with all the instances of a specific class. This is because numbering all the instances and more importantly managing a scrollable window - require at least loading all the references in advance - and that is slow if you have millions of objects.

Currently the workaround is to use queries with max results limit.

Viewing all the objects of a class without a query and max result limit is not a common feature in database viewers. Any idea how this can be done technically without retrieving all the object ids from the database in advance may assist. At least for the tree view - it is possible to split the results by id ranges (adding additional top levels to the tree, similarly to viewing a very large array in the debugger). This will limit retrieval to open branches in the tree.

> 3. ... add a checkbox to request the Results either as a table or tree.

Good idea. Meanwhile, notice that you can also run queries by Window > Open Table Window or by clicking the Table toolbar button when the focus is in the query tab.

> 5. Evaluation of expressions when editing fields

ObjectDB can evaluate expressions in the execution tab. Try the following query: SELECT 2 + 3. This is currently an undocumented feature that could be extended and may be also supported during editing.

Only expressions that are valid in queries (JPQL, JDOQL and ObjectDB extensions)  are supported. SHA1.hash("mypassword") is currently not supported. Please provide a list of additional expressions that you need and how exactly to calculate them, and these expressions may be added to query processing.

Again, many thanks for this important feedback.

For #2 - For me, the use case for the Query tab is when I am interested in a specific piece of data that I will search for. The Schema tab, on the other hand, is quite useful during development when I want to see examples of data, but I don't really care which record I'm looking at.

I would recommend adding a field where you can enter the maximum number of records 'N' (default to some small number like 10) to display. Then at the bottom of the table or tree, have a link to fetch the next N.

This should also apply to collections that are opened while navigating the results. My initial result might have only a few records, but they may contain collections (explicit or derived) with a large number of elements. Again, most likely I'm only interested in seeing an example of the data, rather than all of it.

For #5 - I was talking about using a Java expression to set a value when editing a field, not a JPQL expression in a query.

For example, I double click on a String field in an entity. Right now, all I can do is type some text. I suggest adding a small button to the right of the textfield which will open an 'advanced' dialog for setting the value. This dialog will allow me to enter a Java expression which will return the value I want to set in the field. This value might be something I want to compute, such as SHA1.hash("mypassword"). Or it might be a large amount of data that I could not easily type, such as HttpUtils.fetchContent("http://www.cnn.com/").

In the above examples, I am using static methods to generate the result (which, of course, must match the type of the field I am editing). Another possibility is to put a reference 'this' in the symbol table referring to the current object, which would allow us use to use the object graph stored in objectdb (and any methods in those objects) to produce values.

The other really useful thing about this would be that you could set a value other than a String or primitive. Right now you can only edit text or primitives (and I noticed you have a Date editor too). But suppose I want to set the value of a field of type 'A' where A is an @Entity or @Embeddable. I could do this using a Java expression that returns the correct type.

Using BeanShell, it should be fairly easy to do what I'm describing.

Another possibility for this advanced editor would be to use the result of a JPQL query to set a value of an @Entity or @Embeddable field. ie. I want to set the value of a field of type 'A' where A is an @Entity or @Embeddable. So I enter a JPQL  query which returns a single object. Or I have a field which is an explicit collection. Then I could enter a JPQL query returning any number of objects (of the correct type) and add those to the collection.

Thanks again!

The Explorer has some capabilities that you are probably not aware of.

You can edit references to entity / embeddable objects:

• Using the Edit > Set Reference... commands (also available in the context menu when right clicking a reference).
• Using copy and paste - copy any reference from any window and paste it on a field of a matching type.

You can also edit collections - adding, removing and reordering elements. You can even copy the results of a query and paste it into a collection of a matching type (e.g. copy strings and then paste them into a List<String> collection).

The features above are described in the Explorer page in the new 2.3 manual.

In addition, in the query panel you can invoke static methods. Try for example:

select Math.random()

Fully qualified class names have to be specified for classes that are not in java.lang, and to run your own static methods you have to run the Explorer with classpath that contains your classes.

Again, you can copy the query result (e.g. the random number) and paste it on a matching field.

Build 2.3.1_03 fixes the two bugs that are described in #1.

Thanks again for the report.

I'd like to add 2 more feature requests for the explorer:

1) ability to export JSON (e.g. to Mongo / Couchdb).

ie: it would be terrific to be able to export an object tree to JSON.

2) ability to rename the Entity Classes (especially the package names).

One further point: I'm finding that Groovy works brilliantly with ObjectDb - you could use it rather than Beanshell, as described above.

You can rename packages, classes and fields by using the configuration.

This is preferred over renaming in the Explorer, because:

• Same changes can be applied automatically on more than one database.
• You don't have the Explorer available in every environment (e.g. at end users when applicable).
• Changes must be synchronized with code refactoring and this is less error prone.