Implementing an External Database Interface for MIRC Clinical Trials

The article is intended for software developers interfacing their MIRC sites to specialized databases for clinical trials. The reader is expected to be familiar with Java and with the RSNA MIRC Source Code.

1 The Database Export Service

The Database Export Service is a component of a MIRC Storage Service. It provides a queuing mechanism for submitting files to a database interface, relieving the interface from having to manage the queue. It attempts to parse each file to determine its type and calls the overloaded process method of the interface with one of four objects, encapsulating an XML file, a zip file, a DICOM file, or a file of unparsable contents. Each of the objects includes methods providing access to the internals of the file, allowing the interface to interrogate the files to obtain some or all of their data to insert into an external system.

The Database Export Service dynamically loads the database interface class, obtaining the name of the class from the Storage Service’s trial/trial.xml configuration file. This file is typically configured by the trial administrator through the Admin Service.

To fully understand this article, you may wish to reference the RSNA MIRC Source Code.

2 The Object Classes

As noted above, MIRC provides four classes to encapsulate files of various types. See The MIRC Object Classes for details. The classes are:

DicomObject
XmlObject
ZipObject
FileObject

3 The DatabaseAdapter Class

The DatabaseAdapter class, org.rsna.mircsite.util.DatabaseAdapter, is a base class for building an interface between the Database Export Service and an external database. To be recognized and loaded by the Database Export Service, an external database interface class must be an extension of DatabaseAdapter.

The DatabaseAdapter class provides a set of methods allowing the Database Export Service to perform various functions, all of which are explained in the Javadocs. The basic interaction model is:

When the Database Export Service detects that files are in its queue, it verifies that the database interface class is loaded and loads it if it is not.
It then calls the database interface’s connect() method.
For each file in the queue, it instantiates an object matching the file’s contents and calls the database interface’s process() method. There are four overloaded process methods, one for each object class. The process() methods have two arguments: the metadata object and the URL of the MIRCdocument which references the file.
When the queue is empty, it calls the database interface’s disconnect() method.
If the Database Export Service is restarted after the database interface has been instantiated, it calls the interface’s reset() method.
When the Database Export Service is completely finished with an instance of the database interface and is about to discard the instance before either exiting or instantiating a new instance of the interface, it calls the interface’s shutdown() method.

All the methods of the DatabaseAdapter class return an integer status value. Static values for the statuses are defined in the class. There are three values:

STATUS_OK means that the operation succeeded completely.
STATUS_FAIL means that the operation failed and trying again will also fail. This status value indicates a problem with the object being processed.
STATUS_WAIT means that the operation failed but trying again later may succeed. This status value indicates a temporary problem accessing the external database.

All the methods of the DatabaseAdapter class return the value STATUS_OK.

4 Extending the DatabaseAdapter Class

To implement a useful interface to an external database, you must extend the DatabaseAdapter class. The build.xml file provides an example of such an implementation in the testdatabase target. It builds a jar containing org.rsna.database.TestDatabase. This implementation only logs method calls.

Since the DatabaseAdapter class implements dummy methods returning STATUS_OK, your class that extends DatabaseAdapter only has to override the methods that apply to your application. If, for example, you only care about XML objects, you can just override the process(XmlObject xmlObject) method and let DatabaseAdapter supply the other process() methods, thus ignoring objects of other types.

The database interface should not rely on the reset() and shutdown() methods. They are called when possible, but there are certain situations in which they cannot be called (a power failure, for example), and you should make sure that the data is protected in those situations. Similarly, since one connect() call is made for possibly multiple process() method calls, it is possible that a failure could result in no disconnect() call. Thus, depending on the design of the external system, it may be wise to commit changes in each process() call.

If you want to log information that will be visible through the Show Log button on the admin page, you can use the static methods in the org.rsna.mircsite.log.Log class. Remember, however, that the Log class only maintains a circular buffer of the past 200 entries, so if you make many calls, you may be removing log entries made by other objects.

5 Connecting Your Database Class to MIRC

After building your database interface and producing a jar file for it, you have to install it and configure it. The Database Export Service is part of a Storage Service, and each storage service can have its own (possibly different) external database. Therefore, the interface is installed and configured at the level of the Storage Service.

5.1 Installation

The database interface jar file must be installed in the Tomcat/webapps/[storageservice]/WEB-INF/lib directory, where [storageservice] is the name of the storage service for the clinical trial. Do not place the jar file in the Tomcat/shared/lib directory. Doing so will cause the class not to load.

The MIRCsite-installer program always deletes the Tomcat/webapps/[storageservice]/WEB-INF/lib directory during an upgrade to prevent old libraries from contaminating a new release. This has the unfortunate effect of deleting the database interface jar file. You must therefore be careful to restore the database interface jar file after an upgrade.

5.2 Configuring MIRC for Your Database Class

The Admin Service provides a web interface for setting the configuration parameters. The interface is accessed by clicking the Update Configuration button in the DICOM Service column on the admin page. In the Database Export Service section of the resulting page, there are four parameters:

Mode determines whether objects that are received are automatically forwarded to the Database Export Service. In auto mode, they are. In QC mode, they are not. In QC mode, the assumption is made that the administrator will review the MIRCdocument that contains the received objects and verify that they are appropriate for sending to the database. If they are, he can click the Export to database button in the Images column at the bottom of the displayed document’s Document tab. Doing so queues all the objects in the document for export. An important subtlety is that the only objects that make it into MIRCdocuments are ones that contain a Study Instance UID. Thus, a FileObject will never be sent to the Database Export Service (since a FileObject’s contents cannot be parsed by MIRC). In auto mode, however, all received objects are exported, so if you plan to receive unparseable files as part of your clinical trial, you must operate the Database Export Service in auto mode.
Anonymizer enabled determines whether the objects are anonymized after being sent to the database. This operation is a little counter-intuitive at first sight. The operational model is that objects are anonymized before they are transmitted to the MIRC site. Typically, this would be done by the MIRC FieldCenter application which transmits them to the MIRC site. Actual anonymization takes place at that time, but in some cases, it is desirable to have the anonymizer include provenance information that is intended for the database but not for viewers of the MIRCdocument or its referenced objects. In that situation, the FieldCenter’s anonymizer is configured to add the provenance information, the object is sent to and received by the MIRC site, which stores the file in the directory for the Study Instance UID of the file and references it in the associated MIRCdocument. The file is then passed to the Database Export Service with the provenance information intact. If the anonymizer is enabled, then after the process() method of the database interface class returns, the Storage Service’s anonymizer is called to remove the provenance information from the DICOM object.
Sleep interval determines the frequency at which the Database Export Service checks its queue. The interval is specified in milliseconds, and a typical interval is 10,000, or 10 seconds. The Database Export Service enforces a reasonability criterion on the value, so if it is very short or very long, it uses a 10 second default.
Database class name specifies the fully qualified name of the class, for instance, org.rsna.database.TestDatabase. If this field is empty, the Database Export Service will operate as if it were disabled.

If you want to have references to received metadata files placed in the MIRCdocuments that are automatically created for clinical trial studies, you should add a metadata-refs element to the Tomcat/webapps/[storageservice]/trial/template.xml file. This element is explained in the MIRCdocument Schema and an example is provided in MIRC Templates.