Difference between revisions of "Implementing an External Preprocessor for MIRC Clinical Trials"

Latest revision as of 19:05, 31 July 2009

The article is intended for software developers working on clinical trials which require the imposition of special rules on the types of objects which are accepted into the trial. To fully understand this article, you may wish to reference The RSNA MIRC Source Code, The MIRC Object Classes, and Implementing an External Database Interface for MIRC Clinical Trials.

1 The ObjectProcessor

The ObjectProcessor is a component of a MIRC Storage Service. It processes files which are received from DICOM and HTTP sources and either creates or updates MIRCdocuments to reference them. The processing can include:

External preprocessing.
Anonymization.
Queuing for export to DICOM or HTTP destinations.
Queuing for processing by an external database interface.

This article describes how to develop a preprocessor and connect it to the ObjectProcessor. A preprocessor is a Java class to which the ObjectProcessor passes an object encapsulating a file. The preprocessor returns a result indicating whether the object is acceptable. If it is acceptable, the Object processor continues processing the file. If it is not, the ObjectProcessor quarantines it. The ObjectProcessor dynamically loads the preprocessor's class, obtaining the name of the class from the Storage Service’s trial/trial.xml configuration file. This file is typically configured by the trial administrator through the Admin Service.

2 The Object Classes

MIRC provides four classes to encapsulate files of various types. See The MIRC Object Classes for details. The classes are:

DicomObject
XmlObject
ZipObject
FileObject

3 The PreprocessorAdapter Class

The PreprocessorAdapter class, org.rsna.mircsite.util.PreprocessorAdapter, is a base class for building an external preprocessor. To be recognized and loaded by the ObjectProcessor, an external preprocessor class must be an extension of PreprocessorAdapter.

All the methods of the PreprocessorAdapter class return a boolean result:

true means that the object meets the external criteria for inclusion in the study.
false means that the object is to be excluded from the study, in which case the ObjectProcessor will quarantine it.

All the methods of the PreprocessorAdapter class return the value true.

4 Extending the PreprocessorAdapter Class

To implement a useful preprocessor, you must extend the PreprocessorAdapter class. Since the PreprocessorAdapter class implements dummy methods returning true, your class that extends PreprocessorAdapter only has to override the methods that apply to your application. If, for example, you only care about XML objects, you can just override the process(XmlObject xmlObject) method and let PreprocessorAdapter supply the other process() methods, thus ignoring objects of other types.

If you want to reject objects of certain types, you must override the process() methods for those types and return false.

The preprocessor can be also be used to modify objects before any further processing takes place. The various object classes provide access to the data, and you can programmatically do anything you want with it (it's great to be king).

If you want to log information to the Tomcat/logs/stdout.log directory, the PreprocessorAdapter class provides a static org.apache.log4j.Logger object called logger. The MIRC site typically runs at the logging level WARN, so unless you change that, you should make calls like:

logger.warn("...");

If you want to log information that will be visible through the Show Log button on the admin page, you can use the static methods in the org.rsna.mircsite.log.Log class. Remember, however, that the Log class only maintains a circular buffer of the past 200 entries, so if you make many calls, you may be removing log entries made by other objects.

5 Connecting Your Preprocessor Class to MIRC

After building your preprocessor and producing a jar file for it, you have to install it and configure it. The preprocessor is part of a Storage Service, and each storage service can have its own (possibly different) external preprocessor. Therefore, the interface is installed and configured at the level of the Storage Service.

5.1 Installation

The preprocessor jar file must be installed in the Tomcat/webapps/[storageservice]/WEB-INF/lib directory, where [storageservice] is the name of the storage service for the clinical trial. Do not place the jar file in the Tomcat/shared/lib directory. Doing so will cause the class not to load.

The MIRCsite-installer program always deletes the Tomcat/webapps/[storageservice]/WEB-INF/lib directory during an upgrade to prevent old libraries from contaminating a new release. This has the unfortunate effect of deleting the database interface jar file. You must therefore be careful to restore the preprocessor jar file after an upgrade.

5.2 Configuring MIRC for Your Preprocessor Class

The Admin Service provides a web interface for setting the configuration parameters. The interface is accessed by clicking the Update Configuration button in the DICOM Service column on the admin page. In the Http Import Service section of the resulting page, there are two parameters which control preprocessing:

Preprocessor enabled determines whether objects are to be preprocessed.
Preprocessor class name specifies the fully qualified name of the class. If this field is empty, the system will operate as if preprocessing is disabled.

@@ Line 1: / Line 1: @@
-The article is intended for software developers working on clinical trials which require the imposition of special rules on the types of objects which are accepted into the trial. The reader is expected to be familiar with Java and with [[The_RSNA_MIRC_Source_Code|the RSNA MIRC Source Code]].
+The article is intended for software developers working on clinical trials which require the imposition of special rules on the types of objects which are accepted into the trial. To fully understand this article, you may wish to reference [[The RSNA MIRC Source Code]], [[The MIRC Object Classes]], and [[Implementing an External Database Interface for MIRC Clinical Trials]].
 ===The ObjectProcessor===
 The ObjectProcessor is a component of a MIRC Storage Service. It processes files which are received from DICOM and HTTP sources and either creates or updates MIRCdocuments to reference them. The processing can include:
-* Preprocessing to determine whether the file is acceptable for subsequent processing.
+* External preprocessing.
 * Anonymization.
 * Queuing for export to DICOM or HTTP destinations.
@@ Line 9: / Line 9: @@
 This article describes how to develop a preprocessor and connect it to the ObjectProcessor.  A preprocessor is a Java class to which the ObjectProcessor passes an object encapsulating a file. The preprocessor returns a result indicating whether the object is acceptable. If it is acceptable, the Object processor continues processing the file. If it is not, the ObjectProcessor quarantines it. The ObjectProcessor dynamically loads the preprocessor's class, obtaining the name of the class from the Storage Service’s <b>trial/trial.xml</b> configuration file. This file is typically configured by the trial administrator through the Admin Service.
-To fully understand this article, you may wish to reference [[the RSNA MIRC Source Code]] and [[Implementing an External Database Interface for MIRC Clinical Trials]].
 ===The Object Classes===
-MIRC provides four classes to encapsulate files of various types. The classes are all in the <b>org.rsna.mircsite.util</b> package.
+MIRC provides four classes to encapsulate files of various types. See [[The MIRC Object Classes]] for details. The classes are:
-====FileObject====
+* DicomObject
-The FileObject class encapsulates a file of unknown contents. It is the parent class of the other three classes and provides a method for obtaining a java.io.File object pointing to the file, as well as methods for moving, renaming, and copying the file. It also provides dummy methods for obtaining key information about the file. These methods are overridden by subclasses that can parse their corresponding file types to obtain meaningful values. The FileObject class also includes a factory method, <b>getObject(java.io.File)</b>, which parses a file and returns an instance of the matching FileObject subclass or, if no subclass matches the file, a FileObject itself.
+* XmlObject
+* ZipObject
+* FileObject
-====XmlObject====
+===The PreprocessorAdapter Class===
-The XmlObject class encapsulates an XML file, parsing the XML in the constructor and throwing an Exception if the file does not parse. In addition to the methods of the FileObject, it provides methods for accessing the XML DOM object and reading its elements and attributes. For many purposes, the <b>getValue(String path)</b> method makes it easy to obtain the necessary data to insert into a database, but if necessary, you can use the entire XML DOM.
+The PreprocessorAdapter class, <b>org.rsna.mircsite.util.PreprocessorAdapter</b>, is a base class for building an external preprocessor. To be recognized and loaded by the ObjectProcessor, an external preprocessor class must be an extension of PreprocessorAdapter.
-The Javadocs explain how specific types of data (the <b>uid</b>, the <b>study-uid</b>, the file <b>description</b>) are obtained from the XML DOM object. When designing XML metadata files for a clinical trial, if you put the data in those places, MIRC will be able to manage the files and pass them to your database interface more efficiently.
+All the methods of the PreprocessorAdapter class return a boolean result:
-====ZipObject====
+*<b>true</b> means that the object meets the external criteria for inclusion in the study.
-The ZipObject class encapsulates a zip file, parsing the file in the constuctor and throwing an Exception if the file does not parse. In addition to the methods of the FileObject, it provides methods for accessing the individual files within the zip file. It also provides methods for accessing the contents of a special file, <b>manifest.xml</b>, that contains data necessary to identify the zip file and relate it to other clinical trial data.
+*<b>false</b> means that the object is to be excluded from the study, in which case the ObjectProcessor will quarantine it.
+All the methods of the PreprocessorAdapter class return the value <b>true</b>.
-The ZipObject allows a clinical trial to collect a group of related objects, for example multiple output files from an analytical program for a single analysis, and to manage them as a single object.
+===Extending the PreprocessorAdapter Class===
+To implement a useful preprocessor, you must extend the <b>PreprocessorAdapter</b> class. Since the PreprocessorAdapter class implements dummy methods returning <b>true</b>, your class that extends PreprocessorAdapter only has to override the methods that apply to your application. If, for example, you only care about XML objects, you can just override the <b>process(XmlObject xmlObject)</b> method and let PreprocessorAdapter supply the other <b>process()</b> methods, thus ignoring objects of other types.
-The Javadocs explain how identifying data are obtained from the <b>manifest.xml</b> file. The ZipObject is much less flexible than the XmlObject in the placement of identifying data. When designing zip metadata files for a clinical trial, you should try to construct manifests exactly by the rules described in the Javadocs.
+If you want to reject objects of certain types, you must override the <b>process()</b> methods for those types and return <b>false</b>.
-====DicomObject====
-The DicomObject class encapsulates a DICOM dataset, parsing the file in the constructor and throwing an Exception if the file does not parse. In addition to the methods of the FileObject, it provides access to all the DICOM attributes, as well as image conversion methods returning JPEG images of any size from the DICOM image.
-===The PreprocessorAdapter Class===
+The preprocessor can be also be used to modify objects before any further processing takes place. The various object classes provide access to the data, and you can programmatically do anything you want with it (it's great to be king).
-The PreprocessorAdapter class, <b>org.rsna.mircsite.util.PreprocessorAdapter </b>, is a base class for building an external preprocessor. To be recognized and loaded by the ObjectProcessor, an external preprocessor class must be an extension of PreprocessorAdapter.
-All the methods of the PreprocessorAdapter class return an integer status value. Static values for the statuses are defined in the class. There are two values:
+If you want to log information to the <b>Tomcat/logs/stdout.log</b> directory, the PreprocessorAdapter class provides a static <b>org.apache.log4j.Logger</b> object called <b>logger</b>. The MIRC site typically runs at the logging level WARN, so unless you change that, you should make calls like:
-*<b>STATUS_OK</b> means that the object meets the external criteria for inclusion in the study.
-*<b>STATUS_REJECT</b> means that the object is to be excluded from the study, in which case the ObjectProcessor will quarantine it.
-All the methods of the PreprocessorAdapter class return the value <b>STATUS_OK</b>.
-===Extending the PreprocessorAdapter Class===
+:logger.warn("...");
-To implement a useful preprocessor, you must extend the <b>PreprocessorAdapter</b> class. Since the PreprocessorAdapter class implements dummy methods returning STATUS_OK, your class that extends PreprocessorAdapter only has to override the methods that apply to your application. If, for example, you only care about XML objects, you can just override the <b>process(XmlObject xmlObject)</b> method and let PreprocessorAdapter supply the other <b>process()</b> methods, thus ignoring objects of other types.
 If you want to log information that will be visible through the <b>Show Log</b> button on the admin page, you can use the static methods in the <b>org.rsna.mircsite.log.Log</b> class. Remember, however, that the <b>Log</b> class only maintains a circular buffer of the past 200 entries, so if you make many calls, you may be removing log entries made by other objects.
@@ Line 51: / Line 46: @@
 ====Configuring MIRC for Your Preprocessor Class====
-The Admin Service provides a web interface for setting the configuration parameters. The interface is accessed by clicking the <b>Update Configuration</b> button in the <b>DICOM Service</b> column on the admin page. In the <b>Preprocessor</b> section of the resulting page, there are two parameters:
+The Admin Service provides a web interface for setting the configuration parameters. The interface is accessed by clicking the <b>Update Configuration</b> button in the <b>DICOM Service</b> column on the admin page. In the <b>Http Import Service</b> section of the resulting page, there are two parameters which control preprocessing:
-*<b>Enabled</b> determines whether objects are preprocessed.
+*<b>Preprocessor enabled</b> determines whether objects are to be preprocessed.
 *<b>Preprocessor class name</b> specifies the fully qualified name of the class. If this field is empty, the system will operate as if preprocessing is disabled.
-====Operational Note====
-When a file is received, the PreprocessorAdapter attempts to figure out the type of file by parsing it. It first tries to parse the file based on the extension of its filename. If that fails, it tries all the untried classes in the order:
-#DicomObject
-#ZipObject
-#XmlObject
-If none of the objects can be instantiated, a FileObject is created for the file. If the file is a binary object that is neither a DICOM file nor a zip file, the attempt to instantiate the XmlObject will cause the Xerces parser to encounter a fatal, but not serious, error, which it logs to the console stream where it can be seen in the log viewer or by accessing the <b>Tomcat/logs/stdout.log</b> file. The error is not serious because although the Xerces parser fails, it is reinstantiated whenever it is needed, so there is no effect on the operation of the system. This error will never occur in trials which produce only DICOM, zip, or XML files.