View source for MIRC CTP

This article describes the stand-alone processing application for clinical trials data using MIRC components and the MIRC internet transport mechanism.

==Background==
MIRC supports clinical trials through two applications, one for data acquisition at an imaging center (FieldCenter) and one for management of the data at a principal investigator's site (MIRC).

The FieldCenter application acquires images via the DICOM protocol, anonymizes them, and transfers them (typically via HTTP, although DICOM is also supported) to a principal investigator's MIRC site. It also supports other types of data files and includes an anonymizer for XML files as well. FieldCenter also contains a client for the Update Service of a MIRC site, allowing the application to save data on, and obtain software updates from, the principal investigator's site.

The MIRC site software contains a partially configurable processing pipeline for clinical trials data, consisting of:

:;HttpImportService
::A receiver for HTTP connections from FieldCenter applications transferring data files into the processing pipeline.
:;DicomImportService
::A receiver for DICOM datasets from modalities, PACS, workstations, etc. for insertion into the processing pipeline.
:;Preprocessor
::A user-defined component for processing data received by the HttpImportService before it is further processed by other components.
:;Anonymizer
::A component for anonymizing DICOM objects or XML objects.
:;DatabaseExportService
::A component providing queue management and submission of data objects to a user-defined interface to an external database management system.
:;HttpExportService
::A component in the DicomImportService pipeline providing queue management and transmission of data objects to one or more external systems using the HTTP protocol.
:;DicomExportService
::A component in the HttpImportService pipeline providing queue management and transmission of data objects to one or more external systems using the DICOM protocol.

The processing pipelines for the HttpImportService and DicomImportService are different.  They are not symmetrical. For example, the HttpImportService does not have access to the anonymizer except as part of the DatabaseExportService. Another limitation is that objects received via one protocol can only be exported via the other. While these limitations are consistent with the requirements of most trials, it became clear that a more general design would provide better support for trials requiring complex processing while still satisfying the normal requirements.

==Clinical Trial Processor (CTP)==
CTP is a stand-alone program that provides all the processing features of a MIRC site for clinical trials in a highly configurable and extensible application. It connects to FieldCenter applications and can also connect to MIRC sites when necessary. CTP has the following key features:
#Single-click installation.
#Support for multiple pipelines.
#Processing pipelines supporting multiple configurable stages.
#Support for multiple quarantines for data objects which are rejected during processing.
#Pre-defined implementations for key components:
#*HTTP Import
#*DICOM Import
#*DICOM Anonymizer
#*XML Anonymizer
#*File Storage
#*Database Export
#*HTTP Export
#*DICOM Export
#*FTP Export
#Web-based monitoring of the application's status, including:
#*configuration
#*logs
#*quarantines
#*status

===Installation===
The [http://mirc.rsna.org/CTP/CTP-installer.jar installer] for CTP is available on the [http://mirc.rsna.org RSNA MIRC site]. To run the installer, the Java 1.6 (or better) JRE must be present on the system.

To run the installer, double-click the <tt><b>CTP-installer.jar</b></tt> file and choose a directory in which to install the program. The installer can also be run in a command window using the command:

:<b><tt>java -jar CTP-installer.jar</tt></b>

To run the CTP program, the <b>Java Advanced Imaging ImageIO Tools</b> must be present on the system. Java and all its components are available on the java.sun.com website. When obtaining the ImageIO Tools, pay close attention to the fact that the Java Advanced Imaging component is not the same as the Java Advanced Imaging ImageIO Tools. Only the latter are required.

CTP has no user interface. It can be run by double-clicking the <b><tt>CTP.jar</tt></b> file, or it can be run in a command window. To do so, open a command window, navigate to the directory in which the program was installed, and enter the command:

:<b><tt>java -jar CTP.jar</tt></b>

If large images are to be acquired, it is generally advisable to run the program with a large memory pool. When running from a command window, the parameters can be set using the command:

:<b><tt>java -Xmx512m -Xms128m -jar CTP.jar</tt></b>

When the program starts, it runs without intervention. Status and other information can be obtained through the program's integrated webserver. Accessing the server with no path information displays a page presenting buttons for each of the servlets. 

When the program is first installed, a single user is provided with the name <b>admin</b> and password <b>password</b>. After logging in as this user, the UserManagerServlet can be used to change the name and/or password and to create other users if necessary.

To stop the program, log in as an admin user and click the <b>Shutdown</b> button on the main page.

===Configuration Files===
The program uses two configurable files: <b><tt>config.xml</tt></b>, which is located in the same directory as the program itself, and <b><tt>index.html</tt></b>, which is located in the server's <b><tt>ROOT</tt></b> directory. Both files are intended to be configured for the specific application. The installer does not overwrite these files when it runs; instead, it installs two example files: <b><tt>example-config.xml</tt></b> and <b><tt>example-index.html</tt></b>. When CTP starts, it looks to see if the non-example files are missing, and if so, it copies the example files into the non-example ones. This process allows upgrades to be done without losing any configuration work. After installing the program the first time, it should be run once in order to make the copies, and then the copies can be configured. Configuration is done by hand with any text editor (e.g., TextPad or NotePad). Care should be taken, especially with config.xml, to keep it well-formed. Opening it with a program like InternetExplorer will check it for errors.

===Server===
To provide access to the status of the components, the application includes an HTTP server which serves files and provides servlet-like functionality. Files are served from a directory tree whose root is named <b><tt>ROOT</tt></b>. The <b><tt>ROOT</tt></b> directory contains a file, <b><tt>index.html</tt></b>, which provides buttons which link to several servlets providing information about the operation of the program. This file is intended to be configured with logos, additional links, etc., and upgrades do not overwrite it. The standard servlets are:

*<b>LoginServlet</b> allows a user to log into the system.
*<b>UserManagerServlet</b> allows an admin user to create users and assign them privileges.
*<b>ConfigurationServlet</b> displays the contents of the configuration file.
*<b>StatusServlet</b> displays the status of all pipeline stages.
*<b>LogServlet</b> provides web access to all log files in the <b>logs</b> directory.
*<b>QuarantineServlet</b> provides web access to all quarantine directories and their contents.
*<b>SysPropsServlet</b> displays the Java system properties.
*<b>DicomAnonymizerServlet</b> allows the user to configure any DICOM anonymizers in the pipelines.
*<b>ShutdownServlet</b> allows an admin user to shut the program down.

The configuration element for the HTTP server is:
<pre>
        <Server 
            port="80"
            ssl="no"
            requireAuthentication="no"
            usersClassName="org.rsna.ctp.server.UsersXmlFileImpl" />
</pre>
where:
*<b>port</b> is the port number on which the HTTP server listens for connections.
*<b>ssl</b> determines whether the HTTP server uses SSL. Values are "yes" and "no". The default is "no". 
*<b>requireAuthentication</b> determines whether users are forced to log in to the HTTP server. Values are "yes" and "no". The default is "no". (The HTTP server is typically operated without requiring authentication, thus allowing users to monitor the status of the system without having to log in.)
*<b>usersClassName</b> specifies the Java class to be used for authentication of users and their privileges. If this attribute is missing, the standard CTP implementation (shown above) is employed. This attribute should only be included if a special mechanism has been implemented on the site (for example, using LDAP).

===Pipelines===
A pipeline is a manager that moves data objects through a sequence of processing stages. Each stage in the pipeline performs a specific function on one or more of the four basic object types supported by MIRC:
:*FileObject
:*DicomObject
:*XmlObject
:*ZipObject
Each pipeline must contain at least one ImportService. Each pipeline stage may be provided access to a quarantine directory into which the stage places objects that it rejects, thus removing them from the pipeline and aborting further processing. Quarantine directories may be unique to each stage or shared with other stages. At the end of the pipeline, the manager calls the ImportService which provided the object to remove it from its queue. 

There are four types of pipeline stages. Each is briefly described in subsections below.

====ImportService====
An ImportService receives objects via a protocol and enqueues them for processing by subsequent stages.

====Processor====
A Processor performs some kind of processing on an object. Processors are not intended to be queued. In the context of the current MIRC implementation, a Preprocessor is a Processor, as is an Anonymizer. The result of a processing stage is an object that is passed to the next stage in the pipeline.

====StorageService====
A StorageService stores an object in a file system. It is not queued, and it therefore must complete before subsequent stages can proceed. A StorageService may return the current object or the stored object in response to a request for the output object, depending on its implementation.

====ExportService====
An ExportService provides queued transmission to an external system via a defined protocol. Objects in the queue are full copies of the objects submitted; therefore, subsequent processing is not impeded if a queue is paused, and modifications made subsequently do not affect the queue entry, even if they occur before transmission. (Note: This behavior is different from that of the current MIRC implementation.) After entering an object in its queue, an ExportService returns immediately.

===System Configuration===
The CTP configuration is specified by an XML file called <tt><b>config.xml</b></tt> located in the same directory as the program. There can be one <b>Server</b> element specifying the port on which the HTTP server is to operate, and multiple <b>Pipeline</b> elements, each specifying the stages which comprise it. The name of the element defining a stage is irrelevant and can be chosen for readability; each stage in a pipeline is actually defined by its Java class, specified in the <b>class</b> attribute. Stages are loaded automatically when the program starts, and the loader tests the stage's class to determine what kind of stage it represents. It is possible to extend the application beyond the pre-defined stages available in the implementation as described in [[#Extending CTP| Extending CTP]].

The following is an example of a simple configuration that might be used at an image acquisition site. It contains one pipeline which receives objects via the DICOM protocol, stores the objects locally, anonymizes them, and transmits them via HTTP (using secure sockets layer for encryption) to a principal investigator's site.
<pre>
<Configuration>
   <Server port="80" />
   <Pipeline name="Main Pipeline">
        <ImportService
            name="DICOM Import"
            class="org.rsna.ctp.stdstages.DicomImportService"
            root="roots/dicom-import"
            port="1104" />
        <StorageService
            name="Storage"
            class="org.rsna.ctp.stdstages.FileStorageService"
            root="storage"
            returnStoredFile="no"
            quarantine="quarantines/storage" />
        <Anonymizer
            name="Anonymizer"
            class="org.rsna.ctp.stdstages.DicomAnonymizer"
            root="roots/anonymizer"
            script="roots/anonymizer/scripts/da.script"
            quarantine="quarantines/anonymizer" />
        <ExportService
            name="HTTP Export"
            class="org.rsna.ctp.stdstages.HttpExportService"
            root="roots/http-export"
            url="https://university.edu:1443" />
   </Pipeline>
</Configuration>
</pre>

The configuration above is the default file which is installed when the program is first run. Changes to the configuration made subsequently are not overwritten during an upgrade. Note that because the storage service appears in the pipeline before the anonymizer, the objects which are stored contain the PHI which was originally received, and because the anonymizer appears before the export service, anonymized objects are exported.

The following is an example of a simple configuration that might be used at a principal investigator's site. It contains one pipeline which receives objects via the HTTP protocol, stores them, and exports them to a DICOM destination:
<pre>
<Configuration>
   <Server port="80" />
   <Pipeline name="Main Pipeline">
        <ImportService 
            name="HTTP Import"
            class="org.rsna.ctp.stdstages.HttpImportService"
            root="roots/http-import"
            ssl="yes"
            port="1443" />
        <StorageService
            name="Storage"
            class="org.rsna.ctp.stdstages.FileStorageService"
            root="D:/storage" 
            returnStoredFile="no"
            quarantine="quarantines/StorageServiceQuarantine" />
        <ExportService
            name="PACS Export"
            class="org.rsna.ctp.stdstages.DicomExportService"
            root="roots/pacs-export" 
            url="dicom://DestinationAET:ThisAET@ipaddress:port" />
   </Pipeline>
</Configuration>
</pre>
Note that in the example above, non-DICOM objects are stored in the StorageService, but they are not exported by the DicomExportService. Each pipeline stage is responsible for testing the class of the object which it receives and processing (or ignoring) the object accordingly.

Multiple <b>Pipeline</b> elements may be included, but each must have its own ImportService element, and their ports must not conflict.

Each pipeline stage class has a constructor that is called with its configuration element, making it possible for special processor implementations to be passed additional parameters from the configuration. See [[#Implementing a Pipeline Stage | Implementing a Pipeline Stage]] for details.

===Standard Stages===
The application includes several built-in, standard stages which allow most trials to be operated without writing any software. The sections below show all the configuration attributes recognized by the standard stages. 

Attributes which specify directories can contain either absolute paths (e.g., <tt><b>D:/TrialStorage</b></tt>) or relative paths (e.g., <tt><b>quarantines/http-import-quarantine</b></tt>). Relative paths are relative to the directory in which the CTP application is located.

All standard stages have an optional <b>id</b> attribute which, if present, must uniquely identify the stage across all pipelines. This attribute is required only when the stage must be accessed by another stage, for example, when a DatabaseExportService must interrogate a FileStorageService to determine the URL under which an object is stored.

Some standard stages have attributes which determine which object types are to be accepted by the stage. These attributes are:
*acceptDicomObjects
*acceptXmlObjects
*acceptZipObjects
*acceptFileObjects
The allowed values are "yes" and "no". The default value for all these attributes is "yes". Stages which are restricted to specific object types (e.g. DicomImportService, DicomAnonymizer, XmlAnonymizer, DicomFilter, DicomExportService) ignore the values of these attributes.

If a standard ImportService receives an object which it is not configured to accept, it quarantines the object, or if no quarantine has been defined for the stage, it discards the object.

If a Processor, StorageService, or ExportService receives an object that it is not configured to accept, it either ignores the object or passes it unmodified to the next pipeline stage. Thus, if an anonymizer which is not configured to anonymize XmlObjects receives an XmlObject, it passes the object on without anonymization.

====Import Services====
=====HttpImportService=====
The HttpImportService listens on a defined port for HTTP connections from FieldCenter applications and receives files transmitted using the HTTP protocol with Content-Type equal to <b><tt>application/x-mirc</tt></b>. The configuration element for the HttpImportService is:
<pre>
        <ImportService 
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.HttpImportService"
            root="root-directory"
            port="7777"
            ssl="yes"
            zip="no"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the ImportService for internal storage and queuing.
*<b>port</b> is the port on which the ImportService listens for connections.
*<b>ssl</b> determines whether the port uses secure sockets layer (<b>yes</b>) or unencrypted http (<b>no</b>). The default is <b>no</b>.
*<b>zip</b> determines whether files will be unzipped after reception (<b>yes</b>) or not (<b>no</b>). The default is <b>no</b>. This feature is intended for use with zipped transmissions from the HttpExportService.
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be enqueued when received.
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be enqueued when received.
*<b>acceptZipObjects</b> determines whether ZipObjects are to be enqueued when received.
*<b>acceptFileObjects</b> determines whether FileObjects are to be enqueued when received.
*<b>quarantine</b> is a directory in which the ImportService is to quarantine objects that it receives but is configured not to accept.

=====PollingHttpImportService=====
The PollingHttpImportService obtains files by initiating HTTP connections to an external system. This ImportService is designed to work in conjunction with the PolledHttpExportService to allow penetration of a firewall without having to open an inbound port, as described in [[#Security Issues | Security Issues]]. The configuration element for the PollingHttpImportService is:
<pre>
        <ImportService 
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.PollingHttpImportService"
            root="root-directory"
            url="http://ip:port"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the ImportService for internal storage.
*<b>url</b> is the URL of the PolledHttpExportService.
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be enqueued when received.
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be enqueued when received.
*<b>acceptZipObjects</b> determines whether ZipObjects are to be enqueued when received.
*<b>acceptFileObjects</b> determines whether FileObjects are to be enqueued when received.
*<b>quarantine</b> is a directory in which the ImportService is to quarantine objects that it receives but is configured not to accept.
Notes: 
*The protocol part of the <b>url</b> must be <b>http</b>, although the actual protocol used by the PollingHttpImportService is not HTTP.
*The PollingHttpImportService does not support SSL.
*The PollingHttpImportService does not support a proxy server for its connections.

=====DicomImportService=====
The DicomImportService listens on a defined port for HTTP connections from FieldCenter applications and receives files transmitted using the DICOM protocol. The DicomImportService accepts all Application Entity Titles. The configuration element for the DicomImportService is:
<pre>
        <ImportService 
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomImportService"
            root="root-directory" 
            port="port number" 
            calledAETTag="00097770" 
            callingAETTag="00097772" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the ImportService for internal storage and queuing.
*<b>port</b> is the port on which the ImportService listens for connections.
*<b>calledAETTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the AE Title which was used by the sender to specify the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the AE Title of the receiver in the received DICOM object.
*<b>callingAETTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the AE Title which was used by the sender to identify itself in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the AE Title of the sender in the received DICOM object.

=====DirectoryImportService=====
The DirectoryImportService watches a directory and imports any files it finds in it. After the files are passed down the pipeline, they are moved to the quarantine directory, if configured, or deleted from the import directory if no quarantine is available. The purpose of this ImportService is to allow manual input of objects to the pipeline. The configuration element for the HttpImportService is:
<pre>
        <ImportService 
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DirectoryImportService"
            root="root-directory"
            minAge="5000"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory monitored by the ImportService for files to import.
*<b>minAge</b> is the minimum age (in milliseconds) for files which are imported from the root directory. The default value is <b>5000</b>; the minimum accepted value is <b>1000</b>. The purpose of this attribute is to ensure that files are completely stored in the root directory before being imported.
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be accepted.
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be accepted.
*<b>acceptZipObjects</b> determines whether ZipObjects are to be accepted.
*<b>acceptFileObjects</b> determines whether FileObjects are to be accepted.
*<b>quarantine</b> is a directory in which the ImportService is to quarantine objects that it has handled (whether they were accepted for importing or not).

====Processors====
=====DicomFilter=====
The DicomFilter is a processor stage that interrogates a DicomObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If a DicomObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (XmlObject, ZipObject, FileObject) are passed on unmodified. The configuration element for the DicomFilter is:
<pre>
        <Processor
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomFilter"
            root="root-directory" 
            script="dicom-filter.script"
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the DicomFilter for temporary storage.
*<b>script</b> specifies the path to the script for the DicomFilter.
*<b>quarantine</b> is a directory in which the DicomFilter is to quarantine objects that do not meet the criteria specified in the script file.

For information on specifying acceptance criteria in the script, see [[The CTP DICOM Filter]].

=====XmlFilter=====
The XmlFilter is a processor stage that interrogates an XmlObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If an XmlObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (DicomObject, ZipObject, FileObject) are passed on unmodified. The configuration element for the XmlFilter is:
<pre>
        <Processor
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.XmlFilter"
            root="root-directory" 
            script="xml-filter.script"
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the XmlFilter for temporary storage.
*<b>script</b> specifies the path to the script for the XmlFilter.
*<b>quarantine</b> is a directory in which the XmlFilter is to quarantine objects that do not meet the criteria specified in the script file.

For information on specifying acceptance criteria in the script, see [[The CTP XML and Zip Filters]].

=====ZipFilter=====
The ZipFilter is a processor stage that interrogates the manifest of a ZipObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If a ZipObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (DicomObject, XmlObject, FileObject) are passed on unmodified. The configuration element for the XmlFilter is:
<pre>
        <Processor
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.ZipFilter"
            root="root-directory" 
            script="zip-filter.script"
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the ZipFilter for temporary storage.
*<b>script</b> specifies the path to the script for the XmlFilter.
*<b>quarantine</b> is a directory in which the ZipFilter is to quarantine objects that do not meet the criteria specified in the script file.

For information on specifying acceptance criteria in the script, see [[The CTP XML and Zip Filters]].

=====DicomAnonymizer=====
The DicomAnonymizer is a processor stage that anonymizes DicomObjects and passes all other object types unmodified. The DicomAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the DicomAnonymizer is:
<pre>
        <Anonymizer
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomAnonymizer"
            root="root-directory" 
            lookupTable="lookup-table.properties"
            script="dicom-anonymizer.properties"
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the DicomAnonymizer for temporary storage.
*<b>lookupTable</b> specifies the path to the lookup table used by the DicomAnonymizer.
*<b>script</b> specifies the path to the script for the DicomAnonymizer.
*<b>quarantine</b> is a directory in which the DicomAnonymizer is to quarantine objects that generate quarantine calls during processing.

Notes:
# If the <b>lookupTable</b> attribute is missing, the <b>lookup</b> anonymizer function is disabled.

=====XmlAnonymizer=====
The XmlAnonymizer is a processor stage that anonymizes XmlObjects and passes all other object types unmodified. The XmlAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the XmlAnonymizer is:
<pre>
        <Anonymizer
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.XmlAnonymizer"
            root="root-directory" 
            script="xml-anonymizer.script" 
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the Anonymizer for temporary storage.
*<b>script</b> specifies the path to the script for the XmlAnonymizer.
*<b>quarantine</b> is a directory in which the XmlAnonymizer is to quarantine objects that generate quarantine calls during processing.

=====ZipAnonymizer=====
The ZipAnonymizer is a processor stage that anonymizes ZipObjects and passes all other object types unmodified. The ZipAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the ZipAnonymizer is:
<pre>
        <Anonymizer
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.ZipAnonymizer"
            root="root-directory" 
            script="zip-anonymizer.script" 
            quarantine="quarantine-directory" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the Anonymizer for temporary storage.
*<b>script</b> specifies the path to the script for the ZipAnonymizer.
*<b>quarantine</b> is a directory in which the ZipAnonymizer is to quarantine objects that generate quarantine calls during processing.

====Storage Services====
=====FileStorageService=====
The FileStorageService stores objects in a file system. It automatically defines subdirectories beneath its root directory and populates them accordingly. The configuration element for the StorageService is:
<pre>
        <StorageService
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.FileStorageService"
            root="D:/storage" 
            type="month"
            timeDepth="0"
            acceptDuplicateUIDs="yes"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            returnStoredFile="yes"
            setWorldReadable="no"
            setWorldWritable="no"
            fsNameTag="00097770"
            autoCreateUser="no"
            port="85"
            ssl="no"
            requireAuthentication="no"
            quarantine="quarantine-directory" >

            <jpeg wmax="10000" wmin="96" q="-1" />

        </StorageService>
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is the root directory of the storage tree.
*<b>type</b> determines the structure of the storage tree. The allowed values are:
**<b>year</b>: root/FSNAME/year/studyName, e.g. root/2008/123.456.789
**<b>month</b>: root/FSNAME/year/month/studyName, e.g. root/2008/06/123.456.789
**<b>week</b>: root/FSNAME/year/week/studyName, e.g. root/2008/36/123.456.789
**<b>day</b>: root/FSNAME/year/day/studyName, e.g. root/2008/341/123.456.789
**<b>none</b>: root/FSNAME/studyName, e.g. root/123.456.789
(FSNAME is the name of the file system to which the study belongs. See the <b>fsNameTag</b> attribute below for additional information.)
*<b>timeDepth</b> specifies the length of time in days that studies are stored. The default value is <b>0</b>, which means forever. If timeDepth is greater than zero, studies older than that value are automatically removed from storage.
*<b>acceptDuplicateUIDs</b> determines whether objects with duplicate UIDs are to be stored under separate names or if a newer duplicate object is to overwrite an older one. Values are "yes" and "no". The default is "yes".
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be stored.  Values are "yes" and "no". The default is "yes".
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be stored. Values are "yes" and "no". The default is "yes".
*<b>acceptZipObjects</b> determines whether ZipObjects are to be stored. Values are "yes" and "no". The default is "yes".
*<b>acceptFileObjects</b> determines whether FileObjects are to be stored. Values are "yes" and "no". The default is "yes".
*<b>returnStoredFile</b> specifies whether the original object or a new object pointing to the file in the storage system is to be returned for processing by subsequent stages. Values are "yes" and "no". The default is "yes".
*<b>setWorldReadable</b> determines whether the FileStorageService makes all files and directories readable by all users. Values are "yes" and "no". The default is "no". This attribute should only be used if users or other programs are to be allowed to access files without using the FileStorageService web server. This feature is <b>not</b> recommended.
*<b>setWorldWritable</b> determines whether the FileStorageService makes all files and directories writable by all users. Values are "yes" and "no". The default is "no". This attribute should only be used if users or other programs are to be allowed to write files into the FileStorageService without using the FileStorageService web server. This feature is <b>not</b> recommended.
*<b>fsNameTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is used to specify the name of the root directory's child under which to store a received object. If the attribute is missing from the configuration or if the specified element is missing from the received object, or if the contents of the specified element are blank, the object is stored under the "__default" tree.
*<b>autoCreateUser</b> determines whether the StorageService is to create a user for each new value of the element specified by the fsNameTag. The default is "no". The user is created with both the username and the password set to the value of the element specified by the fsNameTag.
*<b>port</b> specifies the port on which a web server is to be started to provide access to the stored studies. If the attribute is missing, no web server is started for the FileStorageService.
*<b>ssl</b> determines whether the web server uses SSL. Values are "yes" and "no". The default is "no". 
*<b>requireAuthentication</b> determines whether users are forced to log in to the web server. Values are "yes" and "no". The default is "no".
*<b>quarantine</b> is a directory in which the StorageService is to quarantine objects that cannot be stored.

Fro more information on the embedded web server in the FileStorageService, see [[The CTP FileStorageService Web Server]] and [[The CTP FileStorageService Access Mechanism]].

Notes:
# Files are stored in a tree of directories with the root of the tree as defined in the root attribute of the configuration element. 
# Below the root are directories called FileSystems. A FileSystem may be defined by the value of an element in the object being stored by using the fsNameTag attribute. If no FileSystem is defined by the object, it is stored in the default FileSystem (<b>__default</b>).
# Within a FileSystem, studies are grouped depending on the value of the type attribute. For example, if the type attribute has the value "month", studies are organized by year and month, e.g. "2007/09".
# At the bottom of the hierarchy are directories organized by StudyInstanceUID, thus grouping all objects received for a specific study together in one directory. 
# Objects are stored with standard file extensions:
#*.dcm for DicomObjects
#*.xml for XmlObjects
#*.zip for ZipObjects
#*.md for FileObjects
# Any object not containing a StudyInstanceUID or StudyUID is stored in the <b>bullpen</b> directory.
# The <b>fsNameTag</b> attribute can be used to create storage trees based on element values like PatientID. It can also access elements in private groups, as might be done if the <b>calledAETTag</b> attribute of the DicomImportService is used to pass destination information. Similarly, the <b>callingAETTag</b> attribute of the DicomImportService could be used to pass source information, allowing separation of objects into FileSystems based on the sending system.

Notes on the <b>jpeg</b> child element:
# The optional <b>jpeg</b> child element causes the FileStorageService to create a JPEG image for each DICOM image when it is stored. 
# The <b>jpeg</b> element should be included <b>only</b> when pre-computed JPEG images are required by an external application which directly references the disk drive containing the stored files (for example, the NCIA application).
# When images are accessed through the FileStorageService web server's Storage Servlet or Ajax Servlet, they are produced dynamically, and <b>jpeg</b> elements are not required.
# Multiple <b>jpeg</b> elements may appear if multiple images must be created (with different parameters) for each stored DICOM image.
# The attributes specify the width and quality parameters of the created image:
#* If the width of the parent DICOM image lies between the values of <b>wmax</b> and <b>wmin</b>, the width of the created image will be equal to the width of the parent.
#*<b>wmax</b> specifies the maximum width of the created image. The default is 10000.
#*<b>wmin</b> specifies the minimum width of the created image. The default is 96.
#*The height of the created image is automatically computed to provide the same aspect ratio as the parent.
#*<b>q</b> specifies the compression quality for the created image. Allowed values are <b>1</b> through <b>100</b>, with larger values producing better quality (and larger file sizes). The system default is triggered by specifying <b>-1</b>, which generally produces a good image.
# A JPEG image file created in response to a <b>jpeg</b> child element is stored in the same directory as the parent DICOM image.
# The filename of a created image is constructed from:
#* the name of the parent file, 
#* a suffix in square brackets identifying the parameters used to create it, 
#* and a <b>.jpeg</b> extension. 
# The parameters appear in the order: <b>wmax</b>, <b>wmin</b>, <b>q</b>, and are separated by semicolons. 
# For example, a JPEG image created from <b>FO-29126.dcm</b> might have the name <b>FO-29126.dcm[400;96;-1].jpeg</b>.
# If a 96-pixel wide thumbnail is required for an external application, the following <b>jpeg</b> child element could be specified:
<pre>
        <jpeg wmax="96" wmin="96" q="-1" />
</pre>

=====BasicFileStorageService=====
The BasicFileStorageService stores objects in a file system. It provides no organization of the files and no means of access to them. It is intended for use in situations where direct file access is provided through an external application like NCIA. The configuration element for the StorageService is:
<pre>
        <StorageService
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.BasicFileStorageService"
            index="D:/storage"
            root="D:/storage/root" 
            nLevels="3" 
            maxSize="200" 
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            returnStoredFile="yes"
            quarantine="quarantine-directory" >

            <jpeg wmax="10000" wmin="96" q="-1" />

        </StorageService>
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>index</b> is the directory in which the index is stored.
*<b>root</b> is the root directory of the storage tree.
*<b>nLevels</b> defines the depth of the storage tree. The default is 3.
*<b>maxSize</b> defines the maximum number of files or directories in each node of the storage tree. The default is 200.
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be stored.  Values are "yes" and "no". The default is "yes".
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be stored. Values are "yes" and "no". The default is "yes".
*<b>acceptZipObjects</b> determines whether ZipObjects are to be stored. Values are "yes" and "no". The default is "yes".
*<b>returnStoredFile</b> specifies whether the original object or a new object pointing to the file in the storage system is to be returned for processing by subsequent stages. Values are "yes" and "no". The default is "yes".
*<b>quarantine</b> is a directory in which the StorageService is to quarantine objects that cannot be stored.

Notes:
# Files are stored in a tree of directories with the root of the tree as defined in the root attribute of the configuration element. 
# The index directory must not appear under the root directory. I convenient approach is shown in the example above, where the root directory appears under the index directory.
# The number of files which will be stored under any directory in the root is maxSize**(nLevels-1). For the default values of the nLevels and maxSize parameters (3 and 200), this is 40,000. 
# Directories are created at the top level (root) when existing directories are full. There is no maximum number of top-level directories; however, it is wise to consider the number of objects which are expected to be stored and to select values of nLevels and maxSize which would keep the number of top-level directories below 1000.
# For storage requirements of 10 million files, the default values of nLevels and maxSize are fine.
# For storage requirements of 10 billion files, nLevels="4" and maxSize="300" would work well
# Files are stored as leaves at the bottom of the tree.
# No organization into related groups (e.g. by StudyInstanceUID) is provided. 
# Files are indexed by UID (e.g., SOPInstanceUID).
# A duplicate object (e.g., one whose UID matches the UID of an object already stored) overwrites the stored object in the same place in the storage system (e.g., the same directory and the same filename), and any required <b>jpeg</b> images are recreated, overwriting the previously stored ones.
# FileObjects, which do not contain UIDs, are not stored; they are simply passed to the next stage.
# Files are stored with standard file extensions:
#*.dcm for DicomObjects
#*.xml for XmlObjects
#*.zip for ZipObjects
# See the section on the FileStorageService for a description of the <b>jpeg</b> child element.
# If <b>jpeg</b> child elements appear, the files which they create are <b>not</b> counted against the maxSize parameter. Thus, if maxSize is 200 and two <b>jpeg</b> child elements appear, the bottom directories in the tree could contain 600 files. In situations where a given choice of maxSize could result in more than 1000 files in one directory, it is advisable to reduce maxSize and increase nLevels.

====Export Services====
=====HttpExportService=====
The HttpExportService queues objects and transmits them via HTTP with Content-Type <b>application/x-mirc</b>. The configuration element for the HttpExportService is:
<pre>
        <ExportService
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.HttpExportService"
            root="root-directory" 
            url="http://ipaddress:port/path"
            zip="no"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes"
            dicomScript="root-directory/scripts/df.script"
            xmlScript="root-directory/scripts/xf.script"
            zipScript="root-directory/scripts/zf.script"
            interval="10000"
            proxyIPAddress="..."
            proxyPort="..."
            proxyUsername="..."
            proxyPassword="..." />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
*<b>url</b> specifies the destination system's URL.
*<b>zip</b> determines whether files will be zipped before transmission (<b>yes</b>) or not (<b>no</b>). The default is <b>no</b>. This feature is intended for use with the HttpImportService.
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be exported.
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be exported.
*<b>acceptZipObjects</b> determines whether ZipObjects are to be exported.
*<b>acceptFileObjects</b> determines whether FileObjects are to be exported.
*<b>dicomScript</b> specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
*<b>xmlScript</b> specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
*<b>zipScript</b> specifies the path to a script which examines the contents of a ZipObject and determines whether the object is to be exported.
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue.
*<b>proxyIPAddress</b> is the IP address of the network proxy server, if present.
*<b>proxyPort</b> is the port of the network proxy server, if present.
*<b>proxyUsername</b> is the username of the network proxy server, if one is required.
*<b>proxyPassword</b> is the password of the network proxy server, if one is required.

Notes: 
*The default interval is 10 seconds. The minimum allowed value is one second. The maximum allowed value is 20 seconds.
*The protocol part of the <b>url</b> can be <b>http</b> or <b>https</b>, the latter causing connections to be initiated using secure sockets layer.
*If a proxy server is not in use, the proxy attributes must be omitted.
*For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") <u>and</u> the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] and [[The CTP XML and Zip Filters]] for information about the script languages.

=====PolledHttpExportService=====
The PolledHttpExportService queues objects and transmits them in the HTTP response stream of a received connection. Files are transmitted with Content-Type equal to <b>application/x-mirc</b>. This ExportService is designed to work in conjunction with the PollingHttpImportService to allow penetration of a firewall without having to open an inbound port, as described in [[#Security Issues | Security Issues]]. The configuration element for the Polled HttpExportService is:
<pre>
        <ExportService
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.PolledHttpExportService"
            root="root-directory" 
            port="listening-port"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes"
            dicomScript="root-directory/scripts/df.script"
            xmlScript="root-directory/scripts/xf.script"
            zipScript="root-directory/scripts/zf.script" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
*<b>port</b> is the port on which the ExportService listens for connections.
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be exported.
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be exported.
*<b>acceptZipObjects</b> determines whether ZipObjects are to be exported.
*<b>acceptFileObjects</b> determines whether FileObjects are to be exported.

Notes:
*For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") <u>and</u> the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] and [[The CTP XML and Zip Filters]] for information about the script languages.
*The PolledHttpExportService does not support SSL.

=====DicomExportService=====
The DicomExportService queues objects and transmits them to a DICOM Storage SCP. The configuration element for the DicomExportService is:
<pre>
        <ExportService
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomExportService"
            root="root-directory" 
            url="dicom://DestinationAET:ThisAET@ipaddress:port"
            dicomScript="root-directory/scripts/df.script"
            interval="10000" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage. This attribute is required only when the stage is accessed by other stages. Its value, when supplied, must be unique across all pipelines.
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
*<b>url</b> specifies the destination DICOM Storage SCP's URL.
*For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") <u>and</u> the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] for information about the script language.
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue.
Note: The default interval is 10 seconds. The minimum allowed value is one second. The maximum allowed value is 20 seconds.

=====FtpExportService=====
The FtpExportService queues objects and transmits them to an FTP server. The configuration element for the FtpExportService is:
<pre>
        <ExportService
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.FtpExportService"
            root="root-directory" 
            url="ftp://ipaddress:port/path"
            username="..."
            password="..."
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes"
            dicomScript="root-directory/scripts/df.script"
            xmlScript="root-directory/scripts/xf.script"
            zipScript="root-directory/scripts/zf.script"
            interval="10000" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
*<b>url</b> specifies the destination FTP server's URL:
**<b>ipaddress</b> can be a numeric address or a domain name.
**<b>port</b> is the port on which the FTP listener listens. The default port is 21.
**<b>path</b> is the base directory on the FTP server in which the FtpExportService will create StudyInstance directories.
*<b>username</b> specifies the username under which the FtpExportService will log in to the FTP server.
*<b>password</b> specifies the password to be used in the login process.
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be exported.
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be exported.
*<b>acceptZipObjects</b> determines whether ZipObjects are to be exported.
*<b>acceptFileObjects</b> determines whether FileObjects are to be exported.
*For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") <u>and</u> the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] and [[The CTP XML and Zip Filters]] for information about the script languages.
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue.
Notes: 
#The default interval is 10 seconds. The minimum allowed value is one second. The maximum allowed value is 20 seconds.
#The FtpExportService stores files in subdirectories of the <b>path</b> part of the URL, organized by StudyInstanceUID (or StudyUID in the case of non-DICOM files). Files not containing a StudyUID are stored in the <b>bullpen</b> directory under the <b>path</b> directory.
#If the directory specified by the <b>path</b> does not exist, it is created.
#Files are stored within their directories with names that consist of the date and time (to the millisecond) when they were transferred to the server.
#If a file is transmitted multiple times, multiple copies of the file will appear with its directory, each with the date/time of the transfer.
#No index of the studies and files is created on the server.

=====DatabaseExportService=====
The DatabaseExportService queues objects and submits them to a DatabaseAdapter class, which must be written specially for the database in question. The configuration element for the DatabaseExportService is:
<pre>
        <ExportService
            name="stage name"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DatabaseExportService"
            adapterClass="org.myorg.MyDatabaseAdapter"
            poolSize="1"
            fileStorageServiceID="ID of referenced FileStorageService"
            root="root-directory"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes"
            interval="10000" />
</pre>
where:
*<b>name</b> is any text to be used as a label on configuration and status pages.
*<b>id</b> is any text to be used to uniquely identify the stage.
*<b>adapterClass</b> is the class name of the database's adapter class. See [[#Implementing a DatabaseAdapter | Implementing a DatabaseAdapter]] for more information.
*<b>poolSize</b> specifies the number of subordinate export threads. The default is <b>1</b>. The allowed values are from <b>1</b> to <b>10</b>.
*<b>fileStorageServiceID</b> is the ID of the FileStorageService which manages objects referenced by the data stored in the database. This attribute is required only when the database must have direct access to the files on the FileStorageService.
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be exported.
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be exported.
*<b>acceptZipObjects</b> determines whether ZipObjects are to be exported.
*<b>acceptFileObjects</b> determines whether FileObjects are to be exported.
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue.
Note: The default interval is 10 seconds. The minimum allowed value is one second. The maximum allowed value is 20 seconds.

==Extending CTP==
CTP is designed to be extended with pipeline stages of new types. Stages implement one or more Java interfaces. It is useful to obtain the source code and build it in order to obtain the Javadocs, even though in principle you don't need to modify the code itself.

===Obtaining the Source Code===
The software for CTP is open source. All the software written by the RSNA for the project is released under the [http://mirc.rsna.org/rsnapubliclicense RSNA Public License]. It is maintained on a CVS server at RSNA headquarters. To obtain the source code, configure a CVS client as follows:
<pre>
Protocol:           Password server (:pserver)
Server:             mirc.rsna.org
Port:               2401
Repository folder:  /RSNA
Username:           cvs-reader
Password:           cvs-reader
Module:             ClinicalTrialProcessor
</pre>

Together, this results in the following CVSROOT (which is constructed automatically if you use something like Tortoise-CVS on a Windows system):

:<tt>:pserver:cvs-reader@mirc.rsna.org:2401/RSNA</tt>

This account has read privileges, but it cannot write into the repository, so it can check out but not commit. If you wish to be able to commit software to the CVS library, contact the MIRC project manager.

===Building the Software===
When you check out the <b>ClinicalTrialProcessor</b> module from CVS, you obtain a directory tree full of the sources and libraries for building the application. The top of the directory tree is <tt><b>ClinicalTrialProcessor</b></tt>. It contains several subdirectories. The source code is in the <tt><b>source</b></tt> directory, which has two subdirectories, one each for the Java sources and the files required by the application.

Building CTP requires the Java 1.6 JDK and Ant. Running CTP requires the JDK or JRE and the JAI ImageIO Tools.

The Ant build file for CTP is in the <tt><b>ClinicalTrialProcessor</b></tt> directory and is called <tt><b>build.xml</b></tt>. To build the software on a Windows system, launch a command window, navigate to the <tt><b>ClinicalTrialProcessor</b></tt> directory, and enter <tt><b>ant all</b></tt>.

The build file contains several targets. The <tt><b>all</b></tt> target does a clean build of everything, including the Javadocs, which are put into the <tt><b>documentation</b></tt> directory. The Javadocs can be accessed with a browser by opening the file:

:<tt>ClinicalTrialProcessor/documentation/index.html</tt>

The default target, <tt><b>ctp-installer</b></tt>, just builds the application and places the installer in the <tt><b>products</b></tt> directory.

===The Object Classes===
CTP provides four classes to encapsulate files of various types. The classes are located in the <b>org.rsna.ctp.objects</b> package:
* <b>DicomObject</b> - a DICOM dataset
* <b>XmlObject</b> - an XML file containing identifiers relating the data to the trial and the trial subject
* <b>ZipObject</b> - a zip file containing a <b>manifest.xml</b> file providing identifiers relating the zip file's contents to the trial and the trial subject
* <b>FileObject</b> - a generic file of unknown contents and format
Each class provides methods allowing pipeline stages or database adapters to access the internals of an object without having to know how to parse it. See the Javadocs for a list of all the methods provided by these classes.

===Implementing a Pipeline Stage===
To be recognized as a pipeline stage, a class must implement the <b>org.rsna.ctp.pipeline.PipelineStage</b> interface. An abstract class, <b>org.rsna.ctp.pipeline.AbstractPipelineStage</b>, is provided to supply some of the basic methods required by the PipelineStage interface. All the standard stages extend this class.

Each stage type must also implement its own interface. The interfaces are:
*org.rsna.ctp.pipeline.ImportService
*org.rsna.ctp.pipeline.Processor
*org.rsna.ctp.pipeline.StorageService
*org.rsna.ctp.pipeline.ExportService

The Javadocs explain the methods which must be implemented in each stage type.

Each stage class must have a constructor which takes its configuration file XML Element as its argument. The constructor must obtain any configuration information it requires from the element. While it is not required that all configuration information be placed in attributes of the element, the <b>getConfigHTML</b> method provided by <b>AbstractPipelineStage</b> expects it, and if you choose to encode configuration information in another way, you must override the <b>getConfigHTML</b> method to make that information available to the configuration servlet.

===Implementing a DatabaseAdapter===
The DatabaseExportService pipeline stage provides a queuing mechanism for submitting files to a database interface, relieving the interface from having to manage the queue. It calls the overloaded <b>process</b> method of the interface with one of the four object types. Each of the objects includes methods providing access to the internals of its file, allowing the interface to interrogate objects to obtain some or all of their data to insert into an external system.

The DatabaseExportService dynamically loads the database interface class, obtaining the name of the class from the configuration element's <b>adapterClass</b> attribute. 

====The DatabaseAdapter Class====
The DatabaseAdapter class, <b>org.rsna.ctp.stdstages.database.DatabaseAdapter</b>, is a base class for building an interface between the DatabaseExportService and an external database. To be recognized and loaded by the DatabaseExportService, an external database interface class must be an extension of DatabaseAdapter.

The DatabaseAdapter class provides a set of methods allowing the DatabaseExportService to perform various functions, all of which are explained in the Javadocs. The basic interaction model is:
*When the DatabaseExportService detects that files are in its queue, it determines whether the database interface class is loaded and loads it if necessary.
*It then calls the database interface’s <b>connect()</b> method.
*For each file in the queue, it instantiates an object matching the file’s contents and calls the database interface’s <b>process()</b> method. There are four overloaded process methods, one for each object class.
*When the queue is empty, it calls the database interface’s <b>disconnect()</b> method.

All the methods of the DatabaseAdapter class return a static instance of the <b>org.rsna.ctp.pipeline.Status</b> class to indicate the result. The values are:
*<b>Status.OK</b> means that the operation succeeded completely.
*<b>Status.FAIL</b> means that the operation failed and trying again will also fail. This status value indicates a problem with the object being processed.
*<b>Status.RETRY</b> means that the operation failed but trying again later may succeed. This status value indicates a temporary problem accessing the external database.
All the methods of the base DatabaseAdapter class return the value <b>Status.OK</b>.

====Extending the DatabaseAdapter Class====
To implement a useful interface to an external database, you must extend the <b>DatabaseAdapter</b> class.

Since the DatabaseAdapter class implements dummy methods returning Status.OK, your class that extends DatabaseAdapter only has to override the methods that apply to your application. If, for example, you only care about XML objects, you can just override the <b>process(XmlObject xmlObject)</b> method and let DatabaseAdapter supply the other <b>process()</b> methods, thus ignoring objects of other types.

Although the DatabaseAdapter class includes <b>reset()</b> and <b>shutdown()</b> methods, they are not called by the DatabaseExportService because restarts are not done in CTP and there is no notice of an impending shutdown. You should therefore ensure that the data is protected in the event of, for example, a power failure. Similarly, since one <b>connect()</b> call is made for possibly multiple <b>process()</b> method calls, it is possible that a failure could result in no <b>disconnect()</b> call. Thus, depending on the design of the external system, it may be wise to commit changes in each <b>process()</b> call.

====Connecting Your Database Interface Class to CTP====
The easiest way to connect your class into CTP is to create a package for it under the <b>source/java</b> tree and then build the entire application. This will avoid your having to change the manifest declaration in the <b>build.xml</b> file, and it will ensure that the class is included in the installation without having to add a <b>jar</b> file.

==Security Issues==
In a clinical trial, transmission of data from image acquisition sites to the principal investigator site involves penetrating at least one firewall. Since the image acquisition site initiates an outbound connection, this only rarely requires special action. At the principal investigator's site, where the connection is inbound, some provision must be made to allow the connection to reach its destination. There are two basic solutions. The simplest solution is to open a port in the firewall at the principal investigator's site and route connections for that port to the computer running the CTP application. In some institutions, however, security policies prohibit this solution. The alternative is to use the PolledHttpExportService and PollingHttpImportService to allow data to flow without having to open any ports on the internal network to inbound connections.

Using this latter solution requires two computers, each running CTP. One computer is placed in the border router's DMZ with one port open to the internet, allowing connections to the HttpImportService of the program running in that computer. That program's pipeline includes a PolledHttpExportService which queues objects and waits for a connection before passing them onward. The second computer is placed on the internal network. Its program has a pipeline which starts with a PollingHttpImportService. That ImportService is configured to make outbound connections to the DMZ computer when a file is requested. This allows files to pass through the firewall on the response stream of the outbound connection without having to open any ports to the internal network.

==Notes==

===Important Note for Unix/Linux Platforms===
Unix and its derivatives require that applications listening on ports with numbers less than 1024 have special privileges. On such systems, it might be best to put all import services and all web servers on ports above this value. The default configuration puts the web server on port 80 because that is the default port for the web. This can be changed before the program is run for the first time by editing the <b><tt>example-config.xml</tt></b> file, or after it has been run the first time by editing the <b><tt>config.xml</tt></b> file.