Difference between revisions of "MIRC CTP"

From MircWiki
Jump to: navigation, search
(Building the Software)
(DirectoryImportService)
 
(669 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This article describes the stand-alone processing application for clinical trials data using MIRC components and the MIRC internet transport mechanism.
+
{| align="right"
 +
  | __TOC__
 +
  |}
 +
This article describes the RSNA MIRC Clinical Trials Processor (CTP), a stand-alone image processing application for imaging clinical trials data.
  
==Background==
+
==Clinical Trial Processor (CTP)==
MIRC supports clinical trials through two applications, one for data acquisition at an imaging center (FieldCenter) and one for management of the data at a principal investigator's site (MIRC).
+
CTP is a stand-alone program that provides all the processing features of a MIRC site for clinical trials in a highly configurable and extensible application. It connects to FieldCenter applications and can also connect to MIRC sites when necessary. CTP has the following key features:
 
+
The FieldCenter application acquires images via the DICOM protocol, anonymizes them, and transfers them (typically using HTTP, although DICOM is also supported) to a principal investigator's MIRC site. It also supports other types of data files and includes an anonymizer for XML files as well. FieldCenter also contains a client for the Update Service of a MIRC site, allowing the application to save data on, and obtain software updates from, the principal investigator's site.
+
 
+
The MIRC site software contains a partially configurable processing pipeline for clinical trials data, consisting of:
+
 
+
:;HttpImportService
+
::A receiver for HTTP connections from FieldCenter applications transferring data files into the processing pipeline.
+
:;DicomImportService
+
::A receiver form DICOM datasets for iinsertion into the processing pipeline.
+
:;Preprocessor
+
::A user-defined component for processing data received by the HttpImportService before it is further processed by other components.
+
:;Anonymizer
+
::A component for anonymizing DICOM objects or XML objects.
+
:;DatabaseExportService
+
::A component providing queue management and submission of data objects to a user-defined interface to an external database management system.
+
:;HttpExportService
+
::A component in the DicomImportService pipeline providing queue management and transmission of data objects to one or more external systems using the HTTP protocol.
+
:;DicomExportService
+
::A component in the HttpImportService pipeline providing queue management and transmission of data objects to one or more external systems using the DICOM protocol.
+
 
+
The processing pipelines for the HttpImportService and DicomImportService are different.  They are not symmetrical. For example, the HttpImportService does not have access to the anonymizer except as part of the DatabaseExportService. Another limitation is that objects received via one protocol can only be exported via the other. While these limitations are consistent with the requirements of most trials, it became clear that a completely symmetrical design would provide better support for more sophisticated trials while still satisfying the requirements of simple ones.
+
 
+
==ClinicalTrialProcessor==
+
ClinicalTrialProcessor is a stand-alone application that provides all the features of a MIRC site for clinical trials in a highly configurable and extensible way. It connects to FieldCenter applications and can also connect to MIRC sites when necessary. ClinicalTrialProcessor has the following key features:
+
 
#Single-click installation.
 
#Single-click installation.
 
#Support for multiple pipelines.
 
#Support for multiple pipelines.
Line 40: Line 19:
 
#*HTTP Export
 
#*HTTP Export
 
#*DICOM Export
 
#*DICOM Export
 +
#*FTP Export
 
#Web-based monitoring of the application's status, including:
 
#Web-based monitoring of the application's status, including:
 
#*configuration
 
#*configuration
Line 45: Line 25:
 
#*quarantines
 
#*quarantines
 
#*status
 
#*status
#Support for the FieldCenter Update Service client.
+
 
 +
===Installation===
 +
The installer for CTP is available on the [http://mirc.rsna.org RSNA MIRC site]. Click the <b>Download Software</b> link in the left pane of the MIRC home page to obtain a list of all the available software.
 +
 
 +
Download the installer and place it on the disk on which you intend to install or upgrade the CTP program.
 +
 
 +
To run the installer, the Java 1.7 (or better) JRE must be present on the system. Java 1.8 is recommended because earlier versions are being sunset by Oracle/Sun. Java and all its components are available through the [http://www.oracle.com/technetwork/java/javase/downloads/index.html Java] website. Note that only the JRE is required, not the JDK. If you plan to run CTP as a Windows service or if you plan to use the <b>Java Advanced Imaging ImageIO Tools</b> (see below), then you must install the 32-bit Java, even on a 64-bit computer.
 +
 
 +
For convenience, it is recommended (but not required) that a folder called <b>JavaPrograms</b> be created in the root of the disk drive. The installer does not create this folder, but if it is present, the installer will very quickly find it and suggest installing CTP in a subdirectory of <b>JavaPrograms</b>. This is especially helpful when upgrading.
 +
 
 +
Certain CTP pipeline stages (FileStorageService, BasicFileStorageService, DicomDecompressor, and others) require that the <b>[[Java Advanced Imaging ImageIO Tools]]</b>  be present on the system. If you already have the ImageIO Tools installed, note that it is critically important that version 1.1 of the ImageIO Tools be installed rather than version 1.0.  Parenthetically, note that the Java Advanced Imaging component is not the same as the Java Advanced Imaging ImageIO Tools. Only the latter component is required. (Macintosh users should read [[#ImageIO_Tools_for_Macintosh|ImageIO Tools for Macintosh]] in the Notes section.) When the CTP installer runs, it checks several parameters of the system and highlights ones that are not correct for the running of CTP. One such parameter is the version of the ImageIO Tools. The ImageIO Tools need not be present in order to run the installer, and they may be installed later if required, without having to re-install CTP.
 +
 
 +
Note also that the CTP installer includes the ImageIO Tools for Windows 32-bit Java only, so on such systems, you can skip the installation of the ImageIO Tools, but that will make the tools only available for CTP, not for other applications. If you are planning to install certain other RSNA DICOM tools (for example, DicomEditor), it is best to install the ImageIO Tools directly in Java using the installer provided in [[Java Advanced Imaging ImageIO Tools]].
 +
 
 +
To run the CTP installer, double-click the <tt><b>CTP-installer.jar</b></tt> file and choose a directory in which to install CTP. The installer can also be run in a command window using the command:
 +
 
 +
:<b><tt>java -jar CTP-installer.jar</tt></b>
 +
 
 +
The installer creates a directory called <b>CTP</b> in the selected location and places several files and directories in it. Two files of special importance are:
 +
 
 +
* <b>Launcher.jar</b> - the program that launches CTP with a user interface
 +
* <b>Runner.jar</b> - the program that starts or stops CTP with no user interface
 +
 
 +
===Running CTP===
 +
There are three ways to start CTP:
 +
* <b><tt>Launcher.jar</tt></b> - a program for manually starting and stopping CTP through a dialog window. This program is normally used when CTP is installed on an individual user's computer for occasional use.
 +
* <b><tt>Runner.jar</tt></b> - a program for starting and stopping CTP with no user interface. This program is normally used when CTP is installed on a Linux or Mac system for institutional use, running as an automatic service. See [[Running CTP as a Linux Service]] for instructions.
 +
* CTP can also be run as a Windows service. See [[Running CTP as a Windows Service]] for instructions.
 +
 
 +
When learning to use CTP or when developing a new pipeline configuration, it is best to start by running CTP from the Launcher.
 +
 
 +
The launcher displays a dialog providing start/stop buttons and fields for controlling several parameters used by the system.
 +
 
 +
The <b>Server port</b> field allows you to change the port on which the server runs.
 +
 
 +
The default memory configuration is sufficient for all but the very largest sites. For sites with 2000 or more teaching file cases, the <b>Maximum memory pool</b> parameter should be increased to 500. For sites with more than 3000 cases, 750 is recommended. To change the memory configuration for the Windows service, see the article.
 +
 
 +
The <b>Extensions directory</b> parameter is intended for special applications in which third-party software is added into the system. One such application is the NCI's National Biomedical Image Archive (NBIA). Normal teaching file systems do not require this parameter.
 +
 
 +
Across the top of the dialog are several tabs providing access to information about the versions of the components, the system parameters in use by Java as the program runs, and any messages output by the program either directly or through its log file. These are only present as a convenience; they are not typically used in normal operation.
 +
 
 +
As a convenience, the <b>CTP Home Page</b> button launches the user's browser and goes directly to the main MIRC page.
 +
 
 +
CTP has no user interface. When the program starts, it runs without intervention. Status and other information can be obtained through the program's integrated webserver. Accessing the server with no path information displays a page presenting buttons for each of the servlets.
 +
 
 +
When the program is first installed, two users are provided. One user, with the name <b>admin</b> and password <b>password</b>, is intended for general system administration. The other user, with the name <b>king</b> and password <b>password</b>, has the ability to shut down the server through the browser interface. Both users have the ability to create and modify users and assign privileges through the User Manager, but only a user with the shutdown privilege can grant the shutdown privilege to another user.
 +
 
 +
To stop the program, click the <b>Stop</b> button on the Launcher, or log in as a user with the shutdown privilege and click the <b>Shutdown</b> button on the main page. As a convenience, a user with the admin privilege can also shut the server down if the user's browser is running on the same computer as the server.
 +
 
 +
===Configuration Files===
 +
The program uses two configurable files: <b><tt>config.xml</tt></b>, which is located in the same directory as the program itself, and <b><tt>index.html</tt></b>, which is located in the server's <b><tt>ROOT</tt></b> directory. Both files are intended to be configured for the specific application. The installer does not overwrite these files when it runs; instead, it installs two example files: <b><tt>example-config.xml</tt></b> and <b><tt>example-index.html</tt></b>. When CTP starts, it looks to see if the non-example files are missing, and if so, it copies the example files into the non-example ones. This process allows upgrades to be done without losing any configuration work. After installing the program the first time, it should be run once in order to make the copies, and then the copies can be configured. Configuration is done by hand with any text editor (e.g., TextPad or NotePad). Care should be taken, especially with config.xml, to keep it well-formed. Opening it with a program like InternetExplorer will check it for errors.
 +
 
 +
===Server===
 +
To provide access to the status of the components, the application includes an HTTP server which serves files and provides servlet-like functionality. Files are served from a directory tree whose root is named <b><tt>ROOT</tt></b>. The <b><tt>ROOT</tt></b> directory contains a file, <b><tt>index.html</tt></b>, which provides buttons which link to several servlets providing information about the operation of the program. This file is intended to be configured with logos, additional links, etc., and upgrades do not overwrite it. The standard servlets are:
 +
 
 +
*<b>LoginServlet</b> allows a user to log into the system.
 +
*<b>UserManagerServlet</b> allows an admin user to create users and assign them privileges.
 +
*<b>ConfigurationServlet</b> displays the contents of the configuration file.
 +
*<b>SysPropsServlet</b> displays the system properties which define the environment in which CTP is running, and provides an admin user the ability to force a garbage collection.
 +
*<b>StatusServlet</b> displays the status of all pipeline stages.
 +
*<b>LogServlet</b> provides web access to all log files in the <b>logs</b> directory.
 +
*<b>QuarantineServlet</b> provides web access to all quarantine directories and their contents.
 +
*<b>IDMapServlet</b> allows an admin user to access a database of PHI and anonymized replacements for patient IDs accession numbers, and UIDs.
 +
*<b>ObjectTrackerServlet</b> allows an admin user to access a database of the identifiers of objects which have been processed, organized by date, patient ID, Study UID, Series UID, and Instance UID.
 +
*<b>DBVerifierServlet</b> allows an admin user to access a database which tracks the status of objects as they are inserted into an external database (which may be in a remote instance of CTP).
 +
*<b>DicomAnonymizerServlet</b> allows an admin user to configure any DICOM anonymizers in the pipelines.
 +
*<b>ScriptServlet</b> allows an admin user to configure the scripts for script-based pipeline stages, including the various Filter stages and the XML and Zip anonymizers.
 +
*<b>LookupServlet</b> allows an admin user to configure lookup tables used by the anonymizers.
 +
*<b>ShutdownServlet</b> allows an admin user to shut the program down.
 +
 
 +
The configuration element for the HTTP server is:
 +
<pre>
 +
        <Server
 +
            port="80"
 +
            ssl="no"
 +
            requireAuthentication="no"
 +
            usersClassName="org.rsna.server.UsersXmlFileImpl">
 +
          <ProxyServer
 +
              proxyIPAddress=""
 +
              proxyPort=""
 +
              proxyUsername=""
 +
              proxyPassword=""/>
 +
          <SSL
 +
            keystore=""
 +
            keystorePassword=""
 +
            truststore=""
 +
            truststorePassword=""/>
 +
        </Server>
 +
 
 +
</pre>
 +
where:
 +
*<b>port</b> is the port number on which the HTTP server listens for connections.
 +
*<b>ssl</b> determines whether the HTTP server uses SSL. Values are "yes" and "no". The default is "no".
 +
*<b>requireAuthentication</b> determines whether users are forced to log in to the HTTP server. Values are "yes" and "no". The default is "no". (The HTTP server is typically operated without requiring authentication, thus allowing users to monitor the status of the system without having to log in.)
 +
*<b>proxyIPAddress</b> is the IP address of the proxy server. If this attribute is missing or blank, no proxy server is used. If a proxy server is configured, it is shared by all pipeline stages and servlets.
 +
*<b>proxyPort</b> is the port of the proxy server. If this attribute is missing or blank, no proxy server is used.
 +
*<b>proxyUsername</b> is the username which is to be supplied to the proxy server if proxy authentication is required. If this attribute is missing or blank, no credentials will be supplied to the proxy server.
 +
*<b>proxyPassword</b> is the password which is to be supplied to the proxy server if proxy authentication is required. If this attribute is missing or blank, no credentials will be supplied to the proxy server.
 +
*<b>keystore</b> specifies the path to the keystore file. If this attribute is missing or blank, the value <b><tt>keystore</tt></b> is supplied.
 +
*<b>keystorePassword</b> is the password for access to the keystore. It this attribute is missing or blank, the value <b><tt>ctpstore</tt></b> is used.
 +
*<b>truststore</b> specifies the path to the truststore file. If this attribute is missing or blank, the corresponding property is removed from the system properties.
 +
*<b>truststorePassword</b> is the password for access to the truststore.
 +
*<b>usersClassName</b> specifies the Java class to be used for authentication of users and their privileges. If this attribute is missing, the standard CTP implementation (shown above) is employed. This attribute should only be included if a special mechanism has been implemented on the site (for example, using LDAP or OpenAM - see [[CTP Authentication Using LDAP]] or [[CTP Authentication Using OpenAM]]).
 +
 
 +
Notes:
 +
*The ProxyServer element is only required when the system is behind a proxy server.
 +
*The SSL element is only required when accessing a site-specific keystore or truststore instead of the default stores supplied in a CTP installation.
 +
*When an external authentication system is used for authentication, the Server element may have a child element containing its parameters. Examples are the LDAP and OpenAM child elements. See [[CTP Authentication Using LDAP]] or [[CTP Authentication Using OpenAM]].
 +
 
 +
===Plugins===
 +
A plugin is a component that adds functionality to CTP outside the context of a pipeline. Plugins can add servlets to the server as well as provide capabilities that may be accessed by pipeline stages.
  
 
===Pipelines===
 
===Pipelines===
Line 53: Line 143:
 
:*XmlObject
 
:*XmlObject
 
:*ZipObject
 
:*ZipObject
Each pipeline must contain one ImportService as its first stage. Each pipeline stage is provided access to a quarantine directory, which may be unique to the stage or shared with other stages, into which the pipeline places objects that are rejected by a stage, thus aborting further processing. At the end of the pipeline, the manager calls the ImportService to remove the processed object from its queue.
+
Each pipeline must contain at least one ImportService. Each pipeline stage may be provided access to a quarantine directory into which the stage places objects that it rejects, thus removing them from the pipeline and aborting further processing. Quarantine directories may be unique to each stage or shared with other stages. At the end of the pipeline, the manager calls the ImportService which provided the object to remove it from its queue.
 +
 
 +
There are four types of pipeline stages. Each is briefly described in subsections below.
  
 
====ImportService====
 
====ImportService====
 
An ImportService receives objects via a protocol and enqueues them for processing by subsequent stages.
 
An ImportService receives objects via a protocol and enqueues them for processing by subsequent stages.
 +
 +
====Processor====
 +
A Processor performs some kind of processing on an object. Processors are not intended to be queued. In the context of the current MIRC implementation, a Preprocessor is a Processor, as is an Anonymizer. The result of a processing stage is an object that is passed to the next stage in the pipeline.
  
 
====StorageService====
 
====StorageService====
 
A StorageService stores an object in a file system. It is not queued, and it therefore must complete before subsequent stages can proceed. A StorageService may return the current object or the stored object in response to a request for the output object, depending on its implementation.
 
A StorageService stores an object in a file system. It is not queued, and it therefore must complete before subsequent stages can proceed. A StorageService may return the current object or the stored object in response to a request for the output object, depending on its implementation.
 
====Processor====
 
A Processor performs some kind of processing on an object. It is not queued. A processor exposes methods with calling signatures that are unique to the object type. In the context of the current MIRC implementation, a Preprocessor is a Processor, as is an Anonymizer. The result of a processing stage is an object that is passed to the next stage in the pipeline.
 
  
 
====ExportService====
 
====ExportService====
An ExportService provides queued transmission to an external system via a defined protocol. Objects in the queue are full copies of the objects submitted; therefore, subsequent processing is not impeded if a queue is paused, and modifications made subsequently do not affect the queue entry, even if they occur before transmission. (Note: This is different from the current MIRC implementation.)
+
An ExportService provides queued transmission to an external system via a defined protocol. Objects in the queue are full copies of the objects submitted; therefore, subsequent processing is not impeded if a queue is paused, and modifications made subsequently do not affect the queue entry, even if they occur before transmission. (Note: This behavior is different from that of the current MIRC implementation.) After entering an object in its queue, an ExportService returns immediately.
  
===Configuration===
+
===System Configuration===
The ClinicalTrialProcessor configuration is specified by an XML file. There can be one <b>Server</b> element specifying the port on which the HTTP server is to operate, and multiple <b>Pipeline</b> elements, each specifying the stages which comprise it. The name of the element defining a stage is irrelevant and can be chosen for readability; each stage in a pipeline is actually defined by its Java class, specified in the <b>class</b> attribute. Stages are loaded automatically when the program starts, and the loader tests the stage's class to see what kind of stage it represents. It is possible to extend the application beyond the pre-defined stages available in the implementation as described in the section on Extending ClinicalTrialProcessor.
+
The CTP configuration is specified by an XML file called <tt><b>config.xml</b></tt> located in the same directory as the program. There can be one <b>Server</b> element specifying the port on which the HTTP server is to operate, and multiple <b>Pipeline</b> elements, each specifying the stages which comprise it. The name of the element defining a stage is irrelevant and can be chosen for readability; each stage in a pipeline is actually defined by its Java class, specified in the <b>class</b> attribute. Stages are loaded automatically when the program starts, and the loader tests the stage's class to determine what kind of stage it represents. It is possible to extend the application beyond the pre-defined stages available in the implementation as described in [[Extending CTP]].
  
The following is an example of a simple configuration with one pipeline which receives objects via the HTTP protocol, stores them, and exports them to a DICOM destination:
+
The following is an example of a simple configuration that might be used at an image acquisition site. It contains one pipeline which receives objects via the DICOM protocol, stores the objects locally, anonymizes them, and transmits them via HTTP (using secure sockets layer for encryption) to a principal investigator's site.
 
<pre>
 
<pre>
 
<Configuration>
 
<Configuration>
 
   <Server port="80" />
 
   <Server port="80" />
 
   <Pipeline name="Main Pipeline">
 
   <Pipeline name="Main Pipeline">
         <ImportService
+
         <DicomImportService
             name="HTTP Import"
+
             name="DicomImportService"
             class="org.rsna.ctp.stdstages.HttpImportService"
+
             class="org.rsna.ctp.stdstages.DicomImportService"
             root="roots/http-import"  
+
             root="roots/dicom-import"
             port="7777"  
+
             port="1104" />
             quarantine="quarantines/HttpImportQuarantine" />
+
        <FileStorageService
         <StorageService
+
            name="FileStorageService"
             name="Storage"
+
            class="org.rsna.ctp.stdstages.FileStorageService"
             class="org.rsna.ctp.stdstages.StorageService"
+
            root="storage"
             root="D:/storage"  
+
            returnStoredFile="no"
             return-stored-file="no"
+
             quarantine="quarantines/storage" />
             quarantine="quarantines/StorageQuarantine" />
+
         <DicomAnonymizer
         <ExportService
+
             name="DicomAnonymizer"
             name="PACS Export"
+
             class="org.rsna.ctp.stdstages.DicomAnonymizer"
             class="org.rsna.ctp.stdstages.DicomExportService"
+
             root="roots/anonymizer"
             root="roots/pacs-export"  
+
             script="scripts/da.script"
             dest-url="dicom://DestinationAET:ThisAET@ipaddress:port"
+
             quarantine="quarantines/anonymizer" />
            quarantine="quarantines/PacsExportQuarantine" />
+
         <HttpExportService
 +
             name="HttpExportService"
 +
             class="org.rsna.ctp.stdstages.HttpExportService"
 +
             root="roots/http-export"
 +
             url="https://university.edu:1443" />
 
   </Pipeline>
 
   </Pipeline>
 
</Configuration>
 
</Configuration>
 
</pre>
 
</pre>
Note that in the example above, non-DICOM objects are stored in the StorageService, but they are not exported by the DicomExportService. Each pipeline stage is responsible for testing the class of the object which it receives and processing (or ignoring) the object accordingly.
 
  
The following is an example of a more complex configuration. This configuration receives objects, passes them to a trial-specific Processor stage to test whether the object is appropriate for the trial, anonymize objects which make it through the preprocessor, export them to a database, and then anonymize them again to remove information which is not intended for storage, and finally store them.
+
Note that because the storage service appears in the pipeline before the anonymizer, the objects which are stored contain the PHI which was originally received, and because the anonymizer appears before the export service, anonymized objects are exported.
 +
 
 +
The following is an example of a simple configuration that might be used at a principal investigator's site. It contains one pipeline which receives objects via the HTTP protocol, stores them, and exports them to a DICOM destination:
 
<pre>
 
<pre>
 
<Configuration>
 
<Configuration>
 
   <Server port="80" />
 
   <Server port="80" />
 
   <Pipeline name="Main Pipeline">
 
   <Pipeline name="Main Pipeline">
         <ImportService
+
         <HttpImportService
             name="HTTP Import"
+
             name="HttpImportService"
 
             class="org.rsna.ctp.stdstages.HttpImportService"
 
             class="org.rsna.ctp.stdstages.HttpImportService"
             root="roots/http-import"  
+
             root="roots/http-import"
             port="7777"  
+
             ssl="yes"
             quarantine="quarantines/HttpImportQuarantine" />
+
             port="1443" />
         <Processor
+
         <FileStorageService
             name="The Preprocessor"
+
             name="FileStorageService"
            class="org.myorg.MyPreprocessor"
+
             class="org.rsna.ctp.stdstages.FileStorageService"
            quarantine="quarantines/PreprocessorQuarantine" />
+
             root="D:/FileStorageService"  
        <Processor
+
             returnStoredFile="no"
            name="Main Anonymizer"
+
             quarantine="quarantines/StorageServiceQuarantine" />
            class="org.rsna.trials.Anonymizer"
+
        <DicomExportService
            dicom-script="dicom-anonymizer-1.properties"
+
             name="DicomExportService"
            xml-script="xml-anonymizer-1.script"
+
            zip-script="zip-anonymizer-1.script"
+
            quarantine="quarantines/MainAnonymizerQuarantine" />
+
        <ExportService
+
            name="Database Export"
+
            class="org.rsna.trials.DatabaseExportService"
+
            adapter-class="org.myorg.MyDatabaseAdapter"
+
            root="roots/database-export"
+
            quarantine="quarantines/DatabaseExportQuarantine" />
+
        <Processor
+
            name="Provenance Remover"
+
             class="org.rsna.ctp.stdstages.Anonymizer"
+
            dicom-script="dicom-anonymizer-2.properties"
+
            xml-script="xml-anonymizer-2.script"
+
            zip-script="zip-anonymizer-2.script"
+
            quarantine="quarantines/ProvenanceRemoverQuarantine" />
+
        <StorageService
+
            name="Storage"
+
            class="org.rsna.ctp.stdstages.StorageService"
+
             root="D:/storage"  
+
             return-stored-file="no"
+
             quarantine="quarantines/StorageQuarantine" />
+
        <ExportService
+
             name="PACS Export"
+
 
             class="org.rsna.ctp.stdstages.DicomExportService"
 
             class="org.rsna.ctp.stdstages.DicomExportService"
 
             root="roots/pacs-export"  
 
             root="roots/pacs-export"  
             dest-url="dicom://DestinationAET:ThisAET@ipaddress:port"
+
             url="dicom://DestinationAET:ThisAET@ipaddress:port" />
            quarantine="quarantines/PacsExportQuarantine" />
+
  </Pipeline>
    </Pipeline>
+
 
</Configuration>
 
</Configuration>
 
</pre>
 
</pre>
 +
Note that in the example above, non-DICOM objects are stored in the StorageService, but they are not exported by the DicomExportService. Each pipeline stage is responsible for testing the class of the object which it receives and processing (or ignoring) the object accordingly.
 +
 
Multiple <b>Pipeline</b> elements may be included, but each must have its own ImportService element, and their ports must not conflict.
 
Multiple <b>Pipeline</b> elements may be included, but each must have its own ImportService element, and their ports must not conflict.
  
 
Each pipeline stage class has a constructor that is called with its configuration element, making it possible for special processor implementations to be passed additional parameters from the configuration.
 
Each pipeline stage class has a constructor that is called with its configuration element, making it possible for special processor implementations to be passed additional parameters from the configuration.
  
===Server===
+
===Standard Plugins===
To provide access to the status of the components, the application includes an HTTP server which serves files and provides servlet-like functionality. The web pages are served from a directory tree whose root is named <b>ROOT</b>.
+
The application includes built-in, standard plugins to provide features that are required in some clinical trials. The sections below show all the configuration attributes recognized by the standard plugins. Note that the configuration element for a plugin always has the tag name <b><tt>Plugin</tt></b>.
  
====ConfigurationServlet====
+
=====AuditLog=====
The ConfigurationServlet displays the contents of the configuration file.
+
The AuditLog plugin provides a permanent log repository to meet the logging requirements of a 21CFR11-compliant clinical trial. Certain pipeline stages can be configured to make entries in the log repository.
  
====StatusServlet====
+
The AuditLog plugin installs a servlet to provide browser access to the repository for admin users. The servlet is accessed with a path equal to the value of the AuditLog plugin's <b><tt>id</tt></b> attribute. It is possible to configure multiple AuditLog Plugin elements (with different <b><tt>id</tt></b> attributes) if multiple trials are serviced by a single CTP instance and it is desired to keep the log repositories separate. The configuration element for the AuditLog is:
The StatusServlet displays the status of all the pipeline stages.
+
<pre>
 +
        <Plugin
 +
            name="log name"
 +
            id="pluginID"
 +
            class="org.rsna.ctp.stdplugins.AuditLog"
 +
            root="root-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is text to be used to uniquely identify the plugin. Note that since the value of the attribute is used as the context of the servlet that provides access to the repository, it must be a single word (that is, it must not contain whitespace).
 +
*<b>root</b> is a directory for use by the plugin for internal storage of the database.
  
====LogServlet====
+
=====Redirector=====
The LogServlet provides web access to all the log files in the <b>logs</b> directory.
+
The Redirector plugin redirects non-secure HTTP connections to another port using SSL. It is intended for use on CTP installations that configure the main server to use SSL. Such installations typically put the server on port 443. As a convenience for users, the Redirector can be configured to listen on port 80 and redirect the user to port 443, switching the protocol.
  
====UpdateService====
+
The configuration element for the Redirector is:
The UpdateService supports the Update Service clients in FieldCenter applications, serving software updates and saving the remapping tables in trials configured to use it.
+
<pre>
 +
        <Plugin
 +
            name="Redirector"
 +
            class="org.rsna.ctp.stdplugins.Redirector"
 +
            httpPort="80"
 +
            httpsHost="mirc.mysecuresite.myuniversity.edu"
 +
            httpsPort="443" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>httpPort</b> is the port on which the Redirector listens for connections.
 +
*<b>httpsHost</b> is the host to which the Redirector redirects the connection request.
 +
*<b>httpsPort</b> is the port to which the Redirector redirects the connection request.
  
===The Standard Stages===
+
Notes:
The application includes standard stages which allow most trials to be operated without writing any software.
+
* The redirection only occurs on HTTP GET requests.  
====HttpImportService====
+
* If the <b>httpsHost</b> attribute is missing or empty, the value of the HOST header is used in the target URL.
====DicomImportService====
+
* The query string, if any, is included in the target URL.
====Anonymizer====
+
====StorageService====
+
====HttpExportService====
+
====DicomExportService====
+
====DatabaseExportService====
+
==Extending ClinicalTrialProcessor==
+
ClinicalTrialProcessor is designed to be extended with additional pipeline stages. Stages implement one or more Java interfaces, so it is necessary to get the source code for ClinicalTrialProcessor in order to extend it, even though in principle you don't need to modify the code itself.
+
  
===Obtaining the Source Code===
+
===Standard Stages===
The software for ClinicalTrialProcessor is open source. All the software written by the RSNA is released under the [http://mirc.rsna.org/rsnapubliclicense RSNA Public License]. It is maintained on a CVS server at RSNA headquarters. To obtain the source code, configure a CVS client as follows:
+
The application includes several built-in, standard stages which allow most trials to be operated without writing any software. The sections below show all the configuration attributes recognized by the standard stages.
 +
 
 +
Attributes which specify directories can contain either absolute paths (e.g., <tt><b>D:/TrialStorage</b></tt>) or relative paths (e.g., <tt><b>quarantines/http-import-quarantine</b></tt>). Relative paths are relative to the directory in which the CTP application is located.
 +
 
 +
All standard stages have a <b>root</b> attribute which defines a directory for use by the stage for internal storage. All <b>root</b> directories must be unique, e.g. not shared with other stages.
 +
 
 +
All standard stages have an optional <b>id</b> attribute which, if present, must uniquely identify the stage across all pipelines. This attribute is required only when the stage must be accessed by another stage, for example, when a DatabaseExportService must interrogate a FileStorageService to determine the URL under which an object is stored.
 +
 
 +
Some standard stages have attributes which determine which object types are to be accepted by the stage. These attributes are:
 +
*acceptDicomObjects
 +
*acceptXmlObjects
 +
*acceptZipObjects
 +
*acceptFileObjects
 +
The allowed values are <b>yes</b> and <b>no</b>. The default value for all these attributes is <b>yes</b>. Stages which are restricted to specific object types (e.g. DicomImportService, DicomAnonymizer, XmlAnonymizer, DicomFilter, DicomExportService) ignore the values of these attributes.
 +
 
 +
If a standard ImportService receives an object which it is not configured to accept, it quarantines the object, or if no quarantine has been defined for the stage, it discards the object.
 +
 
 +
If a Processor, StorageService, or ExportService receives an object that it is not configured to accept, it either ignores the object or passes it unmodified to the next pipeline stage. Thus, if an anonymizer which is not configured to anonymize XmlObjects receives an XmlObject, it passes the object on without anonymization.
 +
 
 +
====Import Services====
 +
=====HttpImportService=====
 +
The HttpImportService listens on a defined port for connections from HTTP clients and receives files transmitted using the HTTP protocol with Content-Type equal to <b><tt>application/x-mirc</tt></b>. The configuration element for the HttpImportService is:
 
<pre>
 
<pre>
Protocol:          Password server (:pserver)
+
        <HttpImportService
Server:             mirc.rsna.org
+
             name="HttpImportService"
Port:              2401
+
            id="stage ID"
Repository folder:  /RSNA
+
            class="org.rsna.ctp.stdstages.HttpImportService"
User name:          cvs-reader
+
            root="root-directory"
Password:          cvs-reader
+
            port="7777"
Module:             ClinicalTrialProcessor
+
            ssl="yes"
 +
            zip="no"
 +
            requireAuthentication="no"
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            acceptFileObjects="yes"
 +
            logConnections="no"
 +
            logDuplicates="no"
 +
            quarantine="quarantine-directory" >
 +
 
 +
             <accept ip="..."/>
 +
            <reject ip="..."/>
 +
 
 +
        </HttpImportService>   
 
</pre>
 
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ImportService for internal storage and queuing.
 +
*<b>port</b> is the port on which the ImportService listens for connections.
 +
*<b>ssl</b> determines whether the port uses secure sockets layer (<b>yes</b>) or unencrypted http (<b>no</b>). The default is <b>no</b>.
 +
*<b>zip</b> determines whether files will be unzipped after reception (<b>yes</b>) or not (<b>no</b>). The default is <b>no</b>. This feature is intended for use with zipped transmissions from the HttpExportService.
 +
*<b>requireAuthentication</b> determines whether transmissions are required to have headers which identify a user which has the <b>import</b> privilege. The allowed values are <b>yes</b>) and <b>no</b>. The default is <b>no</b>.
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be enqueued when received.
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be enqueued when received.
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be enqueued when received.
 +
*<b>acceptFileObjects</b> determines whether FileObjects are to be enqueued when received.
 +
*<b>logConnections</b> specifies whether to make a log entry for each object that is received. The log entry includes only the IP address of the sender. The values <b>yes</b> and <b>all</b> cause all connections to be logged. The value <b>rejected</b> causes only rejected connections to be logged. The default is <b>no</b>, meaning that no connections are logged. This capability is intended primarily for configuration debugging.
 +
*<b>logDuplicates</b> specifies whether to make a log entry for each object whose UID matches that of any other object in the last 20 that have been received. This capability is intended primarily for configuration debugging.
 +
*<b>quarantine</b> is a directory in which the ImportService is to quarantine objects that it receives but is configured not to accept.
 +
*The <b>accept</b> child element can be used to specify a white list of IP addresses determining which connections are to be accepted. One <b>accept</b> child element must appear for each IP address in the white list. If the white list is empty, the white list is not used to filter connections.
 +
*The <b>reject</b> child element can be used to specify black lists of IP addresses determining which connections are to be rejected. One <b>reject</b> child element must appear for each IP address in the black list. If the black list is empty, the black list is not used to filter connections.
 +
*For a connection to be accepted, the IP address of the remote client must be accepted by the white list and and not rejected by the black list.
  
Together, this results in the following CVSROOT (which is constructed automatically if you use something like Tortoise-CVS on a Windows system):
+
Notes:
 +
*The <b>accept</b> and <b>reject</b> child elements support the <b>regex</b> attribute in place of the <b>ip</b> attribute, allowing a range of IP addresses to be matched. The <b>regex</b> attribute must contain a regular expression. For example, to match any IP address in  the <tt>10.10.*.*</tt> range, the <b>accept</b> element would be coded:
  
:<tt>:pserver:cvs-reader@mirc.rsna.org:2401/RSNA</tt>
+
<pre>
 +
    <accept regex="10\.10\.[0-9]{1,3}\.[0-9]{1,3}"/>
 +
</pre>
  
This account has read privileges, but it cannot write into the repository, so it can check out but not commit. If you wish to be able to commit software to the CVS library, contact the MIRC project manager.
+
=====PollingHttpImportService=====
 +
The PollingHttpImportService obtains files by initiating HTTP connections to an external system. This ImportService is designed to work in conjunction with the PolledHttpExportService to allow penetration of a firewall without having to open an inbound port, as described in [[#Security Issues | Security Issues]]. The configuration element for the PollingHttpImportService is:
 +
<pre>
 +
        <PollingHttpImportService
 +
            name="PollingHttpImportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.PollingHttpImportService"
 +
            root="root-directory"
 +
            url="http://ip:port"
 +
            zip="no"
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            acceptFileObjects="yes"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ImportService for internal storage.
 +
*<b>url</b> is the URL of the PolledHttpExportService.
 +
*<b>zip</b> determines whether files will be unzipped after reception (<b>yes</b>) or not (<b>no</b>). The default is <b>no</b>.
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be enqueued when received.
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be enqueued when received.
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be enqueued when received.
 +
*<b>acceptFileObjects</b> determines whether FileObjects are to be enqueued when received.
 +
*<b>quarantine</b> is a directory in which the ImportService is to quarantine objects that it receives but is configured not to accept.
 +
Notes:
 +
*The protocol part of the <b>url</b> must be <b>http</b>, although the actual protocol used by the PollingHttpImportService is not HTTP.
 +
*The PollingHttpImportService does not support SSL.
 +
*The PollingHttpImportService does not support a proxy server for its connections.
  
===Building the Software===
+
=====DicomImportService=====
When you check out the <b>ClinicalTrialProcessor</b> module from CVS, you obtain a directory tree full of the sources and libraries for building the application. The top of the directory tree is <b>ClinicalTrialProcessor</b>. It contains several subdirectories. The source code is in the <b>source</b> directory, which has two child directories, one each for the Java sources, and any files required by the application.
+
The DicomImportService listens on a defined port for connections from DICOM Storage SCUs and receives files transmitted using the DICOM protocol. The DicomImportService accepts all Application Entity Titles. The configuration element for the DicomImportService is:
 +
<pre>
 +
        <DicomImportService
 +
            name="DicomImportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomImportService"
 +
            root="root-directory"
 +
            port="port number"
 +
            calledAETTag="00097770"
 +
            callingAETTag="00097772"
 +
            connectionIPTag="00097774"
 +
            timeTag="00097776"
 +
            throttle="0"
 +
            logConnections="no"
 +
            logDuplicates="no"
 +
            suppressDuplicates="no" >
  
ClinicalTrialProcessor requires Java 1.5 JDK and the JAI ImageIO Tools.
+
            <accept ip="..."/>
 +
            <reject ip="..."/>
 +
 
 +
            <accept calledAET="..."/>
 +
            <reject calledAET="..."/>
 +
 
 +
            <accept callingAET="..."/>
 +
            <reject callingAET="..."/>
 +
 
 +
            <accept sopClass="..."/>
 +
 
 +
        </DicomImportService>
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ImportService for internal storage and queuing.
 +
*<b>port</b> is the port on which the ImportService listens for connections.
 +
*<b>calledAETTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the AE Title which was used by the sender to specify the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the AE Title of the receiver in the received DICOM object.
 +
*<b>callingAETTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the AE Title which was used by the sender to identify itself in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the AE Title of the sender in the received DICOM object.
 +
*<b>connectionIPTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the IP address of the sender in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the IP address of the sender in the received DICOM object.
 +
*<b>timeTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the system time when the object is received. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the system time in the received DICOM object. (The system time is the time in milliseconds since January 1, 1970.)
 +
*<b>throttle</b> sets the time delay (in milliseconds) before sending a response to the SCP on each transmission. The default is 0 milliseconds.
 +
*<b>logConnections</b> specifies whether to make a log entry for each object that is received. The log entry includes the SOPInstanceUID, the IP address of the sender in the association, and the calledAET and CallingAET. The values <b>yes</b> and <b>all</b> cause all connections to be logged. The value <b>rejected</b> causes only rejected connections to be logged. The default is <b>no</b>, meaning that no connections are logged. This capability is intended primarily for configuration debugging.
 +
*<b>logDuplicates</b> specifies whether to make a log entry for each object whose SOPInstanceUID matches that of any other object in the last 20 that have been received. This capability is intended primarily for configuration debugging.
 +
*<b>suppressDuplicates</b> specifies whether to ignore objects which have the same SOPInstanceUID as any object in the last 10 objects received. Values are <b>yes</b> and <b>no</b>. The default is <b>no</b>. This capability is intended primarily for configuration debugging.
 +
*The <b>accept</b> child elements can be used to specify white lists of values determining which connections are to be accepted. One <b>accept</b> child element must appear for each value in the white list. If the white list is empty, the white list is not used to filter connections. There are four white lists, one each for client IP addresses (ip), SCU application entity titles (callingAET), SCP application entity titles (calledAET), and SOP Classes (sopClass). (Note: SOP Classes can be specified as either the UID value or the dcm4che name. For example, both "1.2.840.10008.5.1.4.1.1.4" and "MRImageStorage" are accepted. Care should be taken when specifying SOP Class names, as unknown names are ignored.)
 +
*The <b>reject</b> child elements can be used to specify black lists of values determining which connections are to be rejected. One <b>reject</b> child element must appear for each value in the black list. If the black list is empty, the black list is not used to filter connections.  There are three black lists, one each for client IP addresses (ip), SCU application entity titles (callingAET), and SCP application entity titles (calledAET).
 +
*For a connection to be accepted, the IP address of the remote client must be accepted by the white list and and not rejected by the black list.
 +
 
 +
=====DicomSTOWRSImportService=====
 +
The DicomSTOWRSImportService listens on a defined port for HTTP or HTTPS connections from clients and receives files transmitted using the DICOM STOW-RS protocol. The configuration element for the DicomSTOWRSImportService  is:
 +
<pre>
 +
        <HttpImportService
 +
            name="DicomSTOWRSImportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomSTOWRSImportService"
 +
            root="root-directory"
 +
            port="7777"
 +
            ssl="yes"
 +
            requireAuthentication="no"
 +
            logConnections="no"
 +
            logDuplicates="no"
 +
            quarantine="quarantine-directory" >
 +
 
 +
            <accept ip="..."/>
 +
            <reject ip="..."/>
 +
 
 +
        </HttpImportService>   
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ImportService for internal storage and queuing.
 +
*<b>port</b> is the port on which the ImportService listens for connections.
 +
*<b>ssl</b> determines whether the port uses secure sockets layer (<b>yes</b>) or unencrypted http (<b>no</b>). The default is <b>no</b>.
 +
*<b>requireAuthentication</b> determines whether transmissions are required to have headers which identify a user which has the <b>import</b> privilege. The allowed values are <b>yes</b>) and <b>no</b>. The default is <b>no</b>.
 +
*<b>logConnections</b> specifies whether to make a log entry for each object that is received. The log entry includes only the IP address of the sender. The values <b>yes</b> and <b>all</b> cause all connections to be logged. The value <b>rejected</b> causes only rejected connections to be logged. The default is <b>no</b>, meaning that no connections are logged. This capability is intended primarily for configuration debugging.
 +
*<b>quarantine</b> is a directory in which the ImportService is to quarantine objects that it receives but is configured not to accept.
 +
*The <b>accept</b> child element can be used to specify a white list of IP addresses determining which connections are to be accepted. One <b>accept</b> child element must appear for each IP address in the white list. If the white list is empty, the white list is not used to filter connections.
 +
*The <b>reject</b> child element can be used to specify black lists of IP addresses determining which connections are to be rejected. One <b>reject</b> child element must appear for each IP address in the black list. If the black list is empty, the black list is not used to filter connections.
 +
*For a connection to be accepted, the IP address of the remote client must be accepted by the white list and and not rejected by the black list.
 +
 
 +
Notes:
 +
*The <b>accept</b> and <b>reject</b> child elements support the <b>regex</b> attribute in place of the <b>ip</b> attribute, allowing a range of IP addresses to be matched. The <b>regex</b> attribute must contain a regular expression. For example, to match any IP address in  the <tt>10.10.*.*</tt> range, the <b>accept</b> element would be coded:
 +
 
 +
<pre>
 +
    <accept regex="10\.10\.[0-9]{1,3}\.[0-9]{1,3}"/>
 +
</pre>
 +
 
 +
=====DirectoryImportService=====
 +
The DirectoryImportService watches a directory and imports any files it finds in it. After the files are passed down the pipeline, they are deleted from the import directory. The purpose of this ImportService is to allow manual input of objects to the pipeline. The configuration element for the DirectoryImportService is:
 +
<pre>
 +
        <DirectoryImportService
 +
            name="DirectoryImportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DirectoryImportService"
 +
            root="root-directory"
 +
            import="import-directory"
 +
            interval="20000"
 +
            fsName="..."
 +
            fsNameTag=""
 +
            filePathTag=""
 +
            fileNameTag=""
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            acceptFileObjects="yes"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ImportService to manage imported files.
 +
*<b>import</b> is the directory monitored by the ImportService for files to import.
 +
*<b>interval</b> is the length of time in milliseconds between polls of the import directory. The default is 20000. If the value is less than 1000, a value of used.
 +
*<b>fsName</b> is the name of the FileSystem to be used by a FileStorageService that receives this object.
 +
*<b>fsNameTag</b> is the name of a DICOM element, specified in hex with no comma separating the group and element numbers, in which to store the value of the fsName attribute.
 +
*<b>filePathTag</b> is the name of a DICOM element, specified as a DICOM keyword or in hex, in which to store the path at which the file was found in the archive. The path starts with the name of the import directory and ends at the name of the directory that contained the file. It does not include the name of the file. The '/' path separator character is used on all platforms.
 +
*<b>fileNameTag</b> is the name of a DICOM element, specified as a DICOM keyword or in hex, in which to store the name of the file that was found in the import directory.
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be accepted.
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be accepted.
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be accepted.
 +
*<b>acceptFileObjects</b> determines whether FileObjects are to be accepted.
 +
*<b>quarantine</b> is a directory in which the ImportService is to quarantine objects that it has handled (whether they were accepted for importing or not).
 +
 
 +
Note: For DicomObjects, if both the <b>fsName</b> and <b>fsNameTag</b> attributes are specified, the stage places the value of the fsName attribute in the element identified by fsNameTag. This allows a DirectoryImportService to mimic the behavior of a DicomImportService which stores the Called AE Title or Calling AE Title in an element for later use by a FileStorageService to assign the object to a FileSystem.
 +
 
 +
Note: When inserting the fsName value into the fsNameTag element in the DicomObject, there is a special feature. If the value of the fsName attribute is <b>@filename</b>, then the name of the file is inserted into the target element. If the filename ends in ".dcm", that string is removed from the end of the name before the name is inserted into the fsNameTag element.
 +
 
 +
=====ArchiveImportService=====
 +
The ArchiveImportService walks the directory tree of a static archive and imports the files it finds. The files are copied from the archive and placed in a separate directory before being passed down the pipeline. When the files have been processed, they are deleted from the directory to which they were copied, but they remain in the archive. The purpose of this ImportService is to allow a complete archive to be processed. The ImportService checkpoints itself as it runs, and restarts where it left off if the program is stopped and restarted. The checkpoint is stored in a file called <b><tt>checkpoint.bin</tt></b> in the stage's root directory. To restart the stage from the tree root, the checkpoint file must be deleted and CTP must be restarted.
 +
 
 +
The configuration element for the ArchiveImportService is:
 +
<pre>
 +
        <ArchiveImportService
 +
            name="ArchiveImportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.ArchiveImportService"
 +
            root="root-directory"
 +
            treeRoot="archive-root-directory"
 +
            expandTARs="no"
 +
            minAge="5000"
 +
            fsName="..."
 +
            fsNameTag=""
 +
            filePathTag=""
 +
            fileNameTag=""
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            acceptFileObjects="yes"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ImportService to manage imported files.
 +
*<b>treeRoot</b> is the root directory of the archive.
 +
*<b>expandTARs</b> determines whether the ImportService is to expand any TAR files it finds in the archive as they are imported. For archives containing TAR files encapsulating DICOM images, this attribute should be set to <b>yes</b>; otherwise, the TAR files will be treated as FileObjects containing no identifiers which would allow them to be connected to other objects.
 +
*<b>minAge</b> is the minimum age (in milliseconds) for files which are imported from the root directory. The default value is <b>5000</b>; the minimum accepted value is <b>1000</b>. The purpose of this attribute is to ensure that files are completely stored in the root directory before being imported.
 +
*<b>fsName</b> is the name of the FileSystem to be used by a FileStorageService that receives this object.
 +
*<b>fsNameTag</b> is the name of a DICOM element, specified in hex with no comma separating the group and element numbers, in which to store the value of the fsName attribute.
 +
*<b>filePathTag</b> is the name of a DICOM element, specified as a DICOM keyword or in hex, in which to store the path at which the file was found in the archive. The path starts with the name of the treeRoot directory and ends at the name of the directory that contained the file. It does not include the name of the file. The '/' path separator character is used on all platforms.
 +
*<b>fileNameTag</b> is the name of a DICOM element, specified as a DICOM keyword or in hex, in which to store the name of the file that was found in the archive.
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be accepted.
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be accepted.
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be accepted.
 +
*<b>acceptFileObjects</b> determines whether FileObjects are to be accepted.
 +
*<b>quarantine</b> is a directory in which the ImportService is to quarantine objects that it has handled (whether they were accepted for importing or not).
 +
 
 +
Note: For DicomObjects, if both the <b>fsName</b> and <b>fsNameTag</b> attributes are specified, the stage places the value of the fsName attribute in the element identified by fsNameTag. This allows an ArchiveImportService to mimic the behavior of a DicomImportService which stores the Called AE Title or Calling AE Title in an element for later use by a FileStorageService to assign the object to a FileSystem.
 +
 
 +
====Processors====
 +
=====ObjectLogger=====
 +
The ObjectLogger is a processor stage that logs the passage of objects as they flow past. Objects are passed on unmodified. The log entries are made in the system log and can be viewed in the log viewer servlet on the main admin server page. This stage is intended for initial configuration testing for a new trial; it creates one entry in the system log for each object it receives (unless the <b>interval</b> attribute is supplied with a value greater than one). Logging every object in a large production system can produce an unwieldy log file. The configuration element for the ObjectLogger is:
 +
<pre>
 +
        <ObjectLogger
 +
            name="ObjectLogger"
 +
            class="org.rsna.ctp.stdstages.ObjectLogger"
 +
            interval="1"
 +
            verbose="yes" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>interval</b> is the interval between objects to log. The default is <b>1</b>. This attribute can be used to measure performance without imposing a large load on the log file by setting the value to, for example, <b>1000</b>.
 +
*<b>verbose</b> increases the amount of information logged.
 +
 
 +
=====ObjectCache=====
 +
The ObjectCache is a processor stage that caches the current object as it flows past. Objects are passed on unmodified. The cached object is saved as a separate file to capture the object before it is changed by downstream processors. This stage is intended for use in conjunction with an audit stage that logs changes to objects or for special uses of the DirectoryExportService stage in the RSNA Image Sharing project. To make the cached object available to subsequent stages, the <b>id</b> attribute is mandatory. The configuration element for the ObjectCache is:
 +
<pre>
 +
        <ObjectCache
 +
            name="ObjectCache"
 +
            class="org.rsna.ctp.stdstages.ObjectCache"
 +
            id="stage ID"
 +
            root="root-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ObjectCache for temporary storage.
 +
 
 +
=====DicomAuditLogger=====
 +
The DicomAuditLogger is a processor stage that makes entries in an AuditLog plugin for DicomObjects. Objects are passed on unmodified. The AuditLog entries contain the values of specified elements in the DicomObject and optionally the corresponding elements in a DicomObject contained in the specified ObjectCache stage. The configuration element for the DicomAuditLogger is:
 +
<pre>
 +
        <DicomAuditLogger
 +
            name="DicomAuditLogger"
 +
            class="org.rsna.ctp.stdstages.DicomAuditLogger"
 +
            root="roots/DicomAuditLogger"
 +
            auditLogID="AuditLog"
 +
            auditLogTags="PatientID;PatientName;SOPInstanceUID"
 +
            cacheID="ObjectCache"
 +
            level="instance" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>root</b> is a directory for use by the DicomAuditLogger for temporary storage.
 +
*<b>auditLogID</b> is the ID of an AuditLog plugin in which entries are to be made.
 +
*<b>auditLogTags</b> is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as dcm4che element names or in standard DICOM notation, e.g. (0008,0050).
 +
*<b>cacheID</b> is the ID of an ObjectCache stage.
 +
*<b>level</b> specifies granularity of the log. Three levels are supported:
 +
:* <b><tt>patient</tt></b>: one entry for each unique PatientID processed by the stage
 +
:* <b><tt>study</tt></b>: one entry for each unique StudyInstanceUID processed by the stage
 +
:* <b><tt>instance</tt></b>: one entry for each unique SOPInstanceUID processed by the stage
 +
 
 +
Notes:
 +
# If the cacheID attribute is not specified, or if it does not point to an ObjectCache stage, the AuditLog entries contain only the values of the DicomObject being processed.
 +
# If the cacheID attribute points to an ObjectCache stage, and that stage can supply a DicomObject, the AuditLog entry contains the values of both the DicomObject being processed and the cached object.
 +
# By caching an object before anonymization and putting a DicomAuditLogger after the anonymizer, you can create AuditLog entries with the PHI and anonymized values. (Note that the AuditLog is visible only to an admin user.)
 +
 
 +
=====MemoryMonitor=====
 +
The MemoryMonitor is a processor stage that provides a way to force garbage collection and/or to log the current memory in use by CTP. Objects are passed on unmodified. The log entries are made in the system log and can be viewed in the log viewer servlet on the main admin server page. This stage is intended for initial configuration testing for a new trial; it creates one entry in the system log for each object it receives (unless the <b>interval</b> attribute is supplied with a value greater than one). Logging every object in a large production system can produce an unwieldy log file. The configuration element for the MemoryMonitor is:
 +
<pre>
 +
        <MemoryMonitor
 +
            name="stage name"
 +
            class="org.rsna.ctp.stdstages.MemoryMonitor"
 +
            interval="1"
 +
            collectGarbage="yes"
 +
            logMemoryInUse="yes" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>interval</b> is the interval between objects to log. The default is <b>1</b>.
 +
*<b>collectGarbage</b> determines whether the stage is to force the garbage collector to run. The allowed values are <b>yes</b> and <b>no</b>. The default is <b>yes</b>.
 +
*<b>logMemoryInUse</b> determines whether the stage is to log the current amount of heap space in use. If garbage collection is enabled, the value logged is the memory in use after garbage collection is complete. The allowed values are <b>yes</b> and <b>no</b>. The default is <b>yes</b>.
 +
 
 +
=====PerformanceLogger=====
 +
The PerformanceLogger is a processor stage that logs the elapsed time taken by each stage in the pipeline during the processing of the current object. Objects are passed on unmodified. The log entries are made in the system log and can be viewed in the log viewer servlet on the main admin server page. This stage is intended for initial configuration testing for a new trial. The configuration element for the PerformanceLogger is:
 +
<pre>
 +
        <PerformanceLogger
 +
            name="stage name"
 +
            class="org.rsna.ctp.stdstages.PerformanceLogger"
 +
            interval="1" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>interval</b> is the interval between objects to log. The default is <b>1</b>. This attribute can be used to measure performance without imposing a large load on the log file by setting the value to, for example, <b>1000</b>.
 +
 
 +
=====DicomFilter=====
 +
The DicomFilter is a processor stage that interrogates a DicomObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If a DicomObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (XmlObject, ZipObject, FileObject) are passed on unmodified. The configuration element for the DicomFilter is:
 +
<pre>
 +
        <DicomFilter
 +
            name="DicomFilter"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomFilter"
 +
            root="root-directory"
 +
            script="scripts/dicom-filter.script"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the DicomFilter for temporary storage.
 +
*<b>script</b> specifies the path to the script for the DicomFilter.
 +
*<b>quarantine</b> is a directory in which the DicomFilter is to quarantine objects that do not meet the criteria specified in the script file.
 +
 
 +
For information on specifying acceptance criteria in the script, see [[The CTP DICOM Filter]].
 +
 
 +
=====XmlFilter=====
 +
The XmlFilter is a processor stage that interrogates an XmlObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If an XmlObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (DicomObject, ZipObject, FileObject) are passed on unmodified. The configuration element for the XmlFilter is:
 +
<pre>
 +
        <XmlFilter
 +
            name="XmlFilter"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.XmlFilter"
 +
            root="root-directory"
 +
            script="scripts/xml-filter.script"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the XmlFilter for temporary storage.
 +
*<b>script</b> specifies the path to the script for the XmlFilter.
 +
*<b>quarantine</b> is a directory in which the XmlFilter is to quarantine objects that do not meet the criteria specified in the script file.
 +
 
 +
For information on specifying acceptance criteria in the script, see [[The CTP XML and Zip Filters]].
 +
 
 +
=====ZipFilter=====
 +
The ZipFilter is a processor stage that interrogates the manifest of a ZipObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If a ZipObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (DicomObject, XmlObject, FileObject) are passed on unmodified. The configuration element for the XmlFilter is:
 +
<pre>
 +
        <ZipFilter
 +
            name="ZipFilter"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.ZipFilter"
 +
            root="root-directory"
 +
            script="scripts/zip-filter.script"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ZipFilter for temporary storage.
 +
*<b>script</b> specifies the path to the script for the XmlFilter.
 +
*<b>quarantine</b> is a directory in which the ZipFilter is to quarantine objects that do not meet the criteria specified in the script file.
 +
 
 +
For information on specifying acceptance criteria in the script, see [[The CTP XML and Zip Filters]].
 +
 
 +
=====IDMap=====
 +
The IDMap is a processor stage that constructs map tables for UID elements, AccessionNumber elements, and PatientID elements. The map tables contain the original values and the replacement values created by the first downstream anonymizer. These tables can be accessed by administrators using the IDMap servlet. The configuration element for the IDMap is:
 +
<pre>
 +
        <IDMap
 +
            name="IDMap"
 +
            class="org.rsna.ctp.stdstages.IDMap"
 +
            root="root-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>root</b> is a directory for use by the IDMap for permanent storage of the map tables.
 +
 
 +
=====ObjectTracker=====
 +
The ObjectTracker is a processor stage that tracks objects by date, PatientID, StudyInstanceUID, SeriesInstanceUID, and SOPInstanceUID. The values tracked are the ones in the objects at the time they arrive at the stage, so if they occur before anonymization, they contain PHI. The tracking tables can be accessed by administrators using the ObjectTracker servlet. The configuration element for the IDTracker is:
 +
<pre>
 +
        <ObjectTracker
 +
            name="ObjectTracker"
 +
            class="org.rsna.ctp.stdstages.ObjectTracker"
 +
            root="root-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>root</b> is a directory for use by the IDTracker for permanent storage of its tracking tables.
 +
 
 +
=====DatabaseVerifier=====
 +
The DatabaseVerifier is a processor stage that tracks objects which have been passed to an external database. The stage periodically polls the DatabaseExportService of the external database and maintains its own internal database indicating the status of the objects. The DBVerifierServlet provides a browser interface to the internal database. There can be no anonymizer stages (either DicomAnonymizers or DicomPixelAnonymizers) between the DatabaseVerifier and the DatabaseExportService. (The reason is that the DatabaseVerifier indexes objects by SOPInstanceUID, and it determines that the object in the remote database matches the one seen earlier by the DatabaseVerifier stage by comparing the hashes of the objects' binary contents.) The configuration element for the DatabaseVerifier is:
 +
<pre>
 +
        <DatabaseVerifier
 +
            name="DatabaseVerifier"
 +
            class="org.rsna.ctp.stdstages.DatabaseVerifier"
 +
            root="root-directory"
 +
            url="http:ip:port"
 +
            username=""
 +
            password=""
 +
            interval="10000"
 +
            maxAge="0" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>root</b> is a directory for use by the DatabaseVerifier for permanent storage of its internal database.
 +
*<b>url</b> specifies the URL of the verification service of the external database's DatabaseExportService. If <b>ssl</b> is enabled on that verification service, the <b>url</b> attribute must start with <tt><b>https://</b></tt>. If this attribute is missing, no verication is performed, although the internal database is still maintained.
 +
*<b>username</b> specifies the username credential for inclusion in verification requests. If this attribute is missing or blank, credentials are not supplied in verification requests.
 +
*<b>password</b> specifies the password credential for inclusion in verification requests. If this attribute is missing or blank, credentials are not supplied in verification requests.
 +
*<b>interval</b> specifies time in milliseconds between queries of the external database's DatabaseExportService. If this attribute is missing or blank, the interval is 10 seconds (10,000 msec).
 +
*<b>maxAge</b> specifies the maximum time in days that an unverified object is allowed to remain in the unverified queue before the DatabaseVerifier removes it and stops trying to verify it with the external database's DatabaseExportService. If the attribute is missing or zero, objects remain in the queue forever until they are verified.
 +
 
 +
=====DicomDecompressor=====
 +
The DicomDecompressor is a processor stage that converts DicomObjects containing encapsulated pixel data into DicomObjects with the Explicit VR Little Endian (EVRLE) transfer syntax. It passes all other object types unmodified. The DicomDecompressor can optionally be configured with a script file which is used like a DicomFilter to select objects for processing. If a script file is either not configured or absent, all objects are processed. The configuration element for the DicomAnonymizer is:
 +
<pre>
 +
        <DicomDecompressor
 +
            name="DicomDecompressor"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomDecompressor"
 +
            root="root-directory"
 +
            skipJPEGBaseline="no"
 +
            script="scripts/dicom-decompressor.script"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the DicomDecompressor for temporary storage.
 +
*<b>skipJPEGBaseline</b> specifies whether to skip objects that are encoded in JPEGBaseline transfer syntax. Allowed values are <b>yes</b> and <b>no</b>. The default is <b>no</b>.
 +
*<b>script</b> specifies the path to a script which can be used to select objects for decompression.
 +
*<b>quarantine</b> is a directory in which the stage is to quarantine objects.
 +
 
 +
Notes:
 +
# The skipJPEGBaseline attribute is just a convenience. Skipping JPEGBaseline objects can also be accomplished by including a script like the following:
 +
:::<tt>!TransferSyntaxUID.equals("1.2.840.10008.1.2.4.50")</tt>
 +
 
 +
=====DicomPlanarConfigurationConverter=====
 +
The DicomPlanarConfigurationConverter is a processor stage that converts DicomObjects from PlanarConfiguration 1 to PlanarConfiguration 0. It passes all other object types unmodified. The configuration element for the DicomPlanarConfigurationConverter is:
 +
<pre>
 +
        <DicomPlanarConfigurationConverter
 +
            name="DicomPlanarConfigurationConverter "
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomPlanarConfigurationConverter"
 +
            root="root-directory"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the DicomPlanarConfigurationConverter for temporary storage.
 +
*<b>quarantine</b> is a directory in which the stage is to quarantine objects.
 +
 
 +
=====DicomPaletteImageConverter=====
 +
The DicomPaletteImageConverter is a processor stage that converts DicomObjects from PhotometricInterpretation PALETTE COLOR to RGB. It passes all other object types unmodified. The configuration element for the DicomPaletteImageConverter is:
 +
<pre>
 +
        <DicomPaletteImageConverter
 +
            name="DicomPaletteImageConverter"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomPaletteImageConverter"
 +
            root="root-directory"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the DicomPaletteImageConverter for temporary storage.
 +
*<b>quarantine</b> is a directory in which the stage is to quarantine objects.
 +
 
 +
=====DicomTranscoder=====
 +
The DicomTranscoder is a processor stage that converts DicomObjects to a specified transfer syntax. It passes all other object types unmodified. The DicomTranscoder can optionally be configured with a script file which is used like a DicomFilter to select objects for processing. If a script file is either not configured or absent, all objects are processed.
 +
 
 +
An example of the use of this stage is a pipeline that decompresses multiframe ultrasound images, blanks regions of the frames containing PHI, and then recompresses the images to save space. Such a pipeline might have these stages in sequence:
 +
* DicomDecompressor
 +
* DicomPixelAnonymizer
 +
* DicomTranscoder
 +
 
 +
The configuration element for the DicomTranscoder is:
 +
<pre>
 +
        <DicomTranscoder
 +
            name="DicomTranscoder"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomTranscoder"
 +
            tsuid="transfer syntax UID"
 +
            quality="100"
 +
            root="root-directory"
 +
            skipJPEGBaseline="no"
 +
            script="scripts/dicom-transcoder.script"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>tsuid</b> is the DICOM transfer syntax UID of the processed DicomObject. These transfer syntaxes have been tested successfully:
 +
:<b><tt>tsuid="1.2.840.10008.1.2.1"</tt></b> (ExplicitVRLittleEndian)
 +
:<b><tt>tsuid="1.2.840.10008.1.2.4.70"</tt></b> (JPEGLossLess)
 +
:<b><tt>tsuid="1.2.840.10008.1.2.4.80"</tt></b> (JPEGLSLossLess)
 +
:<b><tt>tsuid="1.2.840.10008.1.2.4.90"</tt></b> (JPEG2000LossLess)
 +
*<b>quality</b> is an integer from 1 to 100 to indicate the desired quality of the processed DicomObject. This attribute is ignored in the lossless transfer syntaxes.
 +
*<b>skipJPEGBaseline</b> specifies whether to skip objects that are encoded in JPEGBaseline transfer syntax. Allowed values are "yes" and "no". The default is "no".
 +
*<b>root</b> is a directory for use by the DicomTranscoder for temporary storage.
 +
*<b>script</b> specifies the path to a script which can be used to select objects for transcoding.
 +
*<b>quarantine</b> is a directory in which the  is to quarantine objects that generate quarantine calls during processing.
 +
 
 +
Notes:
 +
# If this stage is configured with the ExplicitVRLittleEndian transfer syntax, its function is similar to that of the DicomDecompressor stage. The difference is that although the output of the DicomDecompressor is EVRLE when it performs a conversion, it only converts DicomObjects that contain encapsulated pixel data, leaving all other objects unmodified, while the DicomTranscoder stage modifies all objects.
 +
# The <b>quality</b> attribute is not used in lossless compression transfer syntaxes.
 +
# The JPEGBaseline transfer syntax (<b><tt>tsuid="1.2.840.10008.1.2.4.50"</tt></b>) does not work correctly.
 +
# The skipJPEGBaseline attribute is just a convenience. Skipping JPEGBaseline objects can also be accomplished by including a script like the following:
 +
:::<tt>!TransferSyntaxUID.equals("1.2.840.10008.1.2.4.50")</tt>
 +
 
 +
=====LookupTableChecker=====
 +
The LookupTableChecker is a processor stage that checks all @lookup function calls in the first downstream DicomAnonymizer stage and quarantines any objects for which the function calls will fail. It adds a servlet with the same context as the <b><tt>id</tt></b> attribute, allowing an admin user to insert values into the lookup table. Once the lookup table has been updated, the quarantined objects can be re-queued and processed successfully. The configuration element for the LookupTableChecker is:
 +
<pre>
 +
        <LookupTableChecker
 +
            name="LookupTableChecker"
 +
            class="org.rsna.ctp.stdstages.LookupTableChecker"
 +
            root="root-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>root</b> is a directory for use by the LookupTableChecker for permanent storage of the map tables.
 +
 
 +
=====DicomAnonymizer=====
 +
The DicomAnonymizer is a processor stage that anonymizes DicomObjects and passes all other object types unmodified. The DicomAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the DicomAnonymizer is:
 +
<pre>
 +
        <DicomAnonymizer
 +
            name="DicomAnonymizer"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomAnonymizer"
 +
            root="root-directory"
 +
            lookupTable="scripts/lookup-table.properties"
 +
            script="scripts/dicom-anonymizer.script"
 +
            dicomScript="scripts/dicom-filter.script"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the DicomAnonymizer for temporary storage.
 +
*<b>lookupTable</b> specifies the path to the lookup table used by the DicomAnonymizer.
 +
*<b>script</b> specifies the path to the script for the DicomAnonymizer.
 +
*<b>dicomScript</b> specifies the path to an optional script file (in the same language as the DicomFilter), determining whether to anonymize the object.
 +
*<b>quarantine</b> is a directory in which the DicomAnonymizer is to quarantine objects that generate quarantine calls during processing.
 +
 
 +
Notes:
 +
# If the <b>lookupTable</b> attribute is missing, the <b>lookup</b> anonymizer function will result in the object being quarantined.
 +
# The <b>script</b> attribute specifies the instructions for modifying the individual elements in a DicomObject. If this attribute is missing, DicomObjects are not modified.
 +
# The <b>dicomScript</b> attribute specifies the instructions for determining whether to process the DicomObject. If this attribute is missing, all DicomObjects are accepted for processing. If the attribute is present, the computation must produce the value true for the DicomObject to be processed.
 +
# If the <b>@integer</b> function is used in the DicomAnonymizer script, the starting point can be reset by deleting the <b><tt>integers.db</tt></b> and <b><tt>integers.lg</tt></b> files from the directory specified in the <b>root</b> attribute. This will cause new integers to be assigned starting at <b>1</b>. Deleting the files must be done while CTP is not running.
 +
 
 +
=====DicomCorrector=====
 +
The DicomCorrector is a processor stage that tries to fix problems in certain elements in DicomObjects that may have been coded incorrectly. Specifically, it recursively walks the dataset tree (including SQ item datasets) and finds non-private elements with VR=UN. For such elements defined in the standard to have the VRs FD, FL, or CS, it replaces the elements with the correct VR and VM for the data contained in the element. The DicomCorrector passes non-DicomObjects without modification. The configuration element for the DicomCorrector is:
 +
<pre>
 +
        <DicomCorrector
 +
            name="DicomCorrector"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomCorrector"
 +
            root="root-directory"
 +
            dicomScript="scripts/dicom-filter.script"
 +
            quarantine="quarantine-directory" />
 +
            quarantineUncorrectedMismatches="no" />
 +
            logUncorrectedMismatches="no" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the DicomAnonymizer for temporary storage.
 +
*<b>dicomScript</b> specifies the path to an optional script file (in the same language as the DicomFilter), determining whether to process the object.
 +
*<b>quarantine</b> is a directory in which the DicomCorrector is to quarantine objects that generate quarantine calls during processing.
 +
*<b>quarantineUncorrectedMismatches</b> specifies whether to quarantine objects in which uncorrectable VR mismatches are detected. Values are "yes" and "no". The default is "no".
 +
*<b>logUncorrectedMismatches</b> specifies whether to make a log entry for each uncorrectable VR mismatch that is detected. Values are "yes" and "no". The default is "no".
 +
 
 +
Notes:
 +
# The <b>dicomScript</b> specifies the instructions for determining whether to process the DicomObject. If this attribute is missing, all DicomObjects are accepted for processing. If the attribute is present, the computation must produce the value true for the DicomObject to be processed.
 +
# The computation specified in the <b>dicomScript</b> is performed on the unmodified dataset, so references to elements that may be corrected may produce unexpected results.
 +
 
 +
=====DicomPixelAnonymizer=====
 +
The DicomPixelAnonymizer is a processor stage that blanks regions in DicomObjects which are images and passes all other object types unmodified. The DicomPixelAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the DicomAnonymizer is:
 +
<pre>
 +
        <DicomPixelAnonymizer
 +
            name="DicomPixelAnonymizer"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomPixelAnonymizer"
 +
            root="root-directory"
 +
            log="no"
 +
            script="scripts/dicom-pixel-anonymizer.script"
 +
            setBurnedInAnnotation="no"
 +
            test="no"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the DicomPixelAnonymizer for temporary storage.
 +
*<b>log</b> determines whether the signature matched by the DicomObject is to be listed in the log. Values are "yes" and "no". The default is "no". This feature is intended for use while debugging the script.
 +
*<b>script</b> specifies the path to the script for the DicomPixelAnonymizer.
 +
*<b>setBurnedInAnnotation</b> specifies whether to set the BurnedInAnnotation eledment (0028,0301) to "NO" if as matching signature is found. Values are "yes" and "no". The default is "no". If the value is "no", the element is left unmodified.
 +
*<b>test</b> specifies whether to set the color of blanked regions to black or gray. Values are "yes" and "no". The default is "no". If the value is "no", the regions are set to black. The purpose of this attribute is to make it easy to see what regions are being modified while testing a new script.
 +
*<b>quarantine</b> is a directory in which the DicomPixelAnonymizer is to quarantine objects that generate quarantine calls during processing.
 +
 
 +
For information on the script language, see [[The CTP DICOM Pixel Anonymizer]].
 +
 
 +
<b>Important notes: </b>
 +
* The DicomPixelAnonymizer can process all objects that do not contain encapsulated pixel data.
 +
* The DicomPixel anonymizer can also process objects that contain encapsulated pixel data with the JPEGBaseline transfer syntax (1.2.840.10008.1.2.4.50).
 +
* To allow objects containing encapsulated pixel data with other transfer syntaxes to be processed, include a DicomDecompressor stage before the DicomPixelAnonymizer. When including the DicomDecompressor stage, setting its skipJPEGBaseline attribute to "yes" will preserve the compression of JPEGBaseline objects.
 +
* The DicomPixelAnonymizer does not remove PHI from the non-pixel data in an object. To de-identify that data, include a DicomAnonymizer stage in the pipeline.
 +
 
 +
=====XmlAnonymizer=====
 +
The XmlAnonymizer is a processor stage that anonymizes XmlObjects and passes all other object types unmodified. The XmlAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the XmlAnonymizer is:
 +
<pre>
 +
        <XmlAnonymizer
 +
            name="XmlAnonymizer"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.XmlAnonymizer"
 +
            root="root-directory"
 +
            script="scripts/xml-anonymizer.script"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the Anonymizer for temporary storage.
 +
*<b>script</b> specifies the path to the script for the XmlAnonymizer.
 +
*<b>quarantine</b> is a directory in which the XmlAnonymizer is to quarantine objects that generate quarantine calls during processing.
 +
 
 +
=====ZipAnonymizer=====
 +
The ZipAnonymizer is a processor stage that anonymizes ZipObjects and passes all other object types unmodified. The ZipAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the ZipAnonymizer is:
 +
<pre>
 +
        <ZipAnonymizer
 +
            name="ZipAnonymizer"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.ZipAnonymizer"
 +
            root="root-directory"
 +
            script="scripts/zip-anonymizer.script"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the Anonymizer for temporary storage.
 +
*<b>script</b> specifies the path to the script for the ZipAnonymizer.
 +
*<b>quarantine</b> is a directory in which the ZipAnonymizer is to quarantine objects that generate quarantine calls during processing.
 +
 
 +
=====EmailService=====
 +
The EmailService is a processor stage that sends emails when studies are received. An email is sent for each study, including the numbers of objects, series, and images in the study, as well as other information that is specified in the configuration element below. The configuration element for the EmailService is:
 +
<pre>
 +
        <EmailService
 +
            name="EmailService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.EmailService"
 +
            root="root-directory"
 +
            script="scripts/EmailService.script"
 +
            smtpServer="SMTP server URL"
 +
            username=""
 +
            password=""
 +
            to=""
 +
            from=""
 +
            cc=""
 +
            subject=""
 +
            includePatientName="no"
 +
            includePatientID="no"
 +
            includeModality="no"
 +
            includeStudyDate="no"
 +
            includeAccessionNumber="no"
 +
            logSentEmails="no"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the EmailService for temporary storage.
 +
*<b>script</b> specifies the path to the script for the EmailService. This script determines whether a DicomObject is to be considered by the stage.
 +
*<b>smtpServer</b> is the URL of the SMTP server to be used by the EmailService to send emails.
 +
*<b>username</b> is the username for authentication on the SMTP server. If blank, no authentication is done.
 +
*<b>password</b> is the password for authentication on the SMTP server. If the username is blank, the password is ignored.
 +
*<b>to</b> is the email address of the recipient of the emails. If multiple recipients are necessary, their addresses must be separated by commas.
 +
*<b>from</b> is the email address of the sender.
 +
*<b>cc</b> is the email address of the cc recipients. If multiple cc recipients are necessary, their addresses must be separated by commas.
 +
*<b>subject</b> is the text for the subject line. This can be used to identify the name of the trial.
 +
*<b>includePatientName</b> specifies whether to include the patient name in the email. Allowed values are <b>yes</b> and <b>no</b>. The default is <b>no</b>.
 +
*<b>includePatientID</b> specifies whether to include the patient ID in the email. Allowed values are <b>yes</b> and <b>no</b>. The default is <b>no</b>.
 +
*<b>includeModality</b> specifies whether to include the modality in the email. Allowed values are <b>yes</b> and <b>no</b>. The default is <b>no</b>.
 +
*<b>includeStudyDate</b> specifies whether to include the study date in the email. Allowed values are <b>yes</b> and <b>no</b>. The default is <b>no</b>.
 +
*<b>includeAccessionNumber</b> specifies whether to include the study accession number in the email. Allowed values are <b>yes</b> and <b>no</b>. The default is <b>no</b>.
 +
*<b>quarantine</b> is a directory in which the EmailService is to quarantine objects if necessary.
 +
 
 +
Notes:
 +
* The patient name, patient ID, modality, and study date values are those of the first image received for the study. These values may contain PHI if the EmailService stage occurs before the objects are anonymized, either at the source or in preceding stages in the pipeline.
 +
 
 +
====Storage Services====
 +
=====FileStorageService=====
 +
The FileStorageService stores objects in a file system. It automatically defines subdirectories beneath its root directory and populates them accordingly. The configuration element for the StorageService is:
 +
<pre>
 +
        <FileStorageService
 +
            name="FileStorageService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.FileStorageService"
 +
            root="D:/storage"
 +
            type="month"
 +
            timeDepth="0"
 +
            acceptDuplicateUIDs="yes"
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            acceptFileObjects="yes"
 +
            returnStoredFile="yes"
 +
            setWorldReadable="no"
 +
            setWorldWritable="no"
 +
            fsNameTag="00097770"
 +
            autoCreateUser="no"
 +
            port="85"
 +
            ssl="no"
 +
            requireAuthentication="no"
 +
            exportDirectory=""
 +
            quarantine="quarantine-directory" >
 +
 
 +
            <jpeg frame="first" wmax="10000" wmin="96" q="-1" />
 +
 
 +
        </FileStorageService>
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is the root directory of the storage tree.
 +
*<b>type</b> determines the structure of the storage tree. The allowed values are:
 +
**<b>year</b>: root/FSNAME/year/studyName, e.g. root/2008/123.456.789
 +
**<b>month</b>: root/FSNAME/year/month/studyName, e.g. root/2008/06/123.456.789
 +
**<b>week</b>: root/FSNAME/year/week/studyName, e.g. root/2008/36/123.456.789
 +
**<b>day</b>: root/FSNAME/year/day/studyName, e.g. root/2008/341/123.456.789
 +
**<b>none</b>: root/FSNAME/studyName, e.g. root/123.456.789
 +
::(FSNAME is the name of the file system to which the study belongs. See the <b>fsNameTag</b> attribute below for additional information.)
 +
*<b>timeDepth</b> specifies the length of time in days that studies are stored. The default value is <b>0</b>, which means forever. If timeDepth is greater than zero, studies older than that value are automatically removed from storage.
 +
*<b>acceptDuplicateUIDs</b> determines whether objects with duplicate UIDs are to be stored under separate names or if a newer duplicate object is to overwrite an older one. Values are "yes" and "no". The default is "yes".
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be stored.  Values are "yes" and "no". The default is "yes".
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be stored. Values are "yes" and "no". The default is "yes".
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be stored. Values are "yes" and "no". The default is "yes".
 +
*<b>acceptFileObjects</b> determines whether FileObjects are to be stored. Values are "yes" and "no". The default is "yes".
 +
*<b>returnStoredFile</b> specifies whether the original object or a new object pointing to the file in the storage system is to be returned for processing by subsequent stages. Values are "yes" and "no". The default is "yes".
 +
*<b>setWorldReadable</b> determines whether the FileStorageService makes all files and directories readable by all users. Values are "yes" and "no". The default is "no". This attribute should only be used if users or other programs are to be allowed to access files without using the FileStorageService web server. This feature is <b>not</b> recommended.
 +
*<b>setWorldWritable</b> determines whether the FileStorageService makes all files and directories writable by all users. Values are "yes" and "no". The default is "no". This attribute should only be used if users or other programs are to be allowed to write files into the FileStorageService without using the FileStorageService web server. This feature is <b>not</b> recommended.
 +
*<b>fsNameTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is used to specify the name of the root directory's child under which to store a received object. If the attribute is missing from the configuration or if the specified element is missing from the received object, or if the contents of the specified element are blank, the object is stored under the "__default" tree.
 +
*<b>autoCreateUser</b> determines whether the StorageService is to create a user for each new value of the element specified by the fsNameTag. The default is "no". The user is created with both the username and the password set to the value of the element specified by the fsNameTag.
 +
*<b>port</b> specifies the port on which a web server is to be started to provide access to the stored studies. If the attribute is missing, no web server is started for the FileStorageService.
 +
*<b>ssl</b> determines whether the web server uses SSL. Values are "yes" and "no". The default is "no".
 +
*<b>requireAuthentication</b> determines whether users are forced to log in to the web server. Values are "yes" and "no". The default is "no".
 +
*<b>exportDirectory</b> specifies a directory into which the web server can copy files when the a user with the admin privilege clicks a link on the Study List page. This feature can be used to allow studies to be cached in the FileStorageService and then manually released to the DirectoryImportService of another pipeline for subsequent processing and/or export.
 +
*<b>quarantine</b> is a directory in which the StorageService is to quarantine objects that cannot be stored.
 +
 
 +
For more information on the embedded web server in the FileStorageService, see [[The CTP FileStorageService Web Server]] and [[The CTP FileStorageService Access Mechanism]].
 +
 
 +
Notes:
 +
# Files are stored in a tree of directories with the root of the tree as defined in the root attribute of the configuration element.
 +
# Below the root are directories called FileSystems. A FileSystem may be defined by the value of an element in the object being stored by using the fsNameTag attribute. If no FileSystem is defined by the object, it is stored in the default FileSystem (<b>__default</b>).
 +
# Within a FileSystem, studies are grouped depending on the value of the type attribute. For example, if the type attribute has the value "month", studies are organized by year and month, e.g. "2007/09".
 +
# At the bottom of the hierarchy are directories organized by StudyInstanceUID, thus grouping all objects received for a specific study together in one directory.
 +
# Objects are stored with standard file extensions:
 +
#*.dcm for DicomObjects
 +
#*.xml for XmlObjects
 +
#*.zip for ZipObjects
 +
#*.md for FileObjects
 +
# Any object not containing a StudyInstanceUID or StudyUID is stored in the <b>bullpen</b> directory.
 +
# The <b>fsNameTag</b> attribute can be used to create storage trees based on element values like PatientID. It can also access elements in private groups, as might be done if the <b>calledAETTag</b> attribute of the DicomImportService is used to pass destination information. Similarly, the <b>callingAETTag</b> attribute of the DicomImportService could be used to pass source information, allowing separation of objects into FileSystems based on the sending system.
 +
# The <b>fsNameTag</b> attribute supports a list of element tags in the form <tt><b>fsNameTag=”00400275::00401001”</b></tt>. There is no limit to the number of tags in the sequence. All but the last tag must refer to SQ elements. This feature allows the file system name to be obtained from an element buried in an item dataset that may be several levels down from the root dataset. Only the first item dataset is searched at each level.
 +
 
 +
Notes on the <b>jpeg</b> child element:
 +
# The optional <b>jpeg</b> child element causes the FileStorageService to create one or more JPEG images for each DICOM image when it is stored.
 +
# The <b>jpeg</b> element should be included <b>only</b> when pre-computed JPEG images are required by an external application which directly references the disk drive containing the stored files (for example, the NBIA application).
 +
# When images are accessed through the FileStorageService web server's Storage Servlet or Ajax Servlet, they are produced dynamically, and <b>jpeg</b> elements are not required.
 +
# Multiple <b>jpeg</b> elements may appear if multiple images must be created (with different parameters) for each stored DICOM image.
 +
# The attributes specify the frame, width and quality parameters of the created image:
 +
#* The <b>frame</b> attribute which frame in multi-frame objects is to be saved. It has four allowed values: <b>first, middle, last, all</b>. The default is <b>first</b>. If <b>all</b> is selected and the StorageService supports saving all frames, then one JPEG image is created for each frame in the object. NOTE: The FileStorageService does not support the <b>frame</b> attribute and always behaves as if <b>frame="first"</b> is specified. The BasicFileStorageService fully supports the <b>frame</b> attribute.
 +
#* If the width of the parent DICOM image lies between the values of <b>wmax</b> and <b>wmin</b>, the width of the created image will be equal to the width of the parent.
 +
#*<b>wmax</b> specifies the maximum width of the created image. The default is 10000.
 +
#*<b>wmin</b> specifies the minimum width of the created image. The default is 96.
 +
#*The height of the created image is automatically computed to provide the same aspect ratio as the parent.
 +
#*<b>q</b> specifies the compression quality for the created image. Allowed values are <b>1</b> through <b>100</b>, with larger values producing better quality (and larger file sizes). The system default is triggered by specifying <b>-1</b>, which generally produces a good image.
 +
# A JPEG image file created in response to a <b>jpeg</b> child element is stored in the same directory as the parent DICOM image.
 +
# The filename of a created image is constructed from:
 +
#* the name of the parent file,
 +
#* a suffix in square brackets identifying the parameters used to create it,
 +
#* and a <b>.jpeg</b> extension.
 +
# The parameters appear in the order: <b>wmax</b>, <b>wmin</b>, <b>q</b>, and are separated by semicolons.
 +
# For example, a JPEG image created from <b>FO-29126.dcm</b> might have the name <b>FO-29126.dcm[400;96;-1].jpeg</b>.
 +
# If a 96-pixel wide thumbnail is required for an external application, the following <b>jpeg</b> child element could be specified:
 +
<pre>
 +
        <jpeg wmax="96" wmin="96" q="-1" />
 +
</pre>
 +
 
 +
=====BasicFileStorageService=====
 +
The BasicFileStorageService stores objects in a file system. It provides no organization of the files and no means of access to them. It is intended for use in situations where direct file access is provided through an external application like NBIA. The configuration element for the StorageService is:
 +
<pre>
 +
        <BasicFileStorageService
 +
            name="BasicFileStorageService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.BasicFileStorageService"
 +
            index="D:/storage"
 +
            root="D:/storage/root"
 +
            nLevels="3"
 +
            maxSize="200"
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            returnStoredFile="yes"
 +
            logDuplicates="no"
 +
            rejectDuplicates="no"
 +
            acceptClones="yes"
 +
            quarantine="quarantine-directory" >
 +
 
 +
            <jpeg frame="first" wmax="10000" wmin="96" q="-1" />
 +
 
 +
        </BasicFileStorageService>
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>index</b> is the directory in which the index is stored.
 +
*<b>root</b> is the root directory of the storage tree.
 +
*<b>nLevels</b> defines the depth of the storage tree. The default is 3.
 +
*<b>maxSize</b> defines the maximum number of files or directories in each node of the storage tree. The default is 200.
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be stored.  Values are "yes" and "no". The default is "yes".
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be stored. Values are "yes" and "no". The default is "yes".
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be stored. Values are "yes" and "no". The default is "yes".
 +
*<b>returnStoredFile</b> specifies whether the original object or a new object pointing to the file in the storage system is to be returned for processing by subsequent stages. Values are "yes" and "no". The default is "yes".
 +
*<b>logDuplicates</b> specifies whether an entry is to be made in the log whenever a file is received which has the same SOPInstanceUID as a file which has already been stored. Values are "yes" and "no". The default is "no".
 +
*<b>rejectDuplicates</b> specifies whether a file is to be quarantined (and not saved) if it has the same SOPInstanceUID as a file which has already been stored. Values are "yes" and "no". The default is "no". See <b>acceptClones</b> below.
 +
*<b>acceptClones</b> specifies whether a file is to be accepted even if it has the same SOPInstanceUID as a file which has already been stored, provided that it is identical to the previously stored file. Values are "yes" and "no". The default is "yes". See the special notes on this attribute below.
 +
*<b>quarantine</b> is a directory in which the StorageService is to quarantine objects that cannot be stored.
 +
 
 +
Notes:
 +
# Files are stored in a tree of directories with the root of the tree as defined in the root attribute of the configuration element.
 +
# The index directory must not appear under the root directory. A convenient approach is shown in the example above, where the root directory appears under the index directory.
 +
# The number of files which will be stored under any directory in the root is maxSize**(nLevels-1). For the default values of the nLevels and maxSize parameters (3 and 200), this is 40,000.
 +
# Directories are created at the top level (root) when existing directories are full. There is no maximum number of top-level directories; however, it is wise to consider the number of objects which are expected to be stored and to select values of nLevels and maxSize which would keep the number of top-level directories below 1000.
 +
# For storage requirements of 10 million files, the default values of nLevels and maxSize are fine.
 +
# For storage requirements of 10 billion files, nLevels="4" and maxSize="300" would work well.
 +
# Files are stored as leaves at the bottom of the tree.
 +
# No organization into related groups (e.g. by StudyInstanceUID) is provided.
 +
# Files are indexed by UID (e.g., SOPInstanceUID).
 +
# A duplicate object (e.g., one whose UID matches the UID of an object already stored) overwrites the stored object in the same place in the storage system (e.g., the same directory and the same filename), and any required <b>jpeg</b> images are recreated, overwriting the previously stored ones.
 +
# FileObjects, which do not contain UIDs, are not stored; they are simply passed to the next stage.
 +
# Files are stored with standard file extensions:
 +
#*.dcm for DicomObjects
 +
#*.xml for XmlObjects
 +
#*.zip for ZipObjects
 +
# See the section on the FileStorageService for a description of the <b>jpeg</b> child element.
 +
# If <b>jpeg</b> child elements appear, the files which they create are <b>not</b> counted against the maxSize parameter. Thus, if maxSize is 200 and two <b>jpeg</b> child elements appear, the bottom directories in the tree could contain 600 files. If <b>frame="all"</b> is specified and multi-frame objects are to be processed, this could result in many times that number. In situations where a given choice of maxSize could result in more than 1000 files in one directory, it is advisable to reduce maxSize and increase nLevels.
 +
 
 +
Notes on the use of the <b>logDuplicates</b>, <b>rejectDuplicates</b>, and <b>acceptClones</b>:
 +
* The default operation is to overwrite a stored file if another file is subsequently received with the same UID.
 +
* If it is desired to suppress the storage and subsequent processing of files that have the same UIDs as previously stored files, the <b>rejectDuplicates</b> attribute must be set to "yes".
 +
* If it is desired only to suppress the storage and subsequent processing of files that have the same UIDs but are not identical to the previously stored files, then the <b>acceptClones</b> attribute must be set to "no".
 +
* The operation of the <b>rejectDuplicates</b> and <b>acceptClones</b> attributes is not affected gy the settin gof the <b>logDuplicates</b> attribute.
 +
 
 +
=====DirectoryStorageService=====
 +
The DirectoryStorageService stores DicomObjects in a directory structure with no index. It optionally organizes the objects in subdirectories according to a list of element tags. The organization can be obtained either from the current object or from an object that had been cached in another stage. This StorageService is intended for use in situations where files are passed to an external application like the Edge Server in the RSNA Image Sharing Project. It can also be used to pass files to DirectoryImportService stages in other pipelines. The configuration element for the StorageService is:
 +
<pre>
 +
        <DirectoryStorageService
 +
            name="DirectoryStorageService"
 +
            id="stage ID"
 +
            dicomScript="scripts/dss.script"
 +
            cacheID="cache stage ID"
 +
            structure="dir1/dir2/dir3/..."
 +
            defaultString="UNKNOWN"
 +
            whitespaceReplacement="_"
 +
            class="org.rsna.ctp.stdstages.DirectoryStorageService"
 +
            root="D:/storage/root"
 +
            setStandardExtensions="no"
 +
            filenameTag=""
 +
            acceptDuplicates="yes"
 +
            returnStoredFile="yes"
 +
            quarantine="quarantine-directory" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>dicomScript</b> specifies the path to a script that examines the contents of a DicomObject and determines whether the object is to be accepted for storage. If the attribute is missing, all objects are accepted.
 +
*<b>cacheID</b> is the ID of an ObjectCache stage that can supply an object from which to obtain values to populate the directory names in the storage hierarchy. If this attribute is missing, the values are obtained from the current object.
 +
*<b>root</b> is the root directory of the storage tree.
 +
*<b>structure</b> is an optional sequence of directory names specifying the hierarchy of the storage tree. Directories in the sequence are separated by slashes. Each directory name in the sequence can consist of text and/or DICOM tags. If a directory name in the sequence includes one or more DICOM tags, the values of the corresponding elements in the current (or cached) DICOM object are substituted in the directory name for the tags. See the notes below for more details and examples of directory structures. If this attribute is missing, all files are stored in the root directory.
 +
*<b>defaultString</b> specifies a string used in place of a DICOM tag if the corresponding element is either missing or empty. If this attribute is missing, "UNKNOWN" is used.
 +
*<b>whitespaceReplacement</b> specifies a string used to replace whitespace, slash, or backslash characters in a directory name. If this attribute is missing, the underscore character is used.
 +
*<b>setStandardExtensions</b> specifies whether the stored files are to be assigned file extensions based on their file types (".dcm" for DicomObjects, ".xml" for XmlObjects, ".zip" for ZipObjects. and ".md" for all other object types). Values are "yes" and "no". The default is "no".
 +
*<b>filenameTag</b> specifies a DICOM element from which to obtain a filename to use in the storage of the file. If this attribute is missing or if it references an element that is not present (or is empty), the SOPInstanceUID is used for the filename.
 +
*<b>acceptDuplicates</b> specifies whether a file with the same name as an existing file is to overwrite the existing file or is to be saved with a different name. Values are "yes" and "no". The default is "yes", which means that all files are to be kept, creating new names when necessary.
 +
*<b>returnStoredFile</b> specifies whether the original object or a new object pointing to the file in the storage system is to be returned for processing by subsequent stages. Values are "yes" and "no". The default is "yes".
 +
*<b>quarantine</b> is a directory in which the StorageService is to quarantine objects that cannot be stored.
 +
 
 +
Notes:
 +
# The stored object's SOPInstanceUID is used as the file name.
 +
# No file extension is appended to the SOPInstanceUID when creating the file name unless setStandardExtensions is set to "yes".
 +
# In directory names, DICOM tags are hex integers, formatted in parentheses or square brackets. Tags may optionally include commas to separate the group and element numbers. Leading zeroes are good form but not required in group numbers. Leading zeroes are required in element numbers. Commas are removed before parsing tags, so <b><tt>(20,D)</tt></b> is not equivalent to <b><tt>(20,000D)</tt></b>.
 +
# This example shows tags for PatientID/AccessionNumber/StudyInstanceUID: <b><tt>(0010,0020)/[00080050]/(0020,000D)</tt></b>.
 +
# This example shows how text and multiple tags can be combined in a single directory name: <b><tt>(0010,0020)/(0020,000D) - [00080050]</tt></b>. In this case the PatientID is used as the top-level directory, with a subdirectory consisting of the StudyInstanceUID, a space, a hyphen, another space, and the AccessionNumber.
 +
# Whitespace replacement is performed on all the text of a directory name, after substitution of the DICOM element values for any tags in the name, so in the example in the previous note, the separator between the StudyInstanceUID and the AccessionNumber would actually appear in the directory name as "_-_".
 +
# DICOM tags in directory names can include references to elements in sequence element item datasets in the form <tt><b>(0040,0275)::(0040,1001)</b></tt>. There is no limit to the number of tags in the sequence. All but the last tag must refer to SQ elements. Only the first item dataset is searched at each level.
 +
 
 +
====Export Services====
 +
=====HttpExportService=====
 +
The HttpExportService queues objects and transmits them via HTTP with Content-Type <b>application/x-mirc</b>. The configuration element for the HttpExportService is:
 +
<pre>
 +
        <HttpExportService
 +
            name="HttpExportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.HttpExportService"
 +
            root="root-directory"
 +
            url="http://ipaddress:port/path"
 +
            zip="no"
 +
            sendDigestHeader="no"
 +
            username="username"
 +
            password="password"
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            acceptFileObjects="yes"
 +
            dicomScript="scripts/df.script"
 +
            xmlScript="scripts/xf.script"
 +
            zipScript="scripts/zf.script"
 +
            logDuplicates="no"
 +
            auditLogID="id"
 +
            auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber"
 +
            interval="5000" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
 +
*<b>url</b> specifies the destination system's URL.
 +
*<b>zip</b> determines whether files will be zipped before transmission (<b>yes</b>) or not (<b>no</b>). The default is <b>no</b>. This feature is intended for use with the HttpImportService.
 +
*<b>sendDigestHeader</b> determines whether a Digest header is to be included in the connection, allowing the destination to detect errors at the application level. (<b>yes</b> or <b>no</b>). The default is <b>no</b>.
 +
*<b>username</b> specifies the username credential for inclusion in the header during the transmission. This allows an HttpImportService to authenticate transmissions. If the username attribute is not present or has a whitespace value, no header is generated.
 +
*<b>password</b> specifies the password credential for inclusion in the header during the transmission.
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be exported.
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be exported.
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be exported.
 +
*<b>acceptFileObjects</b> determines whether FileObjects are to be exported.
 +
*<b>dicomScript</b> specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
 +
*<b>xmlScript</b> specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
 +
*<b>zipScript</b> specifies the path to a script which examines the contents of a ZipObject and determines whether the object is to be exported.
 +
*<b>logDuplicates</b> specifies whether to make a log entry for each object whose UID matches that of any other object in the last 20 that have been exported. This capability is intended primarily for configuration debugging.
 +
*<b>auditLogID</b> is the value of the <b><tt>id</tt></b> attribute of an AuditLog plugin in which to log exported DICOM objects. If this attribute is missing or if the value does not correspond to an AuditLog plugin, no audit logging is performed.
 +
*<b>auditLogTags</b> is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as <b><tt>dcm4che</tt></b> element names or in standard DICOM notation, e.g. <b><tt>(0008,0050)</tt></b>.
 +
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue.
 +
 
 +
Notes:
 +
#The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
 +
#The protocol part of the <b>url</b> can be <b>http</b> or <b>https</b>, the latter causing connections to be initiated using secure sockets layer.
 +
#If a proxy server is not in use, the proxy attributes must be omitted.
 +
#For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") <u>and</u> the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] and [[The CTP XML and Zip Filters]] for information about the script languages.
 +
 
 +
=====PolledHttpExportService=====
 +
The PolledHttpExportService queues objects and transmits them in the HTTP response stream of a received connection. Files are transmitted with Content-Type equal to <b>application/x-mirc</b>. This ExportService is designed to work in conjunction with the PollingHttpImportService to allow penetration of a firewall without having to open an inbound port, as described in [[#Security Issues | Security Issues]]. The configuration element for the Polled HttpExportService is:
 +
<pre>
 +
        <PolledHttpExportService
 +
            name="PolledHttpExportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.PolledHttpExportService"
 +
            root="root-directory"
 +
            port="listening-port"
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            acceptFileObjects="yes"
 +
            dicomScript="scripts/df.script"
 +
            xmlScript="scripts/xf.script"
 +
            zipScript="scripts/zf.script">
 +
 
 +
            <accept ip="..."/>
 +
            <reject ip="..."/>
 +
 
 +
        </PolledHttpExportService>
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
 +
*<b>port</b> is the port on which the ExportService listens for connections.
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be exported.
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be exported.
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be exported.
 +
*<b>acceptFileObjects</b> determines whether FileObjects are to be exported.
 +
*<b>dicomScript</b> specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
 +
*<b>xmlScript</b> specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
 +
*<b>zipScript</b> specifies the path to a script which examines the contents of a ZipObject and determines whether the object is to be exported.
 +
*The <b>accept</b> child element can be used to specify a white list of IP addresses determining which connections are to be accepted. One <b>accept</b> child element must appear for each IP address in the white list. If the white list is empty, the white list is not used to filter connections.
 +
*The <b>reject</b> child element can be used to specify black lists of IP addresses determining which connections are to be rejected. One <b>reject</b> child element must appear for each IP address in the black list. If the black list is empty, the black list is not used to filter connections.
 +
*For a connection to be accepted, the IP address of the remote client must be accepted by the white list and and not rejected by the black list.
 +
 
 +
Notes:
 +
#For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") <u>and</u> the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] and [[The CTP XML and Zip Filters]] for information about the script languages.
 +
#The PolledHttpExportService does not support SSL.
 +
 
 +
=====DicomExportService=====
 +
The DicomExportService queues objects and transmits them to a DICOM Storage SCP. The configuration element for the DicomExportService is:
 +
<pre>
 +
        <DicomExportService
 +
            name="DicomExportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomExportService"
 +
            root="root-directory"
 +
            quarantine="quarantine-directory"
 +
            auditLogID=""
 +
            auditLogTags=""
 +
            url="dicom://DestinationAET:ThisAET@ipaddress:port"
 +
            associationTimeout="0"
 +
            forceClose="no"
 +
            hostTag="00097774"
 +
            portTag="00097776"
 +
            calledAETTag="00097770"
 +
            callingAETTag="00097772"
 +
            dicomScript="scripts/df.script"
 +
            auditLogID="id"
 +
            auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber"
 +
            throttle="0"
 +
            interval="5000" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage. This attribute is required only when the stage is accessed by other stages. Its value, when supplied, must be unique across all pipelines.
 +
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
 +
*<b>quarantine</b> is a directory in which the DicomExportService is to quarantine objects that fail to be transmitted because they were rejected by the destination DICOM Storage SCP (usually because a presentation context could not be negotiated). Such objects would always fail, so they must be removed from the export queue. If no quarantine directory is specified, failed objects are deleted.
 +
*<b>auditLogID</b> specifies the ID of an AuditLog plugin. If this attribute is not empty, and if an AuditLog plugin is configured with that ID, an entry is made in the AuditLog for each successful transmission.
 +
*<b>auditLogTags</b> specifies the elements from the transmitted objects that are to be included in the AuditLog entry. Elements can be specified by their dcm4che names or their numeric values. Elements must be separated by semicolons. This is an example of several elements,  including one from a private group:
 +
::<tt><b>auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber;(0009,7790)"</b></tt>
 +
*<b>url</b> specifies the destination DICOM Storage SCP's URL.
 +
*<b>associationTimeout</b> specifies the length of time an association is to be left open after a transfer before it is closed automatically. Allowed valuers are <b>yes</b> and <b>no</b>. The default value is <b>no</b>. This parameter can be set to <b>yes</b> if it appears that the destination SCP is resetting the association and causing a problem with transfers.
 +
*<b>forceClose</b> specifies whether the DICOM association is to be closed after each transfer. The value is specified in seconds. A value of zero (the default) means to disable the automatic association closing mechanism.
 +
*<b>hostTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is to be used by the DicomExportService to obtain the host name (or IP address) of the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, or if the identified element does not exist in a DicomObject being exported, the DicomExportService uses the host specified in the <b>url</b> attribute.
 +
*<b>portTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is to be used by the DicomExportService to obtain the port of the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, or if the identified element does not exist in a DicomObject being exported, the DicomExportService uses the host specified in the <b>url</b> attribute.
 +
*<b>calledAETTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is to be used by the DicomExportService to obtain the AE Title which specifies the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, or if the identified element does not exist in a DicomObject being exported, the DicomExportService uses the AE Title specified in the <b>url</b> attribute.
 +
*<b>callingAETTag</b> is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is to be used by the DicomExportService to obtain the AE Title which specifies the sender in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, or if the identified element does not exist in a DicomObject being exported, the DicomExportService uses the AE Title specified in the <b>url</b> attribute.
 +
*<b>dicomScript</b> specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
 +
*<b>auditLogID</b> is the value of the <b><tt>id</tt></b> attribute of an AuditLog plugin in which to log exported objects. If this attribute is missing or if the value does not correspond to an AuditLog plugin, no audit logging is performed.
 +
*<b>auditLogTags</b> is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as <b><tt>dcm4che</tt></b> element names or in standard DICOM notation, e.g. <b><tt>(0008,0050)</tt></b>.
 +
*<b>throttle</b> sets the minimum time (in milliseconds) between exports to the destination. The default is 0 milliseconds. The maximum allowed value is 5 seconds.
 +
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue. The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
 +
 
 +
Notes:
 +
*For an object to be accepted for export, the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] for information about the script language.
 +
 
 +
=====DicomSTOWRSExportService=====
 +
The DicomSTOWRSExportService queues objects and transmits them via the DICOM STOW-RS protocol. The configuration element for the DicomSTOWRSExportService is:
 +
<pre>
 +
        <DicomSTOWRSExportService
 +
            name="DicomSTOWRSExportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.DicomSTOWRSExportService"
 +
            root="root-directory"
 +
            url="http://ipaddress:port/path"
 +
            includeContentDispositionHeader="no"
 +
            username="username"
 +
            password="password"
 +
            dicomScript="scripts/df.script"
 +
            logDuplicates="no"
 +
            auditLogID="id"
 +
            auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber"
 +
            interval="5000" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
 +
*<b>url</b> specifies the destination system's URL.
 +
*<b>includeContentDispositionHeader</b> determines whether a Content-Disposition header is to be included in the connection. This header is not required in the protocol, but in some cases, it may be useful. Allowed values are <b>yes</b> or <b>no</b>. The default is <b>no</b>.
 +
*<b>username</b> specifies the username credential for inclusion in the header during the transmission. This allows an HttpImportService to authenticate transmissions. If the username attribute is not present or has a whitespace value, no header is generated.
 +
*<b>password</b> specifies the password credential for inclusion in the header during the transmission.
 +
*<b>dicomScript</b> specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
 +
*<b>logDuplicates</b> specifies whether to make a log entry for each object whose UID matches that of any other object in the last 20 that have been exported. This capability is intended primarily for configuration debugging.
 +
*<b>auditLogID</b> is the value of the <b><tt>id</tt></b> attribute of an AuditLog plugin in which to log exported DICOM objects. If this attribute is missing or if the value does not correspond to an AuditLog plugin, no audit logging is performed.
 +
*<b>auditLogTags</b> is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as <b><tt>dcm4che</tt></b> element names or in standard DICOM notation, e.g. <b><tt>(0008,0050)</tt></b>.
 +
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue.
 +
 
 +
Notes:
 +
#The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
 +
#The protocol part of the <b>url</b> can be <b>http</b> or <b>https</b>, the latter causing connections to be initiated using secure sockets layer.
 +
#For an object to be accepted for export, the object must be a DicomObject <u>and</u> the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] and [[The CTP XML and Zip Filters]] for information about the script languages.
 +
 
 +
=====FtpExportService=====
 +
The FtpExportService queues objects and transmits them to an FTP server. The configuration element for the FtpExportService is:
 +
<pre>
 +
        <FtpExportService
 +
            name="FtpExportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.FtpExportService"
 +
            root="root-directory"
 +
            url="ftp://ipaddress:port/path"
 +
            username="..."
 +
            password="..."
 +
            acceptDicomObjects="yes"
 +
            acceptXmlObjects="yes"
 +
            acceptZipObjects="yes"
 +
            acceptFileObjects="yes"
 +
            dicomScript="scripts/df.script"
 +
            xmlScript="scripts/xf.script"
 +
            zipScript="scripts/zf.script"
 +
            auditLogID="id"
 +
            auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber"
 +
            interval="5000" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
 +
*<b>url</b> specifies the destination FTP server's URL:
 +
**<b>ipaddress</b> can be a numeric address or a domain name.
 +
**<b>port</b> is the port on which the FTP listener listens. The default port is 21.
 +
**<b>path</b> is the base directory on the FTP server in which the FtpExportService will create StudyInstance directories.
 +
*<b>username</b> specifies the username under which the FtpExportService will log in to the FTP server.
 +
*<b>password</b> specifies the password to be used in the login process.
 +
*<b>acceptDicomObjects</b> determines whether DicomObjects are to be exported.
 +
*<b>acceptXmlObjects</b> determines whether XmlObjects are to be exported.
 +
*<b>acceptZipObjects</b> determines whether ZipObjects are to be exported.
 +
*<b>acceptFileObjects</b> determines whether FileObjects are to be exported.
 +
*<b>dicomScript</b> specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
 +
*<b>xmlScript</b> specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
 +
*<b>zipScript</b> specifies the path to a script which examines the contents of a ZipObject and determines whether the object is to be exported.
 +
*<b>auditLogID</b> is the value of the <b><tt>id</tt></b> attribute of an AuditLog plugin in which to log exported DICOM objects. If this attribute is missing or if the value does not correspond to an AuditLog plugin, no audit logging is performed.
 +
*<b>auditLogTags</b> is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as <b><tt>dcm4che</tt></b> element names or in standard DICOM notation, e.g. <b><tt>(0008,0050)</tt></b>.
 +
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue.
 +
Notes:
 +
#For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") <u>and</u> the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP DICOM Filter]] and [[The CTP XML and Zip Filters]] for information about the script languages.
 +
#The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
 +
#The FtpExportService stores files in subdirectories of the <b>path</b> part of the URL, organized by StudyInstanceUID (or StudyUID in the case of non-DICOM files). Files not containing a StudyUID are stored in the <b>bullpen</b> directory under the <b>path</b> directory.
 +
#If the directory specified by the <b>path</b> does not exist, it is created.
 +
#Files are stored within their directories with names that consist of the date and time (to the millisecond) when they were transferred to the server.
 +
#If a file is transmitted multiple times, multiple copies of the file will appear with its directory, each with the date/time of the transfer.
 +
#No index of the studies and files is created on the server.
 +
 
 +
=====AimExportService=====
 +
The AimExportService queues objects and transmits them to an AIM Data Service. The configuration element for the AimExportService is:
 +
<pre>
 +
        <AimExportService
 +
            name="AimExportService"
 +
            id="stage ID"
 +
            class="org.rsna.ctp.stdstages.AimExportService"
 +
            root="root-directory"
 +
            url="http://ipaddress:port/path"
 +
            username="username"
 +
            password="password"
 +
            xmlScript="scripts/xf.script"
 +
            logResponses="none"
 +
            quarantine="quarantine-directory"
 +
            interval="5000" />
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>id</b> is any text to be used to uniquely identify the stage.
 +
*<b>root</b> is a directory for use by the ExportService for internal storage and queuing.
 +
*<b>url</b> specifies the destination AIM Data Service URL:
 +
**<b>ipaddress</b> can be a numeric address or a domain name.
 +
**<b>port</b> is the port on which the AIM Data Service listens.
 +
**<b>path</b> is the path identifying the AIM Data Service on the destination server.
 +
*<b>username</b> specifies the username credential for inclusion in the header during the transmission. If the username attribute is not present or has a whitespace value, no header is generated.
 +
*<b>password</b> specifies the password credential for inclusion in the header during the transmission.
 +
*<b>xmlScript</b> specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
 +
*<b>logResponses</b> specifies whether the contents of the response from the AIM Data Service are to be entered into the CTP log file. The recognized values are:
 +
** <b>all</b> - log all responses
 +
** <b>failed</b> - log only the responses that indicate that the Data Service rejected the object
 +
** <b>none</b> - do not log responses.
 +
The default, if the attribute is missing or has any other value, is not to log.
 +
*<b>quarantine</b> is a directory in which the AimExportService is to quarantine objects that fail to be transmitted because they were rejected by the destination AIM Data Service. If no quarantine directory is specified, failed objects are deleted.
 +
*<b>interval</b> is the sleep time (in milliseconds) between polls of the export queue.
 +
Notes:
 +
#The AimExportService only transmits XmlObjects. It ignores all other object types.
 +
#The AimExportService only transmits XmlObjects whose root element is "ImageAnnotation".
 +
#The XmlObject must pass the script test. If the <b>xmlScript</b> attribute is not supplied, the test returns true by default and the object is accepted. See [[The CTP XML and Zip Filters]] for information about the script language.
 +
#The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
 +
 
 +
=====DicomDifferenceLogger=====
 +
The DicomDifferenceLogger queues objects containing the changes made to DicomObjects processed by its pipeline and submits them to a LogAdapter class written specially to connect to an external database. The configuration element for the DicomDifferenceLogger is:
 +
<pre>
 +
        <DicomDifferenceLogger
 +
            name="DicomDifferenceLogger"
 +
            class="org.rsna.ctp.stdstages.DicomDifferenceLogger"
 +
            root="roots/DicomDifferenceLogger"
 +
            adapterClass="..."
 +
            cacheID="ObjectCache"
 +
            tags="PatientID;PatientName;SOPInstanceUID"/>
 +
</pre>
 +
where:
 +
*<b>name</b> is any text to be used as a label on configuration and status pages.
 +
*<b>root</b> is a directory for use by the DicomDifferenceLogger for temporary storage.
 +
*<b>adapterClass</b> is the class name of the external log's adapter class. See [[Developing a DicomDifferenceLogger LogAdapter]] for more information.
 +
*<b>tags</b> is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as dcm4che element names (DICOM keywords) or in standard DICOM notation, e.g. (0008,0050).
 +
*<b>cacheID</b> is the ID of an ObjectCache stage.
 +
 
 +
==Security Issues==
 +
In a clinical trial, transmission of data from image acquisition sites to the principal investigator site involves penetrating at least one firewall. Since the image acquisition site initiates an outbound connection, this only rarely requires special action. At the principal investigator's site, where the connection is inbound, some provision must be made to allow the connection to reach its destination. There are two basic solutions. The simplest solution is to open a port in the firewall at the principal investigator's site and route connections for that port to the computer running the CTP application. In some institutions, however, security policies prohibit this solution. The alternative is to use the PolledHttpExportService and PollingHttpImportService to allow data to flow without having to open any ports on the internal network to inbound connections.
 +
 
 +
Using this latter solution requires two computers, each running CTP. One computer is placed in the border router's DMZ with one port open to the internet, allowing connections to the HttpImportService of the program running in that computer. That program's pipeline includes a PolledHttpExportService which queues objects and waits for a connection before passing them onward. The second computer is placed on the internal network. Its program has a pipeline which starts with a PollingHttpImportService. That ImportService is configured to make outbound connections to the DMZ computer when a file is requested. This allows files to pass through the firewall on the response stream of the outbound connection without having to open any ports to the internal network.
 +
 
 +
==Notes==
 +
 
 +
===Java Cryptography Extension===
 +
The anonymizer pipeline stages (DicomAnonymizer, XmlAnonymizer, and ZipAnonymizer) use classes in the Java Cryptography Extension (JCE). The JCE has been included in the Java JRE since at least Java 1.5. To enable the use of the JCE, an entry must appear in the <b><tt>Java/jre/lib/security/java.security</tt></b> file. In the latest Java versions, the entry is automatically included. The most recent versions include a section in the file that looks like this:
 +
<pre>
 +
# There must be at least one provider specification in java.security.
 +
# There is a default provider that comes standard with the JDK. It
 +
# is called the "SUN" provider, and its Provider subclass
 +
# named Sun appears in the sun.security.provider package. Thus, the
 +
# "SUN" provider is registered via the following:
 +
#
 +
#    security.provider.1=sun.security.provider.Sun
 +
#
 +
# (The number 1 is used for the default provider.)
 +
#
 +
# Note: Providers can be dynamically registered instead by calls to
 +
# either the addProvider or insertProviderAt method in the Security
 +
# class.
 +
#
 +
# List of providers and their preference orders (see above):
 +
#
 +
security.provider.1=sun.security.provider.Sun
 +
security.provider.2=sun.security.rsa.SunRsaSign
 +
security.provider.3=com.sun.net.ssl.internal.ssl.Provider
 +
security.provider.4=com.sun.crypto.provider.SunJCE
 +
security.provider.5=sun.security.jgss.SunProvider
 +
security.provider.6=com.sun.security.sasl.Provider
 +
security.provider.7=org.jcp.xml.dsig.internal.dom.XMLDSigRI
 +
security.provider.8=sun.security.smartcardio.SunPCSC
 +
security.provider.9=sun.security.mscapi.SunMSCAPI
 +
</pre>
 +
On older Javas, it appears that there are no <b><tt>security.provider...</tt></b> lines in the file. If no provider is specified, the anonymizer will crash when it tries to load a JCE class. If that happens, you will see an error in the Launcher's Console tab. You can either edit the <b><tt>Java/jre/lib/security/java.security</tt></b> file and add the <b><tt>security.provider...</tt></b> lines shown above or uninstall Java and get the latest version.
  
The Ant build file for ClinicalTrialProcessor is in the <b>ClinicalTrialProcessor</b> directory and is called <b>build.xml</b>. To build the software on a Windows system, launch a command window, navigate to the <b>ClinicalTrialProcessor</b> directory, and enter <b>ant all</b>.
+
===Important Note for Unix/Linux Platforms===
 +
Unix and its derivatives require that applications listening on ports with numbers less than 1024 have special privileges. On such systems, it might be best to put all import services and all web servers on ports above this value. The default configuration puts the web server on port 80 for Windows platforms because that is the default port for the web. On Linux platforms, the default configuration puts the web server on port 1080. This can be changed easily in the CTP-launcher application when CTP is started.
  
The build file contains several targets. The <b>all</b> target does a clean build of everything, including the Javadocs, which are put into the <b>documentation</b> directory. The Javadocs can be accessed with a browser by opening the file:
+
===ImageIO Tools for Macintosh===
 +
The Java Advanced Imaging ImageIO Tools is a component which provides methods for creating and reading image files. It supports many image file types, and it is designed to be extensible. The authors (Gunter Zeilinger, et al.) of the DICOM toolkit (dcm4che) used by CTP extended the ImageIO Tools to support DICOM.
  
:<tt>ClinicalTrialProcessor/documentation/index.html</tt>
+
The ImageIO Tools consists of two parts, a top-level library written in Java and runnable on any platform, and a native library which implements certain compression/decompression functions. The latter library is unique to each platform.
  
The default target, <b>ctp-installer</b>, just builds the application and places the installer in the <b>products</b> directory.
+
The top-level library consists of two files:
 +
* jai_imageio.jar
 +
* clibwrapper_jiio.jar
  
===Implementing a Pipeline Stage===
+
There is no Macintosh installer for the ImageIO Tools. You can obtain the two files above by getting the zip file for a Linux installation and unpacking it. Place the two files into <b><tt>/System/Library/Java/Extensions</tt></b>.  
To be recognized as a pipeline stage, a class must implement the <b>org.rsna.ctp.pipeline.PipelineStage</b> interface. An abstract class, <b>org.rsna.ctp.pipeline.AbstractStage</b>, is provided to supply some of the basic methods required by the PipelineStage interface. All the standard stages extend this class.
+
  
Each stage type must also implement its own interface. The interfaces are:
+
There is no native library available for the Macintosh.  
*org.rsna.ctp.pipeline.ImportService
+
*org.rsna.ctp.pipeline.Processor
+
*org.rsna.ctp.pipeline.StorageService
+
*org.rsna.ctp.pipeline.ExportService
+
  
Each stage must have a constructor which takes an XML Element as its argument. The constructor must obtain any configuration information it requires from the element. While it is not required that all configuration information be placed in attributes of the element, the <b>getConfigHTML</b> method provided by <b>AbstractStage</b> expects it, and if you choose to encode configuration information in another way, you must override the <b>getConfigHTML</b> method.
+
For more information on the ImageIO Tools, see this [http://mircwiki.rsna.org/index.php?title=Java_Advanced_Imaging_ImageIO_Tools article].

Latest revision as of 12:52, 22 October 2017

Contents

This article describes the RSNA MIRC Clinical Trials Processor (CTP), a stand-alone image processing application for imaging clinical trials data.

1 Clinical Trial Processor (CTP)

CTP is a stand-alone program that provides all the processing features of a MIRC site for clinical trials in a highly configurable and extensible application. It connects to FieldCenter applications and can also connect to MIRC sites when necessary. CTP has the following key features:

  1. Single-click installation.
  2. Support for multiple pipelines.
  3. Processing pipelines supporting multiple configurable stages.
  4. Support for multiple quarantines for data objects which are rejected during processing.
  5. Pre-defined implementations for key components:
    • HTTP Import
    • DICOM Import
    • DICOM Anonymizer
    • XML Anonymizer
    • File Storage
    • Database Export
    • HTTP Export
    • DICOM Export
    • FTP Export
  6. Web-based monitoring of the application's status, including:
    • configuration
    • logs
    • quarantines
    • status

1.1 Installation

The installer for CTP is available on the RSNA MIRC site. Click the Download Software link in the left pane of the MIRC home page to obtain a list of all the available software.

Download the installer and place it on the disk on which you intend to install or upgrade the CTP program.

To run the installer, the Java 1.7 (or better) JRE must be present on the system. Java 1.8 is recommended because earlier versions are being sunset by Oracle/Sun. Java and all its components are available through the Java website. Note that only the JRE is required, not the JDK. If you plan to run CTP as a Windows service or if you plan to use the Java Advanced Imaging ImageIO Tools (see below), then you must install the 32-bit Java, even on a 64-bit computer.

For convenience, it is recommended (but not required) that a folder called JavaPrograms be created in the root of the disk drive. The installer does not create this folder, but if it is present, the installer will very quickly find it and suggest installing CTP in a subdirectory of JavaPrograms. This is especially helpful when upgrading.

Certain CTP pipeline stages (FileStorageService, BasicFileStorageService, DicomDecompressor, and others) require that the Java Advanced Imaging ImageIO Tools be present on the system. If you already have the ImageIO Tools installed, note that it is critically important that version 1.1 of the ImageIO Tools be installed rather than version 1.0. Parenthetically, note that the Java Advanced Imaging component is not the same as the Java Advanced Imaging ImageIO Tools. Only the latter component is required. (Macintosh users should read ImageIO Tools for Macintosh in the Notes section.) When the CTP installer runs, it checks several parameters of the system and highlights ones that are not correct for the running of CTP. One such parameter is the version of the ImageIO Tools. The ImageIO Tools need not be present in order to run the installer, and they may be installed later if required, without having to re-install CTP.

Note also that the CTP installer includes the ImageIO Tools for Windows 32-bit Java only, so on such systems, you can skip the installation of the ImageIO Tools, but that will make the tools only available for CTP, not for other applications. If you are planning to install certain other RSNA DICOM tools (for example, DicomEditor), it is best to install the ImageIO Tools directly in Java using the installer provided in Java Advanced Imaging ImageIO Tools.

To run the CTP installer, double-click the CTP-installer.jar file and choose a directory in which to install CTP. The installer can also be run in a command window using the command:

java -jar CTP-installer.jar

The installer creates a directory called CTP in the selected location and places several files and directories in it. Two files of special importance are:

  • Launcher.jar - the program that launches CTP with a user interface
  • Runner.jar - the program that starts or stops CTP with no user interface

1.2 Running CTP

There are three ways to start CTP:

  • Launcher.jar - a program for manually starting and stopping CTP through a dialog window. This program is normally used when CTP is installed on an individual user's computer for occasional use.
  • Runner.jar - a program for starting and stopping CTP with no user interface. This program is normally used when CTP is installed on a Linux or Mac system for institutional use, running as an automatic service. See Running CTP as a Linux Service for instructions.
  • CTP can also be run as a Windows service. See Running CTP as a Windows Service for instructions.

When learning to use CTP or when developing a new pipeline configuration, it is best to start by running CTP from the Launcher.

The launcher displays a dialog providing start/stop buttons and fields for controlling several parameters used by the system.

The Server port field allows you to change the port on which the server runs.

The default memory configuration is sufficient for all but the very largest sites. For sites with 2000 or more teaching file cases, the Maximum memory pool parameter should be increased to 500. For sites with more than 3000 cases, 750 is recommended. To change the memory configuration for the Windows service, see the article.

The Extensions directory parameter is intended for special applications in which third-party software is added into the system. One such application is the NCI's National Biomedical Image Archive (NBIA). Normal teaching file systems do not require this parameter.

Across the top of the dialog are several tabs providing access to information about the versions of the components, the system parameters in use by Java as the program runs, and any messages output by the program either directly or through its log file. These are only present as a convenience; they are not typically used in normal operation.

As a convenience, the CTP Home Page button launches the user's browser and goes directly to the main MIRC page.

CTP has no user interface. When the program starts, it runs without intervention. Status and other information can be obtained through the program's integrated webserver. Accessing the server with no path information displays a page presenting buttons for each of the servlets.

When the program is first installed, two users are provided. One user, with the name admin and password password, is intended for general system administration. The other user, with the name king and password password, has the ability to shut down the server through the browser interface. Both users have the ability to create and modify users and assign privileges through the User Manager, but only a user with the shutdown privilege can grant the shutdown privilege to another user.

To stop the program, click the Stop button on the Launcher, or log in as a user with the shutdown privilege and click the Shutdown button on the main page. As a convenience, a user with the admin privilege can also shut the server down if the user's browser is running on the same computer as the server.

1.3 Configuration Files

The program uses two configurable files: config.xml, which is located in the same directory as the program itself, and index.html, which is located in the server's ROOT directory. Both files are intended to be configured for the specific application. The installer does not overwrite these files when it runs; instead, it installs two example files: example-config.xml and example-index.html. When CTP starts, it looks to see if the non-example files are missing, and if so, it copies the example files into the non-example ones. This process allows upgrades to be done without losing any configuration work. After installing the program the first time, it should be run once in order to make the copies, and then the copies can be configured. Configuration is done by hand with any text editor (e.g., TextPad or NotePad). Care should be taken, especially with config.xml, to keep it well-formed. Opening it with a program like InternetExplorer will check it for errors.

1.4 Server

To provide access to the status of the components, the application includes an HTTP server which serves files and provides servlet-like functionality. Files are served from a directory tree whose root is named ROOT. The ROOT directory contains a file, index.html, which provides buttons which link to several servlets providing information about the operation of the program. This file is intended to be configured with logos, additional links, etc., and upgrades do not overwrite it. The standard servlets are:

  • LoginServlet allows a user to log into the system.
  • UserManagerServlet allows an admin user to create users and assign them privileges.
  • ConfigurationServlet displays the contents of the configuration file.
  • SysPropsServlet displays the system properties which define the environment in which CTP is running, and provides an admin user the ability to force a garbage collection.
  • StatusServlet displays the status of all pipeline stages.
  • LogServlet provides web access to all log files in the logs directory.
  • QuarantineServlet provides web access to all quarantine directories and their contents.
  • IDMapServlet allows an admin user to access a database of PHI and anonymized replacements for patient IDs accession numbers, and UIDs.
  • ObjectTrackerServlet allows an admin user to access a database of the identifiers of objects which have been processed, organized by date, patient ID, Study UID, Series UID, and Instance UID.
  • DBVerifierServlet allows an admin user to access a database which tracks the status of objects as they are inserted into an external database (which may be in a remote instance of CTP).
  • DicomAnonymizerServlet allows an admin user to configure any DICOM anonymizers in the pipelines.
  • ScriptServlet allows an admin user to configure the scripts for script-based pipeline stages, including the various Filter stages and the XML and Zip anonymizers.
  • LookupServlet allows an admin user to configure lookup tables used by the anonymizers.
  • ShutdownServlet allows an admin user to shut the program down.

The configuration element for the HTTP server is:

        <Server 
            port="80"
            ssl="no"
            requireAuthentication="no"
            usersClassName="org.rsna.server.UsersXmlFileImpl">
          <ProxyServer
              proxyIPAddress=""
              proxyPort=""
              proxyUsername=""
              proxyPassword=""/>
          <SSL
            keystore=""
            keystorePassword=""
            truststore=""
            truststorePassword=""/>
        </Server>

where:

  • port is the port number on which the HTTP server listens for connections.
  • ssl determines whether the HTTP server uses SSL. Values are "yes" and "no". The default is "no".
  • requireAuthentication determines whether users are forced to log in to the HTTP server. Values are "yes" and "no". The default is "no". (The HTTP server is typically operated without requiring authentication, thus allowing users to monitor the status of the system without having to log in.)
  • proxyIPAddress is the IP address of the proxy server. If this attribute is missing or blank, no proxy server is used. If a proxy server is configured, it is shared by all pipeline stages and servlets.
  • proxyPort is the port of the proxy server. If this attribute is missing or blank, no proxy server is used.
  • proxyUsername is the username which is to be supplied to the proxy server if proxy authentication is required. If this attribute is missing or blank, no credentials will be supplied to the proxy server.
  • proxyPassword is the password which is to be supplied to the proxy server if proxy authentication is required. If this attribute is missing or blank, no credentials will be supplied to the proxy server.
  • keystore specifies the path to the keystore file. If this attribute is missing or blank, the value keystore is supplied.
  • keystorePassword is the password for access to the keystore. It this attribute is missing or blank, the value ctpstore is used.
  • truststore specifies the path to the truststore file. If this attribute is missing or blank, the corresponding property is removed from the system properties.
  • truststorePassword is the password for access to the truststore.
  • usersClassName specifies the Java class to be used for authentication of users and their privileges. If this attribute is missing, the standard CTP implementation (shown above) is employed. This attribute should only be included if a special mechanism has been implemented on the site (for example, using LDAP or OpenAM - see CTP Authentication Using LDAP or CTP Authentication Using OpenAM).

Notes:

  • The ProxyServer element is only required when the system is behind a proxy server.
  • The SSL element is only required when accessing a site-specific keystore or truststore instead of the default stores supplied in a CTP installation.
  • When an external authentication system is used for authentication, the Server element may have a child element containing its parameters. Examples are the LDAP and OpenAM child elements. See CTP Authentication Using LDAP or CTP Authentication Using OpenAM.

1.5 Plugins

A plugin is a component that adds functionality to CTP outside the context of a pipeline. Plugins can add servlets to the server as well as provide capabilities that may be accessed by pipeline stages.

1.6 Pipelines

A pipeline is a manager that moves data objects through a sequence of processing stages. Each stage in the pipeline performs a specific function on one or more of the four basic object types supported by MIRC:

  • FileObject
  • DicomObject
  • XmlObject
  • ZipObject

Each pipeline must contain at least one ImportService. Each pipeline stage may be provided access to a quarantine directory into which the stage places objects that it rejects, thus removing them from the pipeline and aborting further processing. Quarantine directories may be unique to each stage or shared with other stages. At the end of the pipeline, the manager calls the ImportService which provided the object to remove it from its queue.

There are four types of pipeline stages. Each is briefly described in subsections below.

1.6.1 ImportService

An ImportService receives objects via a protocol and enqueues them for processing by subsequent stages.

1.6.2 Processor

A Processor performs some kind of processing on an object. Processors are not intended to be queued. In the context of the current MIRC implementation, a Preprocessor is a Processor, as is an Anonymizer. The result of a processing stage is an object that is passed to the next stage in the pipeline.

1.6.3 StorageService

A StorageService stores an object in a file system. It is not queued, and it therefore must complete before subsequent stages can proceed. A StorageService may return the current object or the stored object in response to a request for the output object, depending on its implementation.

1.6.4 ExportService

An ExportService provides queued transmission to an external system via a defined protocol. Objects in the queue are full copies of the objects submitted; therefore, subsequent processing is not impeded if a queue is paused, and modifications made subsequently do not affect the queue entry, even if they occur before transmission. (Note: This behavior is different from that of the current MIRC implementation.) After entering an object in its queue, an ExportService returns immediately.

1.7 System Configuration

The CTP configuration is specified by an XML file called config.xml located in the same directory as the program. There can be one Server element specifying the port on which the HTTP server is to operate, and multiple Pipeline elements, each specifying the stages which comprise it. The name of the element defining a stage is irrelevant and can be chosen for readability; each stage in a pipeline is actually defined by its Java class, specified in the class attribute. Stages are loaded automatically when the program starts, and the loader tests the stage's class to determine what kind of stage it represents. It is possible to extend the application beyond the pre-defined stages available in the implementation as described in Extending CTP.

The following is an example of a simple configuration that might be used at an image acquisition site. It contains one pipeline which receives objects via the DICOM protocol, stores the objects locally, anonymizes them, and transmits them via HTTP (using secure sockets layer for encryption) to a principal investigator's site.

<Configuration>
   <Server port="80" />
   <Pipeline name="Main Pipeline">
        <DicomImportService
            name="DicomImportService"
            class="org.rsna.ctp.stdstages.DicomImportService"
            root="roots/dicom-import"
            port="1104" />
        <FileStorageService
            name="FileStorageService"
            class="org.rsna.ctp.stdstages.FileStorageService"
            root="storage"
            returnStoredFile="no"
            quarantine="quarantines/storage" />
        <DicomAnonymizer
            name="DicomAnonymizer"
            class="org.rsna.ctp.stdstages.DicomAnonymizer"
            root="roots/anonymizer"
            script="scripts/da.script"
            quarantine="quarantines/anonymizer" />
        <HttpExportService
            name="HttpExportService"
            class="org.rsna.ctp.stdstages.HttpExportService"
            root="roots/http-export"
            url="https://university.edu:1443" />
   </Pipeline>
</Configuration>

Note that because the storage service appears in the pipeline before the anonymizer, the objects which are stored contain the PHI which was originally received, and because the anonymizer appears before the export service, anonymized objects are exported.

The following is an example of a simple configuration that might be used at a principal investigator's site. It contains one pipeline which receives objects via the HTTP protocol, stores them, and exports them to a DICOM destination:

<Configuration>
   <Server port="80" />
   <Pipeline name="Main Pipeline">
        <HttpImportService
            name="HttpImportService"
            class="org.rsna.ctp.stdstages.HttpImportService"
            root="roots/http-import"
            ssl="yes"
            port="1443" />
        <FileStorageService
            name="FileStorageService"
            class="org.rsna.ctp.stdstages.FileStorageService"
            root="D:/FileStorageService" 
            returnStoredFile="no"
            quarantine="quarantines/StorageServiceQuarantine" />
        <DicomExportService
            name="DicomExportService"
            class="org.rsna.ctp.stdstages.DicomExportService"
            root="roots/pacs-export" 
            url="dicom://DestinationAET:ThisAET@ipaddress:port" />
   </Pipeline>
</Configuration>

Note that in the example above, non-DICOM objects are stored in the StorageService, but they are not exported by the DicomExportService. Each pipeline stage is responsible for testing the class of the object which it receives and processing (or ignoring) the object accordingly.

Multiple Pipeline elements may be included, but each must have its own ImportService element, and their ports must not conflict.

Each pipeline stage class has a constructor that is called with its configuration element, making it possible for special processor implementations to be passed additional parameters from the configuration.

1.8 Standard Plugins

The application includes built-in, standard plugins to provide features that are required in some clinical trials. The sections below show all the configuration attributes recognized by the standard plugins. Note that the configuration element for a plugin always has the tag name Plugin.

1.8.1 AuditLog

The AuditLog plugin provides a permanent log repository to meet the logging requirements of a 21CFR11-compliant clinical trial. Certain pipeline stages can be configured to make entries in the log repository.

The AuditLog plugin installs a servlet to provide browser access to the repository for admin users. The servlet is accessed with a path equal to the value of the AuditLog plugin's id attribute. It is possible to configure multiple AuditLog Plugin elements (with different id attributes) if multiple trials are serviced by a single CTP instance and it is desired to keep the log repositories separate. The configuration element for the AuditLog is:

        <Plugin
            name="log name"
            id="pluginID"
            class="org.rsna.ctp.stdplugins.AuditLog"
            root="root-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is text to be used to uniquely identify the plugin. Note that since the value of the attribute is used as the context of the servlet that provides access to the repository, it must be a single word (that is, it must not contain whitespace).
  • root is a directory for use by the plugin for internal storage of the database.
1.8.2 Redirector

The Redirector plugin redirects non-secure HTTP connections to another port using SSL. It is intended for use on CTP installations that configure the main server to use SSL. Such installations typically put the server on port 443. As a convenience for users, the Redirector can be configured to listen on port 80 and redirect the user to port 443, switching the protocol.

The configuration element for the Redirector is:

        <Plugin
            name="Redirector"
            class="org.rsna.ctp.stdplugins.Redirector"
            httpPort="80"
            httpsHost="mirc.mysecuresite.myuniversity.edu"
            httpsPort="443" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • httpPort is the port on which the Redirector listens for connections.
  • httpsHost is the host to which the Redirector redirects the connection request.
  • httpsPort is the port to which the Redirector redirects the connection request.

Notes:

  • The redirection only occurs on HTTP GET requests.
  • If the httpsHost attribute is missing or empty, the value of the HOST header is used in the target URL.
  • The query string, if any, is included in the target URL.

1.9 Standard Stages

The application includes several built-in, standard stages which allow most trials to be operated without writing any software. The sections below show all the configuration attributes recognized by the standard stages.

Attributes which specify directories can contain either absolute paths (e.g., D:/TrialStorage) or relative paths (e.g., quarantines/http-import-quarantine). Relative paths are relative to the directory in which the CTP application is located.

All standard stages have a root attribute which defines a directory for use by the stage for internal storage. All root directories must be unique, e.g. not shared with other stages.

All standard stages have an optional id attribute which, if present, must uniquely identify the stage across all pipelines. This attribute is required only when the stage must be accessed by another stage, for example, when a DatabaseExportService must interrogate a FileStorageService to determine the URL under which an object is stored.

Some standard stages have attributes which determine which object types are to be accepted by the stage. These attributes are:

  • acceptDicomObjects
  • acceptXmlObjects
  • acceptZipObjects
  • acceptFileObjects

The allowed values are yes and no. The default value for all these attributes is yes. Stages which are restricted to specific object types (e.g. DicomImportService, DicomAnonymizer, XmlAnonymizer, DicomFilter, DicomExportService) ignore the values of these attributes.

If a standard ImportService receives an object which it is not configured to accept, it quarantines the object, or if no quarantine has been defined for the stage, it discards the object.

If a Processor, StorageService, or ExportService receives an object that it is not configured to accept, it either ignores the object or passes it unmodified to the next pipeline stage. Thus, if an anonymizer which is not configured to anonymize XmlObjects receives an XmlObject, it passes the object on without anonymization.

1.9.1 Import Services

1.9.1.1 HttpImportService

The HttpImportService listens on a defined port for connections from HTTP clients and receives files transmitted using the HTTP protocol with Content-Type equal to application/x-mirc. The configuration element for the HttpImportService is:

        <HttpImportService
            name="HttpImportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.HttpImportService"
            root="root-directory"
            port="7777"
            ssl="yes"
            zip="no"
            requireAuthentication="no"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            logConnections="no"
            logDuplicates="no"
            quarantine="quarantine-directory" >

            <accept ip="..."/>
            <reject ip="..."/>

        </HttpImportService>     

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ImportService for internal storage and queuing.
  • port is the port on which the ImportService listens for connections.
  • ssl determines whether the port uses secure sockets layer (yes) or unencrypted http (no). The default is no.
  • zip determines whether files will be unzipped after reception (yes) or not (no). The default is no. This feature is intended for use with zipped transmissions from the HttpExportService.
  • requireAuthentication determines whether transmissions are required to have headers which identify a user which has the import privilege. The allowed values are yes) and no. The default is no.
  • acceptDicomObjects determines whether DicomObjects are to be enqueued when received.
  • acceptXmlObjects determines whether XmlObjects are to be enqueued when received.
  • acceptZipObjects determines whether ZipObjects are to be enqueued when received.
  • acceptFileObjects determines whether FileObjects are to be enqueued when received.
  • logConnections specifies whether to make a log entry for each object that is received. The log entry includes only the IP address of the sender. The values yes and all cause all connections to be logged. The value rejected causes only rejected connections to be logged. The default is no, meaning that no connections are logged. This capability is intended primarily for configuration debugging.
  • logDuplicates specifies whether to make a log entry for each object whose UID matches that of any other object in the last 20 that have been received. This capability is intended primarily for configuration debugging.
  • quarantine is a directory in which the ImportService is to quarantine objects that it receives but is configured not to accept.
  • The accept child element can be used to specify a white list of IP addresses determining which connections are to be accepted. One accept child element must appear for each IP address in the white list. If the white list is empty, the white list is not used to filter connections.
  • The reject child element can be used to specify black lists of IP addresses determining which connections are to be rejected. One reject child element must appear for each IP address in the black list. If the black list is empty, the black list is not used to filter connections.
  • For a connection to be accepted, the IP address of the remote client must be accepted by the white list and and not rejected by the black list.

Notes:

  • The accept and reject child elements support the regex attribute in place of the ip attribute, allowing a range of IP addresses to be matched. The regex attribute must contain a regular expression. For example, to match any IP address in the 10.10.*.* range, the accept element would be coded:
    <accept regex="10\.10\.[0-9]{1,3}\.[0-9]{1,3}"/>
1.9.1.2 PollingHttpImportService

The PollingHttpImportService obtains files by initiating HTTP connections to an external system. This ImportService is designed to work in conjunction with the PolledHttpExportService to allow penetration of a firewall without having to open an inbound port, as described in Security Issues. The configuration element for the PollingHttpImportService is:

        <PollingHttpImportService
            name="PollingHttpImportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.PollingHttpImportService"
            root="root-directory"
            url="http://ip:port"
            zip="no"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ImportService for internal storage.
  • url is the URL of the PolledHttpExportService.
  • zip determines whether files will be unzipped after reception (yes) or not (no). The default is no.
  • acceptDicomObjects determines whether DicomObjects are to be enqueued when received.
  • acceptXmlObjects determines whether XmlObjects are to be enqueued when received.
  • acceptZipObjects determines whether ZipObjects are to be enqueued when received.
  • acceptFileObjects determines whether FileObjects are to be enqueued when received.
  • quarantine is a directory in which the ImportService is to quarantine objects that it receives but is configured not to accept.

Notes:

  • The protocol part of the url must be http, although the actual protocol used by the PollingHttpImportService is not HTTP.
  • The PollingHttpImportService does not support SSL.
  • The PollingHttpImportService does not support a proxy server for its connections.
1.9.1.3 DicomImportService

The DicomImportService listens on a defined port for connections from DICOM Storage SCUs and receives files transmitted using the DICOM protocol. The DicomImportService accepts all Application Entity Titles. The configuration element for the DicomImportService is:

        <DicomImportService
            name="DicomImportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomImportService"
            root="root-directory" 
            port="port number" 
            calledAETTag="00097770" 
            callingAETTag="00097772"
            connectionIPTag="00097774"
            timeTag="00097776"
            throttle="0"
            logConnections="no"
            logDuplicates="no"
            suppressDuplicates="no" >

            <accept ip="..."/>
            <reject ip="..."/>

            <accept calledAET="..."/>
            <reject calledAET="..."/>

            <accept callingAET="..."/>
            <reject callingAET="..."/>

            <accept sopClass="..."/>

        </DicomImportService>

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ImportService for internal storage and queuing.
  • port is the port on which the ImportService listens for connections.
  • calledAETTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the AE Title which was used by the sender to specify the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the AE Title of the receiver in the received DICOM object.
  • callingAETTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the AE Title which was used by the sender to identify itself in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the AE Title of the sender in the received DICOM object.
  • connectionIPTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the IP address of the sender in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the IP address of the sender in the received DICOM object.
  • timeTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, into which to store the system time when the object is received. If the attribute is missing or zero, or if the value does not parse as a hex integer, the DicomImportService does not store the system time in the received DICOM object. (The system time is the time in milliseconds since January 1, 1970.)
  • throttle sets the time delay (in milliseconds) before sending a response to the SCP on each transmission. The default is 0 milliseconds.
  • logConnections specifies whether to make a log entry for each object that is received. The log entry includes the SOPInstanceUID, the IP address of the sender in the association, and the calledAET and CallingAET. The values yes and all cause all connections to be logged. The value rejected causes only rejected connections to be logged. The default is no, meaning that no connections are logged. This capability is intended primarily for configuration debugging.
  • logDuplicates specifies whether to make a log entry for each object whose SOPInstanceUID matches that of any other object in the last 20 that have been received. This capability is intended primarily for configuration debugging.
  • suppressDuplicates specifies whether to ignore objects which have the same SOPInstanceUID as any object in the last 10 objects received. Values are yes and no. The default is no. This capability is intended primarily for configuration debugging.
  • The accept child elements can be used to specify white lists of values determining which connections are to be accepted. One accept child element must appear for each value in the white list. If the white list is empty, the white list is not used to filter connections. There are four white lists, one each for client IP addresses (ip), SCU application entity titles (callingAET), SCP application entity titles (calledAET), and SOP Classes (sopClass). (Note: SOP Classes can be specified as either the UID value or the dcm4che name. For example, both "1.2.840.10008.5.1.4.1.1.4" and "MRImageStorage" are accepted. Care should be taken when specifying SOP Class names, as unknown names are ignored.)
  • The reject child elements can be used to specify black lists of values determining which connections are to be rejected. One reject child element must appear for each value in the black list. If the black list is empty, the black list is not used to filter connections. There are three black lists, one each for client IP addresses (ip), SCU application entity titles (callingAET), and SCP application entity titles (calledAET).
  • For a connection to be accepted, the IP address of the remote client must be accepted by the white list and and not rejected by the black list.
1.9.1.4 DicomSTOWRSImportService

The DicomSTOWRSImportService listens on a defined port for HTTP or HTTPS connections from clients and receives files transmitted using the DICOM STOW-RS protocol. The configuration element for the DicomSTOWRSImportService is:

        <HttpImportService
            name="DicomSTOWRSImportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomSTOWRSImportService"
            root="root-directory"
            port="7777"
            ssl="yes"
            requireAuthentication="no"
            logConnections="no"
            logDuplicates="no"
            quarantine="quarantine-directory" >

            <accept ip="..."/>
            <reject ip="..."/>

        </HttpImportService>     

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ImportService for internal storage and queuing.
  • port is the port on which the ImportService listens for connections.
  • ssl determines whether the port uses secure sockets layer (yes) or unencrypted http (no). The default is no.
  • requireAuthentication determines whether transmissions are required to have headers which identify a user which has the import privilege. The allowed values are yes) and no. The default is no.
  • logConnections specifies whether to make a log entry for each object that is received. The log entry includes only the IP address of the sender. The values yes and all cause all connections to be logged. The value rejected causes only rejected connections to be logged. The default is no, meaning that no connections are logged. This capability is intended primarily for configuration debugging.
  • quarantine is a directory in which the ImportService is to quarantine objects that it receives but is configured not to accept.
  • The accept child element can be used to specify a white list of IP addresses determining which connections are to be accepted. One accept child element must appear for each IP address in the white list. If the white list is empty, the white list is not used to filter connections.
  • The reject child element can be used to specify black lists of IP addresses determining which connections are to be rejected. One reject child element must appear for each IP address in the black list. If the black list is empty, the black list is not used to filter connections.
  • For a connection to be accepted, the IP address of the remote client must be accepted by the white list and and not rejected by the black list.

Notes:

  • The accept and reject child elements support the regex attribute in place of the ip attribute, allowing a range of IP addresses to be matched. The regex attribute must contain a regular expression. For example, to match any IP address in the 10.10.*.* range, the accept element would be coded:
    <accept regex="10\.10\.[0-9]{1,3}\.[0-9]{1,3}"/>
1.9.1.5 DirectoryImportService

The DirectoryImportService watches a directory and imports any files it finds in it. After the files are passed down the pipeline, they are deleted from the import directory. The purpose of this ImportService is to allow manual input of objects to the pipeline. The configuration element for the DirectoryImportService is:

        <DirectoryImportService
            name="DirectoryImportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DirectoryImportService"
            root="root-directory"
            import="import-directory"
            interval="20000"
            fsName="..."
            fsNameTag=""
            filePathTag=""
            fileNameTag=""
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ImportService to manage imported files.
  • import is the directory monitored by the ImportService for files to import.
  • interval is the length of time in milliseconds between polls of the import directory. The default is 20000. If the value is less than 1000, a value of used.
  • fsName is the name of the FileSystem to be used by a FileStorageService that receives this object.
  • fsNameTag is the name of a DICOM element, specified in hex with no comma separating the group and element numbers, in which to store the value of the fsName attribute.
  • filePathTag is the name of a DICOM element, specified as a DICOM keyword or in hex, in which to store the path at which the file was found in the archive. The path starts with the name of the import directory and ends at the name of the directory that contained the file. It does not include the name of the file. The '/' path separator character is used on all platforms.
  • fileNameTag is the name of a DICOM element, specified as a DICOM keyword or in hex, in which to store the name of the file that was found in the import directory.
  • acceptDicomObjects determines whether DicomObjects are to be accepted.
  • acceptXmlObjects determines whether XmlObjects are to be accepted.
  • acceptZipObjects determines whether ZipObjects are to be accepted.
  • acceptFileObjects determines whether FileObjects are to be accepted.
  • quarantine is a directory in which the ImportService is to quarantine objects that it has handled (whether they were accepted for importing or not).

Note: For DicomObjects, if both the fsName and fsNameTag attributes are specified, the stage places the value of the fsName attribute in the element identified by fsNameTag. This allows a DirectoryImportService to mimic the behavior of a DicomImportService which stores the Called AE Title or Calling AE Title in an element for later use by a FileStorageService to assign the object to a FileSystem.

Note: When inserting the fsName value into the fsNameTag element in the DicomObject, there is a special feature. If the value of the fsName attribute is @filename, then the name of the file is inserted into the target element. If the filename ends in ".dcm", that string is removed from the end of the name before the name is inserted into the fsNameTag element.

1.9.1.6 ArchiveImportService

The ArchiveImportService walks the directory tree of a static archive and imports the files it finds. The files are copied from the archive and placed in a separate directory before being passed down the pipeline. When the files have been processed, they are deleted from the directory to which they were copied, but they remain in the archive. The purpose of this ImportService is to allow a complete archive to be processed. The ImportService checkpoints itself as it runs, and restarts where it left off if the program is stopped and restarted. The checkpoint is stored in a file called checkpoint.bin in the stage's root directory. To restart the stage from the tree root, the checkpoint file must be deleted and CTP must be restarted.

The configuration element for the ArchiveImportService is:

        <ArchiveImportService
            name="ArchiveImportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.ArchiveImportService"
            root="root-directory"
            treeRoot="archive-root-directory"
            expandTARs="no"
            minAge="5000"
            fsName="..."
            fsNameTag=""
            filePathTag=""
            fileNameTag=""
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ImportService to manage imported files.
  • treeRoot is the root directory of the archive.
  • expandTARs determines whether the ImportService is to expand any TAR files it finds in the archive as they are imported. For archives containing TAR files encapsulating DICOM images, this attribute should be set to yes; otherwise, the TAR files will be treated as FileObjects containing no identifiers which would allow them to be connected to other objects.
  • minAge is the minimum age (in milliseconds) for files which are imported from the root directory. The default value is 5000; the minimum accepted value is 1000. The purpose of this attribute is to ensure that files are completely stored in the root directory before being imported.
  • fsName is the name of the FileSystem to be used by a FileStorageService that receives this object.
  • fsNameTag is the name of a DICOM element, specified in hex with no comma separating the group and element numbers, in which to store the value of the fsName attribute.
  • filePathTag is the name of a DICOM element, specified as a DICOM keyword or in hex, in which to store the path at which the file was found in the archive. The path starts with the name of the treeRoot directory and ends at the name of the directory that contained the file. It does not include the name of the file. The '/' path separator character is used on all platforms.
  • fileNameTag is the name of a DICOM element, specified as a DICOM keyword or in hex, in which to store the name of the file that was found in the archive.
  • acceptDicomObjects determines whether DicomObjects are to be accepted.
  • acceptXmlObjects determines whether XmlObjects are to be accepted.
  • acceptZipObjects determines whether ZipObjects are to be accepted.
  • acceptFileObjects determines whether FileObjects are to be accepted.
  • quarantine is a directory in which the ImportService is to quarantine objects that it has handled (whether they were accepted for importing or not).

Note: For DicomObjects, if both the fsName and fsNameTag attributes are specified, the stage places the value of the fsName attribute in the element identified by fsNameTag. This allows an ArchiveImportService to mimic the behavior of a DicomImportService which stores the Called AE Title or Calling AE Title in an element for later use by a FileStorageService to assign the object to a FileSystem.

1.9.2 Processors

1.9.2.1 ObjectLogger

The ObjectLogger is a processor stage that logs the passage of objects as they flow past. Objects are passed on unmodified. The log entries are made in the system log and can be viewed in the log viewer servlet on the main admin server page. This stage is intended for initial configuration testing for a new trial; it creates one entry in the system log for each object it receives (unless the interval attribute is supplied with a value greater than one). Logging every object in a large production system can produce an unwieldy log file. The configuration element for the ObjectLogger is:

        <ObjectLogger
            name="ObjectLogger"
            class="org.rsna.ctp.stdstages.ObjectLogger"
            interval="1"
            verbose="yes" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • interval is the interval between objects to log. The default is 1. This attribute can be used to measure performance without imposing a large load on the log file by setting the value to, for example, 1000.
  • verbose increases the amount of information logged.
1.9.2.2 ObjectCache

The ObjectCache is a processor stage that caches the current object as it flows past. Objects are passed on unmodified. The cached object is saved as a separate file to capture the object before it is changed by downstream processors. This stage is intended for use in conjunction with an audit stage that logs changes to objects or for special uses of the DirectoryExportService stage in the RSNA Image Sharing project. To make the cached object available to subsequent stages, the id attribute is mandatory. The configuration element for the ObjectCache is:

        <ObjectCache
            name="ObjectCache"
            class="org.rsna.ctp.stdstages.ObjectCache"
            id="stage ID"
            root="root-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ObjectCache for temporary storage.
1.9.2.3 DicomAuditLogger

The DicomAuditLogger is a processor stage that makes entries in an AuditLog plugin for DicomObjects. Objects are passed on unmodified. The AuditLog entries contain the values of specified elements in the DicomObject and optionally the corresponding elements in a DicomObject contained in the specified ObjectCache stage. The configuration element for the DicomAuditLogger is:

        <DicomAuditLogger
            name="DicomAuditLogger"
            class="org.rsna.ctp.stdstages.DicomAuditLogger"
            root="roots/DicomAuditLogger"
            auditLogID="AuditLog"
            auditLogTags="PatientID;PatientName;SOPInstanceUID"
            cacheID="ObjectCache"
            level="instance" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • root is a directory for use by the DicomAuditLogger for temporary storage.
  • auditLogID is the ID of an AuditLog plugin in which entries are to be made.
  • auditLogTags is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as dcm4che element names or in standard DICOM notation, e.g. (0008,0050).
  • cacheID is the ID of an ObjectCache stage.
  • level specifies granularity of the log. Three levels are supported:
  • patient: one entry for each unique PatientID processed by the stage
  • study: one entry for each unique StudyInstanceUID processed by the stage
  • instance: one entry for each unique SOPInstanceUID processed by the stage

Notes:

  1. If the cacheID attribute is not specified, or if it does not point to an ObjectCache stage, the AuditLog entries contain only the values of the DicomObject being processed.
  2. If the cacheID attribute points to an ObjectCache stage, and that stage can supply a DicomObject, the AuditLog entry contains the values of both the DicomObject being processed and the cached object.
  3. By caching an object before anonymization and putting a DicomAuditLogger after the anonymizer, you can create AuditLog entries with the PHI and anonymized values. (Note that the AuditLog is visible only to an admin user.)
1.9.2.4 MemoryMonitor

The MemoryMonitor is a processor stage that provides a way to force garbage collection and/or to log the current memory in use by CTP. Objects are passed on unmodified. The log entries are made in the system log and can be viewed in the log viewer servlet on the main admin server page. This stage is intended for initial configuration testing for a new trial; it creates one entry in the system log for each object it receives (unless the interval attribute is supplied with a value greater than one). Logging every object in a large production system can produce an unwieldy log file. The configuration element for the MemoryMonitor is:

        <MemoryMonitor
            name="stage name"
            class="org.rsna.ctp.stdstages.MemoryMonitor"
            interval="1"
            collectGarbage="yes"
            logMemoryInUse="yes" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • interval is the interval between objects to log. The default is 1.
  • collectGarbage determines whether the stage is to force the garbage collector to run. The allowed values are yes and no. The default is yes.
  • logMemoryInUse determines whether the stage is to log the current amount of heap space in use. If garbage collection is enabled, the value logged is the memory in use after garbage collection is complete. The allowed values are yes and no. The default is yes.
1.9.2.5 PerformanceLogger

The PerformanceLogger is a processor stage that logs the elapsed time taken by each stage in the pipeline during the processing of the current object. Objects are passed on unmodified. The log entries are made in the system log and can be viewed in the log viewer servlet on the main admin server page. This stage is intended for initial configuration testing for a new trial. The configuration element for the PerformanceLogger is:

        <PerformanceLogger
            name="stage name"
            class="org.rsna.ctp.stdstages.PerformanceLogger"
            interval="1" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • interval is the interval between objects to log. The default is 1. This attribute can be used to measure performance without imposing a large load on the log file by setting the value to, for example, 1000.
1.9.2.6 DicomFilter

The DicomFilter is a processor stage that interrogates a DicomObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If a DicomObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (XmlObject, ZipObject, FileObject) are passed on unmodified. The configuration element for the DicomFilter is:

        <DicomFilter
            name="DicomFilter"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomFilter"
            root="root-directory" 
            script="scripts/dicom-filter.script"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the DicomFilter for temporary storage.
  • script specifies the path to the script for the DicomFilter.
  • quarantine is a directory in which the DicomFilter is to quarantine objects that do not meet the criteria specified in the script file.

For information on specifying acceptance criteria in the script, see The CTP DICOM Filter.

1.9.2.7 XmlFilter

The XmlFilter is a processor stage that interrogates an XmlObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If an XmlObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (DicomObject, ZipObject, FileObject) are passed on unmodified. The configuration element for the XmlFilter is:

        <XmlFilter
            name="XmlFilter"
            id="stage ID"
            class="org.rsna.ctp.stdstages.XmlFilter"
            root="root-directory" 
            script="scripts/xml-filter.script"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the XmlFilter for temporary storage.
  • script specifies the path to the script for the XmlFilter.
  • quarantine is a directory in which the XmlFilter is to quarantine objects that do not meet the criteria specified in the script file.

For information on specifying acceptance criteria in the script, see The CTP XML and Zip Filters.

1.9.2.8 ZipFilter

The ZipFilter is a processor stage that interrogates the manifest of a ZipObject to determine whether it meets criteria specified in a script file. If a script file is either not configured or absent, objects are passed on unmodified. If a ZipObject meets the specified criteria, it is passed on unmodified. If it does not meet the criteria, it is quarantined. If no quarantine is specified, the object is deleted. Objects of any other type (DicomObject, XmlObject, FileObject) are passed on unmodified. The configuration element for the XmlFilter is:

        <ZipFilter
            name="ZipFilter"
            id="stage ID"
            class="org.rsna.ctp.stdstages.ZipFilter"
            root="root-directory" 
            script="scripts/zip-filter.script"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ZipFilter for temporary storage.
  • script specifies the path to the script for the XmlFilter.
  • quarantine is a directory in which the ZipFilter is to quarantine objects that do not meet the criteria specified in the script file.

For information on specifying acceptance criteria in the script, see The CTP XML and Zip Filters.

1.9.2.9 IDMap

The IDMap is a processor stage that constructs map tables for UID elements, AccessionNumber elements, and PatientID elements. The map tables contain the original values and the replacement values created by the first downstream anonymizer. These tables can be accessed by administrators using the IDMap servlet. The configuration element for the IDMap is:

        <IDMap
            name="IDMap"
            class="org.rsna.ctp.stdstages.IDMap"
            root="root-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • root is a directory for use by the IDMap for permanent storage of the map tables.
1.9.2.10 ObjectTracker

The ObjectTracker is a processor stage that tracks objects by date, PatientID, StudyInstanceUID, SeriesInstanceUID, and SOPInstanceUID. The values tracked are the ones in the objects at the time they arrive at the stage, so if they occur before anonymization, they contain PHI. The tracking tables can be accessed by administrators using the ObjectTracker servlet. The configuration element for the IDTracker is:

        <ObjectTracker
            name="ObjectTracker"
            class="org.rsna.ctp.stdstages.ObjectTracker"
            root="root-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • root is a directory for use by the IDTracker for permanent storage of its tracking tables.
1.9.2.11 DatabaseVerifier

The DatabaseVerifier is a processor stage that tracks objects which have been passed to an external database. The stage periodically polls the DatabaseExportService of the external database and maintains its own internal database indicating the status of the objects. The DBVerifierServlet provides a browser interface to the internal database. There can be no anonymizer stages (either DicomAnonymizers or DicomPixelAnonymizers) between the DatabaseVerifier and the DatabaseExportService. (The reason is that the DatabaseVerifier indexes objects by SOPInstanceUID, and it determines that the object in the remote database matches the one seen earlier by the DatabaseVerifier stage by comparing the hashes of the objects' binary contents.) The configuration element for the DatabaseVerifier is:

        <DatabaseVerifier
            name="DatabaseVerifier"
            class="org.rsna.ctp.stdstages.DatabaseVerifier"
            root="root-directory"
            url="http:ip:port"
            username=""
            password=""
            interval="10000"
            maxAge="0" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • root is a directory for use by the DatabaseVerifier for permanent storage of its internal database.
  • url specifies the URL of the verification service of the external database's DatabaseExportService. If ssl is enabled on that verification service, the url attribute must start with https://. If this attribute is missing, no verication is performed, although the internal database is still maintained.
  • username specifies the username credential for inclusion in verification requests. If this attribute is missing or blank, credentials are not supplied in verification requests.
  • password specifies the password credential for inclusion in verification requests. If this attribute is missing or blank, credentials are not supplied in verification requests.
  • interval specifies time in milliseconds between queries of the external database's DatabaseExportService. If this attribute is missing or blank, the interval is 10 seconds (10,000 msec).
  • maxAge specifies the maximum time in days that an unverified object is allowed to remain in the unverified queue before the DatabaseVerifier removes it and stops trying to verify it with the external database's DatabaseExportService. If the attribute is missing or zero, objects remain in the queue forever until they are verified.
1.9.2.12 DicomDecompressor

The DicomDecompressor is a processor stage that converts DicomObjects containing encapsulated pixel data into DicomObjects with the Explicit VR Little Endian (EVRLE) transfer syntax. It passes all other object types unmodified. The DicomDecompressor can optionally be configured with a script file which is used like a DicomFilter to select objects for processing. If a script file is either not configured or absent, all objects are processed. The configuration element for the DicomAnonymizer is:

        <DicomDecompressor
            name="DicomDecompressor"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomDecompressor"
            root="root-directory" 
            skipJPEGBaseline="no"
            script="scripts/dicom-decompressor.script"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the DicomDecompressor for temporary storage.
  • skipJPEGBaseline specifies whether to skip objects that are encoded in JPEGBaseline transfer syntax. Allowed values are yes and no. The default is no.
  • script specifies the path to a script which can be used to select objects for decompression.
  • quarantine is a directory in which the stage is to quarantine objects.

Notes:

  1. The skipJPEGBaseline attribute is just a convenience. Skipping JPEGBaseline objects can also be accomplished by including a script like the following:
!TransferSyntaxUID.equals("1.2.840.10008.1.2.4.50")
1.9.2.13 DicomPlanarConfigurationConverter

The DicomPlanarConfigurationConverter is a processor stage that converts DicomObjects from PlanarConfiguration 1 to PlanarConfiguration 0. It passes all other object types unmodified. The configuration element for the DicomPlanarConfigurationConverter is:

        <DicomPlanarConfigurationConverter 
            name="DicomPlanarConfigurationConverter "
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomPlanarConfigurationConverter"
            root="root-directory" 
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the DicomPlanarConfigurationConverter for temporary storage.
  • quarantine is a directory in which the stage is to quarantine objects.
1.9.2.14 DicomPaletteImageConverter

The DicomPaletteImageConverter is a processor stage that converts DicomObjects from PhotometricInterpretation PALETTE COLOR to RGB. It passes all other object types unmodified. The configuration element for the DicomPaletteImageConverter is:

        <DicomPaletteImageConverter 
            name="DicomPaletteImageConverter"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomPaletteImageConverter"
            root="root-directory" 
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the DicomPaletteImageConverter for temporary storage.
  • quarantine is a directory in which the stage is to quarantine objects.
1.9.2.15 DicomTranscoder

The DicomTranscoder is a processor stage that converts DicomObjects to a specified transfer syntax. It passes all other object types unmodified. The DicomTranscoder can optionally be configured with a script file which is used like a DicomFilter to select objects for processing. If a script file is either not configured or absent, all objects are processed.

An example of the use of this stage is a pipeline that decompresses multiframe ultrasound images, blanks regions of the frames containing PHI, and then recompresses the images to save space. Such a pipeline might have these stages in sequence:

  • DicomDecompressor
  • DicomPixelAnonymizer
  • DicomTranscoder

The configuration element for the DicomTranscoder is:

        <DicomTranscoder
            name="DicomTranscoder"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomTranscoder"
            tsuid="transfer syntax UID"
            quality="100"
            root="root-directory" 
            skipJPEGBaseline="no"
            script="scripts/dicom-transcoder.script"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • tsuid is the DICOM transfer syntax UID of the processed DicomObject. These transfer syntaxes have been tested successfully:
tsuid="1.2.840.10008.1.2.1" (ExplicitVRLittleEndian)
tsuid="1.2.840.10008.1.2.4.70" (JPEGLossLess)
tsuid="1.2.840.10008.1.2.4.80" (JPEGLSLossLess)
tsuid="1.2.840.10008.1.2.4.90" (JPEG2000LossLess)
  • quality is an integer from 1 to 100 to indicate the desired quality of the processed DicomObject. This attribute is ignored in the lossless transfer syntaxes.
  • skipJPEGBaseline specifies whether to skip objects that are encoded in JPEGBaseline transfer syntax. Allowed values are "yes" and "no". The default is "no".
  • root is a directory for use by the DicomTranscoder for temporary storage.
  • script specifies the path to a script which can be used to select objects for transcoding.
  • quarantine is a directory in which the is to quarantine objects that generate quarantine calls during processing.

Notes:

  1. If this stage is configured with the ExplicitVRLittleEndian transfer syntax, its function is similar to that of the DicomDecompressor stage. The difference is that although the output of the DicomDecompressor is EVRLE when it performs a conversion, it only converts DicomObjects that contain encapsulated pixel data, leaving all other objects unmodified, while the DicomTranscoder stage modifies all objects.
  2. The quality attribute is not used in lossless compression transfer syntaxes.
  3. The JPEGBaseline transfer syntax (tsuid="1.2.840.10008.1.2.4.50") does not work correctly.
  4. The skipJPEGBaseline attribute is just a convenience. Skipping JPEGBaseline objects can also be accomplished by including a script like the following:
!TransferSyntaxUID.equals("1.2.840.10008.1.2.4.50")
1.9.2.16 LookupTableChecker

The LookupTableChecker is a processor stage that checks all @lookup function calls in the first downstream DicomAnonymizer stage and quarantines any objects for which the function calls will fail. It adds a servlet with the same context as the id attribute, allowing an admin user to insert values into the lookup table. Once the lookup table has been updated, the quarantined objects can be re-queued and processed successfully. The configuration element for the LookupTableChecker is:

        <LookupTableChecker
            name="LookupTableChecker"
            class="org.rsna.ctp.stdstages.LookupTableChecker"
            root="root-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • root is a directory for use by the LookupTableChecker for permanent storage of the map tables.
1.9.2.17 DicomAnonymizer

The DicomAnonymizer is a processor stage that anonymizes DicomObjects and passes all other object types unmodified. The DicomAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the DicomAnonymizer is:

        <DicomAnonymizer
            name="DicomAnonymizer"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomAnonymizer"
            root="root-directory" 
            lookupTable="scripts/lookup-table.properties"
            script="scripts/dicom-anonymizer.script"
            dicomScript="scripts/dicom-filter.script"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the DicomAnonymizer for temporary storage.
  • lookupTable specifies the path to the lookup table used by the DicomAnonymizer.
  • script specifies the path to the script for the DicomAnonymizer.
  • dicomScript specifies the path to an optional script file (in the same language as the DicomFilter), determining whether to anonymize the object.
  • quarantine is a directory in which the DicomAnonymizer is to quarantine objects that generate quarantine calls during processing.

Notes:

  1. If the lookupTable attribute is missing, the lookup anonymizer function will result in the object being quarantined.
  2. The script attribute specifies the instructions for modifying the individual elements in a DicomObject. If this attribute is missing, DicomObjects are not modified.
  3. The dicomScript attribute specifies the instructions for determining whether to process the DicomObject. If this attribute is missing, all DicomObjects are accepted for processing. If the attribute is present, the computation must produce the value true for the DicomObject to be processed.
  4. If the @integer function is used in the DicomAnonymizer script, the starting point can be reset by deleting the integers.db and integers.lg files from the directory specified in the root attribute. This will cause new integers to be assigned starting at 1. Deleting the files must be done while CTP is not running.
1.9.2.18 DicomCorrector

The DicomCorrector is a processor stage that tries to fix problems in certain elements in DicomObjects that may have been coded incorrectly. Specifically, it recursively walks the dataset tree (including SQ item datasets) and finds non-private elements with VR=UN. For such elements defined in the standard to have the VRs FD, FL, or CS, it replaces the elements with the correct VR and VM for the data contained in the element. The DicomCorrector passes non-DicomObjects without modification. The configuration element for the DicomCorrector is:

        <DicomCorrector
            name="DicomCorrector"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomCorrector"
            root="root-directory" 
            dicomScript="scripts/dicom-filter.script"
            quarantine="quarantine-directory" />
            quarantineUncorrectedMismatches="no" />
            logUncorrectedMismatches="no" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the DicomAnonymizer for temporary storage.
  • dicomScript specifies the path to an optional script file (in the same language as the DicomFilter), determining whether to process the object.
  • quarantine is a directory in which the DicomCorrector is to quarantine objects that generate quarantine calls during processing.
  • quarantineUncorrectedMismatches specifies whether to quarantine objects in which uncorrectable VR mismatches are detected. Values are "yes" and "no". The default is "no".
  • logUncorrectedMismatches specifies whether to make a log entry for each uncorrectable VR mismatch that is detected. Values are "yes" and "no". The default is "no".

Notes:

  1. The dicomScript specifies the instructions for determining whether to process the DicomObject. If this attribute is missing, all DicomObjects are accepted for processing. If the attribute is present, the computation must produce the value true for the DicomObject to be processed.
  2. The computation specified in the dicomScript is performed on the unmodified dataset, so references to elements that may be corrected may produce unexpected results.
1.9.2.19 DicomPixelAnonymizer

The DicomPixelAnonymizer is a processor stage that blanks regions in DicomObjects which are images and passes all other object types unmodified. The DicomPixelAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the DicomAnonymizer is:

        <DicomPixelAnonymizer
            name="DicomPixelAnonymizer"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomPixelAnonymizer"
            root="root-directory" 
            log="no"
            script="scripts/dicom-pixel-anonymizer.script"
            setBurnedInAnnotation="no"
            test="no"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the DicomPixelAnonymizer for temporary storage.
  • log determines whether the signature matched by the DicomObject is to be listed in the log. Values are "yes" and "no". The default is "no". This feature is intended for use while debugging the script.
  • script specifies the path to the script for the DicomPixelAnonymizer.
  • setBurnedInAnnotation specifies whether to set the BurnedInAnnotation eledment (0028,0301) to "NO" if as matching signature is found. Values are "yes" and "no". The default is "no". If the value is "no", the element is left unmodified.
  • test specifies whether to set the color of blanked regions to black or gray. Values are "yes" and "no". The default is "no". If the value is "no", the regions are set to black. The purpose of this attribute is to make it easy to see what regions are being modified while testing a new script.
  • quarantine is a directory in which the DicomPixelAnonymizer is to quarantine objects that generate quarantine calls during processing.

For information on the script language, see The CTP DICOM Pixel Anonymizer.

Important notes:

  • The DicomPixelAnonymizer can process all objects that do not contain encapsulated pixel data.
  • The DicomPixel anonymizer can also process objects that contain encapsulated pixel data with the JPEGBaseline transfer syntax (1.2.840.10008.1.2.4.50).
  • To allow objects containing encapsulated pixel data with other transfer syntaxes to be processed, include a DicomDecompressor stage before the DicomPixelAnonymizer. When including the DicomDecompressor stage, setting its skipJPEGBaseline attribute to "yes" will preserve the compression of JPEGBaseline objects.
  • The DicomPixelAnonymizer does not remove PHI from the non-pixel data in an object. To de-identify that data, include a DicomAnonymizer stage in the pipeline.
1.9.2.20 XmlAnonymizer

The XmlAnonymizer is a processor stage that anonymizes XmlObjects and passes all other object types unmodified. The XmlAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the XmlAnonymizer is:

        <XmlAnonymizer
            name="XmlAnonymizer"
            id="stage ID"
            class="org.rsna.ctp.stdstages.XmlAnonymizer"
            root="root-directory" 
            script="scripts/xml-anonymizer.script" 
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the Anonymizer for temporary storage.
  • script specifies the path to the script for the XmlAnonymizer.
  • quarantine is a directory in which the XmlAnonymizer is to quarantine objects that generate quarantine calls during processing.
1.9.2.21 ZipAnonymizer

The ZipAnonymizer is a processor stage that anonymizes ZipObjects and passes all other object types unmodified. The ZipAnonymizer is configured with a script file. If a script file is either not configured or absent, objects are passed unmodified. The configuration element for the ZipAnonymizer is:

        <ZipAnonymizer
            name="ZipAnonymizer"
            id="stage ID"
            class="org.rsna.ctp.stdstages.ZipAnonymizer"
            root="root-directory" 
            script="scripts/zip-anonymizer.script" 
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the Anonymizer for temporary storage.
  • script specifies the path to the script for the ZipAnonymizer.
  • quarantine is a directory in which the ZipAnonymizer is to quarantine objects that generate quarantine calls during processing.
1.9.2.22 EmailService

The EmailService is a processor stage that sends emails when studies are received. An email is sent for each study, including the numbers of objects, series, and images in the study, as well as other information that is specified in the configuration element below. The configuration element for the EmailService is:

        <EmailService
            name="EmailService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.EmailService"
            root="root-directory" 
            script="scripts/EmailService.script" 
            smtpServer="SMTP server URL"
            username=""
            password=""
            to=""
            from=""
            cc=""
            subject=""
            includePatientName="no"
            includePatientID="no"
            includeModality="no"
            includeStudyDate="no"
            includeAccessionNumber="no"
            logSentEmails="no"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the EmailService for temporary storage.
  • script specifies the path to the script for the EmailService. This script determines whether a DicomObject is to be considered by the stage.
  • smtpServer is the URL of the SMTP server to be used by the EmailService to send emails.
  • username is the username for authentication on the SMTP server. If blank, no authentication is done.
  • password is the password for authentication on the SMTP server. If the username is blank, the password is ignored.
  • to is the email address of the recipient of the emails. If multiple recipients are necessary, their addresses must be separated by commas.
  • from is the email address of the sender.
  • cc is the email address of the cc recipients. If multiple cc recipients are necessary, their addresses must be separated by commas.
  • subject is the text for the subject line. This can be used to identify the name of the trial.
  • includePatientName specifies whether to include the patient name in the email. Allowed values are yes and no. The default is no.
  • includePatientID specifies whether to include the patient ID in the email. Allowed values are yes and no. The default is no.
  • includeModality specifies whether to include the modality in the email. Allowed values are yes and no. The default is no.
  • includeStudyDate specifies whether to include the study date in the email. Allowed values are yes and no. The default is no.
  • includeAccessionNumber specifies whether to include the study accession number in the email. Allowed values are yes and no. The default is no.
  • quarantine is a directory in which the EmailService is to quarantine objects if necessary.

Notes:

  • The patient name, patient ID, modality, and study date values are those of the first image received for the study. These values may contain PHI if the EmailService stage occurs before the objects are anonymized, either at the source or in preceding stages in the pipeline.

1.9.3 Storage Services

1.9.3.1 FileStorageService

The FileStorageService stores objects in a file system. It automatically defines subdirectories beneath its root directory and populates them accordingly. The configuration element for the StorageService is:

        <FileStorageService
            name="FileStorageService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.FileStorageService"
            root="D:/storage" 
            type="month"
            timeDepth="0"
            acceptDuplicateUIDs="yes"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes" 
            returnStoredFile="yes"
            setWorldReadable="no"
            setWorldWritable="no"
            fsNameTag="00097770"
            autoCreateUser="no"
            port="85"
            ssl="no"
            requireAuthentication="no"
            exportDirectory=""
            quarantine="quarantine-directory" >

            <jpeg frame="first" wmax="10000" wmin="96" q="-1" />

        </FileStorageService>

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is the root directory of the storage tree.
  • type determines the structure of the storage tree. The allowed values are:
    • year: root/FSNAME/year/studyName, e.g. root/2008/123.456.789
    • month: root/FSNAME/year/month/studyName, e.g. root/2008/06/123.456.789
    • week: root/FSNAME/year/week/studyName, e.g. root/2008/36/123.456.789
    • day: root/FSNAME/year/day/studyName, e.g. root/2008/341/123.456.789
    • none: root/FSNAME/studyName, e.g. root/123.456.789
(FSNAME is the name of the file system to which the study belongs. See the fsNameTag attribute below for additional information.)
  • timeDepth specifies the length of time in days that studies are stored. The default value is 0, which means forever. If timeDepth is greater than zero, studies older than that value are automatically removed from storage.
  • acceptDuplicateUIDs determines whether objects with duplicate UIDs are to be stored under separate names or if a newer duplicate object is to overwrite an older one. Values are "yes" and "no". The default is "yes".
  • acceptDicomObjects determines whether DicomObjects are to be stored. Values are "yes" and "no". The default is "yes".
  • acceptXmlObjects determines whether XmlObjects are to be stored. Values are "yes" and "no". The default is "yes".
  • acceptZipObjects determines whether ZipObjects are to be stored. Values are "yes" and "no". The default is "yes".
  • acceptFileObjects determines whether FileObjects are to be stored. Values are "yes" and "no". The default is "yes".
  • returnStoredFile specifies whether the original object or a new object pointing to the file in the storage system is to be returned for processing by subsequent stages. Values are "yes" and "no". The default is "yes".
  • setWorldReadable determines whether the FileStorageService makes all files and directories readable by all users. Values are "yes" and "no". The default is "no". This attribute should only be used if users or other programs are to be allowed to access files without using the FileStorageService web server. This feature is not recommended.
  • setWorldWritable determines whether the FileStorageService makes all files and directories writable by all users. Values are "yes" and "no". The default is "no". This attribute should only be used if users or other programs are to be allowed to write files into the FileStorageService without using the FileStorageService web server. This feature is not recommended.
  • fsNameTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is used to specify the name of the root directory's child under which to store a received object. If the attribute is missing from the configuration or if the specified element is missing from the received object, or if the contents of the specified element are blank, the object is stored under the "__default" tree.
  • autoCreateUser determines whether the StorageService is to create a user for each new value of the element specified by the fsNameTag. The default is "no". The user is created with both the username and the password set to the value of the element specified by the fsNameTag.
  • port specifies the port on which a web server is to be started to provide access to the stored studies. If the attribute is missing, no web server is started for the FileStorageService.
  • ssl determines whether the web server uses SSL. Values are "yes" and "no". The default is "no".
  • requireAuthentication determines whether users are forced to log in to the web server. Values are "yes" and "no". The default is "no".
  • exportDirectory specifies a directory into which the web server can copy files when the a user with the admin privilege clicks a link on the Study List page. This feature can be used to allow studies to be cached in the FileStorageService and then manually released to the DirectoryImportService of another pipeline for subsequent processing and/or export.
  • quarantine is a directory in which the StorageService is to quarantine objects that cannot be stored.

For more information on the embedded web server in the FileStorageService, see The CTP FileStorageService Web Server and The CTP FileStorageService Access Mechanism.

Notes:

  1. Files are stored in a tree of directories with the root of the tree as defined in the root attribute of the configuration element.
  2. Below the root are directories called FileSystems. A FileSystem may be defined by the value of an element in the object being stored by using the fsNameTag attribute. If no FileSystem is defined by the object, it is stored in the default FileSystem (__default).
  3. Within a FileSystem, studies are grouped depending on the value of the type attribute. For example, if the type attribute has the value "month", studies are organized by year and month, e.g. "2007/09".
  4. At the bottom of the hierarchy are directories organized by StudyInstanceUID, thus grouping all objects received for a specific study together in one directory.
  5. Objects are stored with standard file extensions:
    • .dcm for DicomObjects
    • .xml for XmlObjects
    • .zip for ZipObjects
    • .md for FileObjects
  6. Any object not containing a StudyInstanceUID or StudyUID is stored in the bullpen directory.
  7. The fsNameTag attribute can be used to create storage trees based on element values like PatientID. It can also access elements in private groups, as might be done if the calledAETTag attribute of the DicomImportService is used to pass destination information. Similarly, the callingAETTag attribute of the DicomImportService could be used to pass source information, allowing separation of objects into FileSystems based on the sending system.
  8. The fsNameTag attribute supports a list of element tags in the form fsNameTag=”00400275::00401001”. There is no limit to the number of tags in the sequence. All but the last tag must refer to SQ elements. This feature allows the file system name to be obtained from an element buried in an item dataset that may be several levels down from the root dataset. Only the first item dataset is searched at each level.

Notes on the jpeg child element:

  1. The optional jpeg child element causes the FileStorageService to create one or more JPEG images for each DICOM image when it is stored.
  2. The jpeg element should be included only when pre-computed JPEG images are required by an external application which directly references the disk drive containing the stored files (for example, the NBIA application).
  3. When images are accessed through the FileStorageService web server's Storage Servlet or Ajax Servlet, they are produced dynamically, and jpeg elements are not required.
  4. Multiple jpeg elements may appear if multiple images must be created (with different parameters) for each stored DICOM image.
  5. The attributes specify the frame, width and quality parameters of the created image:
    • The frame attribute which frame in multi-frame objects is to be saved. It has four allowed values: first, middle, last, all. The default is first. If all is selected and the StorageService supports saving all frames, then one JPEG image is created for each frame in the object. NOTE: The FileStorageService does not support the frame attribute and always behaves as if frame="first" is specified. The BasicFileStorageService fully supports the frame attribute.
    • If the width of the parent DICOM image lies between the values of wmax and wmin, the width of the created image will be equal to the width of the parent.
    • wmax specifies the maximum width of the created image. The default is 10000.
    • wmin specifies the minimum width of the created image. The default is 96.
    • The height of the created image is automatically computed to provide the same aspect ratio as the parent.
    • q specifies the compression quality for the created image. Allowed values are 1 through 100, with larger values producing better quality (and larger file sizes). The system default is triggered by specifying -1, which generally produces a good image.
  6. A JPEG image file created in response to a jpeg child element is stored in the same directory as the parent DICOM image.
  7. The filename of a created image is constructed from:
    • the name of the parent file,
    • a suffix in square brackets identifying the parameters used to create it,
    • and a .jpeg extension.
  8. The parameters appear in the order: wmax, wmin, q, and are separated by semicolons.
  9. For example, a JPEG image created from FO-29126.dcm might have the name FO-29126.dcm[400;96;-1].jpeg.
  10. If a 96-pixel wide thumbnail is required for an external application, the following jpeg child element could be specified:
        <jpeg wmax="96" wmin="96" q="-1" />
1.9.3.2 BasicFileStorageService

The BasicFileStorageService stores objects in a file system. It provides no organization of the files and no means of access to them. It is intended for use in situations where direct file access is provided through an external application like NBIA. The configuration element for the StorageService is:

        <BasicFileStorageService
            name="BasicFileStorageService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.BasicFileStorageService"
            index="D:/storage"
            root="D:/storage/root" 
            nLevels="3" 
            maxSize="200" 
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            returnStoredFile="yes"
            logDuplicates="no"
            rejectDuplicates="no"
            acceptClones="yes"
            quarantine="quarantine-directory" >

            <jpeg frame="first" wmax="10000" wmin="96" q="-1" />

        </BasicFileStorageService>

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • index is the directory in which the index is stored.
  • root is the root directory of the storage tree.
  • nLevels defines the depth of the storage tree. The default is 3.
  • maxSize defines the maximum number of files or directories in each node of the storage tree. The default is 200.
  • acceptDicomObjects determines whether DicomObjects are to be stored. Values are "yes" and "no". The default is "yes".
  • acceptXmlObjects determines whether XmlObjects are to be stored. Values are "yes" and "no". The default is "yes".
  • acceptZipObjects determines whether ZipObjects are to be stored. Values are "yes" and "no". The default is "yes".
  • returnStoredFile specifies whether the original object or a new object pointing to the file in the storage system is to be returned for processing by subsequent stages. Values are "yes" and "no". The default is "yes".
  • logDuplicates specifies whether an entry is to be made in the log whenever a file is received which has the same SOPInstanceUID as a file which has already been stored. Values are "yes" and "no". The default is "no".
  • rejectDuplicates specifies whether a file is to be quarantined (and not saved) if it has the same SOPInstanceUID as a file which has already been stored. Values are "yes" and "no". The default is "no". See acceptClones below.
  • acceptClones specifies whether a file is to be accepted even if it has the same SOPInstanceUID as a file which has already been stored, provided that it is identical to the previously stored file. Values are "yes" and "no". The default is "yes". See the special notes on this attribute below.
  • quarantine is a directory in which the StorageService is to quarantine objects that cannot be stored.

Notes:

  1. Files are stored in a tree of directories with the root of the tree as defined in the root attribute of the configuration element.
  2. The index directory must not appear under the root directory. A convenient approach is shown in the example above, where the root directory appears under the index directory.
  3. The number of files which will be stored under any directory in the root is maxSize**(nLevels-1). For the default values of the nLevels and maxSize parameters (3 and 200), this is 40,000.
  4. Directories are created at the top level (root) when existing directories are full. There is no maximum number of top-level directories; however, it is wise to consider the number of objects which are expected to be stored and to select values of nLevels and maxSize which would keep the number of top-level directories below 1000.
  5. For storage requirements of 10 million files, the default values of nLevels and maxSize are fine.
  6. For storage requirements of 10 billion files, nLevels="4" and maxSize="300" would work well.
  7. Files are stored as leaves at the bottom of the tree.
  8. No organization into related groups (e.g. by StudyInstanceUID) is provided.
  9. Files are indexed by UID (e.g., SOPInstanceUID).
  10. A duplicate object (e.g., one whose UID matches the UID of an object already stored) overwrites the stored object in the same place in the storage system (e.g., the same directory and the same filename), and any required jpeg images are recreated, overwriting the previously stored ones.
  11. FileObjects, which do not contain UIDs, are not stored; they are simply passed to the next stage.
  12. Files are stored with standard file extensions:
    • .dcm for DicomObjects
    • .xml for XmlObjects
    • .zip for ZipObjects
  13. See the section on the FileStorageService for a description of the jpeg child element.
  14. If jpeg child elements appear, the files which they create are not counted against the maxSize parameter. Thus, if maxSize is 200 and two jpeg child elements appear, the bottom directories in the tree could contain 600 files. If frame="all" is specified and multi-frame objects are to be processed, this could result in many times that number. In situations where a given choice of maxSize could result in more than 1000 files in one directory, it is advisable to reduce maxSize and increase nLevels.

Notes on the use of the logDuplicates, rejectDuplicates, and acceptClones:

  • The default operation is to overwrite a stored file if another file is subsequently received with the same UID.
  • If it is desired to suppress the storage and subsequent processing of files that have the same UIDs as previously stored files, the rejectDuplicates attribute must be set to "yes".
  • If it is desired only to suppress the storage and subsequent processing of files that have the same UIDs but are not identical to the previously stored files, then the acceptClones attribute must be set to "no".
  • The operation of the rejectDuplicates and acceptClones attributes is not affected gy the settin gof the logDuplicates attribute.
1.9.3.3 DirectoryStorageService

The DirectoryStorageService stores DicomObjects in a directory structure with no index. It optionally organizes the objects in subdirectories according to a list of element tags. The organization can be obtained either from the current object or from an object that had been cached in another stage. This StorageService is intended for use in situations where files are passed to an external application like the Edge Server in the RSNA Image Sharing Project. It can also be used to pass files to DirectoryImportService stages in other pipelines. The configuration element for the StorageService is:

        <DirectoryStorageService
            name="DirectoryStorageService"
            id="stage ID"
            dicomScript="scripts/dss.script"
            cacheID="cache stage ID"
            structure="dir1/dir2/dir3/..."
            defaultString="UNKNOWN"
            whitespaceReplacement="_"
            class="org.rsna.ctp.stdstages.DirectoryStorageService"
            root="D:/storage/root" 
            setStandardExtensions="no"
            filenameTag=""
            acceptDuplicates="yes"
            returnStoredFile="yes"
            quarantine="quarantine-directory" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • dicomScript specifies the path to a script that examines the contents of a DicomObject and determines whether the object is to be accepted for storage. If the attribute is missing, all objects are accepted.
  • cacheID is the ID of an ObjectCache stage that can supply an object from which to obtain values to populate the directory names in the storage hierarchy. If this attribute is missing, the values are obtained from the current object.
  • root is the root directory of the storage tree.
  • structure is an optional sequence of directory names specifying the hierarchy of the storage tree. Directories in the sequence are separated by slashes. Each directory name in the sequence can consist of text and/or DICOM tags. If a directory name in the sequence includes one or more DICOM tags, the values of the corresponding elements in the current (or cached) DICOM object are substituted in the directory name for the tags. See the notes below for more details and examples of directory structures. If this attribute is missing, all files are stored in the root directory.
  • defaultString specifies a string used in place of a DICOM tag if the corresponding element is either missing or empty. If this attribute is missing, "UNKNOWN" is used.
  • whitespaceReplacement specifies a string used to replace whitespace, slash, or backslash characters in a directory name. If this attribute is missing, the underscore character is used.
  • setStandardExtensions specifies whether the stored files are to be assigned file extensions based on their file types (".dcm" for DicomObjects, ".xml" for XmlObjects, ".zip" for ZipObjects. and ".md" for all other object types). Values are "yes" and "no". The default is "no".
  • filenameTag specifies a DICOM element from which to obtain a filename to use in the storage of the file. If this attribute is missing or if it references an element that is not present (or is empty), the SOPInstanceUID is used for the filename.
  • acceptDuplicates specifies whether a file with the same name as an existing file is to overwrite the existing file or is to be saved with a different name. Values are "yes" and "no". The default is "yes", which means that all files are to be kept, creating new names when necessary.
  • returnStoredFile specifies whether the original object or a new object pointing to the file in the storage system is to be returned for processing by subsequent stages. Values are "yes" and "no". The default is "yes".
  • quarantine is a directory in which the StorageService is to quarantine objects that cannot be stored.

Notes:

  1. The stored object's SOPInstanceUID is used as the file name.
  2. No file extension is appended to the SOPInstanceUID when creating the file name unless setStandardExtensions is set to "yes".
  3. In directory names, DICOM tags are hex integers, formatted in parentheses or square brackets. Tags may optionally include commas to separate the group and element numbers. Leading zeroes are good form but not required in group numbers. Leading zeroes are required in element numbers. Commas are removed before parsing tags, so (20,D) is not equivalent to (20,000D).
  4. This example shows tags for PatientID/AccessionNumber/StudyInstanceUID: (0010,0020)/[00080050]/(0020,000D).
  5. This example shows how text and multiple tags can be combined in a single directory name: (0010,0020)/(0020,000D) - [00080050]. In this case the PatientID is used as the top-level directory, with a subdirectory consisting of the StudyInstanceUID, a space, a hyphen, another space, and the AccessionNumber.
  6. Whitespace replacement is performed on all the text of a directory name, after substitution of the DICOM element values for any tags in the name, so in the example in the previous note, the separator between the StudyInstanceUID and the AccessionNumber would actually appear in the directory name as "_-_".
  7. DICOM tags in directory names can include references to elements in sequence element item datasets in the form (0040,0275)::(0040,1001). There is no limit to the number of tags in the sequence. All but the last tag must refer to SQ elements. Only the first item dataset is searched at each level.

1.9.4 Export Services

1.9.4.1 HttpExportService

The HttpExportService queues objects and transmits them via HTTP with Content-Type application/x-mirc. The configuration element for the HttpExportService is:

        <HttpExportService
            name="HttpExportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.HttpExportService"
            root="root-directory" 
            url="http://ipaddress:port/path"
            zip="no"
            sendDigestHeader="no"
            username="username"
            password="password"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes"
            dicomScript="scripts/df.script"
            xmlScript="scripts/xf.script"
            zipScript="scripts/zf.script"
            logDuplicates="no"
            auditLogID="id"
            auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber"
            interval="5000" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ExportService for internal storage and queuing.
  • url specifies the destination system's URL.
  • zip determines whether files will be zipped before transmission (yes) or not (no). The default is no. This feature is intended for use with the HttpImportService.
  • sendDigestHeader determines whether a Digest header is to be included in the connection, allowing the destination to detect errors at the application level. (yes or no). The default is no.
  • username specifies the username credential for inclusion in the header during the transmission. This allows an HttpImportService to authenticate transmissions. If the username attribute is not present or has a whitespace value, no header is generated.
  • password specifies the password credential for inclusion in the header during the transmission.
  • acceptDicomObjects determines whether DicomObjects are to be exported.
  • acceptXmlObjects determines whether XmlObjects are to be exported.
  • acceptZipObjects determines whether ZipObjects are to be exported.
  • acceptFileObjects determines whether FileObjects are to be exported.
  • dicomScript specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
  • xmlScript specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
  • zipScript specifies the path to a script which examines the contents of a ZipObject and determines whether the object is to be exported.
  • logDuplicates specifies whether to make a log entry for each object whose UID matches that of any other object in the last 20 that have been exported. This capability is intended primarily for configuration debugging.
  • auditLogID is the value of the id attribute of an AuditLog plugin in which to log exported DICOM objects. If this attribute is missing or if the value does not correspond to an AuditLog plugin, no audit logging is performed.
  • auditLogTags is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as dcm4che element names or in standard DICOM notation, e.g. (0008,0050).
  • interval is the sleep time (in milliseconds) between polls of the export queue.

Notes:

  1. The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
  2. The protocol part of the url can be http or https, the latter causing connections to be initiated using secure sockets layer.
  3. If a proxy server is not in use, the proxy attributes must be omitted.
  4. For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") and the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See The CTP DICOM Filter and The CTP XML and Zip Filters for information about the script languages.
1.9.4.2 PolledHttpExportService

The PolledHttpExportService queues objects and transmits them in the HTTP response stream of a received connection. Files are transmitted with Content-Type equal to application/x-mirc. This ExportService is designed to work in conjunction with the PollingHttpImportService to allow penetration of a firewall without having to open an inbound port, as described in Security Issues. The configuration element for the Polled HttpExportService is:

        <PolledHttpExportService
            name="PolledHttpExportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.PolledHttpExportService"
            root="root-directory" 
            port="listening-port"
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes"
            dicomScript="scripts/df.script"
            xmlScript="scripts/xf.script"
            zipScript="scripts/zf.script">

            <accept ip="..."/>
            <reject ip="..."/>

        </PolledHttpExportService>

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ExportService for internal storage and queuing.
  • port is the port on which the ExportService listens for connections.
  • acceptDicomObjects determines whether DicomObjects are to be exported.
  • acceptXmlObjects determines whether XmlObjects are to be exported.
  • acceptZipObjects determines whether ZipObjects are to be exported.
  • acceptFileObjects determines whether FileObjects are to be exported.
  • dicomScript specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
  • xmlScript specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
  • zipScript specifies the path to a script which examines the contents of a ZipObject and determines whether the object is to be exported.
  • The accept child element can be used to specify a white list of IP addresses determining which connections are to be accepted. One accept child element must appear for each IP address in the white list. If the white list is empty, the white list is not used to filter connections.
  • The reject child element can be used to specify black lists of IP addresses determining which connections are to be rejected. One reject child element must appear for each IP address in the black list. If the black list is empty, the black list is not used to filter connections.
  • For a connection to be accepted, the IP address of the remote client must be accepted by the white list and and not rejected by the black list.

Notes:

  1. For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") and the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See The CTP DICOM Filter and The CTP XML and Zip Filters for information about the script languages.
  2. The PolledHttpExportService does not support SSL.
1.9.4.3 DicomExportService

The DicomExportService queues objects and transmits them to a DICOM Storage SCP. The configuration element for the DicomExportService is:

        <DicomExportService
            name="DicomExportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomExportService"
            root="root-directory" 
            quarantine="quarantine-directory"
            auditLogID=""
            auditLogTags=""
            url="dicom://DestinationAET:ThisAET@ipaddress:port"
            associationTimeout="0"
            forceClose="no"
            hostTag="00097774" 
            portTag="00097776" 
            calledAETTag="00097770" 
            callingAETTag="00097772"
            dicomScript="scripts/df.script"
            auditLogID="id"
            auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber"
            throttle="0"
            interval="5000" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage. This attribute is required only when the stage is accessed by other stages. Its value, when supplied, must be unique across all pipelines.
  • root is a directory for use by the ExportService for internal storage and queuing.
  • quarantine is a directory in which the DicomExportService is to quarantine objects that fail to be transmitted because they were rejected by the destination DICOM Storage SCP (usually because a presentation context could not be negotiated). Such objects would always fail, so they must be removed from the export queue. If no quarantine directory is specified, failed objects are deleted.
  • auditLogID specifies the ID of an AuditLog plugin. If this attribute is not empty, and if an AuditLog plugin is configured with that ID, an entry is made in the AuditLog for each successful transmission.
  • auditLogTags specifies the elements from the transmitted objects that are to be included in the AuditLog entry. Elements can be specified by their dcm4che names or their numeric values. Elements must be separated by semicolons. This is an example of several elements, including one from a private group:
auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber;(0009,7790)"
  • url specifies the destination DICOM Storage SCP's URL.
  • associationTimeout specifies the length of time an association is to be left open after a transfer before it is closed automatically. Allowed valuers are yes and no. The default value is no. This parameter can be set to yes if it appears that the destination SCP is resetting the association and causing a problem with transfers.
  • forceClose specifies whether the DICOM association is to be closed after each transfer. The value is specified in seconds. A value of zero (the default) means to disable the automatic association closing mechanism.
  • hostTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is to be used by the DicomExportService to obtain the host name (or IP address) of the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, or if the identified element does not exist in a DicomObject being exported, the DicomExportService uses the host specified in the url attribute.
  • portTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is to be used by the DicomExportService to obtain the port of the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, or if the identified element does not exist in a DicomObject being exported, the DicomExportService uses the host specified in the url attribute.
  • calledAETTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is to be used by the DicomExportService to obtain the AE Title which specifies the receiver in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, or if the identified element does not exist in a DicomObject being exported, the DicomExportService uses the AE Title specified in the url attribute.
  • callingAETTag is an optional DICOM element, specified in hex with no comma separating the group and element numbers, which is to be used by the DicomExportService to obtain the AE Title which specifies the sender in the association. If the attribute is missing or zero, or if the value does not parse as a hex integer, or if the identified element does not exist in a DicomObject being exported, the DicomExportService uses the AE Title specified in the url attribute.
  • dicomScript specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
  • auditLogID is the value of the id attribute of an AuditLog plugin in which to log exported objects. If this attribute is missing or if the value does not correspond to an AuditLog plugin, no audit logging is performed.
  • auditLogTags is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as dcm4che element names or in standard DICOM notation, e.g. (0008,0050).
  • throttle sets the minimum time (in milliseconds) between exports to the destination. The default is 0 milliseconds. The maximum allowed value is 5 seconds.
  • interval is the sleep time (in milliseconds) between polls of the export queue. The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.

Notes:

  • For an object to be accepted for export, the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See The CTP DICOM Filter for information about the script language.
1.9.4.4 DicomSTOWRSExportService

The DicomSTOWRSExportService queues objects and transmits them via the DICOM STOW-RS protocol. The configuration element for the DicomSTOWRSExportService is:

        <DicomSTOWRSExportService
            name="DicomSTOWRSExportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.DicomSTOWRSExportService"
            root="root-directory" 
            url="http://ipaddress:port/path"
            includeContentDispositionHeader="no"
            username="username"
            password="password"
            dicomScript="scripts/df.script"
            logDuplicates="no"
            auditLogID="id"
            auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber"
            interval="5000" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ExportService for internal storage and queuing.
  • url specifies the destination system's URL.
  • includeContentDispositionHeader determines whether a Content-Disposition header is to be included in the connection. This header is not required in the protocol, but in some cases, it may be useful. Allowed values are yes or no. The default is no.
  • username specifies the username credential for inclusion in the header during the transmission. This allows an HttpImportService to authenticate transmissions. If the username attribute is not present or has a whitespace value, no header is generated.
  • password specifies the password credential for inclusion in the header during the transmission.
  • dicomScript specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
  • logDuplicates specifies whether to make a log entry for each object whose UID matches that of any other object in the last 20 that have been exported. This capability is intended primarily for configuration debugging.
  • auditLogID is the value of the id attribute of an AuditLog plugin in which to log exported DICOM objects. If this attribute is missing or if the value does not correspond to an AuditLog plugin, no audit logging is performed.
  • auditLogTags is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as dcm4che element names or in standard DICOM notation, e.g. (0008,0050).
  • interval is the sleep time (in milliseconds) between polls of the export queue.

Notes:

  1. The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
  2. The protocol part of the url can be http or https, the latter causing connections to be initiated using secure sockets layer.
  3. For an object to be accepted for export, the object must be a DicomObject and the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See The CTP DICOM Filter and The CTP XML and Zip Filters for information about the script languages.
1.9.4.5 FtpExportService

The FtpExportService queues objects and transmits them to an FTP server. The configuration element for the FtpExportService is:

        <FtpExportService
            name="FtpExportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.FtpExportService"
            root="root-directory" 
            url="ftp://ipaddress:port/path"
            username="..."
            password="..."
            acceptDicomObjects="yes"
            acceptXmlObjects="yes"
            acceptZipObjects="yes"
            acceptFileObjects="yes"
            dicomScript="scripts/df.script"
            xmlScript="scripts/xf.script"
            zipScript="scripts/zf.script"
            auditLogID="id"
            auditLogTags="PatientID;SOPInstanceUID;StudyInstanceUID;InstanceNumber"
            interval="5000" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ExportService for internal storage and queuing.
  • url specifies the destination FTP server's URL:
    • ipaddress can be a numeric address or a domain name.
    • port is the port on which the FTP listener listens. The default port is 21.
    • path is the base directory on the FTP server in which the FtpExportService will create StudyInstance directories.
  • username specifies the username under which the FtpExportService will log in to the FTP server.
  • password specifies the password to be used in the login process.
  • acceptDicomObjects determines whether DicomObjects are to be exported.
  • acceptXmlObjects determines whether XmlObjects are to be exported.
  • acceptZipObjects determines whether ZipObjects are to be exported.
  • acceptFileObjects determines whether FileObjects are to be exported.
  • dicomScript specifies the path to a script which examines the contents of a DicomObject and determines whether the object is to be exported.
  • xmlScript specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
  • zipScript specifies the path to a script which examines the contents of a ZipObject and determines whether the object is to be exported.
  • auditLogID is the value of the id attribute of an AuditLog plugin in which to log exported DICOM objects. If this attribute is missing or if the value does not correspond to an AuditLog plugin, no audit logging is performed.
  • auditLogTags is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as dcm4che element names or in standard DICOM notation, e.g. (0008,0050).
  • interval is the sleep time (in milliseconds) between polls of the export queue.

Notes:

  1. For an object to be accepted for export, the object type must be accepted (e.g., acceptDicomObjects="yes") and the object must pass the script test. If the script attribute is not supplied, the test returns true by default and the object is accepted. See The CTP DICOM Filter and The CTP XML and Zip Filters for information about the script languages.
  2. The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
  3. The FtpExportService stores files in subdirectories of the path part of the URL, organized by StudyInstanceUID (or StudyUID in the case of non-DICOM files). Files not containing a StudyUID are stored in the bullpen directory under the path directory.
  4. If the directory specified by the path does not exist, it is created.
  5. Files are stored within their directories with names that consist of the date and time (to the millisecond) when they were transferred to the server.
  6. If a file is transmitted multiple times, multiple copies of the file will appear with its directory, each with the date/time of the transfer.
  7. No index of the studies and files is created on the server.
1.9.4.6 AimExportService

The AimExportService queues objects and transmits them to an AIM Data Service. The configuration element for the AimExportService is:

        <AimExportService
            name="AimExportService"
            id="stage ID"
            class="org.rsna.ctp.stdstages.AimExportService"
            root="root-directory" 
            url="http://ipaddress:port/path"
            username="username"
            password="password"
            xmlScript="scripts/xf.script"
            logResponses="none"
            quarantine="quarantine-directory"
            interval="5000" />

where:

  • name is any text to be used as a label on configuration and status pages.
  • id is any text to be used to uniquely identify the stage.
  • root is a directory for use by the ExportService for internal storage and queuing.
  • url specifies the destination AIM Data Service URL:
    • ipaddress can be a numeric address or a domain name.
    • port is the port on which the AIM Data Service listens.
    • path is the path identifying the AIM Data Service on the destination server.
  • username specifies the username credential for inclusion in the header during the transmission. If the username attribute is not present or has a whitespace value, no header is generated.
  • password specifies the password credential for inclusion in the header during the transmission.
  • xmlScript specifies the path to a script which examines the contents of an XmlObject and determines whether the object is to be exported.
  • logResponses specifies whether the contents of the response from the AIM Data Service are to be entered into the CTP log file. The recognized values are:
    • all - log all responses
    • failed - log only the responses that indicate that the Data Service rejected the object
    • none - do not log responses.

The default, if the attribute is missing or has any other value, is not to log.

  • quarantine is a directory in which the AimExportService is to quarantine objects that fail to be transmitted because they were rejected by the destination AIM Data Service. If no quarantine directory is specified, failed objects are deleted.
  • interval is the sleep time (in milliseconds) between polls of the export queue.

Notes:

  1. The AimExportService only transmits XmlObjects. It ignores all other object types.
  2. The AimExportService only transmits XmlObjects whose root element is "ImageAnnotation".
  3. The XmlObject must pass the script test. If the xmlScript attribute is not supplied, the test returns true by default and the object is accepted. See The CTP XML and Zip Filters for information about the script language.
  4. The default interval is 5 seconds. The minimum allowed value is one second. The maximum allowed value is 10 seconds.
1.9.4.7 DicomDifferenceLogger

The DicomDifferenceLogger queues objects containing the changes made to DicomObjects processed by its pipeline and submits them to a LogAdapter class written specially to connect to an external database. The configuration element for the DicomDifferenceLogger is:

        <DicomDifferenceLogger
            name="DicomDifferenceLogger"
            class="org.rsna.ctp.stdstages.DicomDifferenceLogger"
            root="roots/DicomDifferenceLogger"
            adapterClass="..."
            cacheID="ObjectCache"
            tags="PatientID;PatientName;SOPInstanceUID"/>

where:

  • name is any text to be used as a label on configuration and status pages.
  • root is a directory for use by the DicomDifferenceLogger for temporary storage.
  • adapterClass is the class name of the external log's adapter class. See Developing a DicomDifferenceLogger LogAdapter for more information.
  • tags is a semicolon-separated list of DICOM element tags whose values are to be included in the AuditLog entry. Tags can be specified as dcm4che element names (DICOM keywords) or in standard DICOM notation, e.g. (0008,0050).
  • cacheID is the ID of an ObjectCache stage.

2 Security Issues

In a clinical trial, transmission of data from image acquisition sites to the principal investigator site involves penetrating at least one firewall. Since the image acquisition site initiates an outbound connection, this only rarely requires special action. At the principal investigator's site, where the connection is inbound, some provision must be made to allow the connection to reach its destination. There are two basic solutions. The simplest solution is to open a port in the firewall at the principal investigator's site and route connections for that port to the computer running the CTP application. In some institutions, however, security policies prohibit this solution. The alternative is to use the PolledHttpExportService and PollingHttpImportService to allow data to flow without having to open any ports on the internal network to inbound connections.

Using this latter solution requires two computers, each running CTP. One computer is placed in the border router's DMZ with one port open to the internet, allowing connections to the HttpImportService of the program running in that computer. That program's pipeline includes a PolledHttpExportService which queues objects and waits for a connection before passing them onward. The second computer is placed on the internal network. Its program has a pipeline which starts with a PollingHttpImportService. That ImportService is configured to make outbound connections to the DMZ computer when a file is requested. This allows files to pass through the firewall on the response stream of the outbound connection without having to open any ports to the internal network.

3 Notes

3.1 Java Cryptography Extension

The anonymizer pipeline stages (DicomAnonymizer, XmlAnonymizer, and ZipAnonymizer) use classes in the Java Cryptography Extension (JCE). The JCE has been included in the Java JRE since at least Java 1.5. To enable the use of the JCE, an entry must appear in the Java/jre/lib/security/java.security file. In the latest Java versions, the entry is automatically included. The most recent versions include a section in the file that looks like this:

# There must be at least one provider specification in java.security.
# There is a default provider that comes standard with the JDK. It
# is called the "SUN" provider, and its Provider subclass
# named Sun appears in the sun.security.provider package. Thus, the
# "SUN" provider is registered via the following:
#
#    security.provider.1=sun.security.provider.Sun
#
# (The number 1 is used for the default provider.)
#
# Note: Providers can be dynamically registered instead by calls to
# either the addProvider or insertProviderAt method in the Security
# class.
#
# List of providers and their preference orders (see above):
#
security.provider.1=sun.security.provider.Sun
security.provider.2=sun.security.rsa.SunRsaSign
security.provider.3=com.sun.net.ssl.internal.ssl.Provider
security.provider.4=com.sun.crypto.provider.SunJCE
security.provider.5=sun.security.jgss.SunProvider
security.provider.6=com.sun.security.sasl.Provider
security.provider.7=org.jcp.xml.dsig.internal.dom.XMLDSigRI
security.provider.8=sun.security.smartcardio.SunPCSC
security.provider.9=sun.security.mscapi.SunMSCAPI

On older Javas, it appears that there are no security.provider... lines in the file. If no provider is specified, the anonymizer will crash when it tries to load a JCE class. If that happens, you will see an error in the Launcher's Console tab. You can either edit the Java/jre/lib/security/java.security file and add the security.provider... lines shown above or uninstall Java and get the latest version.

3.2 Important Note for Unix/Linux Platforms

Unix and its derivatives require that applications listening on ports with numbers less than 1024 have special privileges. On such systems, it might be best to put all import services and all web servers on ports above this value. The default configuration puts the web server on port 80 for Windows platforms because that is the default port for the web. On Linux platforms, the default configuration puts the web server on port 1080. This can be changed easily in the CTP-launcher application when CTP is started.

3.3 ImageIO Tools for Macintosh

The Java Advanced Imaging ImageIO Tools is a component which provides methods for creating and reading image files. It supports many image file types, and it is designed to be extensible. The authors (Gunter Zeilinger, et al.) of the DICOM toolkit (dcm4che) used by CTP extended the ImageIO Tools to support DICOM.

The ImageIO Tools consists of two parts, a top-level library written in Java and runnable on any platform, and a native library which implements certain compression/decompression functions. The latter library is unique to each platform.

The top-level library consists of two files:

  • jai_imageio.jar
  • clibwrapper_jiio.jar

There is no Macintosh installer for the ImageIO Tools. You can obtain the two files above by getting the zip file for a Linux installation and unpacking it. Place the two files into /System/Library/Java/Extensions.

There is no native library available for the Macintosh.

For more information on the ImageIO Tools, see this article.