The CTP FileStorageService Web Server

This article describes the web server which is included in the CTP FileStorageService. The primary intended audience for this article is programmers who wish to write applications to access stored objects, but the article may also be useful for CTP administrators. A prerequisite for fully understanding this article is MIRC Clinical Trial Processor.

1 The FileStorageService

The FileStorageService is a CTP pipeline stage which stores objects in a file storage device. Files are stored in a hierarchy of directories, grouping objects by study. An object is associated with a study by its study UID.

As described indescribed in MIRC Clinical Trial Processor, CTP manages four types of objects:

• DicomObject
• XmlObject
• ZipObject
• FileObject

The first three contain unique identifiers for the study and for the object itself. The FileObject, which contains data of unknown format, cannot provide either a study UID or an object UID, so the FileStorageService assigns a FileObject to a study called "__bullpen" and assigns a unique filename to the object, which is used as its object UID.

The type attribute of the FileStorageService determines the directory structure above the study directories.

• At the top, there may be one or more directories which are called FileSystems. Objects (and their studies) are assigned to a FileSystem based on the fs-name-tag attribute. If the fs-name-tag attribute is supplied in the configuration, a DicomObject is interrogated for the contents of the corresponding element and the contents are used as the name of the FileSystem. If no fs-name-tag is supplied in the configuration, the FileStorageService assigns all objects to the "__default" FileSystem.
• If the type attribute is supplied in the configuration, studies are divided among intermediate directories based on the date on which the study was received:
  • type="year" creates one level of directories based on the year.
  • type="month" creates one level of directories based on the year, and within each year, a level based on the month within the year.
  • type="week" creates one level of directories based on the year, and within each year, a level based on the week number within the year.
  • type="day" creates one level of directories based on the year, and within each year, a level based on the day number within the year.

The purpose of the type attribute is to allow FileStorageServices to handle a large number of objects without any directory growing too large for the operating system to handle efficiently.

2 The FileStorageService Web Server

The port attribute, when present in the configuration element of the FileStorageService, starts a web server. This web server is independent of the one which CTP always provides to allow the operation of the system to be monitored.

The root directory of the web server is the same as the root directory of the FileStorageService. The web server can serve files from any directory in the tree. Since the directory structure, as defined by the type parameter, is transparent to the user, the web server provides two servlets which navigate the indexes of the FileSystems and studies to provide access to objects more conveniently.

• The Storage Servlet is primarily intended to provide users a way to navigate the stored objects with a browser.
• The Ajax Servlet is primarily intended to provide programmatic access to the data in the indexes.

The next sections describe the URLs which are recognized by these servlets.

A brief aside: It is important to remember that when a URL path invokes a servlet, the path is not necessarily a file path on the web server. The servlets use the URL path elements as pointers into indexes at each level in the hierarchy, filling in intermediate levels from configuration information when required. This makes URL paths shorter and absolves the user or programmer from having to know about the internals of the storage hierarchy.

3 The Storage Servlet

The Storage Servlet is invoked when the first path element in the URL is "storage". The servlet only provides an HTTP GET method.

The path "/storage" returns an HTML page listing all the FileSystems. If there is only one file system, this URL acts as if the path were "/storage/{FileSystemName}".

The path "/storage/{FileSystemName}" returns an HTML page listing all the studies in the identified FileSystem. The page contains two links for each study, one which lists the objects in the study and one which displays the DicomObjects in the study with a simple viewer.

The path "/storage/{FileSystemName}/{StudyName}" accesses the objects in the identified study. The servlet recognizes a format query string:

• If the query string "?format=list" is supplied, the returned HTML page is an object listing.
• If the query string "?format=display" is supplied, the returned HTML page is a simple viewer.
• If the query string "?format=zip" is supplied, the servlet returns a zip file containing the objects in the study.
• If the query string "?format=delete" is supplied, the servlet deletes the study from the FileSystem. If the requesting user does not have the admin privilege, this request is rejected and an HTTP 403 error (Not Authorized) is returned.
• If no query string is supplied, the object listing is returned.

The object listing contains a line for each object in the study. Depending on the object type, one or more links are provided.

• The link identifying the file name allows the object's file to be downloaded.
• For DicomObjects, the "List" link returns an HTML page containing a listing of all the elements in the object except the pixels themselves.
• For DicomObjects which are images, the "Display" link returns a JPEG version of the image.

The path "/storage/{FileSystemName}/{StudyName}/{FileName}" returns the identified file in any of these formats:

• With no query string, the object's file is returned.
• With the query string "?format=list", a DicomObject returns an element listing; other object types return the file.
• With the query string "format=jpeg", a DicomObject which is an image returns a JPEG version of the image; other object types return the file.

A note on the StudyName path element: While DICOM UIDs are valid URL path elements, the same cannot be said for UIDs obtained from XmlObjects or ZipObjects. Before using a study UID, the FileStorageService filters it to ensure that it can be used as a valid path element. The result is called a study name. For DicomObjects, the study name and the StudyInstanceUID are identical. For other object types, they may not be. When planning a trial or research activity which uses XmlObjects or ZipObjects, it is wise to ensure that UIDs are always constructed along the lines of a DICOM standard UID.

4 The Ajax Servlet

The Ajax Servlet is invoked when the first path element in the URL is "ajax". The servlet only provides an HTTP GET method.

The second path element identifies a method. Four methods are provided, all of which return XML objects.

The path "/ajax/listFileSystems" returns an XML object listing all the FileSystems. This is an example of the returned object:

<FileSystemList>
    <FileSystem name="__default" /> 
</FileSystemList>

The path "/ajax/listStudies/{FileSystemName}" returns an XML object listing all the studies in the identified FileSystem. This is an example of the returned object:

<index fileSystemName="__default">
    <study>
        <dir>2008\06\ST-56617</dir> 
        <patientName>DBW-WHIMS-54</patientName> 
        <patientID>5400001</patientID> 
        <date>20050125</date> 
        <studyName>2.16.840.1.113662.4.4142577143398.1106696834.2813456275582390951</studyName> 
        <studyUID>2.16.840.1.113662.4.4142577143398.1106696834.2813456275582390951</studyUID> 
    </study>
</index>

The path "/ajax/listObjects/[FileSystemName]/[StudyName]" returns an XML object listing all the objects in the identified study. This is an example of the returned object:

<index  date="20050125" 
        fileSystemName="__default" 
        patientID="5400001" 
        patientName="DBW-WHIMS-54" 
        studyName="2.16.840.1.113662.4.4142577143398.1106696834.2813456275582390951">
    <DicomObject type="image">
        <file>FO-56618.dcm</file> 
        <uid>2.16.840.1.113662.4.4142577143398.1106697462.244347808948193328</uid> 
        <series>3</series> 
        <acquisition>6</acquisition> 
        <instance>71</instance> 
        <rows>256</rows> 
        <columns>256</columns> 
    </DicomObject>
</index>

The path "/ajax/findObject/{UID}" returns an XML object listing all objects found anywhere in the storage system which match the UID. This is an example of the returned object:

<ObjectList uid="2.16.840.1.113662.4.4142577143398.1106697462.244347808948193328">
    <Object 
        fileSystemName="__default" 
        patientID="5400001" 
        patientName="DBW-WHIMS-54" 
        studyName="2.16.840.1.113662.4.4142577143398.1106696834" 
        url="/storage/__default/2.16.840.1.113662.4.4142577143398.1106696834/FO-56618.dcm">
        <DicomObject type="image">
            <file>FO-56618.dcm</file> 
            <uid>2.16.840.1.113662.4.4142577143398.1106697462.244347808948193328</uid> 
            <series>3</series> 
            <acquisition>6</acquisition> 
            <instance>71</instance> 
            <rows>256</rows> 
            <columns>256</columns> 
        </DicomObject>
    </Object>
</ObjectList>

Note that each object returned supplies the URL which can be used to access the object via the Storage Servlet. A query string can be appended to the URL to invoke the display or list features of the servlet.