Thursday, December 15, 2011

Today we are going to diagnose a communication problem between two DICOM applications and hopefully find the reason for the problem and solve it. I know, we didn’t even start talking about the DICOM network protocol, but hey, we’re not going to read all this 3,000 pages standard together before getting our hands dirty, right?
In this post we'll discuss:

Application Entities (AE’s) – the nodes in the DICOM network and their name – AE Title

Association – a network peer-to-peer session between two DICOM applications

Association Negotiation – The first part of the association in which the two AE’s agree on what can and can’t be done during the Association

The Verification Service using the C-ECHO command – a DICOM Service Class that is used to verify a connection, sort of application level ‘ping’.

The Storage Service using the C-STORE command – a DICOM Service that allows one AE to send a DICOM object to another AE

The C in C-ECHO and C-STORE commands stands for Composite. If you remember, in chapter 4 when discussing the DICOM Data Model, we said that DICOM applications exchange composite objects (the DICOM images that we already know) that are composites of modules from different IE's where IE's are the information entities of the Normalized DICOM data model.

Here's the story:

Complaint 20123

Burt Simpson from Springfield Memorial Hospital reports that he can’t send the screen capture to the PACS. He kept clicking the green “Send” button but he always gets the same error: “Operation Failed!”. The log file Burt copied from the system is attached.

You may ask yourself, what’s the point in analyzing a log of an application that we are never going to use? Well, the truth is that all DICOM logs look alike. Actually, most DICOM applications are quite similar because DICOM software implementations have common ancient ancestors. If it’s a C library it may be the DICOM test node, CTN. If it’s Java than it might be dcm4che. Even if it's PHP or other newer languages, the libraries were transcribed and ported from the old C implementations so all DICOM logs are similar.

The log file in this case, named DICOM-20111207-093017.log, is 250MB long and when you double click it notepad hangs for couple of minutes before crashing. When you open the log using EXCEL you see the same pattern repeating 100 times, one time for every click Burt made, and after isolating one repetition you see this relatively short pattern with exactly four log entries that we’re going to analyze together.

The problem is clearly stated in the third log entry and marked as error in the fourth entry. It says that the peer application, the one we want to send our image to refuse to store this type of object. Some toolkits do provide additional helpful information. However, we could have guessed that this will be the problem already in the second entry of the log in the association request response where the presentation context for secondary captured was marked as not supported by the called AE.

The log above is from the following short C# function that I’ve written for this post:

publicvoidSendSCImage(DCXOBJ o)

{

DCXAPP app = newDCXAPP();

app.LogLevel
= LOG_LEVEL.LOG_LEVEL_INFO;

app.StartLogging("DICOM.log");

try

{

DCXREQ req = newDCXREQ();

req.SendObject("RZDCX", "PACS", "localhost",
6104, o);

}

catch (Exception e)

{

MessageBox.Show(e.Message);

}

app.StopLogging();

}

Together with the function CreateSCImage that we’ve written together in chapter 4 we have this little program that creates a Secondary Image in memory and then attempts sending it:

DCXOBJ o =
CreateSCImage();

SendSCImage(o);

Before analyzing the log, let’s go over the code of SendSCImage and make sure that we understand it.
The first three lines creates a DCXAPP class, sets the log level to one less than the highest level (which is ‘Debug’). The DCXAPP class is used to control RZDCX’s global settings. Once it goes out of scope, the settings remains.
Then we have the try-catch block that is very straight forward. We create a DCXREQ class and use it to send the object we’ve created using the SendObject method. DCXREQ is a DICOM requester – a DICOM application that initiates DICOM network with another application and sends DICOM commands. SendObject takes five (5) parameters and encapsulates the whole world of DICOM networking in it. All this log was generated by this single method because it does all the work of DICOM networking for you and that’s exactly what’s unique in RZDCX, that you don’t have to deal with all these details. Still, it’s good to know what’s inside so when things gets messy you’ll have a clue about what might have gone wrong.
Like all the other networking methods of DCXREQ, the first four (4) parameters of SendObject are used to establish the DICOM network connection with the remote DICOM application.
The first parameter is our Application Entity Title. In the DICOM network every node is an Application Entity (AE) and the node name is AE Title. You might ask why do we need an AE title if we have a server name or IP address and the answer is that an AE Title is sort of alias for the combination of IP address and port number. We can run many DICOM applications on a single server. I can run two instances of my PACS on the same computer, one listening on port 104 which is the standard TCP/IP port reserved for DICOM communication and another one listening on port 1104. Each application can be completely independent of the other. I can run as many DICOM applications as I like all having the same IP address.
BTW, DICOM is almost always used in LAN environment and I strongly discourage anyone from using DICOM in WAN environment though I know some people do this but it’s really not a good idea. DICOM protocol is internal, private, in your local network, preferably in its own dedicated subnet.
AE Titles are case sENsItIVE, 16 characters max.
The second parameter is the AE title of the application that we would like to connect to. We sometime call it the target application or called AE Title or responding AE or simply the peer.
The third parameter is the server name (or IP address) of the server that the called AE runs on.
The fourth parameter is the port number the called AE listens on.
That concludes the parameters that are common to all DCXREQ network methods. With these parameters we can start an ‘Association’ with the called AE.
The new term that’s interesting here is Association. What’s that? That’s like a network session. It’s a frame that the conversation with the called AE is going to take place in.
We can divide the DICOM network communication into two parts. The first part is setting up the Association and the second part is exchanging DICOM commands.

99% of the difficulties in DICOM networking are related the first part – the association negotiation.

Even if this stage passed and we start exchanging commands, the chances are that problems are because of faults in the first part.

The fifth parameter is the object we would like to send.
SendObject does the following:

Start a TCP/IP connection

Negotiates the association parameters to agree what can be done during the association

Send the DICOM object

Close the association

Close the TCP/IP connection

Let’s go back to the log and have a look at the first part of the log now. Here it is:

2011-12-1022:22:25.906000 1508 INFO Association Request Parameteres:

Our Implementation Class UID: 2.16.124.113543.6021.2

Our Implementation Version Name: RZDCX_2_0_1_8

Their Implementation Class UID:

Their Implementation Version Name:

Application Context Name: 1.2.840.10008.3.1.1.1

Calling Application Name: RZDCX

Called Application Name: PACS

Responding Application Name: resp AP Title

Our Max PDU Receive Size: 32768

Their Max PDU Receive Size: 0

This part of the log is a textual dump of the first information that was sent to the called AE and is called Association Request. It’s a collection of parameters that describe our application, its capabilities and its intentions in this session.
Every log entry in RZDCX log starts with a timestamp, a thread ID (1508 in this case) and the log level of the entry (INFO in this case). In the complete log above I’ve highlighted the timestamps at the beginning of every log entry.
The first element of in the association request identifies our DICOM implementation.

Our Implementation Class UID: 2.16.124.113543.6021.2

Our Implementation Version Name: RZDCX_2_0_1_8

In this case it’s the RZDCX UID and version number. It’s always interesting because DICOM toolkits and systems have their own little glitches so if you know some system has a problem you already identified and you see that you are dealing with the same implementation, you know how to deal with it. It’s also important when communicating with the other application vendor to report the application version.
In the request dump we see only our implementation info but further down the log in the response dump we will see the identification of the called AE.
Then we have the application context name. This is a UID that is reserved for DICOM. It’s always the same.

Application Context Name: 1.2.840.10008.3.1.1.1

Next we have the AE titles: the calling AE title and the called AE title.

Calling Application Name: RZDCX

Called Application Name: PACS

Note that this is just the request so it’s the values we passed to SendObject. In the response we will have also what they sent us back. Usually the application that respond to the association request should check that the called AE is matching to its own AE and that the calling AE is something that is found in its configuration file or database. If it doesn’t match, than the called AE can reject the association.
Then we have the Max PDU Size. PDU is an application level ‘packet’ that says how big is the buffer we are willing to consume for each request.

Our Max PDU Receive Size: 32768

In this case we propose no more than 32K. One known problem is that some applications send an association request so big that the called AE can’t consume. We’ll see in a minute why they do that and how to avoid it.
The next chunk of the log is still part of the first entry in the log. The association request includes a list of DICOM services. The items in this list are called presentation contexts:

Presentation Contexts:

Context
ID: 1 (Proposed)

Abstract
Syntax: =VerificationSOPClass

Proposed
SCP/SCU Role: Default

Accepted SCP/SCU Role: Default

Proposed
Transfer Syntax(es):

=LittleEndianExplicit

=BigEndianExplicit

=LittleEndianImplicit

Context
ID: 3 (Proposed)

Abstract
Syntax: =SecondaryCaptureImageStorage

Proposed
SCP/SCU Role: Default

Accepted
SCP/SCU Role: Default

Proposed
Transfer Syntax(es):

=LittleEndianExplicit

Requested Extended Negotiation: none

Accepted Extended Negotiation: none

We’ve sent a list with two items. Each item is a presentation context and identifies a DICOM Service that we wish to use during this association. The presentation contexts are oddly numbered. The first is 1, the second is 3 and a third would have been 5. Why? I don’t know. That’s the way it is. As I said, they are oddly numbered.
So the first service we’ve asked for is Verification. It is performed using the DICOM command C-ECHO. In the log we see this:

Abstract Syntax:
=VerificationSOPClass

Every service has a UID. In the log file UID’s that are known are replaced by their name. The verification service is a sort of high level ping. It’s a DICOM command called C-ECHO that when sent the peer should respond with a success status. Note that we have not sent a C-ECHO command yet. We just asked the called AE in our association request to use it in the second part. We also didn’t say we will send a C-ECHO. A DICOM application that listens on a port and waits for incoming connections must always implement the verification service. Our little application is not listening on any port yet. At this stage, we only play the client role here and connect to another application. As a client, It’s always a good habit to ask for the verification service. If we don’t ask for it and the application we connect to does not support any of the other services that we ask for than it will hang up on us. By adding the verification to our request we force the server to say yes on at least one thing we ask for.

The second service we’ve asked for is Secondary Capture Image Storage:

Abstract Syntax:
=SecondaryCaptureImageStorage

If you remember when we talked about SOP Class UID in chapter 4, I said that SOP is a pair of a service and an object definition. So here we have this combination. We are asking the peer application to store an object that we are going to send and we tell it that the object is going to be a Secondary Capture Image. If we also had another object type, for example a CT Image, than we would have had to ask for a third presentation context for it. The called AE can allow or disallow each one of the services. So it’s possible to create an application that accepts specific types of objects. For example, if we are writing a 3D reconstruction workstation for CT scans we can accept only CT images and thus force the sending application to send us only that type of objects. However, this is not such a good idea because applications tend to send complete studies and there may be in a study images of different classes, for example there may be one series with a CT scan and another one with a report and another one with radiation dose report and if we limit our workstation to accept only CT images than the application that were implemented to send complete studies will keep reporting failures because they can’t send the other objects even though the CT images that we needed has arrived. A better design would be to allow all object types and ignore the ones we don’t need.

This mechanism of negotiating every type of object led some vendors to the very bad habit of simply requesting all the possible objects they know. This can lead to a 50K long association request and if the called AE implementation can read only 32K long requests it can easily crash on the simplest buffer overflow bug. Additionally, sending a 50K association request every time you just want to check a connection by using a C-ECHO command is pure waste of time.

RZDCX’s SendObject negotiates only the required SOP Class UID’s. The DCXREQ Send method sends a set of DICOM files. First it goes over all the files, create a list of all their SOP Classes and then negotiates this list with the called AE.

This concludes our association request. We identified ourselves and stated what we are calling for. Now let’s see what the called AE is going to say. After the association request is sent, the called AE reads the request and sends back an association response. It is almost identical to the request. The called AE simply fills in the form we sent. The second entry in this log is a dump of this response.

From the timestamp you can see that it came back just 1 tenth of a second after the request was sent and that the response status is Normal (I highlighted the parameters that before were empty or has changed). You can also see that now their implementation identification is filled in with the value softlink_jdt103 which identifies a very handy Java utility package from Tiani. Their AE title is indeed “PACS” and they have accepted our association request. There is couple of cases here. One case is that they simply don’t answer. In this case our request will time out without getting any response. Another case is what we have here that is the association request was accepted and we are now connected to the called AE. The third case is that the called AE decides it doesn’t want to talk to us and sends an Association Reject response. For example if its AE title is not “PACS” so it would probably say “Wrong called AE title”. The reason for rejection is encoded in the response status and sometimes has an additional textual explanation.
We also got the list of services back. The verification was accepted but the Secondary Capture Storage was not. This means that if we like we can send a C-ECHO command but we can’t send our Secondary Capture Image using a C-STORE. Because this was what we wanted to do in this association you see the next two log entries:

It means that we can’t store the object because the peer doesn’t support this service. Yippy! We actually could figure out what’s wrong ha?! Now Burt can go to the PACS admin and ask him why his PACS can’t store Secondary Captures and the PACS admin is probably going to ask Burt to which server he tried connecting and to what port and then say that port 6104 is the Worklist Manager that serve Modality Worklist and Performed Procedure Step requests (which are DICOM services we’ll learn about later on) and that if we want to send something to the PACS we should try connecting to port 104. Case solved.
OK, let’s run this again and this time connect to port 104. It’s a good idea to have the AE title, IP address and port number of the called AE configurable in our application so we don’t have to compile every time. Most DICOM applications have such configuration. Usually it’s a table with at least the columns: AE Title, host and port and maybe an id and a comment. Here’s the log of a successful sending, this time the log level was set to Debug.

The storage command did pass but we got back a warning status (45056 = 0xB000) instead of success (0x0000). We also got a warning comment that the called AE changed Instance Number element from null to 0, maybe in order to index it properly in its database.

We should have talked about transfer syntaxes but this is already a long post so I’ll leave transfer syntaxes for another time.

Let’s summarize what we’ve covered in this post.

The nodes in the DICOM network are called Application Entities (AE) and are identified using a case sensitive name called AE Title.

DICOM communication is always between two AE’s i.e. it is peer-to-peer.

The DICOM ‘session’ is called Association

The association is divided into two stages. The first stage is called Association Negotiation. In the second stage the two AE’s exchange DICOM commands.

In the Association Negotiation, the requesting AE sends a list of presentation contexts that identify DICOM services it wishes to use and the responding AE sends back the same list marked with which services it accepted and can be used and which it declined and can’t be used in this association.

The verification service is an application level service used to verify communication between two AE's

The storage service is used to transfer DICOM objects between AE's. The storage service is negotiated separately for every SOP Class. For example an application can allow storage of CT image and forbid storage of MR images. This is a not a good design though.

That’s it. I hope you still believe me that DICOM is Easy. As always, comments are most welcome.

Placing the real-world scenario into the article was brilliant. I can see now after reading this tutorial that I may become the goto DiCom guy at my company, especially if I can speed up our custom viewer which is god awful.

Thank you! The problem with things like dcm4che is that it is horribly undocumented. dcm4che assumes that you know everything about DICOM from reading the 3000 page spec. Thank you for taking the time to define a bunch of these terms.

Hi,Really a good article,but I have a question,it s possible to have 2 same AET in the same local network? I mean :192.168.1.1 AET: DEVICE1 hostname : ONEconnected to those AETs:192.168.1.2 AET: DEVICE hostname : TWO192.168.1.3 AET: DEVICE hostname : THREE

Hi Franz,Possible? Yes. Reasonable? No!Lets say one of the "DEVICE" (e.g. the one on 192.168.1.2) makes a C-MOVE request from a PACS.The PACS reads the Target AE Title parameter and scratch its head: Which "DEVICE" asked this? The one on 192.168.1.2 or the one on 192.168.1.3Not a good idea.Roni

Really thank you for your answer ;)!Yeah I understand your answer, but the DICOM standard allows that ? I mean if a modality is DICOM conformed, it's possible...? (personnally I think like you, not a good idea :) )

Hi, Roni!I am PACS application speciliast and not so happy with DICOM details. Sometimes setting up DICOM communication with two hosts I see in the logfile messages with related dicom association error codes like 25, 26 etc. How to read them? Do you have a list of defined Dicom-related error codes? Thanks

Excellent article.Your catch line "99% of the difficulties in DICOM networking are related the first part – the association negotiation." helped me not only to resolve most of my issues but also in performance improvement. If one negotiate correct SOP Classes with proper transfer syntaxes, considerable performance improvement will be observed.

Great article (same comment as many other) ... :) ... However i am facing a different issue...

I am using dcm4chee java toolkit to send (using dcmsnd programmatically) a dicom file to a PACS. While i am able to send it to several PACS (dcm4che, Osirix) i am unable when i am trying to send it to some other and i am getting:

Hi expert, DICOM client can get datas from server. However, when the VPN was enable in the network, connection timeout occurs:ERROR: Failed to establish association:java.net.SocketTimeoutException: connect timed out

thanks again for the great article. A question about: """DICOM is almost always used in LAN environment and I strongly discourage anyone from using DICOM in WAN environment though I know some people do this but it’s really not a good idea"""But what if we need to perform a C-store to a DICOM endpoint on a remote network, is VPN robust enough to coop with this? Or do you suggest other protocols?

VPN is good. Secure DICOM (TLS) is also very good. Over WAN there are many alternatives of secure RESTFull API's, WADO and the list is long.DICOM Protocol's domain is narrowing to communication with Imaging Devices (Ultrasound, MRI, CT Machines). Once the images are in the PACS the IT system takes over and http based protocols dominate.

Is there anyway to find out which transfer syntaxes are supported before attempting to send a file?

For example, I have an AVI file that I want to send to the PACS server. I'd like to send it as MPEG-4 or MPEG-2 to save space but if neither are supported then send it as multi-frame data (PixelData).

I was thinking it might be possible to construct an object containing a list of valid transfer syntaxes and one invalid one to ensure the association request fails. But there doesn't appear to be a way to access the list of (un)supported presentation contexts.

Hi. There is a way. All you need to do is to propose the same sop class once with every transfer syntax and see which of the presentation contexts are accepted. Then use one of the accepted that you prefer. We don't provide this option in rzdcx because it's "low level" so to call it. You can do what you suggest to work around this of course.

Hey there! I just wanted to ask if you ever have any problems with hackers? My last blog (wordpress) was hacked and I ended up losing several weeks of hard work due to no data backup.Do you have any solutions to protect against hackers?