Pinned topicDataPower "binary" node type (binaryNode)

DataPower "binary" node type (binaryNode)

DataPower node types

DataPower does support the 7 node types from XPath 1.0 data model [1]:

root, element

text

attribute, namespace

processing instruction, comment

In addition DataPower provides a proprietary node type called binaryNode.
This node type is used for containing binary data while processing XML files, eg. for crypto purposes.

binaryNode is not a first class member of the DataPower node types.
You may not do everything with it and expect it to work.
For example passing a binaryNode as argument to a user defined func:function or a template might not give the results you expect.
This is working as designed as binaryNode is intended for specific uses only.

Examples of how a binaryNode can be created

xsl:copy-of and dp:binary-encode

binaryNode nodes can be copied by xsl:copy-of.
And they can be encoded by dp:binary-encode() which base64 encodes the binary data and therefore returns a xs:string.

xsl:value-of for binaryNode

Trying to access the value of a binaryNode by

<xsl:value-of select=
"dp:binary-decode('VGVzdA==')"/>

you will not get "Test" as you might expect ("VGVzdA==" is base64 encoding of "Test") but get "***BINARY NODE***" instead.
Since a binaryNode may contain arbitrary binary data this is the designed behaviour.

how to get the "real" value

As part of the "hex-to-text" template on this [2] slide of last years WSTE presentation a method was shown how to get the "real" value out of a binaryNode:
1) store the binaryNode into a xsl:variable ($raw)
2) do <xsl:value-of select="$raw"/> then

Security considerations -- validation

If you have the contents of the binary data under control you may just use above method.
But some binary data may affect the stability of the DataPower appliance if accessed that way.
Therefore eg. getting a HTML page which you do not have unter control via

<dp:url-open target=
"http://..." response=
"binaryNode" .../>

is not a good idea since getting the value of the binaryNode return by above method bypassed the input validation of DataPower
(UTF-8 validation is the default validation if no other encoding is specified in the input XML file).

Therefore below two functions dp:value-base64() and dp:value-hex() allow to pass the request for
validation against an encoding as 2nd argument (either "ascii" or "utf-8").
The validation is done by regular expression matching against the hex representation string.

dp:value-base64(base64string, encoding)

The three stylesheets and binary input files (41, c3a4, c080) are contained in attached .zip file.

The binary input files just contain the binary data indicated by their name, eg:

$ od -Ax -tx1 c3a4 000000 c3 a4 000002 $

These are the different outputs generated by dp:value-base64(), with no validation or validation against UTF-8 or ASCII:

.

41

c3a4

c080

value-base64.xsl

"A"

"&#228;"

"&#0;"

value-base64-utf-8.xsl

"A"

"&#228;"

""

value-base64-ascii.xsl

"A"

""

""

Btw, c080 is an overlong representation of the 0x00 byte and therefore not valid UTF-8, but valid modified UTF-8 [3].

This is the stylesheet demonstrating UTF-8 validation as part of dp:value-base64() -- if validation fails an empty string is returned:

DataPower "binary" node type (binaryNode)

DataPower node types

DataPower does support the 7 node types from XPath 1.0 data model [1]:

root, element

text

attribute, namespace

processing instruction, comment

In addition DataPower provides a proprietary node type called binaryNode.
This node type is used for containing binary data while processing XML files, eg. for crypto purposes.

binaryNode is not a first class member of the DataPower node types.
You may not do everything with it and expect it to work.
For example passing a binaryNode as argument to a user defined func:function or a template might not give the results you expect.
This is working as designed as binaryNode is intended for specific uses only.

Examples of how a binaryNode can be created

xsl:copy-of and dp:binary-encode

binaryNode nodes can be copied by xsl:copy-of.
And they can be encoded by dp:binary-encode() which base64 encodes the binary data and therefore returns a xs:string.

xsl:value-of for binaryNode

Trying to access the value of a binaryNode by
<pre class="jive-pre">
<xsl:value-of select=
"dp:binary-decode('VGVzdA==')"/>
</pre>
you will not get "Test" as you might expect ("VGVzdA==" is base64 encoding of "Test") but get "***BINARY NODE***" instead.
Since a binaryNode may contain arbitrary binary data this is the designed behaviour.

how to get the "real" value

As part of the "hex-to-text" template on this [2] slide of last years WSTE presentation a method was shown how to get the "real" value out of a binaryNode:
1) store the binaryNode into a xsl:variable ($raw)
2) do <xsl:value-of select="$raw"/> then

Security considerations -- validation

If you have the contents of the binary data under control you may just use above method.
But some binary data may affect the stability of the DataPower appliance if accessed that way.
Therefore eg. getting a HTML page which you do not have unter control via
<pre class="jive-pre">
<dp:url-open target=
"http://..." response=
"binaryNode" .../>
</pre>
is not a good idea since getting the value of the binaryNode return by above method bypassed the input validation of DataPower
(UTF-8 validation is the default validation if no other encoding is specified in the input XML file).

Therefore below two functions dp:value-base64() and dp:value-hex() allow to pass the request for
validation against an encoding as 2nd argument (either "ascii" or "utf-8").
The validation is done by regular expression matching against the hex representation string.

dp:value-base64(base64string, encoding)

The three stylesheets and binary input files (41, c3a4, c080) are contained in attached .zip file.

store:///pkcs7-convert-input.ffd

You may use this file in dp:input-mapping as in value-base64.xsl, value-base64_ascii.xsl or value-base64_utf-8.xsl above.
<pre class="jive-pre">
... <!-- This FFD converts the input into an XML tree like
this: <object> <message>***binary data***</message> </object> --> ...
</pre>

But you may also use it as dp:output-mapping like in magnifyingGlass.xsl or zero.xsl above.

store:///pkcs7-convert-input.ffd is used by five strore:///pkcs7... stylesheets.

> ...
> The following XSLT returns only half of the entire PDF/binary
> content. ...
> Not sure at this point what could be wrong. I need to return the pdf.
>

please read section "Termination at first 0x00 byte" in my previous posting.
Whenever a 0x00 byte gets introduced somehow in the data (0x00 of &​#0; is not a valid XML character,
can only be introduced by the $raw trick from above posting) the first occurence is interpreted as end of data as described in that section.

> ...
> The following XSLT returns only half of the entire PDF/binary
> content. ...
> Not sure at this point what could be wrong. I need to return the pdf.
>

please read section "Termination at first 0x00 byte" in my previous posting.
Whenever a 0x00 byte gets introduced somehow in the data (0x00 of &​#0; is not a valid XML character,
can only be introduced by the $raw trick from above posting) the first occurence is interpreted as end of data as described in that section.

Is there a way i can preserve the input binary to output. The attached is my input.txt. However the generated output doesnt have the input characters as it is. At the end there are @ characters(4 in output and 5 in input at the end). I have used dp:value-base64() as the third step. If I use dp:decode(_,'base-64') as the third step, getting "Transforming (possibly binary) INPUT with xsl results stored in OUTPUT' failed: Valid base64 passed to dp:decode resulted in non UTF8: "

Attached here with is the inputfile. Here is the xsl file, same as base-64.xsl.

Is there a way i can preserve the input binary to output. The attached is my input.txt. However the generated output doesnt have the input characters as it is. At the end there are @ characters(4 in output and 5 in input at the end). I have used dp:value-base64() as the third step. If I use dp:decode(_,'base-64') as the third step, getting "Transforming (possibly binary) INPUT with xsl results stored in OUTPUT' failed: Valid base64 passed to dp:decode resulted in non UTF8: "

Attached here with is the inputfile. Here is the xsl file, same as base-64.xsl.

Re: DataPower "binary" node type (binaryNode)

Is there a way i can preserve the input binary to output. The attached is my input.txt. However the generated output doesnt have the input characters as it is. At the end there are @ characters(4 in output and 5 in input at the end). I have used dp:value-base64() as the third step. If I use dp:decode(_,'base-64') as the third step, getting "Transforming (possibly binary) INPUT with xsl results stored in OUTPUT' failed: Valid base64 passed to dp:decode resulted in non UTF8: "

Attached here with is the inputfile. Here is the xsl file, same as base-64.xsl.

In October I gave two complete webcasts on Non-XML data processing and forgot to add the references here.
The webcasts contain the presentation .pdf, the recording .mp3 as well as a .zip containing all sample stylesheets and files.

Re: DataPower "binary" node type (binaryNode)

In October I gave two complete webcasts on Non-XML data processing and forgot to add the references here.
The webcasts contain the presentation .pdf, the recording .mp3 as well as a .zip containing all sample stylesheets and files.

Re: DataPower "binary" node type (binaryNode)

Hermann,
I was able to generate the ouput, like the way you explained. However, if I use the below xsl to store the binary contents in a file, under temp(again by your guidance in another post), getting
*****binary node**** as the contents of file.

Is this possible, to generate the text file with the actual contents of the input. As always your guidance helps a lot

Hermann,
I was able to generate the ouput, like the way you explained. However, if I use the below xsl to store the binary contents in a file, under temp(again by your guidance in another post), getting
*****binary node**** as the contents of file.

Is this possible, to generate the text file with the actual contents of the input. As always your guidance helps a lot

Re: DataPower "binary" node type (binaryNode)

Hermann,
I was able to generate the ouput, like the way you explained. However, if I use the below xsl to store the binary contents in a file, under temp(again by your guidance in another post), getting
*****binary node**** as the contents of file.

Is this possible, to generate the text file with the actual contents of the input. As always your guidance helps a lot

> Liv2luv wrote:
> dp:dump-nodes outputs a node-set into the temporary:/// directory. therefore there is no need of a url-open extension function.
>
Maneesh said "Is there a way i can preserve the input binary to output".
He wants to dump the data AND pass it unmodified.
So the dp:url-open is needed.

> Also if the output text is not a node-set; that needs to be converted into a nodeset - probably like below:
>
>

>
> Finally, dp:output-mapping is not needed in this case.
>
again, it is needed as he wants to pass data back.

Also Maneesh's data seems to be binary data, not UTF-8 encoded text.
So dp:decode() as used in your sample will complain on Non-UTF8 data.

What I think Maneesh wants is dump-nodes.xsl below.
It does an identity copy of his backend response data to the client.
In addition, it dumps the binary data base64 encoded in temporary folder.

This is a simple demo only.
What is missing is incorporation transaction ID into dump file name,
otherwise each transaction just overwrites the previous dump.
AND being careful to make sure not running out of temp space.
This should be used for development or debugging situations only.

> Liv2luv wrote:
> dp:dump-nodes outputs a node-set into the temporary:/// directory. therefore there is no need of a url-open extension function.
>
Maneesh said "Is there a way i can preserve the input binary to output".
He wants to dump the data AND pass it unmodified.
So the dp:url-open is needed.

Re: DataPower "binary" node type (binaryNode)

> Liv2luv wrote:
> dp:dump-nodes outputs a node-set into the temporary:/// directory. therefore there is no need of a url-open extension function.
>
Maneesh said "Is there a way i can preserve the input binary to output".
He wants to dump the data AND pass it unmodified.
So the dp:url-open is needed.

>
> The error is
>
> "Cannot convert this value to binary data" and the xsl, where the above code is there, fails
>
sorry, binary contexts are different, their binary content cannot be accessed that way.

For logging above you did choose dp:binary-encode() which is fine.

You can do what you want in a binary transform action like toBase64b.xsl.
I tested this and it works, Input context of binary transform actions is "MyBinaryvalue":

> First in the XSL:
>
> it is input-mapping instead of output-mapping
>
Hi Suresh, it is output-mapping since he wants to output text data received by dp:url-open (binaryNode).
Pradeep, please see "getgif.xsl" on slide 18 of this WSTE webcast (you do not need your own FFD as long as your test-text-file is UTF-8 encoded):http://www-01.ibm.com/support/docview.wss?uid=swg27022979

You can find all sample files in "Samples File" attachment.
Just try out "getgif.xsl" as it is -- it will get you a small animated .gif file from my private web server ...

Re: DataPower "binary" node type (binaryNode)

> First in the XSL:
>
> it is input-mapping instead of output-mapping
>
Hi Suresh, it is output-mapping since he wants to output text data received by dp:url-open (binaryNode).
Pradeep, please see "getgif.xsl" on slide 18 of this WSTE webcast (you do not need your own FFD as long as your test-text-file is UTF-8 encoded):http://www-01.ibm.com/support/docview.wss?uid=swg27022979

You can find all sample files in "Samples File" attachment.
Just try out "getgif.xsl" as it is -- it will get you a small animated .gif file from my private web server ...

But my requirement is slightly different. i need to add to string fields at the end of text file and the concatenated string should go to the backend server

Content of test-text-file is
hello world

trail1 ="string1" and trail2="string2"

Final output would be hello world string1 string2
As i am unable to hold the binary output into xsl variable therefore i am unable to achieve this string concatenation.

But if i do dp:binary-encode and dp:decode then xsl variable is holding the content of test-text-file.
My only objective is skip the encode and decode because my file is already having UTF-8 encoded content.

I believe , i am still missing some thing in binaryNode as i am getting some weird outputs ( If i use simple.ffd)

It demonstrates how to append arbitrary binary data, but of course you can do the same with text strings.
In your case you can use this approach of getting yout trail strings appended:
<pre class="jive-pre">
... <xsl:variable name=
"str" select=
"dp:decode(dp:binary-encode($resp/result/binary/node()),'base-64')"/> ... <object> <message> <xsl:copy-of select=
"dp:binary-decode(dp:encode(concat($str,$trail1,$trail2),'base-64'))"/> </message> </object> ...
</pre>

It demonstrates how to append arbitrary binary data, but of course you can do the same with text strings.
In your case you can use this approach of getting yout trail strings appended:
<pre class="jive-pre">
... <xsl:variable name=
"str" select=
"dp:decode(dp:binary-encode($resp/result/binary/node()),'base-64')"/> ... <object> <message> <xsl:copy-of select=
"dp:binary-decode(dp:encode(concat($str,$trail1,$trail2),'base-64'))"/> </message> </object> ...
</pre>