Extract Variables policy

What

Extract content from the request or response messages, including headers, URI paths, JSON/XML payloads, form parameters, and query parameters. The policy works by applying a text pattern to the message content and, upon finding a match, sets a variable with the specified message content.

After extracting the specified message content to the variable, you can reference the variable in other policies as part of processing a request and response.

Where

This policy can be attached in the following locations.

ProxyEndpoint

TargetEndpoint

PreFlow

Flow

PostFlow

PreFlow

Flow

PostFlow

Request

→

•

•

•

•

•

•

•

•

•

•

•

•

←

Response

PostFlow

Flow

PreFlow

PostFlow

Flow

PreFlow

Samples

These policy code samples illustrate how to extract variables from the following types of artifacts:

Consider the sample policy code above. The <URIPath> element tells the Extract Variables policy to extract information from the URI path. The <Pattern> element specifies the pattern to apply to the URI path. The pattern is treated as a simple template, with the curly braces denoting the varying portion of the URI path.

The name of the variable to be set is determined by the value specified in the <VariablePrefix> element, as well as the value enclosed in curly braces {} in the <Pattern> element. The two values are joined by an intervening dot, resulting in a variable name of urirequest.id for example. If there is no <VariablePrefix> element, then the variable name is just the value enclosed in curly braces.

Consider the sample policy code above working with the following incoming request:

GET http://org1-test.apigee.net/accounts/12797282

When Apigee Edge applies the Extract Variables policy code above to this incoming request, it sets the variable urirequest.id to 12797282. After Apigee Edge executes the policy, subsequent policies or code in the processing flow can refer to the variable named urirequest.id to get the string value 12797282.

You can now access the variable urirequest.id in your proxy. For example, the following AssignMessage policy copies it to the payload of the request:

Consider the sample policy code above. Suppose that your API design stipulates that incoming requests must carry a query parameter named code. code holds a term that looks like DBNXXXXX, where DBN is fixed and the XXXXX denotes a varying string. You can use this policy to extract the varying string.

Consider the sample policy code above working with the following incoming request:

GET http://org1-test.apigee.net/accounts/12797282?code=DBN88271

When Apigee Edge applies the Extract Variables policy code above to this incoming request, it sets the variable queryinfo.dbncode to 88271. After Apigee Edge executes the policy, subsequent policies or code in the processing flow can refer to the variable named queryinfo.dbncode to get the string value 88271.

You can now access the variable queryinfo.code in your proxy. For example, the following AssignMessage policy copies it to the payload of the request:

Suppose your API design allows you to specify multiple query paramaters with the same name. You can use this policy to extract the value of multiple instances of the query parameter "w". ​To reference these query parameters in the Extract Variables policy, you use indexes, where the first instance of the query parameter has no index, the second is at index 2, the third at index 3, etc.

Consider the sample policy code above working with the following incoming request:

GET http://org1-test.apigee.net/weather?w=Boston&w=Chicago

When Apigee Edge applies the Extract Variables policy code above to this incoming request, it sets the variable queryinfo.firstWeather to Boston and the variable queryInfo.secondWeather to Chicago.

You can now access the variable queryinfo.firstWeather and queryinfo.secondWeather in your proxy. For example, the following AssignMessage policy copies it to the payload of the request:

The Extract Variables policy can extract values from complex structures, such as JSON messages. The sample policy code above shows how to extract a variable from a portion of a JSON message payload. The <JSONPayload> element tells the policy to extract a variable from a JSON payload. You specify the portion to extract using a JSON path expression in which the $ character refers to the root node of the JSON message.

When Apigee Edge applies the Extract Variables policy code above to this JSON message, it sets two variables: geocoderesponse.latitude and geocoderesponse.longitude. Both variables use the same variable prefix of geocoderesponse. The suffix for these variables is specified explicitly by the <Variable> element's name attribute.

The Extract Variables policy can extract values from complex structures, such as XML messages. The sample policy code above shows how to extract a variable from a portion of a XML message payload. The <XMLPayload> element tells the policy to extract a variable from an XMLpayload. You specify the portion to extract using XPath and explicitly named variables.

When Apigee Edge applies the Extract Variables policy code above to this XML message, it sets three variables: directionsresponse.travelmode, directionsresponse.duration, and directionsresponse.timeunit. All variables use the same variable prefix of directionsresponse. The suffix for these variables is specified explicitly by the <Variable> element's name attribute.

About the Extract Variables policy

API developers build API proxies that behave differently based on the content of messages, including headers, URI paths, payloads, and query parameters. Often, the proxy extracts some portion of this content for use in a condition statement. Use the Extract Variables policy to do this. Choose the names of the variables to be set, the source of the variables, and how many variables to extract and set. The policy then applies a text pattern to the content and upon finding a match sets a designated variable with the selected content.

Other policies and code can then consume those variables to enable dynamic behavior, or to send business data to Analytics Services.

If you do not see the newly extracted variable show up in the Trace tool, try removing the <VariablePrefix> element if you added it. For more information, refer to this Apigee Community topic.

About matching and variable creation

The Extract Variables policy extracts information from a request or response and writes that information to a variable. For each type of information that you can extract, such as URI path or XML data, you specify the pattern to match and the name of the variable used to hold the extracted information.

However, the way pattern matching works depends on the source of the extraction. The following sections describe the two basic categories of information that you can extract.

When extracting information from a URI path, query parameters, headers, form parameters, and variables you use the <Pattern> tag to specify one or more patterns to match. For example, the following policy example shows a single matching pattern for the URI path:

The urirequest.pathSeg1 variable is set to "b" and the urirequest.pathSeg2 variable is set to "d".

Matching multiple instances in the pattern

You can also match patterns when there are multiple instances of an item with the same name. For example, you can make a request that contains multiple query parameters or multiple headers with the same name. The following request contains two query parameters named "w":

http://myCo.com/a/b/c/d?w=1&w=2

To reference these query parameters in the Extract Variables policy, you use indexes, where the first instance of the query parameter has no index, the second is at index 2, the third at index 3, etc. For example, the following policy extracts the value of the second query parameter named "w" in the request:

The urirequest.secondW variable is set to "2". If the second query parameter is omitted from the request, then the urirequest.secondW variable is empty. Use indexing any time there are multiple items with the same name in the request.

Using special characters in the pattern

When matching URI paths, you can use the "*" and "**" wildcard characters in the pattern, where:

"*" matches any one segments of the path

"**" matches multiple segments of the path

For example, you specify patterns to the <URIPath> element as shown below:

The first pattern matches URIs such as "/a/b/c", "/a/foo/bar", etc. The second pattern matches any number of path segments after "/a/", such as "/a/foo/bar/baz/c", as well as "/a/b/c" and "/a/foo/bar".

When specifying patterns to query parameters, headers, and form parameters, the "*"character specifies to match any number of characters. For example, when matching a header, specify the pattern as:

*;charset={encoding}

This pattern matches the values "text/xml;charset=UTF-16" and "application/xml;charset=ASCII".

If the value passed to the Extract Variables policy contains a special character, such as "{", use the "%" character to escape it. The following example escapes the "{" and "}" characters in the pattern because they are used as literal characters in the value of the query parameter:

In this example, the pattern matches the value "{user} Steve" but not the value "user Steve".

Matching JSON and XML

When extracting data from JSON and XML, you specify one or more <Variable> tags in the policy. The <Variable> tag specifies the name of the destination variable where the extracted information is stored, and the JsonPath (JSON) or XPATH (XML) to the extracted information.

All <Variable> tags in the policy are evaluated, so that you can populate multiple variables from a single policy. If the <Variable> tag does not evaluate to a valid field in the JSON or XML, then the corresponding variable is not created.

The following example shows an Extract Variables policy that populates two variables from the JSON body of a response:

Writing to the same variable in multiple places

The policy executes sequentially from the first extraction pattern to the last. If the policy writes a value to the same variable from multiple places, the last write in the policy determines the value of the variable.

For example, you want to extract a token value that can be passed either in a query parameter or in a header, as shown below:

<!-- If token only in query param, the query param determines the value.
If token is found in both the query param and header, header sets value. -->
<QueryParam name="token">
<Pattern ignoreCase="true">{tokenValue}</Pattern>
</QueryParam>
<!-- Overwrite tokenValue even if it was found in query parameter. -->
<Header name="Token">
<Pattern ignoreCase="true">{tokenValue}</Pattern>
</Header>

Controlling what happens when no match occurs

If the pattern does not match, then the corresponding variable is not created. Therefore, if another policy references the variable, it can cause an error.

One option is to set <IgnoreUnresolvedVariables> to true in a policy that references the variable to configure the policy to treat any unresolvable variable as an empty string (null):

<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>

Element reference

The element reference describes the elements and attributes of the Extract Variables policy.

The internal name of the policy. Characters you can use in the name are restricted to: A-Z0-9._\-$ %. However, the Edge management UI enforces additional restrictions, such as automatically removing characters that are not alphanumeric.

Optionally, use the <DisplayName> element to label the policy in the management UI proxy editor with a different, natural-language name.

N/A

Required

continueOnError

Set to false to return an error when a policy fails. This is expected behavior for most policies.

Set to true to have flow execution continue even after a policy fails.

false

Optional

enabled

Set to true to enforce the policy.

Set to false to "turn off" the policy. The policy will not be enforced even if it remains attached to a flow.

true

Optional

async

Note: This attribute does not make the policy execute asynchronously.

When set to true, policy execution is offloaded to a different thread, leaving the main thread free to handle additional requests. When the offline processing is complete, the main thread comes back and finishes handling the message flow. In some cases, setting async to true improves API proxy performance. However, overusing async can hurt performance with too much thread switching.

<DisplayName> element

Use in addition to the name attribute to label the policy in the management UI proxy editor with a different, natural-language name.

<DisplayName>Policy Display Name</DisplayName>

Default:

N/A

If you omit this element, the the value of the policy's name attribute is used.

Presence:

Optional

Type:

String

<Source> element

Specifies the message to be parsed. The value of <Source> defaults to message. The message value is context-sensitive. In a request flow, message resolves to the request message. In a response flow, message resolves to the response message.

If <Source> cannot be resolved, or resolves to a non-message type, the policy will fail to respond.

<Source> can resolve to a variable containing a message generated by another policy, such as one generated from a ServiceCallout.

<Source clearPayload="true|false">request</Source>

Default:

message

Presence:

Optional

Type:

String

Attributes

Attribute

Description

Default

Presence

Type

clearPayload

Set to true if you want to clear the request payload after the request is sent to the HTTP target.

Use the <clearPayload> option only if the request message is not required after the ServiceCallout is executed because <clearPayload> allocates memory during message processing.

false

Optional

Boolean

<VariablePrefix> element

The complete variable name is created by joining the <VariablePrefix>, a dot, and the name you define in {curly braces} in the <Pattern> element or <Variable> element. For example: myprefix.id, myprefix.dbncode, or myprefix.oauthtoken.

<VariablePrefix>myprefix</VariablePrefix>

For example, suppose the value of name is "user".

If <VariablePrefix> is not specified, the extracted values are assigned to a variable named user.

If <VariablePrefix> is specified as myprefix, the extracted values are assigned to a variable named myprefix.user.

Default:

N/A

Presence:

Optional

Type:

String

<IgnoreUnresolvedVariables> element

Set to true to treat any unresolvable variable as an empty string (null). Set to false if you want the policy to throw an error when any referenced variable is unresolvable.

<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>

Default:

False

Presence:

Optional

Type:

Boolean

<URIPath> element

Extracts a value from the specified URI path of a request source message. If the source message resolves to a message type of response, then this element does nothing.

Optional. However, you must include at least one of the following: <URIPath>, <QueryParam>, <Header>, <FormParam>, <JSONPayload>, or <XMLPayload>.

Type:

N/A

Attributes

Attribute

Description

Default

Presence

Type

name

Specifies the name of the query parameter. If multiple query parameters have the same name, use indexed referencing, where the first instance of the query parameter has no index, the second is at index 2, the third at index 3, etc.

N/A

Required

String

<Header> element

Extracts a value from the specified HTTP header of the specified request or response message.

Optional. However, you must include at least one of the following: <URIPath>, <QueryParam>, <Header>, <FormParam>, <JSONPayload>, or <XMLPayload>.

Type:

N/A

Attributes

Attribute

Description

Default

Presence

Type

name

Specifies the name of the header from which you extract the value. If multiple headers have the same name, use indexed referencing, where the first instance of the header has no index, the second is at index 2, the third at index 3, etc.

N/A

Required

String

<FormParam> element

Extracts a value from the specified form parameter of the specified request or response message. Form parameters can be extracted only when the contentType of the specified message is application/x-www-form-urlencoded.

<JSONPayload>/<Variable>/<JSONPath> element

<XMLPayload> element

Specifies the XML-formatted message from which the value of the variable will be extracted. XML payloads are extracted only when the contentType of the message is text/xml, application/xml, or application/*+xml.