Key Value Map Operations policy

What

Provides policy-based access to a key/value map (KVM) store available in Apigee Edge. Key/value pairs can be stored, retrieved, and deleted from named existing maps by configuring KeyValueMapOperations policies that specify PUT, GET, or DELETE operations. (At least one of these operations must be performed by the policy.)

Samples

When the following policy runs, it creates a Key Value Map named FooKVM, then creates a key named FooKey_1 with two values set with literal strings (not set with values extracted from variables): foo,bar. (When you GET the key in the next example, you specify an index number to retrieve the value you want.)

Notice that the scope is "environment". That means you can see the KVM in the management UI under APIs > Environment Configuration > Key Value Maps. The KVMs shown on that page are all scoped to the selected environment.

This policy looks at the FooKVM map from the previous example, gets the second value (index="2") from the FooKey_1 key, and stores it in a variable called foo_variable.

A simple example of a useful KeyValueMap is a URL shortening service. The KeyValueMap could be configured to store shortened URLs along with corresponding full URLs.

This policy sample creates a KeyValueMap. The policy PUTs a key with two associated values into a key/value map named "urlMapper".

The key in this example, urlencoding.requesturl.hashed, is an example of a custom variable. The hashed request URL would be generated by code (JavaScript or Java, for example) and then stored in this variable, where the KeyValueMapOperations policy can access it.

For each key, requesturl.hashed, two values are stored:

The contents of the custom variable named urlencoding.longurl.encoded

The contents of the predefined variable request.queryparam.url

For example, when the policy executes at runtime, the values of the variables may be as follows:

A simple example of a useful KeyValueMap is a URL 'shortening' service. The KeyValueMap could be configured to store shortened URLs along with corresponding full URLs.

To retrieve the value of key/value map entry, such as the one covered on the KeyValueMapOperations PUT tab, configure a policy to GET the KeyValueMap, as shown above.

When this policy is executed, if the value of the urlencoding.requesturl.hashed variable is ed24e12820f2f900ae383b7cc4f2b31c402db1be, then the custom variable named urlencoding.shorturl will be set with the value http://tinyurl.com/38lwmlr.

Now that the data has been retrieved, other policies and code can access it by extracting the value from those variables.

If a key value map is encrypted, retrieve values by using the "private." prefix in the assignTo attribute value. In the example above, the variable private.encryptedVar holds the decrypted value of the key value map's foo key. For information on creating encrypted key value maps, see the "create" topics of the Key/Value Maps management API.

Now that the data has been retrieved, other policies and code can access it by extracting the value from that variable.

Element reference

The element reference describes the elements and attributes of the KeyValueMapOperations policy.

<KeyValueMapOperations> attributes

This attribute specifies the name of the KVM you want to interact with.

Attribute

Description

Default

Presence

mapIdentifier

Specifies an identifier to be used when accessing, via APIs, a map created by this policy or in the management UI.

If you exclude this attribute, a KVM named kvmap is used.

Within a scope of organization/environment/apiproxy, you can use the mapIdentifier attribute to specify your own map name.

N/A

Optional.

The following attributes are common to all policy parent elements.

Attribute

Description

Default

Presence

name

The internal name of the policy. Characters you can use in the name are restricted to: A-Z0-9._\-$ %. However, the Edge management UI enforces additional restrictions, such as automatically removing characters that are not alphanumeric.

Optionally, use the <DisplayName> element to label the policy in the management UI proxy editor with a different, natural-language name.

N/A

Required

continueOnError

Set to false to return an error when a policy fails. This is expected behavior for most policies.

Set to true to have flow execution continue even after a policy fails.

false

Optional

enabled

Set to true to enforce the policy.

Set to false to "turn off" the policy. The policy will not be enforced even if it remains attached to a flow.

true

Optional

async

This attribute is deprecated.

false

Deprecated

<DisplayName> element

Use in addition to the name attribute to label the policy in the management UI proxy editor with a different, natural-language name.

<DisplayName>Policy Display Name</DisplayName>

Default:

N/A

If you omit this element, the value of the policy's name attribute is used.

Presence:

Optional

Type:

String

<Delete> element

Deletes the specified key/value pair. (At least one of <Get>, <Put>, or <Delete> must be used.)

Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element.

<ExclusiveCache> element

<ExpiryTimeInSecs> element

Specifies the duration in seconds after which Edge refreshes its cached value from the specified KVM.

A value of 0 or -1, or excluding this element, means the default value of 300 seconds is used.

<ExpiryTimeInSecs>600</ExpiryTimeInSecs>

Default:

300 (5 minutes)

Presence:

Optional

Type:

Integer

A KVM is a long-term persistence mechanism that stores keys and values in a NoSQL database. Because of this, reading from a KVM at runtime can potentially slow proxy performance. To improve performance, Edge has a built-in mechanism for caching KVM keys/values in memory during runtime. This KVM Operations policy always reads from cache for GET operations.

The <ExpiryTimeInSecs> element lets you control how long the keys/values used in the policy are stored in cache before they're refreshed again from the KVM. However, there are some differences between how GET and PUT operations affect cache expiration.

GET - The very first time a KVM GET operation executes, the requested keys/values from the KVM (whose name is specified in the policy's root mapIdentifier attribute) are loaded into cache, where they remain for subsequent GET operations until one of the following occurs:

The number of seconds specified in <ExpiryTimeInSecs> expires.
or

A PUT operation in a KVM policy overwrites the existing values (explained next).

PUT - A PUT operation writes keys/values to the specified KVM. If the PUT writes to a key that already exists in cache, that cache is immediately refreshed and now holds the new value for the number of seconds specified in the policy's <ExpiryTimeInSecs> element.

Example - A KVM named "MyKVM" has a key : value of "rating" : "10"

A GET operation retrieves the value of "rating," which adds the value "10" to cache. The <ExpiryTimeInSecs> on the policy is 60.

30 seconds later, the GET policy executes again and retrieves "10" from the cache.

5 seconds later, a PUT policy updates the value of "rating" to "8", and the <ExpiryTimeInSecs> on the PUT policy is 20. The cache is immediately refreshed with the new value, which is now set to remain in cache for 20 seconds. (If the PUT hadn't happened, the cache originally populated by the first GET would still exist for another 30 seconds, left over from the original 60 seconds.)

15 seconds later, another GET executes and retrieves a value of "8".

Notes

For best performance, balance cache expiry time with the amount of API traffic you expect. With heavy traffic, for example, you may want to set longer expiry times to avoid frequent cache-population reads from the NoSQL database.

There is no way to manually reset the KVM cache. However, you can create a temporary policy that executes a one-time PUT operation with a short <ExpiryTimeInSecs>. This will reset the cache, which contains the value you PUT, for the number of seconds you specify.

Updating a KVM with the management UI or API does not reset the KVM cache.

Don't add the PUT and GET operations in the same policy, because the PUT automatically resets the cache expiration, which doesn't allow the GET operation to read from a cache with a desired expiry time.

<Get> element

Retrieves the value for the key specified. (At least one of <Get>, <Put>, or <Delete> must be used.)

Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element.

Don't add PUT and GET operations in the same policy, because the PUT automatically resets the cache expiration, which doesn't allow the GET operation to read from a cache with a desired expiry time. See the ExpiryTimeInSecs element for more information.

Attributes

If the key value map is encrypted, begin the assignTo name with "private.". For example:

<Get assignTo="private.myvar">

The policy throws an error if you try to retrieve an encrypted key value map without using the prefix. The prefix, which is required for basic security purposes during debugging, hides the encrypted values from API proxy Trace and debug sessions.

The index number (in a 1-based index) of the item to fetch from a multi-valued key. For example, specifying index=1 will return the first value and assign it to the assignTo variable. If no index value is specified, all the values of that entry are assigned to the variable as a java.util.List.

<InitialEntries> element

Seed values for KeyValueMaps, which are populated in the KeyValueMap when it is initialized. Be sure to specify the name of the KVM with the mapIdentifier attribute on the parent element.

When using this element, when you save the policy in the management UI on a deployed version of the proxy, or deploy the API proxy bundle containing the policy with this element, the key(s) are automatically created in the KVM (as unencrypted). If the values in the policy are different than the values in the KVM, the values in the KVM are overwritten when the proxy is deployed. Any new keys/values are added to the existing KVM alongside the existing keys/values.

Keys and values populated by this element must be literals. For example, <Parameter ref="request.queryparam.key"> isn't supported within this element.

<Key> element

Specifies the key in a key/value map entry. A key can be composite, which means that more than one parameter can be appended to create the key. For example, userID and role might be combined to create a key.

Be sure to see <Parameter> for specifics about how to set the key name.

When the Key element includes multiple Parameter elements, the effective key string is the concatenation of the values of each parameter, joined with a double underscore. For example, in the above example, if the apiproxy.name variable has the value "abc1", then the effective key will be targeturl__abc1__weight.

Whether you're getting, updating, or deleting a key/value entry, the key name must match the name of the key in the key value map. See Specifying and retrieving key names for guidelines.

Default:

N/A

Presence:

Required

Type:

String

Attributes

Attribute

Description

Default

Presence

ref

Specifies the name of a variable whose value contains the exact name of the key you want to create, get, or delete.

N/A

Required if no literal value is given between the opening and closing tags. Prohibited if a literal value is given.

<Put> element

Writes a key/value pair to a key value map, whether the key value map is encrypted or unencrypted. If the key value map specified in the mapIdentifier attribute on the parent element doesn't exist, the map is automatically created (as unencrypted). If the key value map already exists, the key/value are added to it. To create an encrypted key value map, use the management API; or use the management UI to create encrypted environment-scoped KVMs.

Don't add PUT and GET operations in the same policy, because the PUT automatically resets the cache expiration, which doesn't allow the GET operation to read from a cache with a desired expiry time. See the ExpiryTimeInSecs element for more information.

If the policy PUTs a key/value in a KVM using a null key, the following key is auto-created: __$$_EDGE_KVM_EMPTY_KEY_$$__

This situation would happen, for example, if you're populating a key using a variable reference and the referenced variable is null. As new PUTs with null keys are attempted, only that key is used, so existing values for that key are potentially overwritten each time the policy is executed.

Attributes

Attribute

Description

Default

Presence

override

If set to true, it overrides the value for a key.

false

Optional

<Scope> element

Defines the boundary of accessibility for KeyValueMaps. The default scope is environment, meaning that, by default, maps entries are shared by all API proxies running in an environment (for example, test or prod). If you set the scope to apiproxy, then entries in the KeyValueMap are accessible only by the API proxy that writes the values to the map.

Note that when accessing a map or map entry, you must specify the same scope value you used when the map was created. For example, if the map was created with a scope of apiproxy, you must use the apiproxy scope when retrieving its values, putting changes, or deleting entries.

<Scope>environment</Scope>

Default:

environment

Presence:

Optional

Type:

String

Valid values:

organization

environment

apiproxy

policy (API proxy revision)

<Value> element

Specifies the value of a key. You can specify the value as either a literal string or, using the ref attribute, as a variable to be retrieved at run time.

<!-- Specify a literal value -->
<Value>literal<Value>

OR

<!-- Specify the name of variable value to be populated at run time. -->
<Value ref="variable_name"/>

You can also include multiple <Value> elements to specify a multi-part value. Values are combined at run time.

Schemas

Usage notes

A key value map store provides a lightweight persistence mechanism for data formatted as key/value pairs. You can access these at runtime through policies or code. A map contains any arbitrary data in the format key=value.

In organizations with Core Persistence Services (CPS) enabled, key size is limited to 2 KB. In non-CPS organizations, a key value map has a 15 MB limit. Writing to the map may fail close to or beyond these limits.

For example localhost=127.0.0.1, zip_code=94110, or first_name=felix. In the first example, localhost is a key, and 127.0.0.1 is a value. Each key/value pair is stored as an entry in a key value map. A key value map can store many entries.

For example, say you need to store a list of IP addresses associated with various backend environments. You could create a key value map called ipAddresses that contains a list of key/value pairs as entries. For example, this JSON can represent such a map:

You could use this structure to create a store of IP addresses that could be used by policies at runtime to enforce IP whitelisting or blacklisting, to dynamically select a backend target address, and so on. Typically, the KeyValueMapOperations policy is used to store or retrieve long-lived information that needs to be reused over multiple request/response transactions.

Key/value maps can be manipulated via the KeyValueMapOperations policy, or directly via the Apigee Edge management API. Refer to the management API reference for details on the Organization key/value maps API API. You can use the API to, for example, upload large data sets to the key/value store, or creating scripts to manage key/value map entries. You will need to create a key/value map with the API before accessing it with the KeyValueMapOperations policy.

Specifying and retrieving key names

With the <Parameter> and <Value> elements, you can specify either a literal value (where the value is between the opening and closing tags) or use the ref attribute to specify the name of a variable whose value should be used at runtime.

The Parameter element deserves special mention, because it determines the name of the key that gets created, as well as the key name you want to retrieve or delete. Following are two examples. The first specifies a key name literally, and the second specifies a key name using a variable. Let's assume the following are used to create keys in a KVM:

In the first instance, the literal value of "key_name_literal" is stored in the KVM as the key name. In the second instance, whatever value is in the key.name.variable becomes the name of the key in the KVM. For example, if the key.name.variable contained the value foo, the key would be named "foo".

When you want to retrieve the key and a key value with a GET operation (or delete with a DELETE operation), the <Parameter> setting needs to match the key name in the KVM. For example, if the key name in the KVM is "foo", you can either specify the literal value with <Parameter>foo</Parameter> or specify a variable that contains the exact value "foo", like this: <Parameter ref="variable.containing.foo"/>.