Managing Bulk Operations

Bulk operations are a flexible, convenient tool designed to help you automate and keep tight control over every part of many common activities. They allow you to use built-in or custom actions to automate a wide range of common activities, whether you use Cisco CPEs or third party CPEs.

Note Certain bulk operations are limited when using third-party CPEs. Contact Cisco Customer Support if you have questions.

Bulk operations allow you to run actions against some or all of your CPEs, either passively or actively. If the action runs passively, each CPE is affected by the bulk operation as the CPE calls into the ACS during its regular inform. If the action runs actively, then, based on processing availability, the server solicits CPEs to call in for the update.

The Bulk Operations interface uses simple drag-and-drop functionality, allowing you to quickly create customized search criteria. Search criteria can be based on CPE information, subscriber information, or a combination of both. When you run a bulk operation action, the search criteria allow the action to run on only the CPEs and subscribers that match your specific requirements.

Preparing for Bulk Operations

Prior to running a bulk operation, verify that there are no other bulk operations scheduled to run at a time that interferes with this bulk operation’s schedule.

You can prevent an individual CPE from being selected for any bulk operation by going to the Customer Support tab, locating the device, displaying the Device window, and unchecking the
Participates In Bulk Operations
check box.

Before doing a bulk upgrade:

Always test your bulk firmware update process in a lab first, then run a limited test on a subset of the target population before deploying.

Notify your subscribers that the update is coming. Tell subscribers to make sure they leave their modem on.

Have a plan to ship spare CPEs in the rare event of a failure. Understand which users have what services when you schedule updates, as some actions reboot the CPE.

Prepare for an update by planning ahead.

Verify that the firmware version for your CPE type has been loaded in the Administration tab. Review network usage charts for an appropriate maintenance time window. Between 2:00 a.m. and 5:00 a.m. is a suggested time window.

Consider time zones when scheduling.

Inform intervals:

– If the CPEs have an inform interval of 7 hours and a bulk firmware operation has a 5 hour window on all days of the week, all CPEs should be expected to go through the bulk update process in 2 days.

– If the CPEs have an inform interval of 23 hours and a bulk firmware operation has a 2 hour window on all days of the week, all CPEs should be expected to go through the bulk update process in 23 days.

Creating Bulk Operations

Figure 10-1 indicates the tasks that you need to complete when creating a bulk operation.

Figure 10-1 Bulk Operation Creation Process

Examples of Bulk Operations

Two common uses for bulk operations are updating firmware and enabling or disabling a service. This section describes each of these scenarios.

Example 1: Update Firmware on All SR100G CPEs

This example should familiarize you with the overall process; it is not intended to cover specifics.

For this example, say you want to update the firmware on SR100G CPEs on your activation server. You want the update to run once, starting on Monday, November 5, 2012, and you want it to run during your maintenance window, which is Monday through Friday from 12:01 a.m. to 5:59 a.m.

To contact devices without waiting for an inform, check the
Solicit Devices
check box.

To wait for an inform to run the operation, uncheck the
Solicit Devices
check box.

Step 5 (Optional) In the Max Sessions field, enter the number of maximum concurrent sessions. Click
to create a new bulk operation.

Step 6 From the Action drop-down list, choose
Update firmware
.

After making your selection, if the Action requires a parameter, a list opens containing values for that parameter. The Update Firmware action requires you to choose a version.

Figure 10-3 Create Bulk Operation Window

Step 7 From the Parameters/Value menu, choose the appropriate firmware version for the desired CPE.

The Schedule Preview area at the bottom of the page reflects your choices. After making a selection, an expandable area shows exactly which devices are affected by this bulk operation, if the bulk operation runs at this time.

Note If you schedule the bulk operation to start running at a future date, the list might be slightly different when the bulk operation actually runs. New devices that meet the selection criteria might be added to the system, or existing devices might leave or change.

Step 8 From the Run menu, select
Once
.

Step 9 Set the following scheduling parameters:

Beginning date: 11/5/2012

Day of week: Mon, Tues, Wed, Thu, Fri

Run from: 12:01 AM

Run to: 5:59 AM

Step 10 Click
Create
.

The bulk operation is listed in the Find window along with the operation’s scheduled beginning date, frequency, and status (Figure 10-4).

Figure 10-4 Updated Bulk Operations List

Viewing Bulk Operation Progress

Once the update starts, you can click the name to see a dashboard view of the operation’s progress (Figure 10-5).

The dashboard shows the following:

A summary of the bulk operation you are viewing.

The results of the operation. You can filter the results by checking the
Pending
,
Active
,
Success
,
Failure
, and
Cancelled
check boxes. If the results are longer than one page, you can scroll through the results using the page controls. You can set how many results display on a page.

A pie chart showing the results, including the number of CPEs with each status.

A pie chart showing the failure results along with error codes.

Figure 10-5 Bulk Operations Dashboard

Example 2: Enable Content Filtering on All CPEs

When you deploy a new service, you might want to enable it on all CPEs as a promotional event. This example describes how to use CPE filtering to enable the Content Filtering service for all subscribers within a ZIP code range of 30300 through 30399. You also decide that you want the service to enable when devices check in rather than soliciting them.

a. From the Available Columns section, drag Postal Code to the Filter Criteria section.

b. From the menu, choose
is between
.

c. In the parameter fields, enter
30300
and
30399
.

d. Click
Apply
.

Figure 10-8 Create Bulk Operation Window—Filter Criteria Section

Step 5 Click
Create
.

As each device checks in, the Content Filtering service is enabled.

Best Practices for Working with Bulk Operations

Caution Bulk Operations is a powerful tool that you should understand before you use. It is possible to adversely affect your subscribers’ services if you do not understand the actions you schedule.

Troubleshooting Failed Bulk Operations

Bulk Operations rarely fail on all CPEs, if you have set up the operation correctly. However, there can be “failed actions” on a particular CPE. In this case, Prime Home does not attempt to rerun the action against that CPE.

If the operation is scheduled to run multiple times, the CPE might “enter the pool” of selected CPEs a second time. Otherwise, determine the reason the action failed and then run another bulk operation that includes that CPE.

Controlling Maximum Throughput/Throttling

There are three base parameters that control the maximum throughput of Bulk Operations:

bulkOp.maximumConcurrentRunningBulkOperations

bulkOp.timeBetweenSolicitingSchedulerSecs

bulkOp.minimumTimeBetweenSolicitsMS

The first, bulkOp.maximumConcurrentRunningBulkOperations, is the maximum number of bulk operations that can run at a given time.

The second and third parameters control the behavior of the bulk operations engine when there are free bulk operation slots. The process is described below.

When a bulk operations starts, there are N number of free slots for bulk operations. The engine retrieves N CPEs from the result set (the list of CPEs that are part of the bulk operation) to solicit. It then solicits the CPEs, and the spacing between those solicits is bulkOp.minimumTimeBetweenSolicitsMS.

For every interval defined in bulkOp.timeBetweenSolicitingSchedulerSecs, the engine checks for free bulk operation slots. If there are free slots, it grabs that number of CPEs from the result set and solicits them.

For passive bulk operations, solicits don’t occur. Instead, when a CPE informs, if it’s in the result set and there are free bulk operation slots, the bulk operation runs.

The maximum number of bulk operations is modified internally by the system, depending on the number of CPE sessions that are active. For example, if tr069.maxConcurrentSessions is set to 100, bulkOp.maximumConcurrentRunning-BulkOperations is set to 50, and there are 50 CPEs checked in, the maximum number of bulk operations is reduced by 50%. Current sessions equal 50% of the maximum sessions, and the number of bulk operations is reduced by that percentage.

Preparing to Run Firmware Update Operations

Preparation is the key to a successful bulk firmware update process. There are several things to consider before updating to a newer firmware version:

Make and model—You can only choose one CPE type per process, but you can schedule multiple bulk operations.

Firmware version—Verify that the firmware version being updated from will go directly to the new version. If the CPEs to be updated are on a firmware release that is very far behind the current release, you might need to perform interim updates prior to updating them to the current version. Contact Cisco Customer Support if you have questions.

Firmware definition—View the firmware definition on the Firmware window of the Administration tab. If any changes need to be made, edit the firmware definition.

Field preparation—Verify that all CPEs in the selected group are informing.

Always run a test update on a small group of CPEs before running an update on a large group of CPEs.

Labels—If you have applied a time zone-related label to your devices, searching by labels is useful when updating CPEs in multiple time zones.

Understanding Bulk Operation Options

You have many choices to make when creating your bulk operations. For information on creating a custom solution, one tailored to your specific needs, contact Cisco Advanced Services.

The section below describes some of the built-in options you have for:

Actions
—The types of operations you can run.

Scheduling
—Allows you to set when you want the action to start running and how often you want Prime Home to select the pool of CPEs to be acted on.

CPE Selection
—The CPEs you want this operation to run on.

Action Options

Actions are designed to carry out a specific set of instructions. Each built-in action is named so it describes what the action does.

Prime Home has built-in actions, but you can also contact Cisco Advanced Services to create custom actions. If you are an advanced user, you can create actions by going to the Scripts window on the Administration tab to add a new script.

Some actions have parameters that refine the action. For example:

Inform Interval
—This action sets the inform interval for all affected CPEs. You need to enter the parameter for the number of seconds between informs.

Enable/Disable Service
—This action allows you to turn a subscriber service on or off. When you choose this option, the Parameters area requires you to select a value, which is the service you want to enable or disable. To create a custom service, contact Cisco Advanced Services.

Schedule Options

Table 10-1 describes the schedule options that are available for bulk operations.

Table 10-1 Bulk Operation Schedule Options

If you choose…

The ACS creates a list of CPEs that meet your criteria…

And the action runs on the list of CPEs…

Now

When you click Create.

When you click Create.

Once

On the beginning date.

On the first scheduled weekday after the beginning date.

Weekly

First time
—On the beginning Date.

Each subsequent time
—On the same weekday every week. For example, if the beginning date is Thursday, February 2nd, a new list of CPEs is chosen each Tuesday after that date.

On the first scheduled weekday after the beginning date and every scheduled day thereafter.

Monthly

First time
—On the beginning date.

Each subsequent time
—On the same calendar day every month. For example, if the beginning date is February 2nd, a new list of CPEs is chosen March 2nd, April 2nd, and so on.

On the first scheduled weekday after the beginning date and every scheduled day thereafter.

CPE Selection Options

You have two options for selecting which CPEs are affected by a bulk operation:

Use the options available from within Bulk Operations as shown in the examples.

If you have Prime Home Reporting, you can design a report to select the desired CPEs and apply a label to all of the selected CPEs. You can then filter the CPEs by label for the bulk operation.

You can view the CPEs selected for a bulk operation by expanding the Matching Devices section at the bottom of the Create Bulk Operation window.