Patent application title: METHOD AND SYSTEM FOR GROUPING STORAGE SYSTEM COMPONENTS

Abstract:

Methods and systems are provided for associating a storage system
component, e.g. a server, client, machine, sub-client, or other storage
system component, with one or more other storage system components into a
group. Storage preferences are defined for the group which is
automatically associated with each component in the group. The storage
preferences define how storage operations are to be performed on the
storage system components.

Claims:

1. A method for grouping networked storage system components, the method
comprising:associating a plurality of storage system components executing
on one or more computer processors into at least first and second
groups;associating the first group with a first storage policy which
defines one or more retention storage preferences to be performed by the
storage system components in the first group;associating the second group
with a second storage policy which defines one or more retention storage
preferences to be performed by the storage system components in the
second group; andreassigning one or more of the storage system components
associated with the first group to both the first group and the second
group, and in the event a first operation associated with the first
storage policy conflicts with a second operation associated the second
storage policy, the storage policy associated with a higher priority
controls.

2. The method of claim 1, wherein the one or more storage system
components associated with the first group that are reassigned to both
the first group and the second group, automatically inherits the second
storage policy associated with the second group while retaining the first
storage policy associated with the first group.

3. The method of claim 1, wherein in the event a non-conflict storage
operation associated with the first storage policy is to be performed on
a reassigned storage system component, performing the non-conflict
storage operation in accordance with the first storage policy.

4. The method of claim 1, wherein in the event a non-conflict storage
operation associated with the second storage policy is to be performed on
a reassigned storage system component, performing the non-conflict
storage operation in accordance with the second storage policy.

5. The method of claim 1, wherein assigning storage system components to
one or more groups comprises selecting storage system components for a
group based on the storage system components having a common
characteristic.

6. The method of claim 1, wherein the storage system components further
comprise sub-client components of a plurality of clients, the clients
being data sources for the storage operation.

7. A computer system for grouping networked storage system components, the
system comprising:multiple storage system components comprising one or
more computer processors, wherein at least a first storage system
component is associated with a first group of storage system components
and at least a second storage system component is associated with a
second group of storage system components;at least a first storage policy
which defines one or more retention storage preferences associated with
the first group;at least a second storage policy defining one or more
retention storage preferences associated with the second group;
andwherein in the event the first storage component is reassigned to both
the first group and the second group, the first storage system component
inherits the second storage policy associated with the second group, and
in the event a first storage operation associated with the first storage
policy conflicts with a second storage operation associated with the
second storage policy, the storage policy associated with a higher
priority controls.

8. The computer system of claim 7, wherein the one or more storage system
components associated with the first group that are reassigned to both
the first group and the second group, automatically inherits the second
storage policy associated with the second group while retaining the first
storage policy associated with the first group.

9. The computer system of claim 7, wherein in the event a non-conflict
storage operation associated with the first storage policy is to be
performed on a reassigned storage system component, performing the
non-conflict storage operation in accordance with the first storage
policy.

10. The computer system of claim 7, wherein in the event a non-conflict
storage operation associated with the second storage policy is to be
performed on a reassigned storage system component, performing the
non-conflict storage operation in accordance with the second storage
policy.

11. The computer system of claim 7, wherein assigning storage system
components to one or more groups comprises selecting storage system
components for a group based on the storage system components having a
common characteristic.

12. The computer system of claim 7, wherein the storage system components
comprise sub-client components of a plurality of clients, the clients
being data sources for the storage operation.

13. A method for configuring a storage system for the performance of
storage operations, the method comprising:assigning a plurality of
storage system components executing in one or more computer processors
into multiple groups, wherein the multiple groups have different storage
policies associated therewith;reassigning one or more storage system
components associated with a first group to at least the first group and
a second group, and in the event a storage operation associated with a
first storage policy of the first group conflicts with a second storage
operation associated with a second storage policy of the second group,
the storage policy associated with a higher priority controls.

14. The method of claim 13, wherein the one or more storage system
components associated with the first group that are reassigned to both
the first group and the second group, automatically inherits the second
storage policy associated with the second group while retaining the first
storage policy associated with the first group.

15. The method of claim 13, wherein in the event a non-conflict storage
operation associated with the first storage policy is to be performed on
a reassigned storage system component, performing the non-conflict
storage operation in accordance with the first storage policy.

16. The method of claim 13, wherein in the event a non-conflict storage
operation associated with the second storage policy is to be performed on
a reassigned storage system component, performing the non-conflict
storage operation in accordance with the second storage policy.

17. The method of claim 13, wherein assigning storage system components to
one or more groups comprises selecting storage system components for a
group based on the storage system components having a common
characteristic.

18. The method of claim 13, wherein the storage system components comprise
sub-client components of a plurality of clients, the clients being data
sources for the storage operation.

19. The method of claim 18, wherein assigning storage system components
comprises assigning the sub-client components to comprise subsets of
client applications that identify common portions of client data.

20. The method of claim 18, wherein assigning storage system components
comprises assigning the sub-client components to a group based on the
sub-clients having data taken from common directory structures on
clients.

Description:

PRIORITY CLAIM

[0001]This application is a continuation of and claims priority to U.S.
application Ser. No. 11/269,519, titled METHOD AND SYSTEM FOR GROUPING
STORAGE SYSTEM COMPONENTS, filed Nov. 7, 2005; and claims the benefit of
U.S. provisional application No. 60/626,076 titled SYSTEM AND METHOD FOR
PERFORMING STORAGE OPERATIONS IN A COMPUTER NETWORK, filed Nov. 8, 2004,
and U.S. provisional application No. 60/625,746 titled STORAGE MANAGEMENT
SYSTEM filed Nov. 5, 2004, each of which is incorporated herein by
reference in its entirety.

COPYRIGHT NOTICE

[0002]A portion of the disclosure of this patent document contains
material, which is subject to copyright protection. The copyright owner
has no objection to the facsimile reproduction by anyone of the patent
document or the patent disclosures, as it appears in the Patent and
Trademark Office patent files or records, but otherwise reserves all
copyright rights whatsoever.

RELATED APPLICATIONS

[0003]This application is related to the following pending applications,
each of which is hereby incorporated herein by reference in its entirety:

[0014]The invention disclosed herein relates generally to data storage
systems in computer networks and, more particularly, to improvements in
allocating system resources by automatically grouping storage system
components such as hardware, software, or data components.

[0015]There are many different computing architectures for storing
electronic data. Individual computers typically store electronic data in
volatile storage devices such as Random Access Memory (RAM) and one or
more nonvolatile storage devices such as hard drives, tape drives, or
optical disks that form a part of or are directly connectable to the
individual computer. In a network of computers such as a Local Area
Network (LAN) or a Wide Area Network (WAN), storage of electronic data is
typically accomplished via servers and storage devices accessible via the
network. These individual network storage devices may be networkable tape
drives, optical libraries, Redundant Arrays of Inexpensive Disks (RAID),
CD-ROM jukeboxes, and other devices.

[0016]There are also many different system processing and management
architectures for such storage systems. One common architecture is a
client-server architecture or two-tier architecture. The client-server
architecture is a network architecture in which servers are used to
manage system operations and components, such as storage operations,
storage devices, databases, network traffic and processors, and other
operations and components, for clients, which are typically computers
running applications.

[0017]In general, when a system component, such as a client or a server,
or other component, is newly added to the system or reallocated according
to system storage requirements, each component must be individually
configured in the network storage system. For example, the component must
be associated with other existing system components or storage
operations, information about network pathways to connect to the new
component must be input into the system, and other system configuration
requirements must also be entered. This requires that a system
administrator set forth a new system configuration each time a new system
component is added. In certain small-scale storage systems, adding each
new component may be practicable, however, in systems haling thousands of
components, adding new components and respective individual
configurations for each becomes onerous. There is therefore a need for
improved methods for adding and configuring storage system components.

BRIEF SUMMARY OF THE INVENTION

[0018]The present invention includes methods and systems for allocating
storage system resources by automatically grouping and configuring
storage system components.

[0019]In one embodiment of the invention, a method and system for grouping
networked storage system components is provided. One part of the method
involves selecting, among a plurality of storage system components, a
first storage system component. In general, the storage system component
may be a machine, client, sub-client, server, or other storage component.
The first storage system component is generally selected from a list, or
other interface, of the system components. The list is preferably
generated by querying an index of a storage manager.

[0020]Further, in accordance with the method, the first storage system
component is associated with a second storage system component into a
group, such as a component group.

[0021]Data relating to the association of the first and second storage
system component is stored as index data, preferably to the index of a
storage manager or a media management component. The first and second
storage system components may be the same type of entity, such as two
clients or two servers. Alternatively, the first and second storage
system components may be different entities, e.g. a client is associated
with a server. Each storage system component is associated with or
assigned at least one storage preference. In one embodiment, the storage
preference is a storage policy, sub-client policy, user preference or
other storage preference, each further described herein. In another
embodiment, the storage preference is defined by a system user or system
administrator. In yet another embodiment, the storage preference is a
default preference.

[0022]According to another embodiment of the invention, a method and
system for automatically associating preferences to a storage system
component is provided by selecting a storage system component. The
selected storage system component is associated with an existing group of
one or more storage system components. The existing group has at least
one preference, such as a storage policy, sub-client policy, user
preference or other storage preference, each further described herein.
The at least one preference is thus automatically associated with the
selected storage system component associated with the corresponding
group. Thus the selected storage system inherits the at least one
preference of the group. Data relating to the association of the selected
storage system component with the group, and data relating to the at
least one preference of the group is typically stored as index data to an
index of a storage manager or media management component.

[0023]According to another embodiment of the invention, a method and
system are provided for automatically associating preferences to a
storage system component by identifying at least one storage system
component. The storage system component is identified, for example, by
querying a storage manager. In addition, characteristics of the storage
system component are detected, as further described herein. In accordance
with a template, such as a master grouping policy, which sets forth
parameters for grouping system components, the at least one storage
system component is selected to be added to a group. The storage system
component is typically automatically selected by comparing the detected
characteristics of the component to the parameters of the template.

[0024]The selected storage system component is then assigned to or
associated with the group. In general, the group has one or more storage
system components and already exists. The group generally has at least
one preference, such as a storage policy, sub-client policy, user
preference or other storage preference associated with the group. The
group's at least one preference is automatically associated with the
selected storage system component. Thus, the selected storage system
component inherits the at least one preference or policy of the group. In
another embodiment of the invention, the association of the group and the
preference is stored as index data in the index of the storage manager or
media management component.

[0025]The present invention further includes methods and systems operating
in conjunction with a modular storage system to enable computers on a
network to share storage devices on a physical and logical level. An
exemplary modular storage system is the GALAXY® backup, retrieval
system, and QiNetix® available from CommVault Systems of New Jersey.
The modular architecture underlying this system is described in the above
referenced patent applications, each of which is incorporated herein.

BRIEF DESCRIPTION OF THE DRAWINGS

[0026]The invention is illustrated in the figures of the accompanying
drawings, which are meant to be exemplary, and not limiting, in which
like references are intended to refer to like or corresponding parts, and
in which:

[0027]FIG. 1 is a flow diagram presenting a method for grouping storage
system components according to one embodiment of the present invention;

[0028]FIG. 2 is a flow diagram presenting a method for automatically
associating preferences of a group of storage system components according
to one embodiment of the present invention;

[0029]FIG. 3 is block diagram showing a high-level view of the network
architecture and components according to one embodiment of the invention;

[0030]FIG. 4 is a block diagram of an example of associated storage system
components according to one embodiment of the invention;

[0031]FIG. 5 is a block diagram depicting an index cache within a storage
manager in accordance with one embodiment of the invention;

[0032]FIG. 6 is a flow chart of a method for initializing storage
operations on electronic data in a computer network according to one
embodiment of the invention;

[0033]FIG. 7 is a flow chart of a method for updating a storage procedure
assigned in a computer network according to one embodiment of the
invention; and

[0034]FIG. 8 is a flow chart of a method for performing a storage
operation according to one embodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0035]In the following description of preferred embodiments, reference is
made to the accompanying drawings that form a part hereof, and in which
are shown by way of illustration embodiments in which the invention may
be practiced. It is to be understood that other embodiments may be
utilized and procedural and structural changes may be made without
departing from the scope of the present invention.

[0036]The present invention involves methods and systems for configuring a
storage system by grouping components of the system into logical
groupings, assigning the groups to storage policies, which define aspects
of storage operations, and applying these storage policies to the
components in the groups to perform the storage operations. The
components upon which these methods may be performed include hardware
components, such as clients, hard drives, backup devices, or identifiable
portions thereof, software components such as applications residing on
clients, and data sources such as data stored within specified
directories on client machines or data associated with specific
applications. Components may include sub-clients, which generally refers
to hardware, software or data portions of clients in the system.
Components may be grouped manually, based on user selections, or may be
grouped in an automated fashion by identifying common characteristics of
components or by applying a template to system components.

[0037]The methods described herein may be performed in any storage system.
The functionality for creating and maintaining data representing group
assignments and associations and with policies, for storing such data and
applying the policies is typically provided by a-storage manager for the
storage system.

[0038]Referring to FIG. 1, then, system components are selected, step 10,
to be grouped. The selection may be performed through manual input, for
example, from a menu, screen or other listing of system components. In
general, a system component listing is provided by querying an index or
table storing identifiers for system components and includes components
such as machines, clients, data agents, sub-clients, or other hardware or
software components. The presence of system or network hardware and
software components may be input into the index by system administrators
or may be automatically discovered by conventional network management
systems as known to those of skill in the art. The list of system
components will typically include a list of existing component groups in
the system, if any. Each system component in the list may be presented
together with information about the components, such as type, location,
applications used, or other information. A user or administrator may want
to group system components, for example, according to component type,
component location, department in which the component is used, its role
in the enterprise, application type, or other category or characteristic.
For manual input, the user or administrator selects the component(s) to
group, for example by highlighting, clicking or otherwise selecting the
components in a user interface.

[0039]The selected components are associated in a group, step 15. The
association of the components in the group is accomplished by storing
data relating to the association of the group components as index data in
an index accessible to a storage manager. Storage preferences are
associated with the group, step 20. Assigning preferences to the group
causes each of the components in the group to inherit the group
preferences. In some embodiments, default storage preferences are
automatically assigned to a new group temporarily until a user or
administrator defines storage preferences for the group. In other
embodiments, a user or administrator manually defines storage preferences
for the group. Data relating to the preferences associated with the group
is typically stored as index data in the index accessible to the storage
manager.

[0040]When a system has existing groups, and a new system component is
added to the existing group, a user or administrator need not
individually configure a new storage policy, sub-client policy, user
preference or other defined storage preferences for each new component.
In yet another embodiment of the invention, a component may be
automatically associated with storage preferences by associating the
component with a group, which has associated storage preferences. Thus,
inheritance is provided for the new components added to an existing
group. Inheritance provides the ability for a component group's
predefined definitions of storage policies, sub-client policies, user
preferences or other storage preferences to be automatically applied to a
client or other system component newly associated with the component
group.

[0041]An example of the method described in FIG. 1 is as follows. A server
is associated with a component group, which has a defined storage policy,
sub-client policy, user preferences or other storage preferences. When
the server is associated with the existing component group, the server
inherits the storage policy, sub-client policy, user preferences or other
storage preferences of the group. Data relating to the association of the
server in the group and the associated storage preferences is stored as
index data. In another embodiment of the invention, the user or
administrator may define or amend the storage preferences of the group at
any time and the storage preferences for each component in the group is
defined or amended accordingly.

[0042]Another example of the method described in FIG. 1 involves the
grouping of certain data structures across a number of clients in the
storage system. For example, if an administrator wished to perform a
storage operation, such as a backup operation, on all email files stored
on clients according to a single set of preferences or rules, those files
would be grouped into a single logical group, which group would then be
associated with the storage policy that defined the preferences and
procedures to be followed by the storage system for performing the
storage operation. Thus, any client device having data belonging to the
group would be automatically associated with this policy, without the
need to separately configure the client device. The data in these types
of sub-clients could be distinct groupings of application data, file
system data, operating system data or any other type of grouping.
Additionally, data may span multiple sub-clients and sub-clients may be
physically located on more than one computer or storage volume.

[0043]Component groups can be automatically created without user input.
Referring to FIG. 2, in another embodiment of the invention components
are assigned to a group in accordance with one or more master grouping
policies or templates. The grouping policy or template sets forth a
general policy or templates for grouping components, which includes, for
example, parameters for grouping components. The policy or template may
be based on characteristics of the component, storage preferences, or
other logic. For example, a component running a particular application or
a certain type of component may be a parameter for creating a group. As
another example, data stored for given software applications or within a
defined set of directories on a client may be parameters for group
creation. In such a case, the template would define how to locate and
identify such components, including for example, by locating certain
pathways on a client device or for looking for files having certain name
extensions. The grouping policy or template is preferably stored as index
data in the index of a storage manager or a media management component.

[0044]In general, the system detects components, step 30, to assign to a
group. For example, the system detects when a new hardware, software or
other system component is added to the system, when a component changes,
such as when a characteristic of the component changes, or when such
component becomes available. According to one embodiment, the system
queries the components or related storage system management components to
detect system components to be assigned to a group. According to another
embodiment, whenever a component, such as a storage device, such as a
client or server, software application or other component is added to the
system, or whenever a component changes characteristics, it is
automatically detected to be added to a group. The changed
characteristics in devices may be detecting by querying storage system
components associated with or responsible for those devices to check
characteristics. Techniques for doing such checking of device
characteristics are described in co-pending patent application Ser. No.
11/269,513, titled METHOD AND SYSTEM FOR MONITORING A STORAGE NETWORK,
filed on even date herewith, which application is incorporated herein by
reference in its entirety.

[0045]The component characteristics are identified, step 32, by the
system. In some embodiments, the storage manager index data supplies
information regarding a system component to be grouped. In other
embodiments, the system automatically detects characteristics of the
component. Component characteristics may include, for example, network
pathways, processor speed, location, user preferences, client application
type, storage type, storage operation type, manager preference, hardware
type, software type, department in which it is used, its role in the
enterprise server type, or other characteristic. Based on the identified
component characteristics, and in accordance with the grouping policy or
template, certain components are selected, step 34. The component(s) are
selected, as appropriate, when the identified characteristics of the
component are equivalent, correspond to, or are otherwise related to the
parameters of the grouping policy or template in accordance with a
threshold or algorithm defined in the template or policy.

[0046]For example, when a new client is added to the system that runs
accounting applications, the new client is detected by the storage
manager and identified as an accounting department related client. Such a
client, or a relevant portion thereof acting as a sub-client, would
likely be added to a group of accounting department components, or other
component group according to the grouping policy or template for clients
running accounting applications. In another example, when a storage
device becomes available for storage operations, or otherwise changes
state, it is detected by the storage manager and identified as being
available, and having particular characteristics, such as storage
capacity. The storage device would likely be added to a component group
requiring its particular storage capacity, or otherwise in accordance
with the grouping policy or template.

[0047]The selected component(s) are associated with the group, step 36, in
accordance with the grouping policy or template. The association of the
selected component(s) is stored as index data in the index of a storage
manager. Any preferences associated with the group are automatically
associated with the selected component(s) added to the group, step 38. In
one embodiment, the association of the group preferences is stored as
index data.

[0048]In another embodiment, the automatic group assignment and
association of storage preferences is a temporary grouping. In this
embodiment, a message or alert is sent to a user or administrator
regarding the addition of the new component to a group. The user can
confirm the grouping, or manually change the grouping if necessary.

[0049]As an example of the process set forth in FIG. 2, suppose an
administrator of a storage network wants to configure the network such
that all data associated with a messaging application such as MICROSOFT
EXCHANGE be stored in accordance with one set of preferences, including
type of backup device, length of storage, encryption used, etc., while
all data representing individual work product of users in the network be
stored according to a different set of such preferences. The
administrator would establish one template, which locates the standard
operating system directories, set up by the messaging application during
installation, and would establish a second template to locate directories
such as MyDocuments or MyPhotos in which users are expected to store
live, production data of their work files. The templates would then be
used in the process to assign these directories on each client machine
with their respective groups, in this case a first group for messaging
sub-clients and a second group for user working files. These groups would
be associated with the respective storage policies generated by the
administrator. When a storage operation is performed, the storage manager
applies the storage policy to the data files in the directories assigned
to each respective group, so that all the messaging data files are stored
according to the messaging storage policy and all the working files are
stored according to the user working file storage policy.

[0050]As indicated above, these processes may be performed in any storage
system configuration. An example of a storage system employing these
methods is shown in FIG. 3. As shown, the system includes a plurality of
clients 50 (of which one is shown), data agents 60, sub-clients 62, 64, a
data store 70, a storage manager (or storage management component) 80, a
jobs agent 85, a storage manager index 90, one or more media management
components (or media agent) 100, one or more media management component
indexes 110, and one or more storage devices 120. Although FIG. 3 depicts
a system having two media management components 100, there may be one
media management component 100, or a plurality of media management
components 100, providing communication between client 50, storage
manager 80 and storage devices 120. In addition, the system may include
one or a plurality of storage devices 120.

[0051]A client 50 may be any networked client 50 and typically includes at
least one attached data store 70. The data store 70 may be any memory
device or local data storage device known in the art, such as a hard
drive, CD-ROM drive, tape drive, RAM, or other types of magnetic,
optical, digital and/or analog local storage. The data store 70 includes
sub-client 162 and sub-client 264. The sub-clients 62, 64 are generally
used to identify portions of data in the data store 70. A sub-client
typically contains a subset of the client applications. Although there
are two sub-clients shown in FIG. 3, one or more sub-clients may be
created for a client computer, as explained further above.

[0052]In some embodiments of the invention, client 50 includes at least
one data agent 60, which is a software module that is generally
responsible for performing storage operations on data of a client 50
stored in data store 70 or other memory location. Storage operations
include, but are not limited to, creation, storage, retrieval, migration,
deletion, and tracking of primary or production volume data, secondary
volume data, primary copies, secondary copies, auxiliary copies, snapshot
copies, backup copies, incremental copies, differential copies, synthetic
copies, HSM copies, archive copies, Information Lifecycle Management
("ILM") copies, and other types of copies and versions of electronic
data. In some embodiments of the invention, the system provides at least
one, and typically, a plurality of data agents 60 for each client, each
data agent 60 is intended to backup, migrate, and recover data associated
with a different application. For example, a client 50 may have different
individual data agents 60 designed to handle Microsoft Exchange data,
Lotus Notes data, Microsoft Windows file system data, Microsoft Active
Directory Objects data, and other types of data known in the art.

[0053]The storage manager 80 is generally a software module or application
that coordinates and controls the system, for example, storage manager 80
manages and controls storage operations performed by the system. The
storage manager 80 communicates with all components of the system
including client 50, data agent 60, media management components 100, and
storage devices 120 to initiate and manage storage operations. The
storage manager 80 preferably has an index 90, further described herein,
for storing data related to storage operations. In general, the storage
manager 80 communicates with storage devices 120 via a media management
component 100. In some embodiments, the storage manager 80 communicates
directly with the storage devices 120 (not shown in FIG. 3).

[0054]The system includes one or more media management components 100. The
media management component 100 is generally a software module that
conducts data, as directed by storage manager 80, between client 50 and
one or more storage devices 120, for example, a tape library, a hard
drive, a magnetic media storage device, an optical media storage device,
or other storage device. The media management component 100 is
communicatively coupled with and controls the storage device 120. For
example, the media management component 100 might instruct a storage
device 120 to perform a storage operation, e.g., archive, migrate, or
restore application specific data. The media management component 100
generally communicates with the storage device 120 via a local bus such
as a Small Computer System Interface (SCSI) adaptor.

[0055]Each media management component 100 maintains an index cache 110,
which stores index data that the system generates during storage
operations as further described herein. For example, storage operations
for Microsoft Exchange data generate index data. Media management index
data includes, for example, information regarding the location of the
stored data on a particular media, information regarding the content of
the data stored such as file names, sizes, creation dates, formats,
application types, and other file-related criteria, information regarding
one or more clients associated with the data stored, information
regarding one or more storage policies, storage criteria, or storage
preferences associated with the data stored, compression information,
retention-related information, encryption-related information,
stream-related information, and other types of information. Index data
thus provides the system with an efficient mechanism for performing
storage operations including locating user files for recovery operations
and for managing and tracking stored data.

[0056]The system generally maintains two copies of the media management
index data regarding particular stored data. A first copy is generally
stored with the data copied to a storage device 120. Thus, a tape may
contain the stored data as well as index information related to the
stored data. In the event of a system restore, the index data stored with
the stored data can be used to rebuild media management component index
110 or other index useful in performing storage operations. In addition,
media management component 100 that controls the storage operation also
generally writes an additional copy of the index data to its index cache
110. The data in media management component index cache 110 is generally
stored on faster media, such as magnetic media, and is thus readily
available to the system for use in storage operations and other
activities without having to be first retrieved from storage device 120.

[0058]A storage policy is generally a data structure or other information,
which includes a set of preferences and other storage criteria for
performing a storage operation. The preferences and storage criteria may
include, but are not limited to: a storage location, relationships
between system components, network pathway to utilize, retention
policies, data characteristics, compression or encryption requirements,
preferred system components to utilize in a storage operation, and other
criteria relating to a storage operation. A storage policy may be stored
to a storage manager index, to archive media as metadata for use in
restore operations or other storage operations, or to other locations or
components of the system.

[0059]Index caches 90 and 110 typically reside on their corresponding
storage component's hard disk or other fixed storage device. For example,
jobs agent 85 of a storage manager component 80 may retrieve storage
manager index 90 data regarding a storage policy and storage operation to
be performed or scheduled for a particular client 50. The jobs agent 85,
either directly or via another system module, communicates with the data
agent 60 at the client 50 regarding the storage operation. In some
embodiments, jobs agent 85 also retrieves from index cache 90 a storage
policy associated with client 50 and uses information from the storage
policy to communicate to data agent 60 one or more media management
components 100 associated with performing storage operations for that
particular client 50 as well as other information regarding the storage
operation to be performed such as retention criteria, encryption
criteria, streaming criteria, etc. The data agent 60 then packages or
otherwise manipulates the client data stored in client data store 70 in
accordance with the storage policy information and/or according to a user
preference, and communicates this client data to the appropriate media
management component(s) 100 for processing. The media management
component(s) 100 store the data according to storage preferences
associated with the storage policy including storing the generated index
data with the stored data, as well as storing a copy of the generated
index data in media management component index cache 110.

[0060]In some embodiments, components of the system may reside and execute
on the same computer. In some embodiments, a client component such as
data agent 60, media management component 100, or storage manager 80
coordinates and directs local archiving, migration, and retrieval
application functions as further described in application Ser. No.
09/610,738. These client components may function independently or
together with other similar client components.

[0061]Referring to FIG. 4, storage system components may be associated
with other storage system components to form a group, such as a component
group. The component group is a logical arrangement of one or more
storage components. Other component groups may also be formed that
represent logical arrangements of one or more system components. For
example, a component group may be formed having one or more client and
server components logically arranged in a group, a storage device group
may be formed having one or more storage devices logically arranged in a
group, or other groups of storage system components, etc. A group of
portions of volumes in clients may also be arranged. In essence, any
identifiable component, which can be operated upon by a storage
operation, can become an element of a storage group.

[0062]Examples of representative component groups described herein include
a client group, client-server group and heterogeneous component group.
Examples of client group 200, client-server group 205 and heterogeneous
component group 208 are shown in FIG. 2. Client group 200 includes
clients 1-5, 170-178 which are logically associated in a group. According
to one embodiment of the invention, each of the clients 1-5, 170-178 are
associated with similar storage preferences, further described herein.
Similarly, clients 6-n, 180-184 and server 160 are logically associated
in a client-server group 205, and storage manager 192, storage device
194, media management component 196, and client X 198 are logically
arranged in a heterogeneous component group 208.

[0063]Each of the components in the component groups 200, 205 and 208 are
associated to have similar storage preferences, further described herein.
As shown, a client group 200, client-server group 205, heterogeneous
group 208, and similarly, any other component group may contain one or
more components. When a component is associated with another component in
a group, data relating to the association of the group is stored as index
data in the storage manager index or media management component index.

[0064]A component group typically has a defined storage policy, sub-client
policy, user preferences or other defined storage preferences. Data
relating to the storage policy, sub-client policy, user preferences or
other defined storage preferences assigned to the component group are
stored in an index, such as the storage manager index. In general, a
storage policy defines preferences and procedures for all devices in the
associated group. User preferences may be any storage operation
preferences set by an administrator or other system user, and which
generally overlap with preferences set forth in a storage policy,
sub-client policy, or other storage preferences. Other storage
preferences that may be set in a storage policy include:

[0065]data security settings;

[0066]encryption settings;

[0067]data retention requirements;

[0068]frequency of storage operations such as back ups;

[0069]types of data for storage operations such as back ups;

[0070]types of storage operations to perform in the component group;

[0071]network pathways for use in performing a storage operation;

[0072]scheduling such as a schedule of storage operations; and

[0073]reports such as automatic generation of system reports regarding the
group.

[0074]Each component in a component group inherits the component group's
defined storage policy, sub-client policy, user preferences or other
defined storage preferences. For example, when a client or other
component is associated with a group, the storage preferences associated
with the group are also associated with the client or component added to
the group. Thus, new clients inherit the new storage policy, sub-client
policy, user preferences or other defined storage preferences. For
example, when a new client is added to a group, the new client inherits
the jobs scheduled for the group to which it was added. The storage
preferences inherited by the new client are updated to the media
management index.

[0075]Several different techniques and data structures may be used to
maintain the associations of clients, sub-clients, and other devices and
software with a group, and to maintain the association of a group with a
storage policy or preference set. In some embodiments, a simple data
structure such as a look up table may be used to match up groups,
identified through names or other identifiers assigned by an
administrator or generated based on the characteristics of the group,
with the component(s) listed to be in that group and the policy or
policies associated with the group. Alternatively, a relational database
may be employed in which, for example, each group has a series of records
in a table identifying the components in the group, and one or more other
tables are associated with the group tables to create links between
groups and storage policies and between groups and other properties. In
an object oriented programming system, the groups may be defined as
classes of objects where each object represents the properties or
characteristics of the group and further contains the storage policy or a
link or pointer to the storage policy associated with the group. Each new
component determined or identified to be associated with the group would
then instantiated as an instance of the object class for that group. One
skilled in the art of programming would recognize that other programming
techniques or data structures may be employed to similar effect and are
included within the scope of the present invention.

[0076]An example data structure for storing groups and their associations
with policies is shown in FIG. 5. As shown in FIG. 5, the index cache 90
accessible to storage manager 80 contains a mapping between logical
storage groups SG1, SG2, SG3, SG4, . . . , SGn,
and respective storage procedures 350, labeled SP1, SP2,
SP3, SP4, . . . , SPn. As shown, group SG1 is mapped
with storage procedure SP1. If the administrator desires to change
the storage procedure associated with all of the data in-group SG1,
the administrator may simply reassign that group to, for example, storage
procedure SP4--such a potential reassignment being shown in FIG. 5
through a dotted line. In this way, instead of requiring the
administrator to enter a change for every portion of data in-group
SG1, the administrator may simply make one change in mapping.
Similarly, if a storage system component changed its characteristics such
that it switched from group SG2 to SG3, the storage policy that
would apply to that component would automatically be switched from
SP2 to SP3. Although not shown in FIG. 5, it is understood that
multiple groups may be mapped to one storage policy, and that one group
may be mapped to multiple storage policies.

[0077]In general, a user or administrator may define a component group to
have any storage policy, sub-client policy, user preference or other
storage preference. In addition, each client can be divided such that one
or more sub-clients or one or more sub sets of a data store of a client
can be associated in different component groups. For example, sub-client
1 of client I may be associated with component group 1, sub-client 2 of
client 1 may be associated with component group 2 and component group 3.
The storage policy, sub-client policy, user preference or other storage
preferences of each component group is inherited by the component
associated with the component group. A particular group of volumes 160
may store similar data for an application, file system, operating system
or any other type of data. When configuring the storage operations for an
application, a plurality of storage groups F, G, H and K are created
which logically group portions 170 of data stores 90. This is
accomplished by assigning a logical name space to the physical location
of portions of data 170 residing on a particular hard disk HD w, x, y and
z.

[0078]Components, such as clients, may be associated with one or more
groups. For example, a client may be associated with a client group, a
client-server group or a heterogeneous group. In the event that a
component is associated with more than one group, the component receives
the storage policies, sub-client policies, user preferences or other
defined storage preferences for each group. A component's association
with more than one group can lead to, for example, overlapping back up
operations or other duplicative storage operations. This overlap is
useful in some systems to provide additional data back up protection.
There are conflicting aspects of the overlapping operations of
preferences in the storage policies to which a component is associated.
To resolve this problem, the groups may be stored in a hierarchy of
groups or otherwise prioritized, so that the policy of the group higher
in the hierarchy than other groups, or otherwise having a higher
priority, would control over groups lower in the hierarchy. In
alternative embodiments, a given component that could be associated with
two or more groups would only be assigned to the group having the higher
priority.

[0079]Referring to FIG. 6, there is shown a flow chart of a method for
initializing a storage operation on electronic data in a computer network
according to an embodiment of the invention. The method in FIG. 6 could
be used with, for example, the systems described with reference to FIGS.
3 and 4. At step S2, an administrator defines portions of data sources
for data stores in the network. These portions could be entire volumes or
portions of volumes and may be on a single computer or across multiple
computers including use of network attached storage. At step S4, the
administrator groups together a plurality of the portions into a logical
group. For example, all the MICROSOFT EXCHANGE application data on data
stores may be grouped together. At step S6, an identifier is assigned to
the group. At step S8, the administrator assigns another respective
identifier for each storage procedure used in the system. At step S10,
the administrator associates the identifier for the group with the
identifier for a desired storage procedure for the group.

[0080]Referring to FIG. 7, there is shown a process for changing storage
procedures in accordance with an embodiment of the invention. The process
shown in FIG. 7 could be implemented using, for example, the system
described in FIGS. 3-4. As shown in FIG. 7, at step S20, a second
identifier is assigned to a second storage procedure. At step S22, the
identifier associated with the first storage procedure is disassociated
from the group defined in step S4. At step S24, the second identifier is
associated with the group.

[0081]From the perspective of an administrator or other system user, users
may administer and track preferences relating to component groups or
other component groups, such as system resource requirements, network
pathways, administrative access to system components, storage cost,
resource use, or other preferences. Tracking storage costs by component
groups provides an easier way to bill a user or client for the resources
utilized. An example of a costing metric is further described in patent
application Ser. No. 60/567,178. Thus, the component groups also provide
an efficient means to monitor and administer a network at a high level.

[0082]As explained above, storage operations are performed in accordance
with the inventions described herein by applying the storage policies
associated with components on which the storage operations are to be
performed. A process for applying storage policies in the context of
sub-clients is illustrated by FIG. 8. A storage operation may be
initiated for specific application data, step 500. In general, the
storage operation may be initiated by a user or administrator, or
automatically initiated by a storage manager, media agent or other system
component based on a storage policy, as described herein. Upon initiation
of the storage operation, a storage manager obtains a storage policy,
such as a sub-client policy, step 510, associated with the specific
application data that is the target of the storage operation and any
other storage policies associated with the storage system components on
which the application data is stored. For example, MICROSOFT EXCHANGE
data may be associated with a sub-client policy which indicates that such
application data is to be copied daily using one or more particular
devices or components and the device on which the EXCHANGE data is stored
may be associated with another storage policy.

[0083]The storage manager retrieves a sub-client location based on the
sub-client policy, step 520. The sub-client location may be identified by
consulting an index entry, such as a look up table, relating to the
sub-client policy. Such an index entry may be a listing of system
components, including sub-clients, or other index entry. Information
relating to the sub-client location and media agent is communicated from
a storage manager to a data agent, step 530. As described further herein,
the data agent is typically associated with a client computer and may be
a software module for performing storage operations on application
specific data generated by the client computer, such as the application
data that is the target of the storage operation. The data agent
retrieves sub-client data from a data store located on the client
computer, step 540. The retrieved sub-client data is transmitted or
otherwise made available to an appropriate media agent, step 550, for
performing the storage operation on a storage device through a copy
operation performed in accordance with any preferences set forth in the
storage policy.

[0084]Systems and modules described herein may comprise software,
firmware, hardware, or any combination(s) of software, firmware, or
hardware suitable for the purposes described herein. Software and other
modules may reside on servers, workstations, personal computers,
computerized tablets, PDAs, and other devices suitable for the purposes
described herein. Software and other modules may be accessible via local
memory, via a network, via a browser or other application in an ASP
context or via other means suitable for the purposes described herein.
Data structures described herein may comprise computer files, variables,
programming arrays, programming structures, or any electronic information
storage schemes or methods, or any combinations thereof, suitable for the
purposes described herein. User interface elements described herein may
comprise elements from graphical user interfaces, command line
interfaces, and other interfaces suitable for the purposes described
herein. Screenshots presented and described herein can be displayed
differently as known in the art to input, access, change, manipulate,
modify, alter, and work with information.

[0085]While the invention has been described and illustrated in connection
with preferred embodiments, many variations and modifications as will be
evident to those skilled in this art may be made without departing from
the spirit and scope of the invention, and the invention is thus not to
be limited to the precise details of methodology or construction set
forth above as such variations and modification are intended to be
included within the scope of the invention.