Document review is often seen as the unwanted and unappreciated, but
necessary, activity of the discovery process. Those legal professionals who
must engage in document review often feel as if they are hastily thrown
into the middle of a data jungle, with only the most rudimentary
instructions for working themselves out of that jungle. On the other hand,
those who manage document reviews often believe that the document reviewers
fail to grasp their larger role in the case. These managers lose sleep with
thoughts of a document reviewer failing to accurately tag a very
significant document, resulting in the inadvertent disclosure of a
privileged document, or the litigation team being caught by surprise by a
significant document at a deposition or other hearing.

Statistics show that these feelings are justified in many respects.
Accounting firm KPMG estimates that first-level document review encompasses
anywhere between 58 percent and 90 percent of total litigation costs, a
large amount by any standards. Studies and anecdotal evidence suggest that
the error rate of document review can be 50 percent or higher, a troubling
number indeed from the perspective of those who recognize the importance of
having a quality discovery process.

Given the increasing amount of electronic information that must be reviewed
for litigation and government investigations, it is critical that those
managing a document review, and those engaging in document review, be
equipped with proper training and guidelines in order to ensure that the
document review is proceeding effectively and efficiently. After discussing
general legal issues related to document review, this article sets forth
principles to assist counsel with the selection and management of a
document review vendor and an electronically stored information (ESI)
vendor, training of reviewers, and quality control of the document review.
With an organized and methodical approach for management of the document
review process, communication between all key parties, and steps along the
way to evaluate the efficiency and accuracy of the review, the document
review process can be improved, and the risks to clients and counsel from
errors in the document review process can be minimized.

Counsel's Duty of Supervision
Legal Duties. It is generally recognized that counsel must take reasonable
steps to ensure that document productions are accurate and complete.
Pursuant to Federal Rule of Civil Procedure 34 (which includes the December
2006 amendments to the Federal Rules), "A party must produce documents
as they are kept in the usual course of business or must organize and label
them to correspond to the categories in the request." In practice,
this means that counsel must supervise a process through which relevant and
responsive, nonprivileged documents are produced on a timely basis.

Potential Errors in Document Review. Errors in the document review process
can result in the inadvertent production of privileged or confidential
documents; the failure to produce all responsive documents; the failure to
meet production deadlines; and the production of data in a nonusable format
(i.e., a format that is not searchable or not able to be read). The
consequences of document review errors can be very significant, including
sanctions, or the finding of a waiver of privilege. These consequences,
which will often dramatically change the outcome and tenor of a matter, can
be minimized or avoided altogether with a thorough, organized, and
well-thought out, strategy for document review.

Strategies for Meeting Legal Duties. To meet their duty of
supervision with respect to the production of documents, counsel should
have a well-thought-out plan for collecting, reviewing, and producing all
potentially responsive information for a litigation or government
investigation. The plan should include procedures for the selection and
management of an ESI vendor and a document review vendor, and for proper
training of reviewers and quality control of the document review.

Care should also be taken to ensure that the legal team devotes sufficient
time to the supervision of the collection, review, and production process.
It can be helpful in this regard to assign lawyers experienced in discovery
the specific responsibility of supervising the process.

Good communication with opposing counsel concerning production of documents
can be very valuable. This includes establishing production parameters that
are realistic for both parties at the initial meet and confer, and
resolving any differences as early as possible, before the time for
production nears. In addition, counsel should consider clawback agreements
with opposing counsel concerning privileged documents. (Under a clawback
agreement, a party claiming privilege or protection can notify opposing
counsel of an inadvertent production. After being notified, a party must
promptly sequester, return, or destroy the information, and must not use or
disclose the information until the claim concerning that information is
resolved. Fed. R. Civ. P. 26(b)(5).) Moreover, counsel should consider
quickly apprising opposing counsel of any potential needs to change the
production schedule; opposing counsel is more likely to acquiesce if given
sufficient notice.

Information-Gathering Stage
A significant first step to any document review is gathering important
information about the matter. Counsel should obtain key information
including: (1) the nature of the case and objectives of the review; (2) the
types of ESI to be reviewed; (3) the volume of ESI to be reviewed; (4) the
form the review will take (electronic, hard copy, or both); (5) the form in
which the ESI will be produced; (6) the establishment and implementation of
quality control procedures during the review; and (7) the anticipated
production schedule.

In addition to assessing the scope of the document review, it is important
to consider the available ESI review options. These include: (1) reviewing
native files in-house; (2) reviewing hardcopy paper; (3) a litigation
support database (e.g., concordance or summation); (4) an online repository
(e.g., vendor-provided or client proprietary); or (5) a hybrid option (a
combination of options listed above).

Determining Review StrategyThe ESI Database. An appropriate ESI database will permit counsel to
take some or all of the following actions: (1) preserve and display e-mail
chains; (2) provide links between e-mails and attachments; (3) enable
document redaction; (4) permit document classification in electronic
folders; (5) give reviewers the ability to annotate or create notes; (6)
de-duplicate identical documents and/or near duplicate similar documents
prior to review; (7) utilize a multilevel search tool and combine search
filters to narrowly refine search results; (8) allow the bulk tagging of
documents by placing all search results in one folder; (9) facilitate the
collaboration of multiple parties with Web-based technology; (10) generate
automated Bates numbering; (11) create privilege logs and custom reports;
and (12) reduce the size of the original data set by removing files or
documents that can quickly be deemed irrelevant prior to review.

Selecting an ESI Vendor. The first step in selecting an ESI vendor is to
compile a short list of ESI vendors to evaluate. After the short list of
ESI vendors has been compiled, detailed information should be gathered from
the candidates. The most common method of gathering this information is
through a request for information (RFI) that requires the ESI vendors to
provide detailed, written answers to specific questions on a wide range of
topics, many of which are set forth below.

The capabilities of the final candidates should be investigated and tested.
This could include, among other things: meeting with personnel at the ESI
vendor who will be responsible for performing services for the client;
discussions with companies and law firms that the ESI vendor has named as
references; demonstration of the ESI vendor's tools, especially the review
platform; evaluation and testing of the ESI vendor's processing platform,
utilizing sample data; evaluation of the condition and security of the ESI
vendor's facilities; and testing of the ESI vendor's security procedures.

Counsel should also investigate the ESI vendor's quality control
procedures. Counsel may learn about the vendor's quality control measures
by asking the vendor to describe that process; how often the vendor
executes its quality control procedures (i.e., daily, weekly, monthly, as
requested); and what type of quality control reports are generated.

Evaluating ESI Vendor Candidates
There are several categories of criteria for evaluating ESI vendor
candidates, including their: (1) search and review capabilities; (2)
processing capabilities; and (3) production capabilities. Each of these
three categories is discussed below.

Search and Review Capabilities. The ESI vendor's search platform should
permit keyword, metadata and full-text searching, and it should permit
reviewers to review electronic data in its native format and/or in an
imaged form such as TIFF or PDF. In addition, the review platform should
permit easy tagging of documents (including bulk tagging, which allows
multiple documents to be tagged at the same time), redaction of documents
(including, if possible, the use of optional redaction background and text
colors), and creation of privilege logs and production logs.

Processing Capabilities. The ESI vendor's platform should permit processing
of standard file types, including Word, Excel, PowerPoint, e-mail and
database files, and processing of files that are encrypted or password
protected as appropriate. In addition, the ESI vendor should have the
capability to work with the client to process proprietary software files
and nonstandard applications and files.

Production Capabilities. The ESI vendor's production platform should be
able to automatically apply headers and footers to documents, and to
automatically generate Bates numbering and legends or stamps. The
production platform should permit tracking in a database of what documents
were produced, when a document was produced, and the specific Bates number
assigned to each document produced. Further, the ESI vendor should be
willing to do a sample production set to confirm the reliability of the
process and quality of produced documents and metadata.

Choosing the Review TeamChoosing the Team. Given the volume of data to be reviewed and the
often limited time in which to accomplish some reviews, many law firms hire
contract lawyers or document review vendors to handle some or all of the
document review. Choosing the appropriate document review vendor is
crucial, as this entity will be responsible for providing the actual
document reviewers and conducting the review under the supervision of
counsel's law firm. The following considerations will help to ensure that
the document review vendor, as well as the reviewers, are sufficiently
vetted and appropriate for the document review needs of the matter.

Selecting the Document Review Vendor. In addition to the document review
vendor's reputation, counsel may consider the following criteria when
selecting a document review vendor: (1) the vendor's experience with
staffing document reviews; (2) the recruiting, screening, and retention
methods for reviewers (which are discussed more fully below); (3) the size
and quality of the candidate pool of reviewers; (4) the familiarity of the
vendor with the reviewer candidate pool; (5) the ability to assemble
quality document review teams on short notice; and (6) the ability to
provide project space, workstations, and connectivity on short notice.

Recruiting, Screening, and Retention of Attorneys. While a good document
review vendor will likely address many of these considerations, counsel
should seek to ensure the attorneys selected to conduct the review have the
following characteristics, to the extent appropriate for a matter: (1) an
active law license and good standing in the jurisdiction, with no
disciplinary hearings pending; (2) prior document review experience; (3)
practice experience; (4) subject matter experience related to the case; and
(5) knowledge of the review tool. Depending upon the complexity of the
subject matter of the case, counsel may want to request document review
attorneys with a specific, relevant experience, or educational
background.

How the document review vendor interfaces with its document review
attorneys can also be indicative of the talent pool the document review
vendor is able to attract. For example, if the vendor provides appropriate
pay in the local market and benefits (e.g., health insurance and 401(k)
benefits) to document reviewers after a certain length of service with that
vendor, it will be more likely that the vendor can attract long-term
reviewers who will be more experienced in conducting document reviews.

Assembling the Review Team. In providing information to the document
review vendor so that it can assemble the review team for a matter, it is
important for counsel to provide as much information to the vendor as
possible concerning the matter and the parameters of the review. Also,
counsel should decide the optimal size of the review team, keeping in mind
that it is usually better initially to overestimate the number of document
review attorneys needed and then subsequently pare down the amount of
document review attorneys if necessary.

Introducing the Matter to the Team
At the outset of the document review, it is important that sufficient time
is given to introducing the document review team and counsel to one
another. It is usually beneficial at this initial meeting for counsel and
the document review vendor to appoint a case manager and day-to-day project
manager, respectively, for the document review. Also during this initial
meeting, the case manager and project manager should provide the reviewers
with a thorough overview of the matter. This should include key discovery
deadlines and timelines.

After the initial meeting, the case manager and project manager should
finalize a work plan for handling the document review. This should include
all the activities associated with the document review, such as policies
and procedures for the document reviewers, project orientation,
productivity expectations, whether there will be an issues log or other
method of providing the review team with guidance and feedback, a schedule
for ongoing, regular meetings with each contract attorney to review
performance, and quality control procedures.

Training for Document Reviewers
Once all relevant documents are loaded and ready for review, those legal
professionals who will conduct the document review should receive training.
It is often useful to conduct training only after documents are ready to be
reviewed, so that the training is fresh in the minds of the reviewers when
they begin.

During the document review training, the case manager and project manager
should ensure that the reviewers have a comprehensive understanding of the
matter and how the document review fits into the discovery plan for the
matter. The reviewers should understand, among other things, how to define
what is responsive to discovery requests and what is privileged. It is
usually beneficial for a memorandum and checklists to be prepared for the
reviewers to assist them in making determinations during the document
review. It is also helpful to discuss specific documents as examples and go
through the appropriate responsiveness and privilege analyses. The
reviewers should also receive documents relevant to the matter (e.g., the
initial complaint, subpoena, or document request, and any other relevant
legal pleadings). The reviewers should also receive background information
about the various coding options for the documents and why those codes were
chosen.

The reviewers should receive a thorough overview of the review tool being
used. This process should be interactive, with the reviewers receiving time
to actually test the review tool functions.

Second-Level Review of Documents
After the reviewers have received training, they can begin reviewing
documents. It is very important for the case manager and project manager to
oversee that documents are being reviewed consistently and accurately. In
the beginning stages of the document review, it may be advisable for there
to be second-level review of documents by counsel. This does not mean that
every document needs be reviewed by a more senior person, but rather that
samples should be taken of each reviewer's documents, and the accuracy of
the coding of those documents should be gauged.

If there are widespread issues where reviewers are miscoding documents, the
case manager will want to consider additional training about those issues.
If there are individual reviewers with low accuracy rates, those reviewers
should receive follow-up training if necessary, to ensure they are up to
speed and that accurate coding is occurring. The documents of those
reviewers should be analyzed more closely until it can be demonstrated that
they are accurately reviewing documents.

The second-level reviewers may also review all documents marked privileged,
as it is critical to ensure that privilege is asserted consistently and
accurately in the matter. In some matters, second-level reviewers may also
review all confidential and hot documents as well.

Conclusion
While document review will never be completely painless, the more planning
and effort counsel puts into gathering information about the matter,
selecting the appropriate document review and ESI vendors, and training and
managing the document reviewers, the better position counsel will be in to
meet production deadlines, manage costs, and avoid mistakes in the document
review process. Also, the legal professionals engaging in the document
review will perform reviews fully apprised of the circumstances of the
matter, the capabilities of the review tool, and the reasons supporting
their coding decisions. Counsel will have heightened confidence that all
parties involved in the document reviewincluding the vendors,
attorneys, and paralegalsare equipped with the resources to
achieve a high standard of accuracy. While perfection may be elusive in the
world of document review, following the principles discussed in this
article can bring that goal into closer reach.

Additional Resources

For more reading on a similar topic, you can retrieve the following
articles on the Business Law Today Web site at
www.abanet.org/buslaw/blt.

All issues since 1995 may be accessed under the "Past Issues"
heading at the bottom of the Web page.