United States: Electronic Discovery – Simple Concepts And Tools To Open The Black Box

A version of this article was originally published in The National Law Journal.

After several high-profile court decisions and, more recently, the imposition of the amended Federal Rules of Civil Procedure relating to electronic evidence, attorneys and their clients are now more than ever under pressure to manage discovery of electronic evidence correctly in any litigation, regulatory inquiry, or law enforcement matter. Of course, legal education does not exactly focus on the technical skills and concepts necessary to effectively manage electronic discovery, so most lawyers rely on technology-focused vendors to help manage and process the data to find responsive records.

This in itself can be nerve-wracking for the responsible attorneys. How are they to know if the evidence advisors are doing a thorough job if overseeing attorneys do not understand how the software works or where various types of documents are or may be within the client’s electronic records? And what happens if on the eve of trial it becomes apparent that hundreds or even thousands of potentially relevant documents were not even identified in the initial searches?

Not to be alarmist, but mistakes can happen without a consistent process in place. Ernst & Young e-discovery professionals often are asked to review and evaluate internal investigations performed on behalf of our clients. In doing so, we have run into several common mistakes in handling electronic data. One of the primary errors we have seen comes in accounting for files after they have been processed or, even worse, filtered – rather than starting with the original body of data. This can greatly reduce the number of files included in the electronic evidence report and makes it impossible to know what was eliminated from the field without starting over.

We also have seen examples of providers arbitrarily defining the universe of documents for processing without discussing the exclusions with the client or attorneys. Such definitions include limiting the processing scope to company e-mail files or Microsoft Office® documents while ignoring personal e-mail files or other file types that may contain relevant information. Further, we have seen e-discovery work in which no attempts to recover password-protected or recoverable deleted files are documented in the work papers. Moreover, in many cases these restrictions on scope are not documented and are found only after careful review of the results and interviews of the teams who performed the work for the client. Similar to the decision to account for records only after they have been processed or filtered, this omission makes it impossible to account for 100% of the data without completely reprocessing the original sources. Obviously, this is expensive for the client, disruptive to its business, and entirely avoidable by doing things right the first time.

Unfortunately, as is usually the case in life, there are no easy fixes to avoid these lapses. One extra comfort to litigators or companies responding to e-discovery requests may be found in hiring e-discovery professionals who operate under an established process. For example, professionals who are members of firms licensed by the American Institute of Certified Public Accountants, work under strict professional guidelines. These guidelines include the principles outlined in the Litigation Services and Professional Standards Report issued by the AICPA. It is also important to note that, in addition to the Certified Public Accountants who are members or Partners of the firm, the firm itself is licensed and held to the highest standards of quality and professionalism.

The AICPA report offers four main guidelines for litigation services. It states that service providers should demonstrate professional competence, exercise due professional care, show adequate planning and supervision, and obtain sufficient relevant data. By following these general guidelines, we believe that many common pitfalls can be avoided.

Clearly, most electronic data professionals are not bound by the AICPA guidelines, and such overt regulation is not necessary. This article simply uses them as a basis for a general discussion of e-discovery protocols in the hopes of giving attorneys a few ideas to use in choosing vendors and establishing expectations for discovery in their cases.

The concept of professional competence is a fairly obvious one for anyone looking to hire a professional for any service. Whether in the market for a doctor, a lawyer, or an electrician, you want someone with the right education and certifications to perform that service well. The same theory applies to electronic discovery service providers.

In addition to relevant experience and publications, several designations are helpful in identifying competent professionals in the electronic discovery arena.

• CISA – Certified Information Systems Auditor
• CISSP – Certified Information Systems Security Professional
• CISM – Certified Information Security Manager
• CITP – Certified Information Technology Professional
• CFE – Certified Fraud Examiner
• CPP - Certified Protection Professional
• SCERS - Seized Computer Evidence Recovery Specialist
• CCE - Certified Computer Examiner
• CFCE - Certified Forensic Computer Examiner
• ENCE - Encase Certified Examiner
• PMP - Project Management Professional
• CRM - Certified Records Manager

Further, in light of Daubert v. Merrill Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), and Kumho Tire Company, Ltd. v. Patrick Carmichael, 526 U.S. 137 (1999), it may be prudent to consider the credibility, reputation, experience, and relevance of the service provider. These factors will come into play should your consultant or vendor have to testify in response to questions about the quality of the data acquisition and processing. Perhaps more importantly, your chosen consultant often will have to respond to the inquiries of regulators, law enforcement officials or your client’s auditors regarding the accuracy and completeness of their work.

Undoubtedly, any professional consultant or service provider who has helped manage electronic evidence for a large litigation or investigation has encountered the daunting task of documenting the process used to find responsive records amid terabytes of data. When data collection occurs in multiple locations including some on the other side of the world; when the individuals key to the investigation or litigation change or expand over time; or when client technology is archaic, inconsistent, or poorly documented, due professional care can be difficult to demonstrate.

Nevertheless, demonstrating the accuracy and completeness of the process used to collect and cull the data is critical to any litigation or investigation. This will only gain in importance as regulators, law enforcement officials, and courts become increasingly sophisticated regarding electronic evidence. Therefore, producing parties must be able to adequately account for 100% of the data collected. This includes explaining all assumptions used in de-duplicating, filtering, rendering, displaying, and exporting the data. Further, they will have to be able to defend any challenges related to spoliation, technologies, conversions, or calculations used in their reports – sometimes under rigorous examination by increasingly savvy regulators, law enforcement officials, courts, and auditors.

In order to make sure the team takes due professional care, it stands to reason that sufficient planning and supervision will be necessary. In planning, the team will need to lay out its objectives and translate them into activities needed to meet those objectives. Further, successful planning includes regular communication with the client and attorneys involved so the team’s activities can change to accommodate and document changes in scope.

In planning an electronic discovery engagement, some traditional project management tools can help. These tools include project charters that describe the engagement’s scope and define communication protocols, time lines, and data collection work plans that document the approach for consistent and thorough data collection.

Supervision helps to support adherence to a defined process and delivery of a quality product. It is important that a supervisor endorse key data processing steps, such as the confirmation of de-duplication settings or keyword terms, prior to processing. Failure to segregate duties during key processing phases, regardless of the individual’s experience, can pose a very dangerous risk to the engagement as anyone can make a typographical error or get distracted.

It is important to document all of the data from the original source (e.g., known system files, database files, non-user documents, etc.) through each collection and culling phase so the records will be as complete as possible. While all files may not need to be processed through the review database, depending on the nature of the engagement and type of data collected, all files should be accounted for and the work thoroughly documented. For example, a document-accounting equation that explains every initial culling step for Tom Smith’s hard drive is set forth below (numbers have been rounded for simplicity):

It also is important to note that all media should have some sort of identification label for tracking in the document review database to fulfill requirements regarding the chain of custody and control. For instance, an assigned barcode number tied to the media may serve this purpose and allows for quick data entry with minimal input errors. When maintained correctly in the review database, a reliable media identification will allow the review team to quickly tie back an important e-mail to the original media source it came from, along with other important metadata about the file path and source.

Electronically stored information⁵ resides in many locations beyond a local hard drive, and investigations and litigation often involve the collection of data from multiple sources – including network shares, backup tapes, personal computers, PDAs, and any other form of digital storage device that may contain relevant information. On a by-custodian basis, e-discovery professionals should document the total number of items⁶ collected in a master report that lists all items from all sources.

Beyond initial processing, accounting for how data was filtered via keywords or other metadata strategies as well as any processing, indexing⁷ or other exceptions that may limit the search is important. Data filtering accounting should break down the number of "search hits" by term or criteria on an aggregate or by custodian basis. Index exceptions reporting should identify all items not properly indexed (hence not available for keyword searching) and the appropriate reason – e.g., password protected, corrupted or unsupported file type. While it may not be appropriate from a cost/benefit perspective to recover all index exceptions, it is important to report that exceptions do exist and reasonable steps were taken to recover, re-index and search potentially relevant files.

Finally, e-discovery professionals should prepare their work papers under the assumption that they will be scrutinized by opposing counsel, a judge, regulators, or others. As a result, the working papers should include all necessary information to demonstrate the accuracy and completeness of their work. The professional ultimately responsible for the report should be able to explain the procedures performed, sources of information, and interrelationships among the working papers.

While attorneys may not need or want to be deeply involved in the technical details of computer systems operations, search techniques, and electronic data discovery processes, they should be armed with the basics required to evaluate the quality and completeness of the work done. Those basics can go a long way in laying the groundwork for a discovery project that produces a full complement of responsive data so the client company and the attorney both live up to their responsibilities to the court and to the process. By conducting projects according to a few simple guidelines, many common mistakes can be avoided, along with the costly setbacks they can represent.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.