Over three years ago, I had the pleasure of being on the acquisition team at eTERA to select a managed review provider. Today, eTERA’s Managed Review and Staffing Division, which was formally Discovery Services, LLC, is one of the leading review solutions in the country. However, as the acquisition process took months and incorporated numerous criteria necessary for us to make the right decision, I thought sharing some of those criteria might be helpful for our clients and prospective clients who are in the process of selecting their own managed review services provider.
When selecting a managed review provider, one of the most important considerations is whether or not they have strong workflows developed. The process and procedures around these workflows should be fully documented, and the individuals that manage those workflows should be diligent about ensuring that they are closely followed. That said, there should also be enough customization possible so that the workflows can adapt to the needs of your matter and adjust accordingly to achieve your legal goals. The best managed review providers will understand how to confidently strike that balance.
Quality Control Procedures
In this industry, quality control is notoriously of the utmost importance. There should be documented quality control procedures from your managed review provider and the quality control should be done no later than the day after a document set has been reviewed. Even in a small review project with 10 reviewers, a week of delay on quality control could mean that as many as 20,000 documents have been reviewed without any quality control being done on them.
Additionally, the managed review provider should assume under normal circumstances that approximately 5-7% of the documents should be put through the quality control procedure. Generally viewed as an industry standard, this percentage is based on the amount of documents that a single quality control reviewer could review, if (s)he were reviewing the documents of 10 reviewers from the previous day.
But also keep in mind that quality control requirements change, depending on the specific matter. In understanding your needs, make sure that the managed review provider has the ability to do a full 100% quality control protocol, if that is what is required.
Over 8 years ago, eTERA developed its initial 1ntelligent One reporting protocols, using state-of-the-art business intelligence software along with solid reporting protocols to provide data information to its clients at every step of the process. We wanted to ensure that commitment to robust reporting stayed consistent with our review division, as well.
When selecting a review provider, take a hard look at how they handle their reporting process. The reporting should include any variables that need to be tracked, but should definitely include the following, as a minimum:
- Review pace for each reviewer (documents reviewed/hour);
- Total number of documents reviewed each day;
- Total number of documents QC reviewed (and any changes to document coding);
- Percentage of responsive documents;
- Issue codes (and number of documents tagged for those issue codes).
These reports should be provided at a frequency that best suits each project. However, daily reports will ensure an effective ongoing handle on the matter, as well as assist in real-time understanding of progress and determining any necessary next steps. Additionally, ask to receive reports on a weekly and monthly basis so that viewing larger trends becomes substantially easier.
In addition to review progress reports, Q&A logs and summary reports can also be essential in evaluating review teams’ understanding of key concepts or requirements in a review, as well as tracking changes in approach or workflow over the course of a project.
Finally, detailed reports, when properly executed, can also satisfy the codes of conduct applicable to lawyers in the US, UK, and elsewhere. For example, in the American Bar Association’s Model Rules of Professional Conduct, Rule 5.3 (Nonlawyer Assistance, at Comment 3) provides that a lawyer must make “reasonable efforts to insure that [non-lawyer] services are provided in a manner that is compatible with the lawyer’s professional obligations”. Thorough and comprehensive managed review reporting will allow you to be sure that you have the tools to meet these requirements.
As a technology-driven company, eTERA sought r a review provider that understood that analytics, predictive coding and other technology assisted review were the new future for the review of documents.
Even where it means that their reviewers will have less billable hours, the best review service providers will understand that technology assistance can save clients substantial time and money. Your review provider should be a partner that understands that it is the long-term relationship that will make them the revenue that they need. Ensure that your review provider is well-versed in the options available around technology assistance and is not afraid to make those recommendations.
Additionally, selecting a review provider that is part of a larger data organization can also help. By having the combined experience of data and review, efficiencies can be found throughout the process that can ensure that review is as cost-effective as possible.
Finally, the importance of tested experience and expertise in a review provider cannot be understated. In looking at Discovery Services, it was apparent that Ashish Prasad and Vazantha Meyers had the expertise that eTERA was looking for. As some of the leading experts in eDiscovery, Ashish and Vazantha brought their expertise from being practicing lawyers in the eDiscovery and Records Management group at Mayer Brown to the policies, procedures and expertise of their review business. Today, they continue to enhance that expertise with the hiring of additional project managers, quality control experts and contract reviewers.
When selecting a review provider, familiarize yourself with the biographies of the individuals who will be staffing projects, managed projects, and running your managed review division. The expertise of these individuals directly correlates to the caliber of reviewers that are going to be working on your documents and increases the likelihood of efficient and accurate document reviews.
Selecting a review provider can be a tough decision. By taking into consideration the factors above and ensuring you have a managed review provider that can truly walk the walk, a cost-effective, highly efficient and superior quality review can be a near certainty.
Written by Todd M. Haley, Vice President, Strategic Solutions, eTERA Consulting. Todd can be reached at firstname.lastname@example.org.