13 May 2015
Predictive coding technology validated in landmark ruling
By Katie Fitzgerald, predictive coding specialist at Kroll Ontrack UK
This past month Ireland has given me cause for celebration due to the decision in Irish Bank Resolution Corporation Ltd & ors v Quinn & ors  IEHC 175, which sanctioned the use of predictive coding technology in the disclosure process.
For those who don’t yet know, this technology learns from human document reviewers and prioritises and categorises documents automatically using algorithms. The use of this technology in discovery has been sanctioned in the US for some time. For the first time, a court closer to home has agreed to the validity of using this technology and the benefits being reaped from it. As an advocate for predictive coding, this makes me about ready to jump for joy.
The judgment is a great read, for anyone inclined to do so, and a shining endorsement of the potential benefits of this technology.
The court held that:
- The rules of court in Ireland do not require a manual document review to be carried out;
- The evidence establishes, that in discovery of large data sets, technology-assisted review (TAR) using predictive coding is at least as accurate as, and, probably more accurate than, the manual or linear method in identifying relevant documents;
- As TAR combines man and machine, the process must contain appropriate checks and balances which render each stage capable of independent verification. The parties need to agree to these;
- Provided the process has sufficient transparency, TAR using predictive coding discharges a party’s discovery obligations;
- Predictive coding will save time and money if used to refine a data set and to limit the pool of documents to be manually reviewed (in this case, 10% of the 680,809 documents would need to be manually reviewed after employing predictive coding); and
- Parties should first agree to the use of predictive coding, run agreed upon keyword searches to initially refine the dataset and then use predictive coding subject to agreed-upon checks and balances. Documents suggested by the software as being potentially relevant should then be reviewed manually by a human review team.
The ruling addresses major concerns expressed about predictive coding and seeks to sway the sceptics. It unequivocally states that predictive coding will save time and money. Although there is no specific reference to proportionality in Irish law, the judgment states that we should be minded that cost should not be a barrier on access to justice.
English lawyers, litigants and courts take heed given the fact that proportionality is a key aspect of the Civil Procedure Rules. The methodology underpinning the use of this technology has been declared sound, as has the benefits of using it.
Is the use of predictive coding technology best practice?
The logic of the TAR process described in the judgment is sound, although the use of both keyword searching and predictive coding might raise a few eyebrows.
There is some debate within the e-discovery industry as to whether or not this is best practice: should we continue to use keywords given the limitations around their use which we now know exist (returning to many or too few documents)? There is also conflicting case law on this topic in the US. For now, the answer is that keywords remain an effective culling tool.
Is predictive coding as accurate as human review?
The court compared and discussed the merits of human review versus the use of predictive coding. Following this discussion, the judge decided that predictive coding is at least as accurate as, if not more than, a human review.
Will using predictive coding technology mean relevant documents will be missed?
The judge addressed these concerns. Common sense counters this; relevant documents will always be missed. There are likely to be mistakes, even in a human review. So long as we take the correct validation steps and our predictive coding relevance scores are sufficiently high, then we have met our obligation.
Does the use of predictive coding necessitate disclosing commercially sensitive, non-relevant information to the other side?
The final, and most interesting point to note, is the discussion of the disclosure of the document set used to train (teach) the predictive coding system, for transparency purposes.
When I talk to people about predictive coding, one of the biggest concerns is that they will have to show the other side commercially sensitive non-relevant information, to prove they have correctly taught the system how to identify relevant documents and categorise them.
The judge held that this is not the case. The training set (with the exclusion of relevant data) would not have to be disclosed; in a linear review the other side would never be allowed access to the ‘not relevant’ material and the same is true for a predictive coding review.
What does the acceptance of predictive coding mean for clients?
This case is a landmark decision in Europe and the judgment tackles with ease the concerns often articulated about predictive coding. It also provides a solid foundation for a protocol on the use of this kind of technology in the disclosure process and states with conviction that predictive coding will save time and money.
Overall, I am thrilled with this decision and the increased confidence with which clients can take advantage of this technology. I anticipate the decision will inspire many litigants to use predictive coding, safe in the knowledge that courts closer to home think the technology works and is a bona fide method to save time and money.