Two weeks ago, I talked about the underused power of off-the-shelf testing – there’s another important tool that states underutilize, which is blind proficiency tests. Until cannabis becomes federally legal, there is no entity better poised to run these tests that are so important to understanding the accuracy of results produced by cannabis laboratories. While there are other, third-party, options available for running proficiency tests there are compelling reasons that the state itself should engage in blind proficiency tests.
What Are Proficiency Tests?
One of the working groups of the National Institute of Standards and Technology (NIST), a government agency, defines proficiency tests as “the evaluation of practitioner performance against pre-established criteria.” It’s like a standardized test to make sure a lab’s results are accurate.
Proficiency tests, or ‘round robin’ tests, offer laboratories the opportunity to (1) confirm that their processes and procedures continue to give precise results, (2) identify possible areas of improvement, and (3) ensure the reliability of their results for their consumers. Intralaboratory comparison is an important tool for external quality assurance of laboratory results.
Types of Proficiency Tests
Proficiency tests can be categorized by who the participants are, and whether they are aware of the test at the time of testing.
Internal Vs. External
When proficiency testing involves only the personnel in a laboratory, it’s considered ‘internal’ proficiency testing. While this testing is adept at informing businesses of whether or not their processes and procedures are being followed within their organization, it is not effective at ensuring the integrity of an entire market. External, or intra-laboratory, proficiency tests involve several laboratories. These present an opportunity for laboratories to compare their results with other laboratories.
Open Vs. Blind
An ‘open’ proficiency test is one in which the participant knows that the test is happening, similar to how people who have pre-employment drug screens know they are being tested for drug use. These types of proficiency tests can give a moderate amount of information about a laboratory’s compliance. The best results of proficiency tests are created by blind testing. In blind proficiency tests, a laboratory is unaware that they are being tested. This helps to ensure that they are behaving the way they would normally behave when running the tests.
Blind, Intra-Laboratory Proficiency Testing is Important
In order to ensure the highest quality lab testing possible, it’s important that states set up blind, intra-laboratory proficiency tests for all licensed cannabis testing laboratories.
It’s imperative to periodically audit the performance of laboratories against each other often in order to ensure the highest integrity of their results. Cross-laboratory comparisons help regulators (1) monitor compliance, (2) identify possible problem areas, and (3) minimize fraudulent testing results.
It’s also important that proficiency testing be performed blind. If laboratories know that there is an upcoming proficiency test, it would be impossible to prevent usage of any of the methods for THC percentage inflation. There’s a reason that financial audits of businesses and drug testing for athletes are done with no scheduling or warning.
The Challenges of Intra-Lab Proficiency Testing For Cannabis
Initially, it seems very easy and straightforward to run a proficiency test, but often the devil is in the details. Many of these challenges are alleviated if the state is the one running proficiency tests.
Samples
Cannabis remains a federally illegal substance and intra-state transportation of cannabis is considered to be drug trafficking. As proficiency tests require the same testing material to be sent to all participants, having a state run proficiency test narrows the samples so that they can only be hemp or hemp-derived and that they don’t have to be sent across state lines.
Lack of Standardized Methods
There’s no widely accepted standardized method that covers how to test cannabis products. This can lead to a lack of consistent test results, as we’ve discussed how seemingly small nuances in testing can have a big impact, especially on potency.
By maintaining the records and performing statistical analysis on the performance of laboratories on these tests over time, the state can gain a lot of information on how effective (or not) certain regulations have been.
Rapidly Evolving Yet Highly Regulated Industry
The cannabis industry in general is still new and it requires a high degree of flexibility and agility. Cannabis lab science is also still very much evolving as the scientific possibilities around cannabis expand following prohibition. This creates an exceptionally difficult environment in which to set up a cohesive laboratory testing program.
Issues with Third-Party Proficiency Tests
There are a lot of third party proficiency testers, but without the power and neutrality of the state their motivations are a bit muddied, which makes their efficacy suspect. For instance, one third-party proficiency tester awarded ACS 82 badges for its exemplary test results. This was at the same time that ACS was fined by the state of Florida for using inappropriate methods to test for mold and yeast. ACS was fined (along with Method) for inflating THC levels.
Third-party proficiency tests also can’t currently use cannabis, and resort to use hemp-based samples to support their testing programs while not running afoul of interstate shipping laws. Using hemp to measure makes proficiency testing far less effective as a tool for examining THC percentage results, as the sample isn’t reflective of the needed test.
Blind Proficiency Tests Are Another Underutilized Tool of the State
Because of their resources, neutrality, and (hopefully) sanctioning ability, it’s vital that the state run blind proficiency tests. Some have already started fledgling programs. It’s important that states use their abilities, access, and resources to run the tests as close to blind as possible. A state-run reference laboratory is the best case arbiter of testing results, but states can still ensure accuracy of tests by performing statistical analysis on the data collected from a well-designed proficiency testing program.
Without using tools like blind proficiency tests and off-the-shelf testing, states are failing to effectively monitor cannabis testing labs.