By Mauricio Rodriguez, August 7, 2020

Whenever you take corrective action and it results in a specification addition or change, you should always ask yourself whether or not you have a design input update; that is the whole premise of design change which is how far back do you need to go in the design history file: You might just have to go to the specifications, but you might also have to go to the input because something was obviously missed in the original design. Always remember that as a part of a corrective action, you need to assess where in that sort of tree from inputs to outputs to V&V (Verification and Validation), you need to make changes.

And then remember, there is a second piece to this which is, what are the unintended consequences of the changes?

Regression Analysis

So, think about it this way; when you change something what else could be impacted by the change? This is equivalent to a “regression analysis” in software – e.g. I changed the dimensions, changed the material, what does this affect? You are not only testing your desired correction here (e.g. that it doesn’t bend or crack or break), but could the change also have some other unintended consequences?

Generally speaking, we do what may be called a “regression analysis”. So, we say, this is what we changed and why we changed it, and we validated that, and it was effective. And then we say, what other parts of this device could have been affected by the changes we made? So, what are they and how they may affect? You do this sort of failure mode analysis based upon the revisions and you ask what else could happen? And then you design an acceptance test for the device based upon the performance parameters that could have been impacted by the change.

Now, your device may have been out in the field for a while, so you have some fielded data. We suggest that you should do testing. Retrospective evaluation using fielded data is very weak, it is not a good approach. We are not saying it cannot add to an approach, but the primary approach should be bench testing to show that the device still meets all of its other dimensional and performance requirements that are unrelated to the desired corrections. There should be some sort of bench test to say “Yes, the rest of the design still performs right in a statistical model and we have fielded data which confirms this, therefore the unintended consequences are then appropriately verified”. We cannot say it enough – engineering analysis and retrospective data is weak. The default position is test.

So, “Regression Analysis” is this idea that, here are all the things that could have changed, these are unimportant or these are detected by the equipment and rendered mute, so it is only this one attribute or these two attributes that can actually be impacted and have an effect on the therapy, therefore we have to test those. That is your rationale in your protocol for what, other than the effectiveness of the change, the unintended consequence analysis justification of what tests to do, but you need to say: This is the world of tests that we would perform if this was a brand new device, here are the changes we made – e.g. they can only affect these 3 things, and of those 3 things 2 of them do not matter because the system detects them, so we have to test just 1. That is the regression analysis, that is the justification that goes in the assumption section of your protocol to justify why you selected this one test or these 2 tests to show no unintended consequences. And again, it is about this sort of analysis, saying, here is the all the tests we would run if this was a new device, here is what could be affected by our changes, and here is the subset of the tests that we will run because there are other factors that mitigate some of them.

Comprehensive testing

That is one way to do it. The other way to do it is to make sure the tests you are doing are comprehensive. So instead of doing a “regression analysis” and doing some subset of the tests, do the complete set you would do if that was a brand-new device and say you covered all of the prior changes. It is like a full catch up. So, let’s take the current design that is in the field and we are going to run all the tests on that design to verify all of its specifications and all of its performance criteria and then we caught ourselves up to the current design. So, we are not suggesting you go back to every device and every piece of the system that you ever made and look at all of the design changes – We are suggesting that for those that are related to CAPAs (Corrective Actions and Preventive Actions), if you stumble on a bunch of design changes that you doubt whether or not they were appropriately verified or validated, your choices are, “let me confirm for myself that those changes were appropriately verified, reasonable”, or “when I am redoing my testing for the changes in the CAPA, I will just re-run the complete set of verification tests and cover everything that was done before”

Key Notes

There is no design change of any kind ever that does not require verification. None. The regulations are clear, the FDA has been enforcing it for 25 years, you cannot change a tolerance, you cannot change a dimension, you cannot change a material, without doing some sort of verification.