The checklist is a popular tool to assist the cGMP auditor in conducting a thorough, systematic and consistent audit. It's important that any list of questions, whether in a "check off" format or open ended questions, be answered by comments and be used as a tool, not the entirety of the audit. If an audit is to serve both its compliance and education tasks, the checklist used should provide assistance not obstacles.
A checklist, however, should not be a substitute for proper planning and cooperative climate-setting prior to starting the audit. Consider the advantages of using a well-prepared checklist. It ensures that the auditor will ask most of the critical items necessary to satisfy cGMP requirements, and if properly prepared, it will assist the auditor in focusing on important issues and not becoming absorbed in small details.
The questions in a prepared list can, by design, be more objective than those asked extemporaneously by an auditor, even an experienced auditor. Prepared questions indicate to those whose departments or functions are being audited that the auditor is prepared to conduct a systematic audit.
Disadvantages to a checklist (or improper use of a checklist) include the following:
- A checklist can become a "crutch" instead of the tool it was intended to be.
- A checklist should not substitute proper planning of an audit, i.e., doing the "homework" necessary to identify the particular objectives of an audit.
- The focus of a prepared checklist may be too narrow to identify problem areas.
- A checklist can be intimidating to those being audited.
- The questions in a checklist may be outdated or call for conclusions that can only be answered by thorough investigation.
Let's explore some of the ways that checklists and their use can be improved. Because of the tendency to obtain the most use from any document, it's not difficult to imagine checklists or lists of prepared questions that are long overdue for updating, reorganizing, or general revision. With the widespread availability of word processing and desktop publishing software, there's no good reason why question sheets or checklists should not be updated or revised as needed. The checklist should be easy to read, easy to use and be organized in a logical manner. Since answers to questions will form the basis of an audit report, organization is doubly important.
Before starting a revision, discuss the current checklist with auditors and obtain their ideas for improvement. The group responsible for auditing should retain a copy of the latest checklist on disk so that future changes can be made without having to prepare an entirely new document.
Unless a company-wide list of terminology is prepared and circulated, an audit checklist may include a list of definitions to ensure the consistency of answers. A checklist should provide adequate space for entering answers, comments, and observations. Checklist questions should include the cGMP reference, either the specific portion of the regulation or the page number of the preamble. This can serve as an invaluable cross-reference for the auditor and can serve to educate those being audited if a requirement is not understood.
A well-prepared checklist will include instruction on how the checklist is to be used. For example, a determination of the adequacy of a written procedure cannot be answered by simple "yes/no" questions. Even correctly determining that a written procedure exists does not verify that the procedure is current, is adequate, or is being followed with proper documentation. The auditor should be instructed to lead with a question such as "show me how [specific] procedure works." By following each step to confirm that the procedure is being followed and is adequate for the specific task, the auditor can then answer specific "yes/no" questions about an identified procedure. Some questions, as pointed out in this example, are not interview questions.
Since it is unusual for every procedure in a particular department or functional area to be examined in an audit, neither department supervision nor corporate management should assume that just because a few procedures selected to auditing are found to be adequate that all procedures are adequate and current.
A quick review of the log of documents and procedures will provide good insight on how often procedures have been updated. If a procedure is more than two years old, an auditor should question if any changes have taken place.
Likewise, if the proper "climate" has been set for the interview, the auditor should obtain cooperation in identifying potential problem areas by asking, "Do you have areas of concern that you'd like me to examine?" Auditors need to observe, compare observations, and enter comments and conclusions in the space provided on the checklist or in a bound notebook. These comments will prove to be invaluable when preparing the audit report, and they go far beyond the information that can be conveyed in the answers to "yes/no" questions.
Avoid the temptation or suggestions to put observations on a rating scale. The auditor should be concerned with the adequacy of facilities, control, procedures, and documentation that meet cGMP requirements (at a minimum) or corporate requirements if they are more stringent.
Rating scales are a potential source of disagreement and a rating can be very misleading. It's difficult to apply a grade to a department or a facility unless one has a fool-proof computer model that gives proper weight to each item being evaluated and detailed criteria for assigning scores. If one serious problem is somehow overlooked during an audit and a false sense of security is thus provided with a high, but inaccurate, score, the deficiency may be overlooked until an FDA inspector identifies it on a Form 483. It's not uncommon for conscientious supervisors to want a quantitative answer to "how are we doing", but that 's a difficult answer to quantify. It's best to compliment such a manager for the areas that are under control and to say that no deficiencies were noted. Obviously, the "politics" of discussing an audit could be the subject of a lengthy article.
Since the control of quality represents a cost to any drug, food, or medical device manufacturer, there's no advantage in striving for "superior" when adequate provides all of the control needed to ensure safety, purity and efficacy of a product. Many manufacturers want something more than an "adequate" system that implies minimal compliance with regulatory requirements. Thus "adequate" or "meets requirement" may mean considerably more in a quality-committed company than in one that strives only for minimum compliance requirements.
Should you use a checklist? The answer is probably "yes", but the checklist used should be thorough and current. Auditor should be trained in the use of a particular checklist and shown how to use it to obtain maximum information by using skillful questioning techniques.