Online Issues

Journal Vol 49 (4)

A Sample Postapproval Monitoring Program in Academia

Ron E. Banks and John N. Norton

Ron E. Banks, DVM, is Director of the Office of Animal Welfare Assurance at Duke University in Durham, North Carolina. John N. Norton, DVM, PhD, is Director of the Division of Laboratory Animal Resources at Duke University Medical Center in Durham.

Address correspondence and reprint requests to Dr. Ron Banks, Director, Office of Animal Welfare Assurance, Duke University, Hock Plaza Suite 1104, 2424 Erwin Road, Durham, NC 27705 or email ron.banks@duke.edu.

Abstract

The primary goal of an animal care and use program (ACUP) should be to ensure animal well-being while fostering progressive science. Both the Animal Welfare Act (and associated regulations) and the Public Health Service (PHS) Policy require the institutional animal care and use committee (IACUC) to provide oversight of the animal program through continuing reviews to ensure that procedures are performed as approved by the committee. But for many committees the semiannual assessment does not provide an opportunity to observe research procedures being performed. Furthermore, IACUC members are typically volunteers with other full-time commitments and may not be able to dedicate sufficient time to observe protocol performance. Postapproval monitoring (PAM) is a tool that the IACUC can use to ensure that the institution fulfills its regulatory obligation for animal program oversight. When performed by attentive and observant individuals, PAM can extend the IACUC's oversight, management, training, and communication resources, regardless of program size or complexity. No defined PAM process fits all institutions or all situations; rather, the monitoring must match the program under review. Nonetheless, certain concepts, concerns, and conditions affect all PAM processes; they are described in this article. Regardless of the style or depth of PAM chosen for a given program, one thing is sure: failure of the IACUC to engage all available and effective oversight methods to ensure humane, compassionate, efficient, and progressive animal care and use is a disservice to the institution, to the research community and to the animals used for biomedical research, testing, or teaching.

Key Words: animal care and use program (ACUP) management; animal use; compliance; institutional animal care and use committee (IACUC); postapproval monitoring (PAM); program audit; quality assurance; regulations; compliance liaison

Introduction

A hallmark of a healthy animal care and use program (ACUP¹) is a strong institutional culture of compliance and a clear commitment to animal-based research that both protects the well-being of the animals and facilitates the scientific needs of the researcher (Klein and Bayne 2007). The primary goal of an ACUP should be to support this commitment by ensuring animal well-being while fostering progressive science. In an environment where animal well-being is the guiding principle, enhanced compliance with regulatory guidance and institutional policies should follow.

All animal research programs, regardless of size or complexity, must be concerned with compliance. To this end, the focus of ACUP managers should not be "Do we have a compliance concern?" but "Where are our compliance concerns?" and "What are we doing to decrease or prevent noncompliant incidents?" There are many reasons to monitor compliance. Risk and hazards are inherent in an ACUP; hazards that affect protocol performance may be definable (e.g., dangerous species, infective or hazardous agents, reactive chemicals) or insidious (e.g., personnel who fail to adhere to the approved protocol, unintended effects of the proposed activity, experimental drift over time that becomes a laboratory procedural standard). Sometimes there are deviations from approved protocols. An institutional animal care and use committee (IACUC¹) may not have sufficient committee resources (e.g., funding, staff, data management resources, time, or skill sets) to effectively accomplish its responsibility for ensuring that only approved activities are performed at the institution.

The institution and the IACUC are responsible under federal regulation for program oversight (including animal housing and use) and thus for ensuring compliance in the performance of research activities as approved by the IACUC. But the IACUC's semiannual inspections provide a snapshot rather than the sustained observation necessary to assess overall protocol performance. Such observation generally requires considerable time, which is often difficult for IACUC members. If the institution is not assessing animal activity at a level to ensure adherence to the approved protocol or standard operating procedure (SOP¹), then it may not be meeting certain of its regulatory obligations for "continuing review of activities." Failure of the IACUC to engage all available and effective oversight mechanisms may be a disservice to the institution, to the research community, and to the animals used for biomedical research, testing, or teaching.

A postapproval monitoring (PAM¹) program is one method of extending the IACUC's oversight and institutional management of the animal program. A PAM program can assist the committee with its obligations and facilitate successful program oversight as required by federal guidelines. It can enhance the committee's ability to monitor approved procedural activities, provide information about programmatic risks and associated issues, and convey practical corrective plans for management of the identified risks. Most importantly, the IACUC's use of PAM can foster and ensure a positive research culture rather than becoming an instrument for punishing noncompliant behavior.

Although a PAM program is not specifically required by regulations or standards pertaining to animal care and use, these documents do describe the oversight responsibility of the IACUC (NIH 2002, 2005). Indeed, inadequate postapproval monitoring was noted as the most frequent concern by inspectors from the United States Department of Agriculture (DeHaven 2002), and the importance of PAM has been described specifically for animal models of movement disorders (Wallace and Sikoski 2007). Thus a defined PAM program can positively enhance IACUC oversight at many institutions.

Some may argue that the nature of science and of experimental investigation naturally result in deviations from procedures originally approved by the IACUC; experimental forecasting is at best an inaccurate activity—indeed, there would be no need to perform an experiment if its outcome were assured a priori. Although most common protocol deviations are a matter of minor drift (e.g., unapproved anesthetic, change in treatment schedule, increase in blood volumes required over a defined time) rather than intentional violation of approved experimental design, the institution should work to limit such drift before it becomes a significant protocol deviation that may compromise the integrity of the individual, the laboratory, or the institution. Effective oversight, focused training, and practical corrective actions are often successful ways to foster a culture of compliance, and PAM is one way to help achieve this desirable outcome.

Obligations of Animal Programs at an Academic Institution

Regulatory agencies have provided the framework for the humane and appropriate operation of animal care and use programs and have assigned responsibility for the oversight and management of such programs to the institution. It is the IACUC's responsibility to ensure adherence to approved protocols, conformance with institutional animal care and use policies, and the implementation of programmatic SOPs for animal care. Obligatory IACUC activities such as semiannual program reviews and facility inspections remain necessary, but singular reliance on the semiannual processes may not provide sufficient robustness to reveal noncompliance. The academic ACUP may find that PAM provides a simple and effective way to protect the institution, facilitate the IACUC's obligatory oversight in a practical and reasonable manner, enhance the participation of the attending veterinarian and clinical care staff, and provide collegial assistance to the researcher.

The specific regulatory requirements and regulatory basis for PAM activity are discussed elsewhere in this issue (Bayne and Garnett 2008; Haywood and Greene 2008). Even so, considered against the backdrop of increasing scientific complexity and depth of regulations as well as self-imposed institutional policies, the responsibility of academic IACUCs to ensure adherence to all regulatory requirements is frequently challenging. For example, academic organizations are not generally designed with the level of oversight common to the industrial sector. They are often highly dependent on research grant support, which means the institution generally benefits from having the researcher in the laboratory rather than performing institutional obligations of animal care or use oversight. Thus academic institutions may look to adjunctive measures that extend and complete the IACUC's oversight capabilities and differ in complexity and organization from ACUPs in other research organizations. A postapproval monitoring program is just such a measure.

The specific obligations of a PAM program are to:

Ensure animal well-being,
Keep the IACUC and institutional official informed about program status and process,
Communicate IACUC positions on matters of animal care and use to researchers,
Provide on-the-spot education and training needs in the laboratory,
Protect the institution's reputation,
Serve as a resource to the research community,
Support the advancement of strong science, and
Facilitate regulatory compliance.

However, the research community does not generally support programs that focus on compliance as the first goal; rather, programs that focus on ensuring quality research, proper animal care, and communication between researchers and the IACUC tend to achieve regulatory compliance as a byproduct. The most successful PAM programs are those where researchers believe they are partners with the IACUC for the success of the institution and their research program. The IACUC can encourage the perception of partnership by using PAM as a tool to foster and ensure a positive research culture rather than to seek and punish noncompliant behavior.

For institutions that perform federally funded research, indirect cost recovery (ICR) funds partially contribute to the institution's infrastructure, of which ACUP oversight is a component. But few institutions use their ICRs to support PAM. They may wish to consider allocating a portion of their ICRs to compensation for the PAM program as it can provide critical process assurance. An additional option is for institutions located reasonably near each other to share personnel associated with the PAM program by, for example, arranging for staff to spend a specified number of hours per month at each institution performing various PAM functions.

Organizational Models of a PAM Program at an Academic Institution

There are three general organizational models for conducting PAM:

Attending veterinarian (AV¹)–based PAM,
IACUC members or staff performing PAM, and
Compliance office–based PAM.

Academic institutions may choose any or even a hybrid of these basic models, depending on their programmatic needs; there is no one ideal program format as each institution has differing needs and problem areas. For each model, certain processes are easier to perform and others somewhat more difficult; a simple approach using clear processes is always best, but the institutional choice should be the method that most effectively ensures the integrity of and support for the ACUP. Generally, the more complex the program or the more intensive the research enterprises, the more likely there will be a benefit in employing a dedicated PAM staff. Whichever method is used, it is critical for the PAM liaison to establish the integrity of the process and a cooperative partnership with the group under review to ensure effective PAM.

AV-Based PAM

Program constraints (e.g., limited staff, small program) may dictate organization of the PAM under the direction of the attending veterinarian (AV). But, although the veterinary staff are experts in animal care, reliance on them as the primary mechanism for ensuring adherence to an IACUC-approved protocol may be self-defeating. Most researchers and laboratory staff consider that the primary goal of the veterinary staff is to assist them with their research while encouraging, improving, and ensuring quality animal care. They may perceive conflicts of interest in the AV's conduct of audits of both animal research and animal care practices.

In addition, the participation of veterinary staff in PAM activities may compromise collegiality and engender the attitude that the veterinary staff are also the institution's animal police. Researchers may, for example, consider a request to the veterinary staff for assistance as an implicit admission of fault in their animal care practices and fear that such knowledge will be reported to the IACUC.

Furthermore, although the AV may provide personnel management or administrative support, PAM liaisons under the AV's direction must not be reluctant—because of expected allegiance to (or fear of retribution from) the AV—to report concerns or observations to the IACUC. There must also be no filtering of content by the AV. It is therefore essential to clearly define mechanisms for reporting PAM observations to the IACUC.

Last, a blurring of compliance and clinical duties could lead to a potential conflict of interest and call into question the integrity of both the program and the AV. The PAM program's operational integrity requires that the process be clearly defined, stable, consistent, and applicable to all animal handlers and users independent of credentials.

For all of these reasons, it is important to ensure that the PAM program independently evaluates animal care as well as animal procedural activities. Generally speaking, however, the cultivation of cooperation between the AV's staff and the investigators may be more desirable than engaging the AV's staff in compliance auditing if possible. The investigator community will support PAM if it is clear that all animal user groups, including the veterinary staff, are held accountable to the expectations of the PAM program.

IACUC-Based PAM

This organizational model of PAM does not mean the other models discussed here are not under the auspices of the IACUC: all PAM is an IACUC tool. Rather, this model concerns the performance of PAM by members of the IACUC or, if there is a dedicated PAM staff, that they report to an IACUC coordinator or administrator (e.g., the committee chair). Advantages of this model include the IACUC's defined organizational entity, in which PAM simply becomes an additional duty of the committee members, who (although usually volunteers) are interested in the best outcomes of the program. A further advantage is the possibility that IACUC members who observe noncompliant activities may effect more rapid change or correction on the spot by explaining the committee's prior position on similar research performance.

But the use of IACUC members as liaisons can present unique challenges as well; indeed, for many institutions, the potential disadvantages may outweigh the advantages. Disadvantages may include the increased workload that PAM places on IACUC members, who most often have research commitments of their own. The level of assessment of a PAM program (observing practices throughout an experiment) may exceed the length of time an IACUC member can devote to the auditing process, thereby giving a false sense of completeness of the protocol or SOP. There may be difficulty in coordinating PAM visits, as they are not part of an IACUC member's primary responsibilities. New committee members, who may lack familiarity or an understanding of restrictions on their authority, may exceed their responsibilities to simply observe and report to the committee, instead instructing the persons being audited with their opinion rather than the IACUC's decision. Last, PAM audits have the potential to create conflict between members of the faculty (IACUC member vs. researcher); because many IACUC members are also investigators, the committee member performing the PAM may be viewed as a scientific competitor, thus leading to a perception of bias in the PAM process. Using a dedicated staff in this model could alleviate many of the potential disadvantages of using committee members to perform PAM activities.

Compliance Office–Based PAM

Many institutions may find that compliance office–based PAM will best meet the needs of institutional compliance, allow the veterinary staff to focus on animal care and support of the principal investigator (PI¹), and assist the IACUC with effective oversight of its regulatory obligations. This option both minimizes any real or perceived conflict of interest and potentially maximizes the availability of personnel to audit research or care practices that may be associated with either the AV- or IACUC-directed models. Following are the specific advantages of this model:

Dedicated PAM staff can devote the time required to observe all aspects of animal care or use, beginning with procurement from the home cage to postprocedure recovery. Thus the same person is available to perform focused auditing over extended periods of time.
Dedicated PAM staff are more likely to be familiar with the institution's policies and procedures.
Dedicated PAM staff are less likely to have a conflict of interest as they do not have an investment in the particular research or allegiance to an animal-using group (or leader). They can accurately state that their allegiance is to the program rather than a person or department in the program.
Dedicated PAM staff can assist the IACUC in fulfilling its responsibilities while reducing the burden on committee members for additional time investment in the program. They enable IACUC members to instead spend their time performing research, writing grants, or preparing the latest manuscript. PAM audits do not require the skills of an IACUC member.
Dedicated PAM staff enable the AV and clinical veterinary staff to focus on animal care and welfare concerns without potentially adversarial situations when interacting with investigators. Because PAM personnel are not clinical veterinarians, investigators, or IACUC members, they are not typically perceived as threatening to the research team, and they can facilitate communication between the research team and the veterinary staff.
Dedicated PAM staff can assist the IACUC with identifying specific training needs from audit findings.
Dedicated PAM staff have the advantage of "institutional memory"—they are familiar with historical concerns in particular laboratories and can direct the IACUC's attention to recurring problems.

These advantages of compliance office–based PAM, which may be more pronounced in a complex academic institution, will ultimately lead to a consistent, detailed, unbiased, and thorough monitoring process.

The Role of the PAM Staff

No matter which model an institution adopts, it is important to determine early in the process the role of the PAM staff. They are essentially the IACUC's "special envoys," spending the time necessary in the research laboratory to completely review practices and educating research associates in a nonthreatening manner. We believe that even the position title can subtly foster a sense of partnering and collegiality, thus the term "liaison" is recommended. The PAM liaison not only reports observations and concerns to the IACUC but also can assist the investigator with development of an appropriate corrective action and ensure its implementation in a manner that meets the expectations of the IACUC.

PAM performed respectfully is an opportunity for communication, education, and relationship development between the user groups (i.e., the PIs, research associates, animal care staff) and the IACUC. The PAM liaison is not a substitute for any of the IACUC's responsibilities or duties, but can assist the committee in performing program and facility review more effectively. The liaison builds bridges, encourages partnership, enhances education, and ensures animal well-being.

Modeling for Institutional Size

Every research institution, regardless of size, has the same obligation: to ensure that the care and use of animals in research, testing, or teaching adhere to the methods and procedures approved by the IACUC. The nature and implementation of PAM may differ among institutions, but all have the same tools to accomplish their obligations. At smaller institutions, the PAM liaison may be part-time and engaged in aspects of the program other than IACUC support activities; at larger institutions, multiple PAM liaisons may be desirable to oversee the animal use enterprise.

The size of the PAM program is based on a number of factors, one of which is the size of the campus animal use program. Larger institutions generally have larger and more complex problems. They may also have access to more financial resources than smaller institutions, but generally the adequate funding of any program, whether large or small, is a concern. Every institution seems to have fewer fiscal resources than the needs of the program or unfunded mandates.

Some institutions may believe that "they do not have a compliance problem." But an animal program administrator's honest assessment of research performance and observation of compliance concerns—many of which require attention, in some cases both critical and immediate—often result in PAM program initiation.

Smaller institutions have on occasion argued that because everyone knows everyone, there is no need for a PAM program. But based on such familiarity, institutions may develop a unique interpretation of committee-approved activities, which if left unchallenged can develop into significant compliance concerns. For example, the IACUC members may decide, because they all know the regulations and each other, to forgo routine audit processes and focus on non-IACUC members. Among other risks, the result of this "compliance" based on familiarity may be a less than favorable regulatory review.

At the other end of the spectrum, a large institution may suggest that there is no need for a PAM program because of its global research awards, internationally respected scientists, or highly paid consultants, which are touted as evidence that the program is free of noncompliance. But size or scientific recognition is not a guarantee of compliant performance. Just one major noncompliance incident may negatively affect the future of research at the institution.

The goal of all PAM programs should be to find deficiencies when they are small and easily correctable. If successful, this approach can secure the foundation for research and prevent costly compliance concerns. Too many large institutions have either been close to program suspension or actually had their research program suspended, outcomes that could have been avoided if the program administration had performed an internal audit (PAM) and found the problem when it was a small deficiency.

Ensuring Complete PAM Coverage

The IACUC is responsible for the entire program of animal care and use, but different components of the animal program may benefit differently from PAM. Researchers, laboratory staff, animal care operations, and breeding activities all have differing concerns but similar requirements for compliant animal activity.

Research that meets the regulatory requirements of the funding agency, oversight groups, and the institution is in the best interests of the researcher. However, in the desire to be productive, allegiance to all regulatory and IACUC required steps may occasionally be forgotten (in our experience, intentional noncompliance is rare). PAM can both help researchers remain compliant and ensure unfettered and continued investigation. In cases where performance has been less than ideal, the PAM liaison can review what was wrong, why it was wrong, and what effective solutions may prevent recurrence. In cases that require corrective actions, the PAM staff can provide counsel on appropriate responses for IACUC consideration. PAM staff can provide the proper ACUP forms or applications as well as clarifications about the role of the safety office and about partnering with the AV's staff.

Veterinary and Husbandry Staff

PAM audits should include activities of the veterinary and husbandry staff, who, like the investigative staff, have responsibilities, obligations, and management privileges with respect to animal care and use. Husbandry and vivarium staff, who focus on animal disease prevention, husbandry support, and facility management, are subject to IACUC oversight; their SOP can be considered the same type of procedural document as a protocol. The purpose of the animal care audit is to ensure that the activities of the prescriber (usually the AV) and the "approver" (the IACUC) are performed as approved in the protocol or outlined in the SOP. The PAM process for evaluating animal care is different from that for research activity but just as critical, as it provides a consistent method for addressing concerns related to animal care or use; for example, PAM can confirm that animal physical examinations are performed as scheduled or in the manner prescribed by the veterinary SOP.

Species and Locations

PAM should cover all species used in research, testing, or teaching and the auditing processes should be consistent, although the specific issues reviewed will depend on the species. There may be differences in species management (e.g., adequate airflow is required for all rodent species but is not a concern for aquatic species) or in facility operations (e.g., distant satellite livestock operations versus vivarium livestock operations, mammal care versus aquatic care), but central issues such as the storage, quality, and appropriate provision of feed and bedding are equally important for all animals. The IACUC's obligation is to ensure that the animals' needs are met in the manner approved (e.g., single housing, restriction of environmental enrichment).

Of particular concern for academic operations are pastured animals or breeding stock, which, at many institutions, are not under the direct management of the central veterinary office. Animal care practices for pastured animals entail conditions that are usually different from biomedical containment or management of animals, but the obligation of oversight to ensure the best interests of the animals and the conduct of procedures as approved by the IACUC remains the same. The use of PAM to audit these activities can clarify for the AV and IACUC what is working well and which management processes require institutional attention. Because many IACUC members at large biomedical institutions are not sufficiently familiar with agricultural species management procedures, PAM should focus on all species- and procedure-appropriate animal care and management options. IACUCs at land grant institutions generally have the skill set for competent assessment of agricultural care practices, but even here the PAM liaison should devote the time necessary for a full assessment and review of practices and procedures. For example, veterinary care or surgical procedures on livestock species often require quite a bit of time; the PAM staff should schedule adequate time to observe the practice or procedure from start to finish in order to get a complete picture of the activity.

Breeding colonies, especially for rodents, often do not fall under direct veterinary oversight and may be managed in a satellite facility under investigator control. The institution should have consistent standards for animal care regardless of location and established standards that are consistent with the type of housing paradigm. PAM can be a mechanism for observation and assessment that compares and contrasts processes between core and satellite operations. It can also provide an independent assessment of the processes used at satellite operations and can alert the IACUC to deviations from the minimum standards required by the committee. For example, if an investigator needs to maintain harem breeding, there is a risk of cage overcrowding (at least at a higher frequency than with pair mating). In a satellite investigator-managed facility, the presence of veterinary staff may be minimal because of their workloads elsewhere. The PAM liaison can visit PI breeding operations to assess adherence to the institution's expectations for breeding management and thus serve as a bridge between the investigator's staff and the core facility animal care staff, ensuring communication between the two groups.

Animals on loan or maintained at distant field sites present special challenges for IACUC oversight. Travel to distant sites to assess these types of studies usually requires a significant time investment (e.g., for travel and overnight stays). IACUC members' and veterinary staff's time is usually better used at the core operations where the bulk of activity occurs, but if the institution has research performed at a distant site it has an obligation to oversee that research. If the IACUC does not perform semiannual inspections of distant or field activities, the PAM staff can serve as a consultant to the committee and provide regular and impartial assessments of activities performed under IACUC approval at less convenient locations.

Although the IACUC has a regulatory obligation to oversee all animal care and use, most committee members may not have authorization to enter restricted areas such as those where Animal Biosafety Level (ABSL) operations take place. However, failure to oversee animal activities in a biohazardous environment may interfere with the institution's ability to provide effective management and the required oversight of the campus program of animal care and use. The PAM staff can assist the committee in such oversight, although special training and preapproval through Select Agent registration may be necessary for entry into and observation of biohazardous activities.

Organization and Management

PAM processes are applicable to all campus administrative organization styles, whether the institution under consideration is a medical school, undergraduate, graduate, land grant, veterinary, or liberal arts university. In some of these cases, and especially in smaller institutions, the PAM program may be administratively housed in one branch (e.g., the veterinary unit) with performance responsibilities in another (e.g., the IACUC). For example, a university with both undergraduate and graduate schools may have an autonomous fiduciary structure for each, with an independent personnel reporting scheme up the organization to the chancellor or president; the PAM program may be administratively housed in either school but with programmatic responsibility for the entire campus, as is the case with the IACUC.

Institutions may have a number of administrative units responsible for animal care and/or use, so PAM audits should include all operational (or management) designs. Although the simplest approach is to have a central vivarium with all animal care in a specific building or buildings, the more common arrangement in academic institutions includes decentralized operations in many locations and buildings. PAM can provide consistent and fair assessments of practices across the campus while ensuring that the various administrative units meet the institutional expectations of the IACUC.

PAM Methods

A single method for PAM auditing is insufficient given the breadth of animal research on most academic campuses. Effective PAM audits should therefore be guided both by the specific content of the IACUC-approved protocol and a "performance approach" focusing on the outcomes (e.g., animal well-being, pain/distress prevention, effective and humane research) of the processes used. It is also important to remember that different people may do the same procedure slightly differently, but all procedures must be performed as described in the protocol; the liaison should confirm the use of protocol-approved methods without discounting alternative ways of accomplishing a task. And throughout the audit, the PAM staff should bear in mind that the goal is to find evidence of good performance and to confirm compliance.

Certain auditing activities are routine (also see Dale 2008) and others depend on the particular type of animal use (e.g., routine or noninvasive procedures, surgery, clinical care). First we consider the common activities and then those that depend on animal use. A PAM audit may include any or all of the following:

Advance review of an approved protocol to understand the scope of the laboratory's efforts and IACUC-approved activity. To maximize the efficiency of the review process and minimize the impact on the laboratory, it is best for the PAM staff to be familiar with the specific procedures approved by the IACUC.
Comparison of animal numbers (whether acquired by purchase or breeding) with the numbers approved in the protocol. The animal usage rate may correlate with certain types of concerns (e.g., a busy lab that uses a lot of animals may be less likely to have outdated medications because the latter are used more frequently).
Entrance interview with researchers (or senior lab managers) to explain the PAM process and potential outcomes. To encourage teamwork and transparency, the PAM staff should begin the audit with an overview of its purpose and a step-by-step review of planned activities.
Inquiries about any unexpected deaths or other adverse events. How was the adverse event managed? Was veterinary counsel sought? Was the event reported to the IACUC?
Review of procedure records for consistency with the approved protocol. If the protocol calls for weekly weights, is there evidence that animals are being weighed weekly? In some cases, performing "less than" the IACUC approved might be acceptable (e.g., the IACUC approved anesthesia doses up to 50 mg/kg, but the laboratory achieved effective anesthesia at 35 mg/kg). In other cases performing "less than" might be unacceptable (e.g., the IACUC approved 20 mg/kg of an analgesic, but the laboratory, concerned that the dosage interfered with the research, provided only 5 mg/kg).
Review of animal use/study areas to assess proper animal care.
Review of laboratory training records to determine what training may be needed for the procedures involved (e.g., the IACUC required the investigator to ensure that staff performing surgery had completed the "Aseptic Technique" training module, or the staff failed to enroll in annual update training).
Confirmation of appropriate medical surveillance participation (PAM staff should not have access to personal health care information, but usually can assess enrollment in the institution's health and safety program).

Although these steps are part of a relatively formal process, they should be performed in a nonthreatening and collegial manner. Performed well, they establish a framework for subsequent detailed assessment. Performed poorly, they complicate the performance of the audit and may interfere with a smooth and effective PAM program.

The steps above are applicable to each of the following PAM audit styles. In addition, for all types of PAM audits the process should be as transparent as possible and should not interfere with the research staff; for example, PAM staff should not engage in extraneous conversation or activity during the audit. Asking too many questions during a procedure or requesting a demonstration of an activity creates artificial variables that may alter the normal practice and actually cause a noncompliant activity, or at least present an unreliable picture of the procedural steps.

Finally, at the conclusion of any type of PAM review, it is critical to share the findings with the senior laboratory manager or investigator. Although discussions may occur of how other laboratories have managed performance failure in such cases, the laboratory staff should understand that any corrective action they may take before the IACUC review of the report will be based solely on the decision of the PI. It is important that the liaison not have implied or perceived powers of obligation—PAM staff cannot actually require or independently authorize any corrective action. To do so would violate their responsibility to serve as the eyes and ears of the IACUC by objectively gathering information.

Procedure Observation (POPAM)

This method of PAM, which should follow the steps described above, entails the observation of all activities that involve animals—for example, a nonsurgical experiment, noninvasive imaging, behavior assessments, a care practice such as cage changing, or a clinical procedure such as TB testing of primates.

A POPAM audit may include any or all of the following:

Observation of the presence and condition of necessary equipment and supplies required to ensure care of the research animals. Is proper caging in use? Are there sufficient restraint devices, medications, or sharps containers?
Husbandry and care practices for animals before, during, and after the procedure. Do the staff exercise compassionate care (e.g., with the provision of sufficient feed, water, heat, cooling, bedding) of the animals in their charge?
Animal manipulations. Are the procedures performed as described in the IACUC-approved protocol? Are procedures performed for which there is no (or insufficient) description in the protocol?
Transport of animals between housing and experimental areas. Are animals protected from the environment (heat, cold, sun)? Are they secured to prevent escape? Are they hidden from the casual observation of passers-by? Are people protected from allergen exposure?
Personnel qualifications in the procedures performed. Do the staff performing the procedure(s) demonstrate expertise? Do they recognize animal distress behaviors? Do they handle hazardous agents or sharps appropriately?
Review confirmation. Confirm with the research staff that the observations are accurate.

Surgical Observation (SOPAM)

The surgical observation style of PAM should follow the steps noted above in observing surgical facilities and surgical activities involving animals. Such activities include surgical experiments, surgical interventions to remove an implanted device, surgical repair of a failed device, or surgical approaches to collect tissues.

A SOPAM audit may include any or all of the following:

Physical plant appropriateness. Does the surgical area have the necessary operative surfaces for satisfactory performance of the required activity? Does the surgeon's preparation area have the materials necessary both for proper surgeon preparation and for the species involved? Are satisfactory plans in place to provide for postprocedure recovery (e.g., heat, cooling, fluids, bedding, medications)?
Housekeeping/sanitation procedures. Has the operative theater been appropriately prepared for the planned activity? Are the animal contact surfaces properly disinfected?
Drug inventory and use. Are the necessary medications on hand? Are they current or expired? Are there sufficient quantities for the procedure? Are the sterilization method and packs appropriate for the procedure? Is the instrumentation adequate and sterilized?
Personnel skills and training. Do the staff exhibit adequate knowledge of the equipment and procedures in the surgical area? Are animals handled expertly? Is animal sedation/anesthesia performed without causing undue distress to the animal?
Emergency protocols for adverse procedural events. Is there a plan for a surgical adverse event? Are "emergency" medications on hand? Are the staff aware of the procedure for quickly contacting a veterinarian (if a veterinarian is not part of the activity)?
Anesthetic equipment and scavenging. Has the inhalant anesthesia machine been certified? Did the staff test it before attaching the animal? Do the staff clearly understand how to operate the anesthesia machine? Is there waste gas–scavenging equipment? Is it functional? Do the staff use it effectively?
Anesthetic and analgesic regimen. Is there a process for assessing depth of anesthesia and animal well-being? What are the steps to modify or adjust anesthetic agent in response to changes in the animal? Was analgesic use as approved in the protocol? Were changes made in response to animal behavior?
Postprocedural monitoring, nursing, and supportive care. Did the animal receive supplemental environmental support (heat/cooling), fluids (if a long procedure), and other postoperative support measures?
Overall conduct of major surgeries. Did the surgical support team demonstrate expertise? Was the animal seamlessly managed from initial cage capture to recovery and postprocedural release?
Review confirmation. Confirm with the research staff that the observations are accurate.

Animal Care (Vivarium) / Clinical Care PAM (ACPAM)

Of the three PAM styles described here, the ACPAM method permits the greatest freedom of scheduling. Whereas research activity is usually a discrete and finite activity, animal care occurs daily in some form, either routine or specific to clinical practice (e.g., TB testing of primates). ACPAM can be performed when time allows and when other research activities are not under way.

An ACPAM audit may include any or all of the following:

Observation of the room checklist, safety sheets, emergency call rosters, or other documentation expected to be at the room level. Are emergency numbers readily accessible and up to date? Is there a process for reporting animal care and use concerns and is it clearly posted in the vivarium areas? Are the appropriate SOPs available?
Review of the animal disease report for indications of animal distress. Are animal disease and health conditions regularly monitored? Are conditions reported to the veterinary staff? Are adverse conditions treated? Have there been unexpected deaths?
Review of animal care areas for proper equipment and resources. Is the equipment necessary for care and clinical use present and serviceable? Is it sanitary? Is it on a regular maintenance schedule?
Review enrichment records. Do all animals have access to enrichment? If not, is there an exception on file?

Unannounced Audits

Certain cases, usually the result of either an allegation against a researcher or an IACUC concern, trigger an unannounced assessment. The unannounced visit should not generally involve the breadth of a routine PAM review; the goal is to find specific information either confirming adherence to IACUC-mandated activity (following an earlier instance of noncompliance) or confirming or disqualifying a recent allegation against the investigator or the laboratory.

Although the methods for this type of PAM audit may differ from those for routine visits, for campus process integrity the audit should draw on the same set of assessment criteria and its focus should remain the same: to observe "evidence of good performance" and gather information for the IACUC's informed decision making. This style of PAM should also be followed by postvisit communication and a report to the IACUC.

Because the unannounced audit may require multiple attempts to visit the same location, it may be better suited to the PAM staff than the AV or IACUC members. If there is a strong sense of fairness among research and laboratory staff based on prior engagements with the PAM liaison, there are rarely significant concerns among the laboratory staff that the unannounced visit is punitive in nature. However, if PAM liaison engagement has been problematic and less collegial, mistrust and combativeness may complicate information gathering during an unannounced visit. In these cases, or if there is concern about such an outcome, a senior representative of the animal program should either accompany the PAM liaison or perform the PAM activity. At Duke, the unannounced PAM or potentially combative PAM reviews are performed by the director of the Office of Animal Welfare Assurance.

Resources Required for a PAM Program

Personnel

PAM staff qualifications should be a reflection of an institution's research program. For example, perioperative experience is encouraged for individuals performing SOPAM, whereas husbandry or clinical care experience is appropriate for ACPAM staff. Institutions that have significant involvement with regulatory agencies may be best served by individuals with regulatory training. Each institution should consider its program strengths and potential weaknesses and select its PAM staff both to match the research emphases of the institution and to compensate where the program is most vulnerable.

The most important criterion for all PAM staff is good observation skills. Can they observe rather than simply see? Are they able to capture their observations and report them clearly and accurately? PAM staff should observe the anesthetic agent used, the dose given, any supplemental anesthetic, intraoperative support steps, and the animal's behavior during recovery. They should know whether the anesthetist used the IACUC-approved anesthetic agent and at the proper dose, whether the patient care parameters were followed as described in the institution's SOP for anesthetized patient care and were recorded, and whether patient support was adjusted if necessary during recovery. The best PAM staff see what others do not—the nuances of animal care and use.

The number of PAM staff should be determined by the program size, complexity, and risks. For most institutions starting a PAM program, beginning small is usually preferable. The complexity of the protocols, number of investigators, and number of locations where care or use is performed all affect the number of individuals necessary to adequately advise and inform the IACUC. As an example, the Duke University program has two PAM program staff—a veterinarian and a registered laboratory animal technologist (RLATG)—who perform over 1200 visits each year. The Duke program's strategic plan for PAM calls for the addition of a liaison who will focus on training and assisting researchers with IACUC-mandated corrective actions.

An alternative method of determining staffing needs is by percentage of initial noncompliance. If 0-30% of the audits are noncompliant, then one individual may be sufficient. If 31-50% of the audits are noncompliant, then two individuals may be required. Although it may seem that a high percentage of noncompliant audits would require more staff, the acceptable compliance limit should be set by the IACUC, since one aim of the PAM program is to ensure the committee's confidence in the performance of IACUC-approved animal procedures. If the PAM staff are also responsible for providing corrective training, then this should also be considered in determining the number of staff necessary.

As noted above, the actual administrative support for the PAM staff can come from any source or segment of the institution's structure. At Duke, for example, the Director of the Office of Animal Welfare Assurance provides daily management for the liaison staff and helps to ensure a complete and accurate audit process. But whatever the source of the administrative support, it must not become a gatekeeper as all audit observations should be reported to the IACUC upon completion of the audit.

Finances

Most institutions are reluctant to spend money on personnel for a problem that has not yet occurred. For many institutions, the motivation for financing a PAM program is too often the result of a significant program failure (e.g., regulatory inspection/review for cause). Each institution is different, but most institutions beginning a PAM process discover problems they did not know about. For the Duke program, a high percentage of PAM audits during the initial months of implementation revealed concerns that required further attention; it was most fortunate that the noncompliant items had not caused any significant negative event.

Computer Resources

The software or information database requirements for an institutional PAM vary among institutions. There are no off-the-shelf products designed for PAM data management, although many companies are developing software to facilitate this program information management. A spreadsheet or computer database can be used to track and trend PAM data. At a minimum, the institution will need data management to:

identify which protocols, SOPs, or activities require assessment
track scheduling of the audit events
capture (and manage) the potential noncompliances noted
track noncompliances according to protocol, investigator, or other core value
report findings to the IACUC
record IACUC corrective actions (as appropriate)
ensure clear and timely communication with the regulatory reporting agency
track trends in noncompliance to offer advice on improving the institution's training or communication program.

Cost Analysis

Because PAM is not a federally funded mandate, its resource requirements cannot be the basis for additional federal funds acquisition. However, PAM staff and support needs may be one of the components of the institution's overall indirect cost recovery (ICR) for administration. PAM costs can be absorbed into institutional support costs or funded out of user fees or other funding streams.

The indirect costs of not having a PAM program are difficult to calculate, but they can be significant. PAM can help prevent such negative indirect institutional costs as

animals in unnecessary pain or suffering,
investigators performing procedures not approved by the IACUC, and
care practices not up to the standards of the institution.

Any of the above (or other types of poor performance) could result in lost public confidence or a regulatory institutional visit for cause (e.g., Office of Laboratory Animal Welfare site visit; also see Bayne and Garnett 2008 about the risks of inadequate oversight).

It takes only a few examples of poor performance to jade public perception of the institution. A strong training program, attentive care staff, and compassionate research staff are crucial to avoiding noncompliant behavior. And engaged and proactive PAM staff as an extension of the IACUC are the best program insurance for academic institutions.

Tips for a Successful PAM Program: Lessons Learned

Following are a few tips, based on the lessons we and others have learned, for building a successful PAM program.

Be a Liaison, Not the Animal Police

A critical aspect of PAM at an academic institution is the development of the concept of "liaison" (the term "animal police" to describe PAM staff is destructive to a progressive animal research program). The responsibility of the PAM liaison is to serve as the eyes and ears of the IACUC, not to act as its decision-making authority.

The goal of PAM is to foster partnership—among the funding agencies, the regulatory agencies, the institution, the IACUC, the researchers, and the public. A research laboratory that views PAM as a supportive institutional activity will communicate when things go awry and will work to effect internal laboratory improvement. A research team that sees the PAM liaison defending the investigators' privilege to do progressive research will work to incorporate effective training and corrective actions in the laboratory. The liaison can help foster a positive, collaborative relationship by being friendly, upbeat, nonjudgmental, and supportive during the audit.

Partner with the Laboratory for Culture Change When Necessary

Some research laboratories have a culture of compliance, whereas others have drifted. Most noncompliance problems are the result of bias, lack of attentiveness to the approved protocol, or resistance to change (e.g., "we have always done it this way") rather than overt disregard for regulation or policy. When problems are single incidents of poor performance, effecting change is usually quick and decisive (usually in laboratories with a strong culture of compliance). When problems are pervasive or deep-seated, laboratory culture may interfere with effective change—and if the research laboratory culture does not change, the same problem is more likely to recur.

For the PAM staff, time spent in the laboratory is a prime opportunity both to help the laboratory staff understand program policies and procedures and to encourage a positive partnership with the IACUC. For laboratories where a positive culture shift is desirable, it is important to remember that changing the laboratory research culture will not occur quickly; it will require consistent encouragement, accurate communication, transparent program policies, clear guidelines, and teamwork. PAM staff can serve as the representatives of a postapproval monitoring program that cares about the laboratory, its staff, and its research.

Keep It Simple

A critical component of successful PAM is simplicity. Find a process that meets the needs of the institution, can be funded by the institution, and benefits the institution, then stick with it. Any number of plans or processes may be used or proposed, but the PAM process really boils down to three steps:

review the protocol (or SOP) to know what was approved,
develop a checklist of key components to use each time, and
use good observation skills (these are critical).

Select Audit Activities Based on Risk Factors and Without Bias

Some institutions have chosen to initiate their PAM program as a randomly selected percentage of active protocols. Such an approach is most useful when the percentage of reviewed protocols per year equals 100% of the approved protocol list over a short number of years—for example, when 50% of the protocol base is audited each year (thus the whole program is audited every 2 years). Auditing a small percentage, say 10%, each year leaves a larger percentage of the approved animal activity without a PAM audit over the 3-year life of a protocol, whereas any audit selection process should include all animal activities within a 3-year period. To develop an initial "ballpark estimate" of the number of audits required, institutions can consider the active protocol pool and choose a reaudit frequency based on perceived risk factors—for example, primate surgery versus mouse breeding, or higher compliance risk (HCR) versus lower compliance risk (LCR).

At Duke, our audit plan is based on protocol types and risk assessments and includes audit processes for common procedures as well as SOPs or procedure plans that may not be part of a specific protocol but are referenced in or performed separate from a protocol. Our institutional goal is an audit of each approved procedure every year, an approach that captures each protocol on our registry as well as SOPs for animal care or use. The same procedure may be used on any number of protocols. A single procedure audit may capture multiple protocols at the same time, but the PAM audits are recorded and tracked against specific protocols. This method requires a detailed tracking system to ensure the review of all procedures on all protocols.

In some cases, the inherent risk of an adverse event and/or significant distress or pain to the animal (e.g., primate survival surgery has a high compliance risk, as opposed to mouse breeding, which has an LCR) may require more frequent audits. A laboratory's history (e.g., repeated noncompliances (HCR) versus a stellar audit history (LCR)) may also suggest the number of audits necessary to ensure compliant care or use, and perhaps additional actions by the IACUC.

Communicate, Communicate, Communicate

Effective communication is vital to the success of a PAM program. The PAM staff should work diligently to ensure that those visited understand the process, receive updates on the progress of the review, and recognize the value of corrective actions. At a minimum, the PAM staff should communicate before the visit (unless it is an unannounced audit for cause or allegation) to allay fears of an "inspection that will shut us down." At the conclusion of the visit, the liaison should share observations with the laboratory staff so errors can be corrected before the report is written. Before the IACUC meeting, the liaison should make sure the PI has had an opportunity to respond to the concerns that will be presented to the IACUC. After the IACUC's decision, the liaison should share the committee's decision and offer to work with the laboratory staff toward resolution of the IACUC-required corrective action(s).

A successful communication method used at Duke University by the Office of Animal Welfare Assurance (OAWA) involves the following steps.

Initial Communication

After selecting a protocol, procedure, or activity, the OAWA contacts the principal investigator or responsible party. A sample introductory email or letter is shown in Box 1, but the early communication itself is more critical than the particular language used.

Postvisit Communication: Full Compliance

After observing the activity and considering all applicable documents, the PAM staff prepares a report to the laboratory or unit audited. The purpose of this postvisit communication is to build rapport, ensure accurate observations for the IACUC report, and keep the process transparent. At Duke, this communication is referred to as the "attaboy/attagirl" letter (Box 2) and it means "you are doing a good job, thank you!" This is one way to provide positive reinforcement to the research staff. Many laboratories at Duke choose to post their "attaboy/attagirl" letters on the laboratory bulletin boards to show pride in their success. We have found that recognizing successful performance is even more critical than identifying performance failures and encourages collegiality and partnership with the program.

Postvisit Communication: Possible Noncompliance

After observing the activity and considering all applicable documents, the PAM staff prepares a report to the laboratory or unit audited. Again, the purpose of this communication is to build rapport, ensure accurate observations for the IACUC report, and keep the process transparent. This communication (Box 3) identifies each of the possible deviations or problematic areas identified and suggests a corrective activity (e.g., return to the approved procedure, submit an amendment for IACUC consideration). An important aspect of a collegial PAM process is the inclusion of reasonable and practical suggestions for correcting the noncompliance. Most researchers will implement these before the IACUC meeting as evidence of their desire to partner with the program.

Oral Report to the IACUC

The IACUC must have accurate and complete information to make an informed decision. Any form of presentation is possible, but a verbal report to the IACUC by the PAM staff is usually best as it provides an opportunity to answer committee members' questions and clearly describe observations. Box 4 provides a sample template for presentation to the IACUC.

Post-IACUC Meeting Decision Memo

Once the IACUC has considered the PAM staff's observations and other documents (e.g., the investigator's letter to the committee, institutional policy), the committee issues a decision. The decision must be shared with the principal investigator or responsible individual, and the manner in which it is shared is almost as critical as the PAM visit. The information must be accurate and must provide clear instructions on how to accomplish the necessary corrective actions. In Box 5 is a sample letter based on a series of noncompliant events observed by the PAM staff during an audit. (Note: All issues described in Box 5 are fictitious, created for the purpose of demonstrating clear communication.)

Trends Report

As a byproduct of improving compliance in animal use procedures, the PAM process creates information useful to the IACUC. Table 1 shows a sample trends report for tracking noncompliant activity over a period of time. The IACUC may use PAM trends (e.g., as part of the semiannual program review process) to assess the effectiveness of the institution's training program or determine whether tailored investigator training would benefit the program. The information in the trends report can also be used to improve the protocol review process by focusing on specific areas of poor performance across the campus.

Getting Started with a PAM Program

Begin with a clear review of your program and its needs, recognizing that it may take 2 or 3 years to become fully functional in PAM auditing.
Identify your institution's high-risk activities and have the IACUC prioritize its review of them.
Establish milestones to gauge the effectiveness of the process. What percentage of compliant audits will be considered acceptable? Unacceptable? How many audits will be performed each month? How will protocols, SOPs, or activities be selected for audit? As with any new task, do not set lofty goals—be realistic!
Build confidence in your auditing, training, and trending process. Select simple activities to review first. Set aside a self-assessment period for each audit. How well did you do? What went well? What did not go well?
Communicate the plans with the research community, by newsletter, seminar, department meetings, or other venues. Assure researchers that the PAM liaisons are there not to slow or impede research but rather to support and establish a strong foundation for continued research.
Encourage PI self-reporting. Programs with a substantial level of self-reporting (at the laboratory level) and self-correction are the best evidence of an engaged culture and a highly successful PAM program.
Obtain "buy-in"! Administration "buy-in" is critical. At some point in the process, a senior or long-tenured researcher will object. The administration must be confident that PAM is in the best interests of the institution, the research community, and the individual researcher.
Know how, when, and who receives regulatory reports. NIH's Guidance on Prompt Reporting to the Office of Laboratory Animal Welfare (OLAW) under the PHS Policy on Humane Care and Use of Laboratory Animals (NIH 2005) provides guidance in determining what, when, and how situations should be reported to OLAW. The PHS Policy (IV.F.3, NIH 2002) requires that "The IACUC, through the Institutional Official, shall promptly provide OLAW with a full explanation of the circumstances and actions taken with respect to:

any serious or continuing noncompliance with this Policy;
any serious deviation from the provisions of the Guide [for the Care and Use of Laboratory Animals]; or
any suspension of an activity by the IACUC."

A comprehensive list of reportable situations is impractical; we refer readers to the NIH Notice (NIH 2005) for examples of reportable incidents. Some observations may be considered by the IACUC but determined not reportable to OLAW. Even in such cases, useful trending information may enable enhanced management of the institution's resources and personnel who provide animal care or use.

Other Efficiencies That Can Be Realized with PAM

Although postapproval monitoring and IACUC semiannual program and facility inspections are quite different processes, they can share information and be mutually beneficial. PAM can serve the IACUC in more than just auditing, by, for example,

—using noncompliance observations for focused education efforts,
—trending observations to drive the semiannual inspection focus, and
—building the training program to address specific negative trends.

The PAM staff may coordinate the IACUC semiannual facility inspections. In this role, the staff can monitor IACUC semiannual facility inspection reports over time and identify problem facilities or problem laboratories that need a more intensive level of PAM review.
The PAM staff may manage the database for all IACUC inspection reports and PAM observations. Using a single database between IACUC and PAM allows for the incorporation of all findings, facilitates program management, and improves problem area attention. Trend reports are easier to develop and can serve as indicators for focused training.
The PAM staff can share information from their audits to advise the IACUC semiannual inspection team of specific areas of concern in the facility under inspection for that day.
The PAM staff can provide follow-up visits to problematic areas identified during semiannual inspection to ensure that the IACUC's recommendations (and PI replies) are fully implemented and appropriate.

Conclusion

Noncompliance incidents may occur due to the many professional and personal demands on individuals associated with an ACUP: an animal care provider may be distracted by the illness of a family member, the researcher who recently arrived on campus may be unfamiliar with local animal use requirements, or the building heating failure over the long holiday weekend may have caused distress to the animals. None of these adverse conditions was intended or predictable, but each had consequences that, depending on the severity, may affect animal well-being, modify research outcomes, and require reporting to regulatory and funding agencies.

To address such lapses before they cause serious noncompliance, every institution can implement a PAM program. PAM provides insurance for the animal program by serving as a bridge to the IACUC's oversight of daily research performance and animal care. A successful PAM program features the following ten characteristics:

The institutional leadership, regulatory agencies, and funding agencies have confidence in the institution's self-management processes.
The IACUC provides clear guidance and directions for PAM.
The PAM program has the support of the institutional official and IACUC.
The PAM program focus is on evidence of good performance, with praise for such performance when found.
The PAM program identifies problems while they are small and facilitates correction.
The PAM staff are viewed as partners (not "animal police").
The PAM staff are known as good observers, fair, and consistent.
The PAM staff foster communication between the IACUC and the research community.
The research and husbandry staff self-report animal incidents.
Corrective actions are viewed as positive educational opportunities.

In considering the development of your institution's PAM, start with the program at present and select a strategic goal. Where does the program wish to go? Focus process development on improved compliance audit outcomes.

It is an institutional obligation to ensure animal welfare and adherence to the activities approved for all animals. Institutions can use various methods to develop an assessment process that assists the IACUC with oversight, ensures a progressive institutional culture, and facilitates investigators' compliance in their research. The key to successful accomplishment of the institutional obligation of animal oversight is a collegial partnership between the animal program management and user community, and an effective PAM program can facilitate such a partnership.

Abbreviations used in this article: ACUP, animal care and use program; AV, attending veterinarian; IACUC, institutional animal care and use committee; PAM, postapproval monitoring; PI, principal investigator; SOPs, standard operating procedures

References

Bayne KA, Garnett NL. 2008. Mitigating risk, facilitating research. ILAR J 49:369-371.

Dale WE. 2008. Postapproval monitoring and the role of the compliance office. ILAR J 49:393-401.

DeHaven WR. 2002. Best practices for animal care and use committees and animal use oversight. ILAR J 43:S59-S62.

Haywood JR, Greene M. 2008. Avoiding an overzealous approach: A perspective on regulatory burden. ILAR J 49:426-434.

Klein HJ, Bayne KA. 2007. Establishing a culture of care, conscience, and responsibility: Addressing the improvement of scientific discovery and animal welfare through science-based performance standards. ILAR J 48:3-11.

NIH [National Institutes of Health]. 2002. Public Health Service Policy on Humane Care and Use of Laboratory Animals. Bethesda: NIH Office of Laboratory Animal Welfare (OLAW). Available online (http://grants.nih.gov/grants/olaw/references/PHSPolicyLabAnimals.pdf), accessed June 4, 2008.

NIH [National Institutes of Health]. 2005. Guidance on Prompt Reporting to OLAW under the PHS Policy on Humane Care and Use of Laboratory Animals. Bethesda: NIH Office of Laboratory Animal Welfare (OLAW). Available online (http://grants.nih.gov/grants/guide/notice-files/not-od-05-034.html), accessed June 4, 2008.

Wallace JM, Sikoski P. 2007. Animal care and use issues in movement disorder research. ILAR J 48:317-322.