Introduction Of The Drug Tacrolimus

Published Date: 02 Nov 2017

AMITY-Logo1

AMITY UNIVERSITY

UTTAR PRADESH

Project on ‘Analysis of Adverse event data of Tacrolimus for signal detection and evaluation’ for the partial fulfillment of M.Tech Biotechnology.

Batch: 2011-2013

Year of submission-2013

Submitted by- Archana Thakran

Roll No: - MTB/11/2054

Acknowledgement

Apart from the efforts of me, the success of this project depends largely on the encouragement and guidelines of many others. I take this opportunity to express my gratitude to the people who have been instrumental in successful completion of this project.

Firstly, I want to pay sincere thanks to my Mother and Father for being the best parents of the world and inspiring me throughout the course of my life and this project.

I owe a deep sense of gratitude to Dr. Arani Chatterjee, Sr.Vice President, for his unflinching interest, valuable advice, close supervision, constructive criticism, healthy encouragement & generosity throughout the course of this study.

I would like to thank my mentor, Dr.Mudgal Kothekar ,Deputy General Manager for his patience, interesting and valuable advices in discussions over a long period and supervision using his precious time to read this report and giving his comments and suggestions about it.

Many thanks to Dr.Soumajit Choudhury, Mrs. Hitasha Malhotra,and Ms.Priya Gupta who willingly helped me out there abilities to understand the minor details of the work on day to day basis.

I would like to thank my internal guide Dr. Pawan Kumar Maurya, Assistant Professor, Amity Institute of Biotechnology for his continuous support and guidance which helped me in completion of this project report.

Contents

INTRODUCTION

The fact that drug can lead to desirable outcome is undoubtedly accepted by all but favorable outcome is hardly seen as it is fraught with increased number of problems like irrationality, resistance, medication errors and lack of root-cause analysis. The delayed reflexes were picked up way back in 1960s with Thalidomide tragedy and then origin of international drug monitoring activities in 1968 making it mandate for manufacturers, stakeholders, regulators, drug authorities and healthcare professionals to vigilantly monitor drug use. Regulations for drugs are proposed by authorities but execution delay interrupted the whole network. Pharmacovigilance network is well sustained in developed countries but still on its way to progress in developing countries. With increased number of new chemical entities (NCE), Pharmacovigilance has become mandatory requirement for pharmaceutical companies. With this view, phase IV studies are critically analyzed and executed with the aim to monitor and capture long term safety outcomes and report ongoing safety review to regulatory authorities in terms of periodic safety update reports (PSUR). It also insists manufacturers to update safety information in product leaflet or summary of product characteristics (SmPC) within stipulated time period.

Pharmacovigilance is an important and integral part of clinical research. So, Pharmacovigilance is "defined as the pharmacological science relating to the detection, assessment, understanding and prevention of adverse effects, particularly long term and short term adverse effects of medicines." An adverse event is defined as any un toward medical occurrence that may present during treatment with a drug but which does not necessarily have a relationship with its use." Spontaneous reporting of adverse drug reaction and adverse events is an important tool for assessing the safety information for early detection.

There is a need to monitor the effects of drugs before and after it’s successfully tested and launched in the market. Pharmacovigilance involves monitoring and assessing the quality of drugs, detection and prevention of any adverse effects of drugs. It also involves evaluating information provided by health care providers, pharmaceutical companies and patients in order to understand the risks and benefits involved with a particular drug. The pharmaceutical companies spend millions of dollars and a considerably long time in developing new drugs.

Signal Detection

Signals may be "qualitative" (based on spontaneously reported data) or quantitative" (based on data mining, epidemiologic data, or trial data). The signal may be a new issue never before seen with this product, or it may be the worsening or changing of a known AE or problem (e.g., a previously unaffected patient group is experiencing this problem, or the incidence has increased, or it is now fatal in those it attacks, whereas before it was not). As noted above, qualitative signals may be based on one single striking case or on a collection of cases. In addition, qualitative signals may also be based on preclinical findings; experience with other similar products in the class ("class signals"); new drug or food interactions; confusion with a product’s name, packaging, or use; counterfeiting issues; quality problems; and more. Thus, the word "signal" is being expanded.

Spontaneous reporting systems for suspected adverse drug reactions (ADRs remain a cornerstone of pharmacovigilance. A primary aim in pharmacovigilance is the timely detection of either new ADRs or a change of the frequency of ADRs that are already known to be associated with the drugs involved, i.e. signal detection. Adequate signal detection solely based on the human intellect (case by case analysis or qualitative signal detection) is becoming time consuming given the increasingly large number of data, as well as less effective, especially in more complex associations such as drug-drug interactions, syndromes and when various covariates are involved.

The detection of early warnings related to potential public health or patient safety issues is the main aim of collecting and analyzing individual case safety reports. In the context of ADR surveillance, the WHO defines a signal as: "Reported information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously. Usually more than a single report is required to generate a signal, depending upon the seriousness of the event and quality of the information."

Need for Signaling

Individual case safety reports provide genuine clinical concerns from observant health professionals .As they are based on actual patients in real world clinical practice, their collection and analysis increase the chance to discover ADRs that are due to drug–drug interaction, affect patients with certain medical predispositions or that belong to patient subgroups that tend to be excluded from pre-marketing clinical trials, such as children or pregnant women. In addition, the large numbers of patients exposed and the unlimited follow-up time available considerably increase the chance to detect ADRs that are rare or that occur only after extended periods of use.

The aim of ADR signal detection is to generate, strengthen and refine hypotheses related to suspected drug toxicity. Hypothesis testing is not possible on account of the inherently nonsystematic nature of data collection and the lack of proper comparison groups. In-depth clinical evaluation and scrutiny of reports remain at the core of the ADR signal detection process.

Source of data and information

The sources for identifying new signals are diverse. They potentially include all scientific information concerning the use of medicinal products including quality, non-clinical, clinical, pharmacovigilance and pharmacoepidemiological data. Specific sources for signals include spontaneous adverse drug reaction (ADR) reporting systems, active surveillance systems, non-interventional studies, clinical trials, scientific literature and other sources of information.

Signals from spontaneous reports may be detected from monitoring of individual case safety reports (ICSRs), ADR databases, articles from the scientific literature or review of information provided by marketing authorization holders in the context of regulatory procedures (e.g. variations, renewals, post-authorization commitments, periodic safety update reports (PSURs), Risk Management Plan (RMP) updates or from other activities related to the on-going benefit-risk monitoring of medicinal products.

Spontaneous reports of ADRs may also be notified to poison centers, teratology information services, vaccine surveillance programs, reporting systems established by marketing authorization holders, and any other structured and organized data collection schemes allowing patients and healthcare professionals to report suspected adverse reactions related to medicinal products.

Methods used for Signal Detection

Databases held by regulatory agencies and by pharmaceutical companies may be used to detect safety signals. A number of different quantitative signal detection methods are currently used by regulatory authorities and pharmaceutical companies. The best known methods are the proportional reporting ratio (PRR), the reporting odds ratio (ROR), empirical Bayes Multi-item Gamma Poisson Shrinker (MGPS) and the Bayesian Confidence Propagation Neural Network (BCPNN).

In this project I have compared the utility of a company safety database with those from one of the large regulatory authority databases in assessing potential safety signals in order to understand how the different sources may be used. Proportional reporting ratios (PRRs) were calculated from the company database for historical events that had led to changes in prescribing information (PI) and were compared to PRRs calculated in external databases.

Proportional reporting ratios (PRRs) are being used increasingly to compare the rate of adverse drug reactions for specific drugs with similar reactions for all drugs in a safety database. The roots of PRRs go back more than 35 years, and the original work by Patwary using 2X2 tables to examine the rate of new reports received for a particular drug was acknowledged by Evans [1] and more recently by Moore [2]. Evans and colleagues [3] have moved the concept forward with a description of how PRRs can be used by a regulatory agency in signal detection. Debate is ongoing about the utility of PRRs compared to other tools such as reporting odds ratios [4] and Bayesian methods, such as Bayesian confidence neural network analysis, which is used by the Uppsala Monitoring Centre [5,6]. Roux et al. [7] compared 10 methods of data mining and concluded that several different methods provide good results. It is becoming accepted that PRRs are, at the very least, a useful tool in detecting signals [8]. The data necessary to calculate PRRs can come from a variety of sources, but large computerized safety databases are the most convenient, and automated signal detection tools are likely to be used more frequently in the future. Pharmaceutical companies hold databases containing spontaneous reports that are sent to them directly or from regulatory agencies and also from clinical trial safety reports. Despite substantial underreporting of spontaneous reports, PRRs have proved to be a useful signal detection tool and do not rely on prescriptions as a denominator, although it must be recognized that they are one of many techniques for signal detection, which must be used alongside other methods available in pharmacovigilance[9].

Introduction of the drug-Tacrolimus

Tacrolimus is a macrolide of fungal origin (produced by Streptomyces Tsukubaenis) with strong immunosuppressive actions. Its primary target appears to be the helper T lymphocytes, with little effect on other aspects of the immune response. However, because it acts early in the process of T cell activation, it has secondary effects on other cell types that are normally activated by factors produced by the T cells.

Overview of T-cell activation

T‐cell activation is a highly co‐ordinated process which requires binding of the T‐cell receptor (TcR)‐CD3 complex to antigen:MHC class II molecules expressed on the surface of antigen‐presenting cells [17], and the provision of cell‐bound and secreted co‐stimulatory molecules which, while not imparting antigenic specificity, significantly augment T‐cell activation. As a result, several signal transduction pathways become operational, leading to induction of cytokine gene expression and stimulation of cellular activation (Figure 1). Whereas initial events of T‐cell activation are calcium‐dependent, later events, in particular those associated with interaction of cytokines with their high‐affinity receptors, are less dependent on calcium [18]. Interruption of any of the events of T‐cell activation by the immunosuppressive drugs CsA, FK506, glucocorticoids, and rapamycin (Sirolimus) results in downstream inhibition of cytokine expression and T‐cell proliferation

http://ndt.oxfordjournals.org/content/15/12/1916/F1.large.jpg

Figure 1

Site of action of cyclosporin A and FK506. Antigen‐specific T‐cell activation pathways, including activation of calcineurin and expression of cytokines and their high‐affinity receptors. CsA and FK506 antagonize calcineurin activation, leading to inhibition of NF‐AT‐supported IL‐2 gene expression. FK506, but not CsA, acts more distally by attenuating the response to cytokine stimulation, possibly by altering the expression of their high‐affinity receptors and/or through antagonizing a key step in cytokine receptor signalling.

Mode of Action

Inhibits the production of interleukin IL-2 by helper T-cells thereby blocking T cell activation and proliferation (amplification of immune response). It is effective both in the prevention and in the treatment of ongoing acute rejection.

The current model for the mechanism of action of tacrolimus (or cyclosporine) suggests that, in the T-cell cytoplasm, tacrolimus binds to a specific binding protein called immunophilin which is actually a cis-trans isomerase. The tacro-immunophilin complex in turn binds to and blocks a phosphatase called calcineurin. The latter is required for the translocation of an activation factor (NF-ATc) from the cytosol to the nucleus, where it would normally bind to and activate enhancers/promotors of certain genes. In the presence of tacro, the cytosolic activation factor is unable to reach the nucleus, and the transcription of IL-2 (and other early activation factors) is strongly inhibited. As a result, T cells do not proliferate, secretion of gamma-interferon is inhibited, no MHC class II antigens are induced, and no further activation of the macrophages occurs.

Chapter 2

Objective of the study

To evaluate the adverse event data of Tacrolimus in the database Panacea Biotec Limited (PBL) and to compare it with other database; Eudravigilance (EMA).

Evaluation of line-listing of Individual Case Safety ReportICSRs with Tacrolimus in the PBL database.

Identification of unlisted events

Evaluation of unlisted events using PRR

Comparison of frequency of common adverse events in database of PBL with that in the Eudravigilance

Generation of potential signal on the basis of identification of an unlisted event or increased frequency

Evaluation of the potential signal with the assistance of medical reviewer

CHAPTER 3

Review of literature

Overview

This chapter presents a review of earlier work done by several researchers about pharmacovigilance.

Pharmacovigilance

Pharmacovigilance is the science and activity relating to the detection, assessment, understanding and prevention of adverse effects or any other possible medicine related problems (WHO, 2002) [19]. It is concerned with the post-marketing surveillance of medicines and the use of the information generated for education and effective drug regulation.

Origin of Pharmacovigilance

The origin of pharmacovigilance can be traced back to over 40 years ago. In 1965 the eighteenth World Health Assembly, WHA 18.42, drew attention to the problem of adverse drug reaction monitoring and following further resolutions in 1966, 1967 and 1970 the International Drug Monitoring Programme came into being. In 2005, 78 member countries participated in this Programme and the last decade has seen the participation of numerous developing countries.

The Programme functions on the basis of National Pharmacovigilance centers coordinated by the WHO Programme for International Drug Monitoring, which consists of the WHO Collaborating Centre for International Drug Monitoring, Uppsala and the Pharmacovigilance Department of WHO, Geneva (WHO, 2002) [19].

According to Pirmohamed, M (2005) [20], If ADRs that are not discovered during clinical trials are to be detected, investigated and communicated, and the appropriate action taken, then it is vital that post-marketing pharmacovigilance of all medicines is comprehensive. Effective pharmacovigilance should take into account trends in use, as well as the occurrence of ADRs, enabling more effective advice to be given to those prescribing and using medications and should ensure better standards of safety and efficacy.

Anabela et al.,(2002) [21] reported that, all surveillance systems involve six key elements such as; Detection and notification of health event, Investigation and confirmation (epidemiological, clinical, laboratory),Collection of data, Analysis and interpretation of data, Feedback and dissemination of results, Response and link to public health programs, specifically actions for prevention and control.

Andrew et al., (2004) [22] discussed the potential use of data mining and knowledge discovery in databases for detection of adverse drug events (ADE) in pharmacovigilance. They studied the details of data mining, signal generation or knowledge discovery in relation to adverse drug reactions or pharmacovigilance in medical databases.

Bennett et al., (2008)[23] described preliminary work to identify and quantify the specific factors that contributed to a decision to prioritize a specific drug-ADR combination for further in-depth review. They applied a tool from the discipline of decision analysis to systematically assess the important attributes of spontaneously reported ADRs. A model was created that integrated these assessments and produces rankings for the signals generated from quantitative signaling methods.

Hauben and Aronson(2005)[24] proposed the definition of a signal of suspected causality information that arises from one or multiple sources (including observations and experiments), which suggests a new potentially causal association, or a new aspect of a known association, between an intervention and an event or set of related events, either adverse or beneficial, which would command regulatory, societal or clinical attention, and is judged to be of sufficient likelihood to justify verifiable and, when necessary, remedial actions.

Ferreira G (2005) [25] studied Prescription-event monitoring (PEM) which is a non-interventional intensive method for post-marketing drug safety monitoring of newly licensed medicines. He described an exploratory investigation of the distortion caused by product-related variations of the number of events to the interpretation of the proportional reporting ratio (PRR) values ("the higher the PRR, the greater the strength of the signal") computed using drug-cohort data. He studied this effect by assessing the agreement between the PRR based on events (event of interest vs all other events) and PRR based on cases (cases with the event of interest vs cases with any other events). 

Huaben M(2007)[26] studied the evaluation, potential utility and limitations of the commonly used DMAs by providing a ‘holistic’ perspective on their use as one component of a comprehensive suite of signal detection strategies incorporating clinical and statistical approaches to signal detection —a marriage of technology and the ‘prepared mind’. Data-mining exercises involving spontaneous reports submitted to the US FDA will be used for illustration.

Hauben and Zhou (2003)[27] found automated signal detection methods transparent to drug safety professionals of various backgrounds. This was accomplished by first providing a brief overview of the evolution of signal detection followed by a series of sections devoted to the methods with the greatest utilization and evidentiary support: proportional reporting rations, the Bayesian Confidence Propagation Neural Network and empirical Bayes screening.

Alan et al (2009) [28], constructed a highly inclusive reference event database of reported adverse events for a limited set of drugs, and utilized that database to evaluate three DMAs for the overall yield of scientifically supported adverse drug effects, with an emphasis on ascertaining false-positive rates as defined by matching to the database, and to assess the overlap among SDRs detected by various DMAs.

Matsushita et al (2007) [29][30] carried out a study with the objective to revise the criteria for signal detection to make them suitable for use by pharmaceutical manufacturers. The study comprised of 40 drugs and 1000 adverse events was constructed based on a spontaneous reporting database provided by a pharmaceutical company and used in a simulation to investigate appropriate criteria for signal detection. In total, 1000 pseudo datasets were generated with this model, and three statistical methods (proportional reporting ratio [PRR], Bayesian Confidence Propagation Neural Network [BCPNN] and multi-item gamma Poisson shrinker [MGPS]) for signal detection were applied to each dataset. The sensitivity and specificity of each method were evaluated using these pseudo datasets. [31]

Waller et al (2005)[32] described a new method of prioritizing signals of potential adverse drug reactions (ADRs) detected from spontaneous reports that is called impact analysis. This was an interim step between signal detection and detailed signal evaluation. Using mathematical screening tools, large numbers of signals may now be detected from spontaneous ADR databases.[33]

Maignen et al (2010) [34] had conducted a preliminary study using the parametric modelling of the time to onset of adverse reactions as an approach to signal detection on spontaneous reporting system databases.   Found that some consistency between the occurrences of the infections with the TNF inhibitors suggests a causal association. There were statistical issues that are important to keep in mind when interpreting the results (the impact of the data quality on the fit of the distributions and the absence of a test of hypothesis linked to the absence of a relevant comparator). The study suggested that the modelling of the reported time to onset of adverse reactions could be a useful adjunct to other signal detection methods.

Hauben and Reich(2004)[35] compared performance of two well described DMAs (proportional reporting ratios [PRRs] and an empirical Bayesian algorithm known as multi-item gamma Poisson shrinker [MGPS]) using commonly recommended thresholds on a diverse data set of adverse events that triggered drug labeling changes. In most instances in which a drug-event combinations (DEC) generated a signal of disproportionate reporting with both DMAs (almost twice as many with PRRs), the signal was generated using PRRs in advance of MGPS. No medically important events were signaled only by MGPS. It is likely that the incremental utilities of DMAs are highly situation-dependent.

Pariente et al(2009)[36] studied the reporting patterns over time concerning suicide with the six SSRIs marketed in the UK as of March 2003 and their potential effect on disproportionality signal detection. For older drugs, the events reported during the high-reporting post-television Programme period were diluted by years of low reporting. Differential effects related to time on market on cumulated reporting of adverse drug reactions were taken into account when analyzing spontaneous reporting databases with automated signal generation methods after an alert has changed the spontaneous reporting patterns. Proper use of measures of disproportionality requires thorough knowledge of potential biases and careful analysis of reporting patterns.

The paper of Mozzicato(2007)[37] describes the features of SMQs that allow for flexibility in their application, such as 'narrow' and 'broad' sub-searches, hierarchical grouping of sub-searches and search algorithms. In addition, as with MedDRA, users can request changes to SMQs. SMQs are maintained in synchrony with MedDRA versions by internal maintenance processes in the MSSO. The list of safety topics to be developed into SMQs is long and comprehensive. The CIOMS Working Group retains a list of topics to be developed and periodically reviews the list for priority and relevance. As of mid-2007, 37 SMQs are in production use and several more are under development. The potential uses of SMQs in safety analysis will be discussed including their role in signal detection and evaluation.

Pearson et al (2009) [38] concluded that the use of HLT and SMQ groupings can improve the percentage of unlabeled supported SDRs in data mining results. The trade-off for this gain is the medically less-specific language of HLTs and SMQs compared to PTs, and the need for the added step in data mining of examining the component PTs of each HLT or SMQ that results in a signal of disproportionate reporting.

Chapter 4

Methodology

This section explores and demonstrates the approaches and methods for achieving the specific objectives of the study.

Data mining statistics

Some of the data mining statistics that are widely used for signal detection are:

EBGM – empirical Bayes geometric mean

IC – information component

PRR – proportional reporting ratio

ROR – reporting odds ratio

All of the statistics methods are designed to uncover the same type of information: a disproportionately high occurrence of reports for an event and drug, when compared to reports for that event in the entire database. As a result, for large sample sizes, the score that each of these statistics produces for any given drug-event combination is likely to be similar. Drugs that are new to the market, or that are prescribed to a small number of patients, may have a small presence in the database. For these drugs, the sample size may be quite low, which tends to also make scores calculated for their drug-event combinations low. Some data mining algorithms include statistical techniques that compensate for disparate sample sizes in the safety database and the extreme scores that can result.

In this report I have used PRR as the data mining algorithm along with chi square test.

The PRR and ROR statistics do not in themselves indicate a level of certainty or uncertainty due to a small sample size. Rather, each drug-event combination is often accompanied by an additional statistic, a chi-squared test statistic, and its corresponding p (significance) value. When we assess a PRR or ROR value, we are supposed to suspect the reliability of large values unless the chi squared value is very large (or, equivalently, the p value is very small). However, there is no clear rule for evaluating a large PRR value that has a moderate p value.

The proportional reporting ratio (PRR)

The PRR is a measure of disproportionality of reporting used to detect ADRs in pharmacovigilance databases such as EudraVigilance. This method makes the assumption that when a SDR (involving a particular adverse event) is identified for a medicinal product (P), this adverse event is reported relatively more frequently in association with this medicinal product P than with other medicinal products. This relative increase in the adverse event reporting for the medicinal product P is reflected in a 2x2 contingency table (Table 2) based on the total number of individual cases contained in a pharmacovigilance database, as follows:

Event (E)

All other events

Total

Medicinal Product (P)

A

B

A+B

All other immunosuppresants

C

D

C+D

Total

A+C

B+D

N=A+B+C+D

Table 2: 2x2 contingency table for the computation of the PRR

In this table the elements counted are the individual cases available in the database. Thus, a given individual case may contribute to only one of the cells of the table, even if the individual case refers to multiple medicinal products or multiple adverse events. The approach of performing the computations of the PRR on the individual case counts instead of number of ADRs has been chosen to keep the independence between the variables used to compute the PRR so that the variance of the PRR will not be underestimated.

The general criteria to calculate the PRR are as follows:

- The value A indicates the number of individual cases with the suspect medicinal product P involving an adverse event E.

- The value B indicates the number of individual cases related to the suspect medicinal product Involving any other adverse events but E.

- The value C indicates the number of individual cases involving event E in relation to any other medicinal products but P.

- The value D indicates the number of individual cases involving any other adverse events but E and any other medicinal products but P.

The PRR is computed as follows:

A/ (A+B)

PRR=

C/(C+D)

The PRR is a very sensitive method which may generate a high number of falsepositive signals particularly when the number of reports is low. Therefore case count thresholds (number of reports > 3) are also used in association with the PRR and Chi-square statistics to reduce the number of false positives.

When the PRR is displayed with the chi2 statistic:

· The PRR > 2

· The chi 2 > 4

· The number of individual cases greater or equal to 3.

Steps of SDR evaluation process

Communicate with relevant stakeholders.

Information available in other parts of the marketing authorisation dossier:

• Initial application

• SUSARs

• PSURs

• Post-authorisation commitments

• Risk management plan

• Other post-authorisation study

• Identification of potential

duplicates

• Data quality check

• Obtain additional information

when appropriate

Signal of disproportionate

reporting

Check the terms of the marketing

authorization (SPC, PL)

For maintaining uniformity, MedDRA terminology was used. The Medical Dictionary for Regulatory Activities (MedDRA) Terminology is the international medical terminology developed under the auspices of the International Conference on Harmonization (ICH) of Technical Requirements for Registration of Pharmaceuticals for Human Use. Prior to the development of MedDRA, there had been no internationally accepted medical terminology for biopharmaceutical regulatory purposes. Most organizations processing regulatory data used one of the international adverse drug reaction terminologies in combination with morbidity terminology. Using different terminologies at various stages in a product’s life complicates data retrieval and analysis, making it difficult to cross-reference data. The use of multiple terminologies also affected communication between companies and clinical research organizations.

In October 1994, the ICH Steering Committee introduced multi-disciplinary regulatory communication initiatives to complement the ongoing safety, quality, and efficacy harmonization topics. The aim of the ICH M1 initiative was to standardize the international medical terminology for regulatory communication. This includes communication in the registration, documentation, and safety monitoring of medical products for use in both pre- and post-marketing phases of the regulatory process.

There may be multiple locations of relevant terms within a system organ class (SOC) and lack of recognition of appropriate group terms; the user may think that group terms are more inclusive than is the case. MedDRA may distribute terms relevant to one medical condition across several primary SOCs. If the database supports the MedDRA model, it is possible to perform multiaxial searching: while this may help find terms that might have been missed, it is still necessary to consider the entire contents of the SOCs to find all relevant terms and there are many instances of incomplete secondary linkages. It is important to adjust for multiaxiality if data are presented using primary and secondary locations. Other sources for errors in searching are non-intuitive placement and the selection of terms as preferred terms (PTs) that may not be widely recognized. Some MedDRA rules could also result in errors in data retrieval if the individual is unaware of these: in particular, the lack of multiaxial linkages for the Investigations SOC, Social circumstances SOC and Surgical and medical procedures SOC and the requirement that a PT may only be present under one High Level Term (HLT) and one High Level Group Term (HLGT) within any single SOC.

Structural elements of the terminology

The MedDRA terminology was developed as a medically validated medical terminology for utilization throughout the regulatory process. The developers of the terminology designed a structure that promotes specific and comprehensive data entry and flexible data retrieval. Figure 2-1 represents the hierarchical structure of the terminology. Relationships between terms in the terminology fall into the following two categories:

Equivalence

The equivalence relationship groups synonymous terms, or equivalent terms, under Preferred Terms.

Hierarchical

The hierarchy provides degrees or levels of super ordination and subordination. The superordinate term is a broad grouping term applicable to each subordinate descriptor linked to it. Hierarchical levels thus represent vertical links in the terminology.

Hierarchies are an important mechanism for flexible data retrieval and for the clear presentation of data. The five-level structure of this terminology provides options for retrieving data by specific or broad groupings, according to the level of specificity required. The Lowest Level Term (LLT) level provides maximum specificity.

The terminology was not developed as a formal classification or taxonomy; each level in the hierarchy may reflect a variable degree of specificity or "granularity" from one System Organ Class to another. High Level Terms (HLTs) and High Level Group Terms (HLGTs) facilitate data retrieval and presentation by providing clinically relevant grouping of terms. Collectively, the HLT and HLGT levels are sometimes referred to as the "grouping terms" in MedDRA.

The 26 System Organ Classes (SOCs) represent parallel axes that are not mutually exclusive. This characteristic, called "multi-axiality," allows a term to be represented in more than one SOC and to be grouped by different classifications (e.g., by etiology or manifestation site), allowing retrieval and presentation via different data sets. Grouping terms are pre-defined in the terminology and not selected on an ad hoc basis by data entry staff. Rather, the terminology is structured so that selection of a data entry term leads to automatic assignment of grouping terms higher in the hierarchy. Multi-axial links of terms are pre-assigned, ensuring comprehensive and consistent data retrieval, irrespective of which SOC is selected at data retrieval.

A PT is a distinct descriptor (single medical concept) for a symptom, sign, disease, diagnosis, therapeutic indication, investigation, surgical, or medical procedure, and medical, social, or family history characteristic.

PTs should be unambiguous and as specific and self-descriptive as possible in the context of international requirements. Therefore, eponymous terms are only used when they are recognized internationally.

System Organ

Class (SOC)

High Level Group

Term (HLGT)

High Level Term (HLT)

Preferred Term (PT)

Lowest Level Term (LLT)

Figure 2-1. Structural Hierarchy of the MedDRA Terminology

Standardized MedDRA Query (SMQ)

Standardized MedDRA (Medical Dictionary for Regulatory Activities) queries (SMQs) are a newly developed tool to assist in the retrieval of cases of interest from a MedDRA-coded database. SMQs contain terms related to signs, symptoms, diagnoses, syndromes, physical findings, laboratory and other physiological test data etc, that are associated with the medical condition of interest. They are being developed jointly by CIOMS and the MedDRA Maintenance and Support Services Organization (MSSO) and are provided as an integral part of a MedDRA subscription. During their development, SMQs undergo testing to assure that they are able to retrieve cases of interest within the defined scope of the SMQ.

Standardized MedDRA Queries (SMQs) are groupings of MedDRA terms, ordinarily at the Preferred Term (PT) level that relate to a defined medical condition or area of interest. The only Lowest Level Terms (LLTs) represented in an SMQ are those that link to a PT used in the SMQ; all others are excluded.

Total number of unique terms at each level:

SOC

26

HLGT

335

HLT

1713

PT

19737

LLT*

70634

Software Used:

There were 2 softwares used for making this project report and for analysis of data:

Oracle Argus b)SPSS (Statistical Product and Service Solutions)

Oracle argus

Safety reports are processed in Argus safety database, and all cases can be extracted from Argus and assessed further. Argus is complete pharmacovigilance software system designed for managing global adverse Events (AE) case data and regulatory reporting. It supports reporting compliance via a comprehensive and robust reporting engine that allows for configuration of specific rules to match regulatory requirements.

Case processing in Argus is done in 4 stages:-

Case entry

Peer review

Medical review

Distribution & reporting

There are 8 tabs in Argus:

Each tab enables us to capture specific information about the case and is designed to capture similar information in each of its subsections.

The General tab

The General tab is designed to capture case information in categorized sections that capture category-specific information. The General tab enables you to enter or view information such as type of report, literature information, and so forth.

The Patient Tab

The Patient Tab section of the Case Form helps you to enter patient information such as the patient's past medical history and current conditions, and laboratory tests and test results. The medical information entered here could be very useful to the person analyzing the event. For example, if the adverse event was a rash that developed after applying a topical product, the knowledge that the patient has a history of allergic reactions could be relevant.

3. The Product Tab

The Product tab contains information related to products of panacea biotec , its indication,QC, dosage regimens, product details like withdrawal, tapering.

4. The Event Tab

The Events tab contain information regarding the events during the course of medication, event coding, nature of event its serious criteria and all the events assessment by peer reviewer.

5. The Analysis Tab

The Analysis tab provides information related to the analysis of case as narrative and as case summary done by peer review personnel. The Analysis tab enables us to generate or view a narrative description of the case, together with other notes. In addition, it also enables us to enter information required for generating the MedWatch 3500A, BfArM, and AFSSaPS reports.The typical users of this tab are responsible for:

Making a medical assessment of the case

Approving the case for completeness and accuracy.

The following is an illustration of the Analysis tab.

6. The Activities Tab

The Activities tab presents detailed information about the contact log, routing comments, action items, and Case Lock/Closure.

The following is an illustration of the Activities tab.

7. The Additional Information tab

The Additional Information tab enables you to attach notes and other items to the case. For example, you could attach a fax message that came in as part of the case and needs to be scanned and attached or an electronic file received by e-mail. It also enables us to set up cross-references to other cases such as links between cases referring to mothers and children. The total number of attachments and references attached to a case display in the header.

The following is an illustration of the Additional Information tab.

8. The Regulatory Reports Tab

The Regulatory Reports tab enables us to:

View all scheduled reports

Schedule new reports

When a new case is created, there are no reports associated with it. As data is entered and the case is saved, the regulatory report scheduling algorithm determines which reports, if any, will be required for that case. The reports determined to be necessary appear in the Regulatory Reports tab. We can manually schedule reports via the Reports menu or by clicking the Regulatory Reports Tab. We can also add comments to the existing reports. The comment section can also be updated to enter the notes for the report even after the report has been submitted.

The following is an illustration of the Regulatory Reports tab.

Procedure for extracting data from Argus Safety database:

1. for extracting cases with respect to SOC and PT:

Reports<Aggregate Reports<System Reports library

2. for extracting total cases of a particular drug:

Reports<Periodic Reports<ICH PSUR

SPSS (Statistical Product and Service Solutions)

SPSS is a comprehensive and flexible statistical analysis and data management solution. SPSS can take data from almost any type of file and use them to generate tabulated reports, charts, and plots of distributions and trends, descriptive statistics, and conduct complex statistical analyses.

Figure showing Screen appearance when SPSS is started

There are two tabs at the bottom of the screen. The Data View tab brings up the Data Editor that is used for the input of data. But to use data properly, SPSS must have a description of that data. Clicking the Variable View tab brings up the window that allows for this description.

Performing Analyses

Once the data is entered, we use SPSS to perform analyses on these data.SPSS uses two basic methods for performing analyses and other tasks. The first is menu driven while the second makes use of one of several programming languages.

Few descriptive statistics were done on the variables. To do this, select Analyze>Descriptive Statistics>Descriptives

To perform the chi-square analysis click Analyze>Descriptive Statistics>Crosstabs

CHAPTER –5

Results and discussion

The data was obtained from Argus safety database for the drugs Tacrolimus and other immunosuppresants like ciclosporin, sirolomus and MMF .

Following is the observation found for each drug.

Tacrolimus:

There were 402(Listed + Unlisted) cases reported in literature from 1 Jan 1990 till 28 Feb 2013.out of 402 cases 1416 ADR’s (serious+non-serious) unlisted events were seen throughout the world where Panacea Biotec is the Market Authorized Holder(MAH).

All events were categorized as per SOC and PT level of MedDRA. After getting the data all events were checked with the innovator’s PI and only those events were included for signal detection which were unlisted according to the latest PI of the innovator of Tacrolimus Astelles .There were 215 adverse events which were unlisted as per PI.

For any event to be considered as potential signal 3 criteria’s were set

The number of individual cases greater or equal to 3

The PRR > 2

The chi 2 > 4

Out of 215 events only 15 events were seen where individual cases were greater or equal than 3.

SOC

PT

No. of events with tacrolimus

No.of events with others immunosuppressants

PRR value

Chi sq. value

Endocrine Disorders

Diabetes Insipidus

3

0

infinite

0.015

Eye disorders

Pupil fixed

4

0

infinite

.0005

Immune system disorders

Food allergy

3

0

infinite

0.015

Infections and infestations

Meningitis cryptococcal

4

4

1.96

.3296

Infections and infestations

Polyomavirus-associated nephropathy

7

14

0.9826

0.9698

Infections and infestations

TB of CNS

3

0

infinite

0.015

Injury, poisoning and procedural complications

Toxicity to various agents

3

7

0.84

0.8031

Investigations

Neutrophil Pelger-Huet anomaly present

12

14

1.6

0.1786

Musculoskeletal and connective tissue disorders

Rhabdomyolysis

5

16

0.614

0.3354

Neoplasms benign, malignant and unspecified (incl cysts and polyps)

Kaposi's sarcoma

3

15

0.394

0.125

Nervous system disorders

Dysarthria

3

3

1.965

0.48

Psychiatric disorders

Mental status changes

3

5

1.179

0.821

Renal and urinary disorders

Focal segmental glomerulosclerosis

5

9

1.09

0.8745

Vascular disorders

Venoocclusive disease

3

0

infinite

0.1520.

General disorders and administration site conditions

Drug interaction

14

40

0.687

0.228

When PRR was calculated for these events only in 7 events the ratio was equal or greater than 2.and no event had chi square value greater than 4. So, it can be concluded that there was no new potential signal found during the study. Results are summarized in the table given above. It can be seen from the table that there are 4 events which are exclusively seen in case of Tacrolimus.Mathematically, infinite value doesnot show any significance but medically it can be considered as the event to be monitored hence forth as it is not been observed during initial trails and nor with other immunosuppressants.

Other significant observations during the study:

Besides signal detection, there were few interesting observation found during carrying out this study. With the help of SPSS software, other variables were compared like Demographic data, gender wise comparison, etc. which are mentioned below.

Maximum events were observed in Following SOC’s:

Infections and infestations> Investigations> Nervous system disorders> Blood and lymphatic system disorders

Following is the table showing listedness of the events observed in the database for the drug Tacrolimus:

System Organ Class * Case Listedness Cross tabulation

Count

Case Listedness

Total

Listed

Unknown

Unlisted

System Organ Class

1

0

0

0

1

Blood and lymphatic system disorders

0

11

0

7

18

BLOOD AND LYMPHATIC SYSTEM DISORDERS

0

12

0

1

13

Cardiac disorders

0

4

0

3

7

Congenital, familial and genetic disorders

0

0

0

1

1

Endocrine disorders

0

1

0

3

4

ENDOCRINE DISORDERS

0

0

0

3

3

Eye disorders

0

2

0

5

7

Gastrointestinal disorders

0

4

0

8

12

GASTROINTESTINAL DISORDERS

0

2

0

3

5

General disorders and administration site conditions

0

9

0

4

13

GENERAL DISORDERS AND ADMINISTRATION SITE CONDITIONS

0

1

0

3

4

Hepatobiliary disorders

0

0

0

3

3

HEPATOBILIARY DISORDERS

0

0

0

1

1

Immune system disorders

0

11

0

4

15

IMMUNE SYSTEM DISORDERS

0

7

0

1

8

Infections and infestations

0

27

0

18

45

INFECTIONS AND INFESTATIONS

0

16

1

8

25

Injury, poisoning and procedural complications

0

4

0

3

7

INJURY, POISONING AND PROCEDURAL COMPLICATIONS

0

0

0

3

3

Investigations

0

5

0

12

17

INVESTIGATIONS

0

0

0

1

1

Metabolism and nutrition disorders

0

5

0

2

7

METABOLISM AND NUTRITION DISORDERS

0

0

0

2

2

Musculoskeletal and connective tissue disorders

0

0

0

4

4

MUSCULOSKELETAL AND CONNECTIVE TISSUE DISORDERS

0

3

0

3

6

Neoplasms benign, malignant and unspecified (incl cysts and polyps)

0

24

0

11

35

NEOPLASMS BENIGN, MALIGNANT AND UNSPECIFIED (INCL CYSTS AND POLYPS)

0

6

0

6

12

Nervous system disorders

0

24

0

15

39

NERVOUS SYSTEM DISORDERS

0

11

0

7

18

Pregnancy, puerperium and perinatal conditions

0

0

0

3

3

PREGNANCY, PUERPERIUM AND PERINATAL CONDITIONS

0

1

0

0

1

Psychiatric disorders

0

0

0

1

1

Renal and urinary disorders

0

4

0

15

19

RENAL AND URINARY DISORDERS

0

8

0

2

10

Respiratory, thoracic and mediastinal disorders

0

10

0

4

14

RESPIRATORY, THORACIC AND MEDIASTINAL DISORDERS

0

1

0

0

1

Skin and subcutaneous tissue disorders

0

3

0

1

4

SKIN AND SUBCUTANEOUS TISSUE DISORDERS

0

0

0

1

1

Vascular disorders

0

5

0

6

11

VASCULAR DISORDERS

0

1

0

1

2

Total

1

222

1

179

403

From the table it can be seen that out of 403 cases, 222 cases were listed and 179 cases were unlisted. This listedness criterion was done by referring the innovators PI. Infections and infestations SOC had the maximum number of unlisted cases.

Graph showing the listedness of events according to SOC:

From the graph, it can be seen that in unlisted SOC renal and urinary disorders, infections and infestations, the number of cases are in maximum number as compared to other classes.

SPSS was used to analyze these descriptive variables. For Gender specific analysis, following observations were seen. Out of 403 cases, 187 cases were female and 208 were males. So it can be seen from below table that the frequency is more in males as compared to females.

In females, Infections and Infestation and nervous disorders cases were seen in high frequency as compared to other SOC’s.while in males infection and infestation along with Neoplasm and nervous disorder showed high frequency as compared with rest of SOC’s.

System Organ Class * gender Cross tabulation

Count

gender

Total

Female

Male

System Organ Class

1

0

0

1

Blood and lymphatic system disorders

1

7

10

18

BLOOD AND LYMPHATIC SYSTEM DISORDERS

0

8

5

13

Cardiac disorders

0

3

4

7

Congenital, familial and genetic disorders

0

0

1

1

Endocrine disorders

0

2

2

4

ENDOCRINE DISORDERS

0

1

2

3

Eye disorders

0

4

3

7

Gastrointestinal disorders

0

8

4

12

GASTROINTESTINAL DISORDERS

0

4

1

5

General disorders and administration site conditions

0

6

7

13

GENERAL DISORDERS AND ADMINISTRATION SITE CONDITIONS

1

2

1

4

Hepatobiliary disorders

0

1

2

3

HEPATOBILIARY DISORDERS

0

1

0

1

Immune system disorders

0

3

12

15

IMMUNE SYSTEM DISORDERS

0

2

6

8

Infections and infestations

2

22

21

45

INFECTIONS AND INFESTATIONS

0

10

15

25

Injury, poisoning and procedural complications

1

5

1

7

INJURY, POISONING AND PROCEDURAL COMPLICATIONS

0

3

0

3

Investigations

0

4

13

17

INVESTIGATIONS

0

0

1

1

Metabolism and nutrition disorders

0

2

5

7

METABOLISM AND NUTRITION DISORDERS

0

1

1

2

Musculoskeletal and connective tissue disorders

0

0

4

4

MUSCULOSKELETAL AND CONNECTIVE TISSUE DISORDERS

0

5

1

6

Neoplasms benign, malignant and unspecified (incl cysts and polyps)

0

10

25

35

NEOPLASMS BENIGN, MALIGNANT AND UNSPECIFIED (INCL CYSTS AND POLYPS)

0

4

8

12

Nervous system disorders

1

22

16

39

NERVOUS SYSTEM DISORDERS

0

9

9

18

Pregnancy, puerperium and perinatal conditions

1

2

0

3

PREGNANCY, PUERPERIUM AND PERINATAL CONDITIONS

0

0

1

1

Psychiatric disorders

0

0

1

1

Renal and urinary disorders

0

10

9

19

RENAL AND URINARY DISORDERS

0

7

3

10

Respiratory, thoracic and mediastinal disorders

0

8

6

14

RESPIRATORY, THORACIC AND MEDIASTINAL DISORDERS

0

1

0

1

Skin and subcutaneous tissue disorders

0

3

1

4

SKIN AND SUBCUTANEOUS TISSUE DISORDERS

0

1

0

1

Vascular disorders

0

5

6

11

VASCULAR DISORDERS

0

1

1

2

Total

8

187

208

403

Data analysis of other immunosuppressants:

Total 4 immunosuppressants which are marketed by Panacea biotec limited were included in the study, .viz Sirolomus, ciclosporin, MMF, MMF-sodium.2783 events were seen in total for all these immunosuppressants.

S.No

PT

Sirolomus

Ciclosporin

MMF

MMF-NA

1

Diabetes Insipidus

0

0

0

0

2

Pupil fixed

0

0

0

0

3

Food allergy

0

0

0

0

4

Meningitis cryptococcal

1

0

0

3

5

Polyomavirus-associated nephropathy

2

2

0

0

6

TB of CNS

0

0

0

0

7

Toxicity to various agents

1

2

4

0

8

Neutrophil Pelger-Huet anomaly present

0

0

14

0

9

Rhabdomyolysis

1

10

5

0

10

Kaposi's sarcoma

1

9

5

0

11

Dysarthria

1

1

2

0

12

Mental status changes

1

2

2

0

13

Focal segmental glomerulosclerosis

1

4

4

0

14

Venoocclusive disease

0

0

0

0

15

Drug interaction

5

18

16

1

Diabetes insipidus, Pupil fixed, food allergy, TB of CNS and Venoocclusive disease are the new events which were exclusively seen in case of Tacrolimus. These events were not seen with the use of other immunosuppressants.

Data Analysis of EMA database for the drug Tacrolimus:

There were 13,793 total events reported in EMA database; Eudravigilance. All those events which fulfilled the criteria of 3 cases in PBL database were compared in Eudravigilance database.

Obeservation in EMA:

Frequency of events is seen more in males as compared to females. Out of 1141 cases of 15 SOC’s in tacrolimus 629 were males and 392 were females.

Graphs showing SOc Vs Males

Out of total number of females it can be seen from the graph that maximum number lies in General disorder SOC(PT level is Drug interaction), followed by infection and infestations(Polyomavirus-associated nephropathy) and so on.

All the comparisons are made according to those events of Tacrolimus which satisfied the criteria where the number of individual cases greater or equal to 3.

.

Summary of total events vs. SOC is given in form of Graph.

In Males also drug interaction was the event whose number was in maximum .there were 224 cases of drug interactions, followed by infection and infestation.

Conclusion

Class specific signals of events associated with tacrolimus were not found potent enough to cause any new signal. However, since the calculated statistics were high albeit not significant, the possibility of event Tacrolimus pairing should still be analyzed from large database. Since there were few events which were exclusively seen in case of Tacrolimus, they should be closely monitored thereafter.

While comparing the data obtained from PBL and EMA, it can be concluded that males are more prone to ADRs as compared to females.

Though there was no potential signal detected during the study but few events which were seen only in case of tacrolimus users can be considered as future risk events which need continuous monitoring. The main reason for not getting any signal could be the small size of PBL database.

R e f e r e n c e s

1. Evans sjw. Pharmacovigilance: a science or fieldingemergencies? Stat med. 2000;19:3199–3209.

2. Moore n, thiessard f, begaud b. The history ofdisproportionality measures (reporting oddsratio, proportional reporting rates) in spontaneousreporting of adverse drug reactions.pharmacoepidemiol drug safety. 2005;14:285–286.

3. Evans sjw, waller pc, davis s. Use of proportionalreporting ratios (prrs) for signal generation from spontaneous adverse drug reaction reports.pharmacoepidemiol drug safety. 2001;10:483–486.

4. Rothman kj, lanes s, sacks st. The reporting odds ratio and its advantages over proportional reporting ratio. Pharmacoepidemiol drug safety. 2004;13:519–523.

5. Van puijenbroek e, bate a, leufkens hgm, et al. A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol drug safety. 2002;11:3–10.

6. Waller p, van puijenbroek e, egberts a, et al. The reporting odds ratio versus the proportional reporting ratio: "deuce." Pharmacoepidemiol drug safety. 2004;13:525–526.

7. Roux e, thiessard f, fourrier a, et al. Evaluation of statistical association measures for the automatic signal generation in pharmacovigilance. Ieee trans inf technol biomed. 2005;9(4):518–527.

8. Almenoff j, tonning jm, gould l, et al. Perspectives in the use of data mining. Pharmacovigilance drug safety. 2005; 28:981–1007.

9. Strom bl, kimmel se, eds. Textbook of pharmacoepidemiology. Chichester, uk: wiley & sons; 2006.

10. Kilgour-christie j, czarnecki a, simmons v. Proportional reporting ratio calculations from a pharmaceutical company database. Pharmacoepidemiol drug safety. 2001;10(suppl 1):s159.

11. Kilgour-christie j, czarnecki a. Safety signals verified using prrs on regulatory authority databases. Pharmacoepidemiol drug safety. 2002;11 (suppl 1):s145–s146.

12. Rodriguez em, staffa ja, graham dj. The role of databases in drug postmarketing surveillance. Pharmacoepidemiol drug safety. 2001;10:407–410.

13. Fda adverse event reporting system (aers). Available at: http://www.fda.gov/cder/aers/default .htm. Accessed september 13, 2005.

14. Wood sm, coulson r. Adverse drug reactions online information tracking (adroit). Pharm med. 1993;7:203–213.

15. Dean ad, dean aj, dicker rc. Epi info version 5: a word processing, database and statistical program For epidemiology on microcomputers. Atlanta, ga: centers for disease control; 1990.

16.Medicines contro lagency .2001.Available at: http://www.accenture.com/countries/uk/services/by_subject/infrastructure_outsourcing/client_successes/medicinesecommerce.htm. Accessed may 2, 2006.

17. Almawi wy, melemedjian ok, rieder mj. An alternate mechanism of glucocorticoid anti‐proliferative effect: promotion of a th2 cytokine‐secreting profile. Clin transplant1999; 13: 365–372

18. Almawi wy, assi jw, chudzik dm, lazarovits ai. Opposing effects of rapamycin and cyclosporin a on activation‐induced ca2+ release. Eur j pharmacol1999; 381: 51–56

19. World health organization. (2002). the importance of pharmacovigilance. International Conference on pharmacovigilance, geneva.

20. Pirmohamed, m. (2005). Anticipating, investigating and managing the adverse events of Drugs. Clinical medicine, 5 (1): 23-5.

21. Anabela, ga.,halperin, a.,isabella, d,.(2002).public health surveillance toolkit.a guide

for busy task managers.a world bank journal, 4(2): 19-20.

22. Pirmohamed m, breckenridge am, kitteringham nr, park bk. Adverse drug reactions. Br med j. 1998;316:1295–8.

23 Levitan b;yee cl;russo I;drug safety:an international journal of medical toxicology and drug experience,2008,vol.31,issue9,p727

24. Aronson jk, hauben re. Clarification of terminology in drug safety. Drug saf 2005; 28(10): 851–70

25. Ferreira g. Prescription-event monitoring. In: strom bl, editor. Pharmacoepidemiology. 4th ed. Chichester, uk: john wiley & sons ltd, 2005: 203–16

26. Hauben m, signal detection in the pharmaceutical industry, drug safety july 2007, volume 30, issue 7, pp 627-630.

27. Manfred hauben, ,xiaofeng zhou , quantitative methods in pharmacovigilance, drug safety march 2003, volume 26, issue 3, pp 159-186

28. Alan m,hochberg,manfred hauben,ronald k.pearson,donald j hara,david.i.goldsmith, an evaluation of three signal-detection algorithms using a highly inclusive reference event database,drug safety, 2009;32(6):509-25

29. Yasuyuki matsushita, , shinpei niwa, satoshi sonehara, chikuma hamada, isao yoshimura,criteria revision and performance comparison of three methods of signal detection applied to the spontaneous reporting database of a pharmaceutical manufacturer,drug saftey 2007;30(8):715-2

30. Lindquist m, edwards ir, bate a, et al. From association to alert: a revised approach to international signal analysis. Pharmacoepidemiol drug saf 1999; 8: s15–25

31. Van puijenbroek ep, bate a, leufkens hgm, et al. A comparison of