Methodological and Ethical Quality Checklist Assessment in Issues of Annals of Family Medicine Clinical Trials (2010-2013)

In the present investigation the Randomized Clinical Trial (RCT) is the “gold standard” for evaluating new therapies or strategies in medicine [1]. A low quality in RCTs design or publication, it could lead to an underestimation of the risk or even false, using treatments that may be less or even harmful, and could produce daily clinical practice consequences [2]. So a higher quality in published clinical trials to help choose the best therapies for patients.


Introduction
In the present investigation the Randomized Clinical Trial (RCT) is the "gold standard" for evaluating new therapies or strategies in medicine [1]. A low quality in RCTs design or publication, it could lead to an underestimation of the risk or even false, using treatments that may be less or even harmful, and could produce daily clinical practice consequences [2]. So a higher quality in published clinical trials to help choose the best therapies for patients.
There are different methods of assessing the ethical quality of Clinical Trials (CTs) publications, amongst which Dechartres et al. identified 26 scales or checklists based on CONSORT (Consolidated Standards of Reporting Trials) [3]. In this way they obtained increased data on methodological issues, but only 29% adequate masked allocation. They also found that quality evaluation still presents significant problems [4]. Katrak et al. also observed heterogeneity in the use of criteria and unclear definition in the scales, which might limit the relevance of the results [5].
No studies have been found assessing the methodological and ethical quality of the CTs published in Annals of Family Medicine journal.
The aim is to analyze quantitative and qualitative methodological and ethical quality of published clinical trials in Annals of Family Medicine journal, between 2010-2013. We chose this journal because it has the highest impact factor in Family Medicine (4.613), published CTs, CONSORT endorsement and has an important general quality. We want to know how clinical trials are performing in primary care, as this will help to improve the design and publication of these.

Materials and Methods
On March 25, 2014 we use a hand search and subsequent checklist to analyze CTs published in Annals of Family Medicine since January 1, 2010 to December 31, 2013. It is a print edition with free online access (no "open access"), which follows the CONSORT standards [6,7] and the ICMJE (International Committee of Medical Journal Editors) recommendations [8].
We follow the CONSORT statement [3] and PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [9], which provide higher quality checklists [8] for CTs publishing [10,11], and have been implemented in over 400 journals and editorial groups as ICMJE [3]. We will also use the most recent edition of the Declaration of Helsinki (2013) [12,13].
The "choice of subjects" is the next section (4th). The subjects are White in 54.3%, 45.7% African Americans, 37.1% Hispanic and in 8 studies Asian. Race is not indicated in 42.9%, not specified in 13 CTs, and in 4, subjects are ethnic minorities such as American Indian and Vietnamese. The inclusion of immigrant or low-income groups has increased in the last three years: 2011 (1), 2012 and 2013 (3). We have not observed the inclusion of "vulnerable" people [24].
In the 5 th section the "sample size" is evaluated. Quantitative variables are reported in a table (Table 3). The "alpha" error is 74.3%, "beta" is 82.9%, the magnitude of the minimum difference to be found is 80%, and the "flowchart" is 85.7%.
Regarding the "Description of treatment" (section 6), with an effective standard treatment, placebo or non-intervention has not been used as a control in 62.9%. A strong and compelling case for the use of placebo or non-intervention for methodological reasons was given in 6 CTs, and the determination of efficacy and safety in two. In 37.1%, there were serious adverse events in general (including death), and it was unclear in 3 studies. The "intention to treat analysis" was conducted in 15 CTs, "not indicated" (to be performed but not mentioned) in 3, and was "unclear" (to be mentioned but not performed properly) in one. According to the Jadad score, 15 studies were of good quality (3), 16 low-grade (1 or 2) and 4 excellent (5).
The 7th section deals with the "development of the trial and assessment the answer". The "primary outcome variable" or "endpoint" is not described in one CT and is "unclear" in 11.4%. "Multivariate analysis" is applied in 42.9%. There is a DSMB in 14.3%. Finally, CONSORT standards mentioned that it followed in 28.6%.

Assessment of other ethical quality parameters
In the 8th section, the most important variables are represented in Table 2. Two CTs included minors as research subjects. Parents or representative completed Informed Consent (IC) in one of them and the other this completion was "unclear". publications or "short reports") for the study of secondary variables or secondary analysis of a previously published CT with the same number of subjects.

Checklist to assess the ethical and methodological quality
We reviewed a checklist [16] based on CONSORT [17] that had 80 items divided into 8 sections. We have updated to a 133-item checklist in eleven sections. It was applied to assess their ethical and methodological quality. Following the Declaration of Helsinki [18], items were modified or created to adapt them to Primary Care (PC) with the assistance of the PRECIS (Pragmatic-Explanatory Continuum Indicator Summary) tool [19]. More explicit variables were created for placebo use. We introduced AHRQ [20] ("Agency for Healthcare Research and Quality") criteria and the Jadad quality scale. We use the assessment by a DSMB (Data and Safety Monitoring Board) and "stopping rules" (Haybittle-Peto, Pocock, O'Brien-Fleming) [21]. We also evaluated sequential trials, optimized evaluated by the CREC (Clinical Research Ethics Committee) or futility analysis. We stressed race and language, incentives, training in Bioethics, types of conflicts of interest and sponsor influence.

Analysis and data extraction
Two tables with frequency and rates for qualitative variables were made. We built a table with the mean, median, mode, minimum, maximum and statistical percentiles (P25, P50, P75) for quantitative variables. Data were calculated using SPSS 20.0.
Comparable variables on the κ, κ means and 95% CI value was obtained [22]. This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent was no necessary to obtain because no individual participants were included in the study.

Results
We found 35 CTs that met the inclusion criteria ( Figure 1). Most were observational studies. Afterwards we eliminated three of these (not meet the inclusion criteria) and final analysis was performed. Table 1 shows the frequencies and rates for the most representative items.

Assessment of methodological quality
In the section on "justification and objectives" of our checklist, we observed that the following appear in all: 1) CT in heading or subheading, 2) related CTs, 3) systematic review cited and 4) a priori hypothesis.
Regarding the "clinical trial type", 68.6% were performed in > 5 centers and 14.3% in ≤ 5. In 80% some authors are epidemiologists or statisticians.
In the 3rd section, the "clinical trial design" is evaluated. In 80%, there were controlled parallel-groups and controlled factorials in 4 CTs. There is a tendency to be more pragmatic than explanatory according to the criteria of AHRQ.
Baseline comparison was made in a table following CONSORT [14] for almost all CTs (34), and "p" in 68.6%. Only the CT by Licciardone et al. [23] is sequential; moreover it uses the O'Brien Fleming stopping rule. The effectiveness of masking and withdrawals due to adverse events were found in only 5 CTs and the latter were uncertain in 6 studies ( Table 2).  The confidentiality of the participants' personal information was found in only 17.1%. In 8.6% of the CTs, people were injured by their participation, but none were compensated. In 22.9% a study protocol is not stated.
In all CTs possible conflicts of interest were stated. The influence by the sponsor was not described in 48.6%. The studies reviewed are registered in 57.1% of the cases.

Calculation of Kappa index
The authors (FGG and ACG) independently reviewed the 35 articles chosen following the checklist. For the completion of the variables they followed detailed instructions and discrepancies were resolved by consensus. In 82 variables we found a κ average of 0.93 (95% CI, 0.90-0.96), optimal) [25]. In Table 4 the variables assessed by κ appear.

Discussion
In current Family Medicine RCT is one of the main tools to compare a new treatment with other standard or a placebo. Therefore the quality of the publication of new RCTs, is crucial for a reliable picture of the results they can obtain in order to implement them safely on clinical practice.
Clinical trials published in "Annals of Family Medicine" journal are most frequently named in Family Medicine. Therefore, our study evaluates the quality of these, for a fairly good idea of how they are developing RCTs in Family Medicine. We assess one journal only to have a minimal publication bias.
We observed a higher general quality if we compare our study with similar. In a related study [16] there is a progressive increase in masked randomization: 33.1% (1993-95), 56.7% (2004) and 68.6% in 2010-13. It could be explained because after CONSORT, it was stressed that it was essential to reduce selection bias. In contrast, double blind trial is less frequent (5.7%) versus earlier revisions (43.9% (1993-1995) and 43.3% (2004) and placebo use as a control element has also decreased: 37.0% (93-95), 16.7% (2004) and 11.4% (2010-13). Probably due to the treatments used less frequent damaging. The figures for "intentionto-treat" and "multivariate" analysis remain stable (47.7% and 44.2%, 42.9% (both), although both were higher in 2004 (70% each). This may be due to the increased statistical power searched in recent years.
Regarding the indicators of ethical quality [16], the number of CTs without a reference to CREC has dropped, particularly in the latest period (25.5% (93-95), 20% (2004) and 5.7% (2010-13). This is perhaps attributable to CONSORT standards. Coinciding with Ndebele et al. [12], CTs on children or disabled persons have been very infrequent in the three studies. A possible explanation could be that in Family Medicine children as research subjects are uncommon.
In Sinha et al. [26], a Jadad score ≥3 was found in 40%, which is slightly higher than the rate found in our study (54.3%). This is perhaps because there are less RCT.
We agree with Agha et al. [27] that the country where most CTs were carried out was the US (15). Possibly because it is the country where more research is conducted.
In contrast with our results, O'Meara et al. [28], in their review observed very limited information on the sample size and masking. We think it could be because most studies reviewed are low quality.
We agree with Laframboise et al. [29] when they describe all the justification and objectives. This description is possibly necessary to assess the CTs results.
Panic et al. [30] grant and sponsors were described in 70%, which is more frequent than in our findings. This could be because "Annals Other a 1 2.9 * Sum of several similar variables. Proportions add up to more than 100% because of multiresponse a An author had worked as consultant for the company provided an electronic tool to help make medical decisions.    Similar to our study, the most common source of funding in Lai et al. [31] was public (58.1%), probably because CTs are very expensive. In this study, the significant adverse events were common (64.9%), possibly by not studying safe medical interventions. Moreover the methodological quality is higher in our study, only appear in this review in 45.9% the primary endpoint, and sample size calculation in 39.2%, masked randomization in 29.7%, blind in 12.2%, and "intentionto-treat" in 27%. Ifeacho et al. [32] observed lower rates than ours in relation to the appearance of CT in the title (40%), description of the design (12%), sample size (20%) and random assignment (32%). Moreover their study obtained less frequent results regarding the type of masking (24%), funding-sponsors (28%) and statistical methods (22%). Poolman et al. [33] found fewer cases of an epidemiologist as VARIABLE NAME κ Was the study identified as a clinical trial in the title or subtitle? 1.00 Author nationality 1.00 Country where the study was performed 1.00 Is any earlier related systematic review quoted? 1.00 Is the "a priori" hypothesis explicitly detailed? 1.00 Are there references to an earlier publication of the same clinical trial? 1.00 Clinical trial type with reference to the number of participating centers 0.94 Are an epidemiologist and/or statistician involved as an author? 0.84 Is the collaboration of an epidemiologist and/or statistician indicated? 0.78 Type of funding 0.83 Assessment of outcome ("endpoint") 1.00 Was the clinical trial finalized sooner than scheduled? 1.00 Why the trial was stopped earlier than scheduled specified 1.00 Is "intention to treat analysis" carried out? 1.00 Are the values of "p" or "confidence intervals" (CI) indicated? 0.82 Does it indicate whether a multivariate analysis was performed? 0.83 Is there an agency for data monitoring and safety such as the DSMB? 0.80 Are there serious adverse events in general? 0.88 Were there any amendments to the protocol during the course of the trial? 1.00 If so, has the ethics committee been informed? 1.00 Is it specified whether the trial follows the CONSORT standards? 1.00 Is it mentioned whether an ethics committee has approved the clinical trial? 1.00 If so, is the appropriate ethics committee mentioned? 1.00 Have the potential subjects included in the study been informed of different aspects of the clinical trial? 1.00 If the answer is yes, have they understood this information and has this been confirmed? 1.00 Does it mention if informed consent was obtained? 1.00 Was the informed consent obtained voluntary? 1.00 If "Yes" to question 95, is the type of consent specified, and is it witnessed and documented? 0.96 Does it indicate whether a parent/legal guardian in the case of minors or incompetent adults gave it? 1.00 If the IC was not obtained from legal representatives (not available), and the CT could not be delayed, was it obtained as soon as possible? 1.00 Has the disagreement of the potential subject, either a minor or incompetent, been respected, and IC conformity been given by his/her legal representative? 1.00 Do incompetent or minor subjects, who cannot give IC, participate as a necessary characteristic of the researched group? 1.00 Does it specify any information received by the patient at the time of obtaining informed consent? 0.95 Has the IC request been made under pressure or dependence? 1.00 Following the previous answer, was a properly qualified unrelated person designated in order to obtain consent? 1.00 Did any individual participant withdraw consent at any time during the investigation? 1.00 Were precautions taken on the confidentiality of personal information of subjects participating in the clinical trial? 1.00 Was informed consent to save and use the data obtained in the future given by subjects participating in clinical trials? 1.00 If the answer was "2. No," did a research ethics committee carry out the clinical trial after approval? 1.00 Are people injured by their participation in the study shown? 1.00 If they have been injured, have they been compensated for the harm they suffered? 1.00 If injury is caused to subjects participating, does the protocol cover the repair of such injury, so there are no side effects for any subject? 1.00 In the protocol, are arrangements outlined for the less beneficial interventions after finishing in order to equate all participants? 1.00 At the end, was the intervention identified as most beneficial provided for other participants who had not received it? 1.00 In accordance with the above answer, they were informed of this fact in the informed consent? 1.00 Have you informed all participating subjects of the overall results of the study? 1.00 Was any incentive offered to researchers, participating centers and/or subjects? 1.00 If so, specify the type and amount 1.00 If an incentive was offered, has it been provided and why so or not? 1.00 Is information given on whether there is a conflict of interest? 1.00 Type of information provided in relation to conflict of interest 1.00 Influence of the sponsor in writing, reviewing and/or publication of results 1.00 Is the clinical trial been registered? 1.00 author (9%), multicenter types and following of CONSORT (13%) standards than in our study. We believe it might be because the journals assessed in these studies did not follow CONSORT standards completely.
Anttila et al. [34] performed after CONSORT, obtained similar results to ours on justification, objectives and design.
In 2011 Kiehna et al. [35] found data with rates lower than our study in the justification and design, in contrast with the description of significant adverse events. The sequence of randomization (41%), the analyses of "intention-to-treat" (33%) and masking (19%) were also significantly lower than those included in our review. Augestad et al. [36] observed less frequency of CT registration (25%), funding (95%), performing in PC (70%) and masking type (15.6%). The mean Jadad score was 2, also below our findings. This is perhaps because they assessed more pharmacological studies with questionable quality.
Coinciding with Baskerville et al. [11] we only have access to the published CT and not the original data (that may differ). There is variability between the studies regarding the measurement of the "endpoints", due to the many types of variables found by the study treatment.
Would be desirable a study based on an analysis of disciplines, specialties or a particular subject in the future.
The authors recommend that a greater emphasis on proper implementation of the CONSORT standards, could take an improvement in quality of Family Medicine clinical trials published. Therefore, a deeper insight into obtain a better methodological and ethical quality, it will lead us to further analysis in the evaluation of a new medical treatment before use with patients. In this way, it could avoid many of the side effects or ineffectiveness of new therapies, so that when they were cast safe and effective into the public. This also would be a great saving for the health system costs and potential demands of patients in the future.

Conclusions
We observe that the most frequent CT published in Annals of Family Medicine (2010-2013) are: multicenter >5, with public funding, cluster controlled open parallel, with decentralized masked randomization using an active treatment as a control, in a 6-12 months period; made up of in a biracial patient group and applying a method of medical intervention with positive results in 50% (measured by questionnaire). In 50% the intention-to-treat analyses are performed, and they show serious adverse events and good quality (Jadad 3). Furthermore, IC (verbal or not mentioned), no incentive and no conflicts of interest occur more frequently than in other publications cited. Lastly, we find optimal κ.
Following the CONSORT standards publication, it has increased the overall quality of the CTs published. But there are some areas for improvement in the methodological and ethical quality of the CTs published from 2010 to 2013 in Annals of Family Medicine.