(1) Criteria for reference selection

References that satisfied all of the following 3 criteria were included:

1. 1) References using Kampo formulations (extract and pill) approved for manufacture and sale in Japan
   (excluding in-house formulations such as decoctions, because of unknown quality of the medicines used)
2. 2) Randomized controlled trials (RCTs), quasi-randomized controlled trials (quasi-RCTs), crossover trials, and meta-analyses
   (including some with randomization procedure not fully indicated. Crossover trials are regarded as RCT)
3. 3) References published in or after 1986
   Those published in or after 1986 obviously using formulations of 1985 or previous quality, which differs from the current quality levels, were excluded throughout the study period.

(2) Search and screening

Searches were performed using the 3 databases listed below. Screening was performed in 2 steps: first, the references that obviously did not satisfy the criteria were excluded by the search staff; then, the remaining references were reviewed in the process of preparation of structured abstracts described below to finally decide which to include/exclude.

1) The Cochrane Library (C)

2) Igaku Chuo Zasshi (I)

3) Database offered by the Japan Kampo Medicines Manufacturers Association (N)

From these 3 databases combined, 549 references were identified, as shown in Table 1. Of these, 416 were compiled as structured abstracts, and 132 were listed as excluded references along with bibliographic items. Of the references compiled as structured abstracts, 4 included 2 RCTs each, and were therefore compiled as structured abstracts separately for each RCT.

(3) Preparation of structured abstracts

The references satisfying the inclusion criteria were compiled as structured abstracts (SA). Studies on SA started in the 1980s. Here, an 8-item structured abstract format of RCTs, as proposed by Altman, et al. and currently used worldwide, was employed.

Altman DG, Gardner MJ. More informative abstracts. Ann Inter Med 1987; 107(5): 790-1.

Aoki T. Kozoka shoroku no kisochisiki (Basic understanding in structured abstracts). In: Tsutani K, Yamazaki S, Sakamaki H (eds). EBM no tameno joho senryaku – ebidensu wo tsukuru, tsutaeru, tsukau – (Information strategies for EBM – generate, transfer and use evidence–). Tokyo: Chugai-igakusha; 2000: 82-93 (in Japanese).

Here, 8 items are referred to as, 1) objectives, 2) design, 3) setting, 4) participants, 5) intervention, 6) main outcome measures, 7) main results, and 8) conclusions.

These 8 items are widely used in medical journals such as JAMA and secondary information journals such as Evidence Based Medicine, ACP Journal Club, as well as in secondary information journals on traditional medicine and complementary and alternative medicine, represented by Focus on Alternative and Complementary Therapies (FACT). The acupuncture part of the journal is available in Japanese (Tsutani K [supervise-trans]. Hari no ebidensu [Evidence for acupuncture – abstracts of articles on clinical evaluation of acupuncture–]. Yokosuka: Ido-no-Nippon-sha; 2003 [in Japanese]. Thereafter, serialized in the journal “IDO-NO NIPPON [The Japanese Journal of Acupuncture & Manual Therapies]”).

Regarding 5) intervention, since the quality may differ among manufacturers, the brand name indicated in the original article was to be used as a rule. When the brand name was changed after the issue of the article for such reason as a change in manufacturer name, the brand name indicated in the article was used.

In the structured abstracts in the present “Evidence Reports of Kampo Treatment,” the following 4 items were added to the above world-standard 8 items: 9) from Kampo medicine perspective; 10) safety assessment in the article; 11) abstractor’s comments; and 12) abstractor and date. These are described below:

9) “From Kampo medicine perspective”means how the unique diagnosis system of Kampo medicine is used. This is applied to 2 processes: design of a clinical trial and analysis after completion of the study. With RCT, this can be referred to as pre-randomization and post-randomization. In the former process of designing clinical trial design, “sho (証, pattern/syndrome)” of Kampo medicine is indicated in the entry criteria and exclusion criteria of participants in the protocol in a manner that participating investigators can understand. The latter process involves stratified analysis, in which participants are stratified by “sho” (with sho or without sho), as well as by age and sex, etc. However, the stratified analysis is associated with “inference multiplicity”; that is, repeated testing of many strata produces false positive results, which indicates a difference when actually there is no difference. Among post-hoc approaches are adjustment fo covariates.

10) Safety assessment in the article” was incorporated since not only efficacy but also safety should always be considered in Kampo medicine as well. Here, the expression of “safety assessment in the article” was used rather than mere “safety assessment” because “safety assessment” is frequently misunderstood. RCTs usually use efficacy-related endpoints to determine appropriate sample sizes, and are not intended to assess safety. For instance, when no adverse drug reactions occurred among 100 subjects receiving a Kampo medicine, the Kampo medicine is apt to be considered “safe because there are no adverse drug reactions when used in as many as 100 subjects.” Certainly, point estimation yields an incidence of 0%; however, interval estimation yields a 95% confidence interval (CI) for the incidence of 0-3%. Especially when a serious adverse drug reaction may infrequently occur, safety should be judged in consideration of the number of participants in the clinical trial in the article. From this perspective, the expression of “safety assessment in the article” was used.

Fig. 1 shows 95% CI in the absence of adverse events by the number of participants. iThe values are 0-26%, 0-14%, 0-6%, 0-3%, and 0-0.6% for 10, 20, 50, 100, and 500 subjects. The “law of 3” approximately holds around over 50 subjects. The confidence limit, values on both sides of a confidence interval, was constantly 0 on the left and 1-0.051/n on the right. For information, adverse event (AE) means “any untoward medical occurrence, whether or not related to the medical product,” and AE with “causal relationship with the medical product that cannot be ruled out” is referred to as adverse drug reaction (ADR).

Here, the above explanation including Fig. 1 was based on the assumption that the incidence of AE was 0. When “AE occurred in m of n subjects,” however, the confidence interval can be determined by the equation below as a binominal distribution. This equation, which utilizes normal approximation of binominal distribution, produces larger error with smaller sample size (n < 25).

95% confidence interval for event incidence＝p±1.96√(p(1-p) /n), where p=m/n.

Although normal approximation was used for the above equation, there are other calculation systems easily available on the Internet, as represented by
 http://www.ec.kagawa-u.ac.jp/~hori/delphistat/binom.html.

In the present evidence report, indication as “safety assessment included in the article” was standardized as follows:

1) Without indication

When safety assessment was not performed or indicated, “none” was indicated.

2) With indication

When safety assessment was performed, even if only slightly, and revealed no adverse drug reactions, that effect was indicated. When adverse drug reactions were specifically indicated, the abstractor in charge indicated this according to the expression used in the reference. When the number of patients with adverse drug reactions was specified in the reference, it was indicated. Note that some indications of adverse drug reactions are not unified.

11)Abstractor’s comments” refers to objective comments on a reference presented as a structured abstract. This helps busy readers not used to critical appraisal correctly and easily judge the value of the article. Abstractors were selected such that the abstractor did not belong to the same group of the authors concerned or have a master-student relationship, in consideration of conflict of interest (COI). How comments should be made was most actively discussed among members of the task force. With the aim of improving and standardizing comments, the 2nd Workshop of the Task Force for Evidence Report “To prepare appropriate comments” was held in conjunction with the Japan Society for Oriental Medicine 58th Annual Meeting held in Hiroshima on 17 Jun. 17, 2007.

Tsuruoka H, Okabe T, Tsutani K. Kampoyaku RCT no kozokashoroku niokeru komento kisai no kaizen － Dai 2-kai ebidensu repoto tasuku fosu wakushoppu hokoku －(Improvement of indication of comments in structured abstracts of RCTs on Kampo medicine－the Report of the 2nd Workshop of the Task Force for Evidence Report－. Nihon Toyo Igaku Zasshi [Kampo Medicine] 2009: 60: 177-84 (in Japanses).

12)Abstractor and date” is intended to clarify the responsibility, which also concerns the abovementioned conflict of interest, and to show the temporary relationship between the comments and related studies and in consideration of the possibility of correction at a later date. When a structured abstract was revised, the date of revision was added.

Structured abstract preparation tasks were assigned to Task Force members in consideration of their specialties. However, since the specialties of members did not cover the whole field, some abstracts were prepared by non-specialized members.

Bibliographic items were indicated in the Vancouver style as a rule, with some modifications, including that the number of authors listed shall be up to 3 and that the name of a journal shall not be abbreviated.

Structured abstracts were arranged in the order of ICD10 (Version 2003) code of diseases. Those with the same code were arranged by date of publication of the main reference evaluated. When more than one ICD code was possible, the one seeming to be generally more understandable was selected. Similarly, excluded references were arranged in the order of ICD10 code. The names of Kampo formulae, etc. that could not be written in Chinese characters for daily use in Japan were written in Katakana. The names of ICD code diseases differ from general names and were therefore read as shown in Table 2 to indicate them in this report.

In preparing structured abstracts, to maintain the quality, a“Structured Abstract Preparation Manual”was prepared, distributed to Task Force members, and updated as appropriate.

PageTop

• Lists of structured abstracts
• Lists of excluded references
• History of version upgrades
• Notes on the current version
• Organization
• Excecutive summary
• 1. Background
• 2. Purpose
• 3. Steps for development
• 4. Included and excluded references
• 5. Relation to other projects
• 6. Conflict of interests
• 7. Acknowledgements
• 8. Contact point
• Full Text (PDF)

• released 4 June 2010
• last update 1 July 2010