Page Start

  1. Home
  2. Medical top
  3. Related information
  4. The Special Committee for EBM
  5. Evidence Reports of Kampo Treatment 2013
  6. 4. Steps for development of structured abstracts

3. Steps for development

(1) Criteria for reference selection

References that satisfied all of the following 3 criteria were included:

  1. 1) References using Kampo formulations approved for manufacture and sale in Japan
    (excluding in-house formulations such as decoctions, because of unknown quality of the medicines used)
  2. 2) Randomized controlled trials (RCTs), quasi-randomized controlled trials (quasi-RCTs), crossover trials, and meta-analyses
    (including some with randomization procedure not fully indicated. Crossover trials are regarded as RCT)
  3. 3) References published in or after 1986
    Those published in or after 1986 obviously using formulations of 1985 or previous quality, which differs from the current quality levels, were excluded throughout the study period.

(2) Search and screening

Searches were performed using the 3 databases listed below. Screening was performed in 2 steps: first, the references that obviously did not satisfy the criteria were excluded by the search staff; then, the remaining references were reviewed in the process of preparation of structured abstracts described below to finally decide which to include/exclude.

1) The Cochrane Library (C)

Kampo RCTs were searched using the Cochrane Central Register of Controlled Trials (CENTRAL), a worldwide RCT database organized by the Cochrane Collaboration. Since the CENTRAL covers all RCTs in the Medline, searches using the Medline were not performed.

On 26 November 2012, searches were performed by the following search formula, limited to publications in and after 1986:

#1 MeSH descriptor Medicine, East Asian Traditional explode all trees
#2 MeSH descriptor Medicine, Kampo explode all trees
#3 MeSH descriptor Medicine, Chinese Traditional explode all trees
#4 MeSH descriptor Drugs, Chinese Herbal explode all trees
#5 MeSH descriptor Herb-Drug Interactions explode all trees
#6 MeSH descriptor Herbal Medicine explode all trees
#7 MeSH descriptor Plants, Medicinal explode all trees
#8 MeSH descriptor Plant Structures explode all trees
#9 MeSH descriptor Plant Extracts explode all trees
#10 MeSH descriptor Materia Medica explode all trees
#11 MeSH descriptor Phytotherapy explode all trees
#12 (Kampo):ti,ab,kw
#13 (Kanpo):ti,ab,kw
#14 (Japanese):ti,ab,kw
#15 (Oriental):ti,ab,kw
#16 (Traditional):ti,ab,kw
#17 (East Asia):ti,ab,kw
#18 (East-Asia):ti,ab,kw
#19 (Herb*):ti,ab,kw
#20 (Chinese):ti,ab,kw
#21 (#1 OR #2 OR #3 OR #4 OR #5 OR #6 OR #7 OR #8 OR #9 OR #10 OR #11 OR #12 OR #13 OR #14 OR #15 OR #16 OR #17 OR #18 OR #19 OR #20), from 1986 to 2012
#22 (HS-EKAT)
#23 (#21 AND NOT #22)

In October 2011, of the RCT papers on Kampo formulations appearing EKAT 2010 under the "Reference(s)" of structured abstracts, those that had not been previously listed in CENTRAL were so listed and after HS-EKAT tagging, were linked to the EKAT SAs on the JSOM website. The present search excludes papers with "HS-EKAT" in the search formula, so as to exclude those papers related SAs.

Of 20,449 search hits, Kampo references were visually selected, resulting in 121 such references. Among Kampo references resulting from the CENTRAL search, 19 were overlapped with the Ichushi Web hits.

Kampo-related reports accounted for approx. 0.6% of the total hits.

Finally, of 121 references, 74 satisfying the inclusion criteria were compiled as structured abstracts, and 47 were not compiled as structured abstracts but cited as excluded references along with bibliographic items and the reason for exclusion.

2) Igaku Chuo Zasshi (I)

On 19 November 2012, randomized controlled trials of Kampo medicines were searched by the following search formula using Igaku Chuo Zasshi (Japana Centra Revuo Medicana [JCRM], Ichushi) Web (in Japanese). Words in square blackets indicate equivalent English words.

Search formula:
(漢方薬 [Kampo medicine]/TH or漢方 [Kampo]/AL) and (メタアナリシス [meta-analysis]/RD or ランダム化比較研究 [randomized controlled trial]/RD or 準ランダム化比較研究 [quasi-randomized controlled trial]/RD) and (DT=1986: 2012)

Since the Ichushi Web tags meta-analyses, randomized controlled trials, and quasi-randomized controlled trials, the present search targeted references that were tagged (メタアナリシス [meta-analysis]/RD or ランダム化比較研究 [randomized controlled trial]/RD or 準ランダム化比較研究 [quasi-randomized controlled trial]/RD), had "漢方薬 [Kampo medicine]" (漢方薬 [Kampo medicine]/TH) as a keyword (control term), or a title or abstract including the term "Kampo [漢方]" (漢方 [Kampo]/AL), and were published between 1986 and 2012 (DT=1986:2012).

As a result, 243 references were hit (19 of them were overlapped with CENTRAL hits). Of these references, 171 satisfying the inclusion criteria were selected and compiled as structured abstracts (170 RCTs and 1 meta-analysis). Of the references compiled as structured abstracts, 2 included 2 RCTs each, and were therefore compiled as structured abstracts separately for each RCT.

The Ichushi Web assigns the keyword "漢方 [Kampo]" to traditional Chinese medicines and food and Indian medicines as well. These references, combined with those with randomization not for evaluation of Kampo medicine, no-clinical articles, and citations of existing references, totaled 66, which were not compiled as structured abstracts but listed as excluded references along with bibliographic items and the reason for exclusion.

3) Database offered by the Japan Kampo Medicines Manufacturers Association (N)

The database offered by the Japan Kampo Medicines Manufacturers Association (JKMA) (unpublished) is the second database on Kampo and crude drugs, constructed by exhaustively searching existing databases including JST, JAPIC, Igaku Chuo Zasshi, Medline, and EMBASE with the keywords "漢方 [Kampo]," "東洋医学 [Oriental medicine]," "和漢 [Wakan]," "生薬 [crude drug]," "Kampo" and "Chinese Medicine," and hand-searching several tens of Kampo-related journals. At present, the database includes approx. 100,000 references.

The database search on 19 November 2012 covered papers between 1986 and 2012 with the following search terms.

Search terms: the search terms are given in their original form (words in parentheses indicate search terms or hits in Roman characters, and the corresponding meanings in English):

メタアナリシス (meta-anarisisu, meta-analysis); メタ解析(meta-kaiseki, meta-analysis); メタ分析(meta-bunseki, meta-analysis); RCT; ランダム(randamu, random); 無作為, 無作意(musakui, random); 封筒(futo, envelope); 来院順 (raiinjun, order of presentation); 受診順 (jushinjun, order of presentation); 診断順 (shindanjun, order of diagnosis); 割付, 割り付け (;waritsuke, allocation); ブラインド (buraindo, blind); 盲検, 盲験(mouken, blind), 遮蔽, 遮へい, しゃへい (shahei, mask), マスク (masuku, mask); マスキング (masukingu, masking); クロスオーバー (kurosuoba, crossover), 交叉, 交差 (kosa, crossover), 比較臨床 (hikaku rinsho, controlled clinical); random; cross over; meta analysis; envelope

The search included the terms メタ解析 (meta-kaiseki, meta-analysis); メタ分析 (meta-bunseki, meta-analysis); 来院順 (raiinjun, order of presentation); 診断順 (shindanjun, order of diagnosis); and envelope, which were not used for EKAT 2010.

The 11,892 hits underwent primary visual screening, which resulted in 314 papers before SAs were prepared for 265 references that did not overlap with the results of CENTRAL or Ichushi Web searches. SAs were not prepared for 49 references that passed primary screening but did not exactly match the selection criteria. These SAs appear in the list of excluded references along with bibliographic information and the reasons for exclusion.

From these 3 databases combined, 653 references were identified, as shown in Table 1. Of these, SAs were prepared for 494, and 159 appear in the excluded references list along with bibliographic information.

(3) Preparation of structured abstracts

The references satisfying the inclusion criteria were compiled as structured abstracts (SA). Studies on SA started in the 1980s. Here, an 8-item structured abstract format of RCTs, as proposed by Altman, et al. and currently used worldwide, was employed.

Altman DG, Gardner MJ. More informative abstracts. Ann Inter Med 1987; 107(5): 790-1.

Aoki T. Kozoka shoroku no kisochisiki (Basic understanding in structured abstracts). In: Tsutani K, Yamazaki S, Sakamaki H (eds). EBM no tameno joho senryaku – ebidensu wo tsukuru, tsutaeru, tsukau – (Information strategies for EBMgenerate, transfer and use evidence–). Tokyo: Chugai-igakusha; 2000: 82-93 (in Japanese).

Here, 8 items are referred to as, (1) objectives, (2) design, (3) setting, (4) participants, (5) intervention, (6) main outcome measures, (7) main results, and (8) conclusions.

These 8 items are widely used in medical journals such as JAMA and secondary information journals such as Evidence Based Medicine, ACP Journal Club, as well as in secondary information journals on traditional medicine and complementary and alternative medicine, represented by Focus on Alternative and Complementary Therapies (FACT). The acupuncture part of the journal is available in Japanese (Tsutani K [supervise-trans]. Hari no ebidensu [Evidence for acupuncture – abstracts of articles on clinical evaluation of acupuncture–]. Yokosuka: Ido-no-Nippon-sha; 2003 [in Japanese]. Thereafter, serialized in the journal “IDO-NO NIPPON [The Japanese Journal of Acupuncture & Manual Therapies]” [in Japanese]).

If one SA has been based on multiple references, all the bibliographic information for references used for the SA appears at the top of the SA, in order of publication, with the principle reference appearing in bold type.

Regarding (5) intervention, since the quality may differ among manufacturers, the brand name indicated in the original article was to be used as a rule. When the brand name was changed after the issue of the article for such reason as a change in manufacturer name, the brand name indicated in the article was used.

In the structured abstracts in the present “Evidence Reports of Kampo Treatment,” the following 4 items were added to the above world-standard 8 items: (9) from Kampo medicine perspective; (10) safety assessment in the article; (11) abstractor’s comments; and (12) abstractor and date. These are described below:

9) “From Kampo medicine perspective” means how the unique diagnosis system of Kampo medicine is used. This is applied to 2 processes: design of a clinical trial and analysis after completion of the study. With RCT, this can be referred to as pre-randomization and post-randomization. In the former process of designing clinical trial design, “sho (証, pattern)” of Kampo medicine is indicated in the entry criteria and exclusion criteria of participants in the protocol in a manner that participating investigators can understand. The latter process involves stratified analysis, in which participants are stratified by “sho” (with sho or without sho), as well as by age and sex, etc. However, the stratified analysis is associated with “inference multiplicity”; that is, repeated testing of many strata produces false positive results, which indicates a difference when actually there is no difference. Among post-hoc approaches are adjustment for covariates.

10) Safety assessment in the article” was incorporated since not only efficacy but also safety should always be considered in Kampo medicine as well. Here, the expression of “safety assessment in the article” was used rather than mere “safety assessment” because “safety assessment” is frequently misunderstood. RCTs usually use efficacy-related endpoints to determine appropriate sample sizes, and are not intended to assess safety. For instance, when no adverse drug reactions occurred among 100 subjects receiving a Kampo medicine, the Kampo medicine is apt to be considered “safe because there are no adverse drug reactions when used in as many as 100 subjects.” Certainly, point estimation yields an incidence of 0%; however, interval estimation yields a 95% confidence interval (CI) for the incidence of 0-3%. Especially when a serious adverse drug reaction may infrequently occur, safety should be judged in consideration of the number of participants in the clinical trial in the article. From this perspective, the expression of “safety assessment in the article” was used.

Fig. 1 shows 95% CI in the absence of adverse events by the number of participants. iThe values are 0-26%, 0-14%, 0-6%, 0-3%, and 0-0.6% for 10, 20, 50, 100, and 500 subjects. The “law of 3” approximately holds around over 50 subjects. The confidence limit, values on both sides of a confidence interval, was constantly 0 on the left and 1-0.051/n on the right. For information, adverse event (AE) means “any untoward medical occurrence, whether or not related to the medical product,” and AE with “causal relationship with the medical product that cannot be ruled out” is referred to as adverse drug reaction (ADR).

Here, the above explanation including Fig. 1 was based on the assumption that the incidence of AE was 0. When “AE occurred in m of n subjects,” however, the confidence interval can be determined by the equation below as a binominal distribution. This equation, which utilizes normal approximation of binominal distribution, produces larger error with smaller sample size (n < 25).

95% confidence interval for event incidence=p±1.96√(p(1-p) /n), where p=m/n.

Although normal approximation was used for the above equation, there are other calculation systems easily available on the Internet, as represented by

Fig. 1 Confidence interval for population mean of event incidence in the absence of events in any

In the present evidence report, indication as “safety assessment included in the article” was standardized as follows:

1) Without indication
When safety assessment was not performed or indicated, "none" was indicated.
2) With indication
When safety assessment was performed, even if only slightly, and revealed no adverse drug reactions, that effect was indicated. When adverse drug reactions were specifically indicated, the abstractor in charge indicated this according to the expression used in the reference. When the number of patients with adverse drug reactions was specified in the reference, it was indicated. Note that some indications of adverse drug reactions are not unified.

11)Abstractor’s comments” refers to objective comments on a reference presented as a structured abstract. This helps busy readers not used to critical appraisal correctly and easily judge the value of the article. Abstractors were selected such that the abstractor did not belong to the same group of the authors concerned or have a master-student relationship, in consideration of conflict of interest (COI). How comments should be made was most actively discussed among members of the task force. With the aim of improving and standardizing comments, the 2nd Workshop of the Task Force for Evidence Report “To prepare appropriate comments” was held in conjunction with the Japan Society for Oriental Medicine 58th Annual Meeting held in Hiroshima on 17 Jun. 17, 2007.

Tsuruoka H, Okabe T, Tsutani K. Kampoyaku RCT no kozokashoroku niokeru komento kisai no kaizen - Dai 2-kai ebidensu repoto tasuku fosu wakushoppu hokoku -(Improvement of indication of comments in structured abstracts of RCTs on Kampo medicine-the Report of the 2nd Workshop of the Task Force for Evidence Report-. Nihon Toyo Igaku Zasshi [Kampo Medicine] 2009: 60: 177-84 (in Japanses).

12)Abstractor and date” is intended to clarify the responsibility, which also concerns the abovementioned conflict of interest, and to show the temporary relationship between the comments and related studies and in consideration of the possibility of correction at a later date. When a structured abstract was revised, the date of revision was added.

Structured abstract preparation tasks were assigned to Task Force members in consideration of their specialties. However, since the specialties of members did not cover the whole field, some abstracts were prepared by non-specialized members.

Bibliographic items were indicated in the Vancouver style as a rule, with some modifications, including that the number of authors listed shall be up to 3 and that the name of a journal shall not be abbreviated.

Structured abstracts were arranged in the order of ICD10 (Version 2003) code of diseases. Those with the same code were arranged by date of publication of the main reference evaluated. When more than one ICD code was possible, the one seeming to be generally more understandable was selected. Similarly, excluded references were arranged in the order of ICD10 code. The names of Kampo formulae, etc. that could not be written in Chinese characters for daily use in Japan were written in Katakana. The names of ICD code diseases differ from general names and were therefore read as shown in Table 2 to indicate them in this report.

Table 2

In preparing structured abstracts, to maintain the quality, a Structured Abstract Preparation Manual was prepared, distributed to Task Force members, and updated as appropriate.


Page End

Page Top