Recommendations for improving the quality of reporting clinical electrochemotherapy studies based on qualitative systematic review

Electrochemotherapy is becoming a well-established non-thermal ablative technique for malignancies of skin and non-skin origin.1,2 The medical applications of electrochemotherapy are based on the principle of electroporation, which dates back to 1982, when sequences of electric pulses were applied to deliver naked DNA molecules within mouse lyoma cells.3 Preclinical studies carried out by several research groups, coupled with technical developments, culminated in the clinical application of electroporation during the early 1990s.4–13 These initial data on electroporative uptake of molecules are viewed as seminal for various biotechnological and medical applications.14,15 The principle of electrochemotherapy is the use of electroporation to enhance chemotherapeutic drug delivery. Two agents, bleomycin or cisplatin, can achieve a several fold increase in their intracellular availability, and consequently cytotoxicity, when the tumor tissue is exposed to reversible electroporation and transient cell membrane permeabilization, thus achieving an optimal intratumor drug distribution.7,16–18 Electrochemotherapy has proven effective for the treatment of different tumor his-totypes, including both skin and non-skin cancers, as well as for the palliation of metastases involving cutaneous and subcutaneous tissues.19–22 The treatment of primary skin tumors is largely restricted to multifocal cutaneous tumors, most notably some selected cases of basal cell carcinoma, when tumor anatomical location and patient medical conditions contraindicate more aggressive treatments.23

The publication of the European Standard Operating Procedures of Electrochemotherapy (ESOPE) in 2006 facilitated a broad acceptance of electrochemotherapy for treatment of cutaneous tumors and metastases.24 Over a number of years, several clinical reports have confirmed its effectiveness. Interestingly, the vast majority of studies used the Standard Operating Procedure (SOP) as a guideline for electrochemotherapy. The availability of SOP allowed for reproducibility and improvement of results in the clinical practice. Several large follow up series confirmed the efficiency of electrochemotherapy. A recent meta-analysis of the use of electrochemotherapy in the treatment of cutaneous metastasis places it well amongst other, more established, treatment options.2 Recently, electrochemotherapy has also been recognized by the National Institute for Health and Care Excellence (NICE) as an integral part of the multidisciplinary treatment for patients with skin metastases of non-skin origin and melanoma (NICE interventional procedure guidance IPG 446, http://www.nice.org.uk/guidance/ipg446). More recently, electrochemotherapy has been introduced into the treatment of deep-seated and endoluminal tumors.25–28 The first clinical report on visceral metastases indicates its effectiveness, and suggests a possible role of electrochemotherapy for the treatment of liver metastases, especially when located close to major blood vessels and when not manageable with surgery or other ablative techniques.29

Overall, literature data from Web of Science database indicate a steady increase in number of publications and their citations under the key word “electrochemotherapy” (Figure 1A,B) and “clinical electrochemotherapy” (Figure 1C,D). Despite a steady increase in the number of published reports, a higher quality and standardization of reported studies is needed to improve and support a truly evidence based practice. In our study we only included papers published after 2006, specifically only to include reports published after the Standard Operating Procedures (SOP).24

Figure 1.

The purpose of this recommendation paper is to provide practical recommendations in order to improve the precision of reported clinical studies on electrochemotherapy (a summary checklist is provided as Supplementary file). This, in turn, we hope will stimulate the scientific community to report research using these guidelines to give comprehensive reports on areas including study design, definition of study endpoints, patient selection criteria, treatment plan and outcome assessment. The adoption of more precision in reporting will enable researchers and clinicians to perform more meaningful outcome comparisons with other ablative techniques, to clarify the direction for future research, and to produce more evidence-based practice. It is our hope that these advancements may improve patient selection, resource allocation, and ultimately patient outcome.

This report was prepared based on initiative of the Steering Committee of the COST TD 1104 Action (www.electroporation.net) and in response to a general call for increased awareness and concern for low quality reporting practice30; moreover, it has been prepared by the committee within the Working group of Medical applications of electroporation, in COST action TD 1104 EP4Bio2Med, and is included in the series of publications addressing the same topic in preclinical research in electroporation as well as in the pulsed electric fields for industrial purposes.31

Several guidelines exist with the aim of assuring sound research practices, and improving the quality of clinical trials and, ultimately, allow for generalizable results. At a basic level, Good Clinical Practice (GCP) represents an international ethical and scientific quality standard for designing, conducting, recording and reporting trials that involve the participation of human subjects. At a higher level, dedicated guidelines and recommendations have been developed according to the specific type of study performed. For instance, the STROBE statement (www.strobe-statement.org) indicates a checklist for details that should be reported in observational trials; the Consolidated Standards of Reporting Trials (CONSORT) statement (www.consort-statement.org/consort-statement/) provides guidance for reporting the aim, methods, results and implications of randomized controlled trials; the PRISMA statement (www.prisma-statement. org) indicates preferred reporting for systematic reviews; finally, the REporting recommendations for tumor MARKer prognostic studies (REMARK, www.equator-network.org/reporting-guidelines/reporting-recommendations-for-tumour-marker-prognostic-studies-remark/) suggest guidelines to provide relevant information about study design, preplanned hypotheses, patient and specimen characteristics, assay methods, and statistical analyses when evaluating tumor markers in oncology. In addition, these guidelines provide helpful suggestions on how to present data and important elements to include in discussions. Although these guidelines provide a fundamental guidance for conducting a valid clinical trial and reporting generalizable findings, nonetheless it is recognized that there is a need for specialty-specific guidelines and that these guidelines will lead to improvement in the quality of reports and to higher impact publications.32,33 In the field of electrochemotherapy, comprehensive meta-analyses or Cochrane style reviews of efficiency are hampered by the lack of some relevant clinical data in published reports. Therefore, we evaluated the published papers on clinical electrochemotherapy and identified possible pitfalls in data reporting. On this basis, we prepared recommendations for improving the quality of future studies and fostering further rational development of electrochemotherapy.

A total of 56 papers were initially identified. Of these, only 33 reports were finally retained in the qualitative synthesis; the reasons for exclusion of the remaining reports are listed in Figure 2.

Figure 2.

A summary of the studies included in the final analysis is presented in Table 2.20–22,29,37–65 The total number of patients across all studies was 1215. Electrochemotherapy protocol was following the SOP as defined in ESOPE study in all but two cases.40,65

The majority (24/33) of reports were single-center studies. There were 24 tumor-specific studies (melanoma, n=8; breast cancer, n=5; head and neck squamous cell carcinoma, n=4; Kaposi sarcoma, n=3; pancreatic cancer, n=1; colorectal cancer, n=1; soft tissue sarcomas, n=1; vaginal squamous cell cancer, n=1) and 9 studies including heterogeneous histologies. Response assessment was based on clinical evaluation in all except 3 studies on pancreatic cancer41, liver metastases from colorectal cancer29, and chest wall recurrence from breast cancer57, where response assessment was radiological (ultrasound scan, magnetic resonance imaging, computed tomography, or fluorine-18-deoxyglucose PET-CT scan). Details of the quality criteria used to assess trial design are presented in Figure 3. Less than half (15/33, 45%) of studies were prospective and only two of them (6%) were entered into a publicly accessible clinical trials registry.29,57 Eighteen percent (6/33) of papers represented the report of a multicenter study. There was a single comparative trial (an internally controlled study with intrapatient randomization of melanoma metastases to intralesional bleomycin versus intralesional bleomycin followed by electric pulses)65; a formal sample size calculation or analysis of “intent-to-treat” population was found in only 4/33 (12%) studies.20,50,57,65

Figure 3.

Details of the quality criteria used to assess the description of patient population are presented in Figure 4. Treated tumors were described in detail in most reports: number of tumors, 94%; tumor location, 100%; tumor histotype, 100%; tumor size, 91%. On the other hand, additional clinical information was less frequently reported: study setting -palliative/curative-, 54%; presence of visceral metastases, 54%; concomitant oncologic treatments, 45%.

Figure 4.

Details of the manuscript quality criteria used to assess the description of treatment delivery and response assessment are presented in Figure 5. Treatment details were accurately described in most reports: type of anaesthesia, 32/33 (97%); drugs, 33/33 (100%); pulse generator, 33/33 (100%); electrode types, 31/33 (93%); electric pulse parameters, 32/33 (97%). The criteria for response assessment were clearly stated in 29/33 (88%) of studies, while toxicity criteria were indicated in only 14/33 (42%) of papers.

Figure 5.

Details of the quality criteria used to assess the analysis of results and their interpretation are presented in Figure 6. The majority of reports included a critical analysis: interpretation of results, 33/33 (100%); comparison to historical control, 25/33 (76%); indication of possible future directions, 33/33 (100%); conflict of interest statement, 27/33 (82%). On the contrary, only a minority of them fulfilled other specific quality criteria: summary of primary and secondary endpoints, 13/33 (39%); indication of predictive factors, 9/33 (27%); additional patient outcome parameters, 9/33 (27%).

Figure 6.

Based on the results of this analysis, the consensus between authors was to recommend some minimal requirements for reporting clinical data in future studies.

Electrochemotherapy was initially used with palliative intent. First trials demonstrated remarkable efficiency in the treatment of skin metastases from malignant melanoma.21,60,63 Subsequently, electrochemotherapy was also evaluated for the treatment of other tumor histotypes (e.g., non-melanoma skin cancers and cutaneous metastases from other tumor histotypes) with equally high success.20,37–38,47,57–58 Reports of small series indicate also its possible usefulness in the treatment of primary basal cell carcinomas23 and a clinical trial is currently ongoing comparing the effectiveness of electrochemotherapy to standard surgical resection and is due to report 5 year follow up data next year (EudraCT Number: 2010-019260-37). A particular advantage of electrochemotherapy is that it is a reliable alternative treatment option for patients who have exhausted more conventional oncological treatments or are judged unfit for or refuse repetitive surgical interventions.47 Therefore, future reports need to include detailed description of patient’s demographic and clinical data including detailed description of previous treatments. A detailed description of tumor location, histotype as well as number and size of the electrochemotherapy target and non-target lesions is paramount. Authors should also specify whether targeted lesions had previously received irradiation or not, whether visceral metastases are present and whether the treatment is intended as palliative or curative. Additionally, since electrochemotherapy is finding its place among other oncologic treatments, and will be increasingly used also in combination with them, an accurate record of concomitant treatments is also advisable.66

Key elements of patient population:

The treatment is applied by performing a procedure conjugating the administration of a drug and local application of electric pulses. In one “session” or “cycle” a single or several tumor nodules can be treated. Since the procedures can be repeated on the same and also on newly emerged tumor nodules, patient treatment may require one or more sessions of electrochemotherapy. Therefore, reports should clearly indicate how many sessions (or cycles) were needed for the treatment of baseline tumors and, overall, for patient management. If retreatment is necessary, the indication should be clear, detailing previous response and disease status in target and non-target tumors. In order to ensure the maximum efficacy, electrochemo-therapy needs two key elements: the presence of a cytotoxic agent within tumor tissue and the adequate coverage of tumor with electric pulses above the threshold of reversible membrane electroporation.67 The results of the ESOPE study and the adoption of SOP that were prepared within the ESOPE project (QLK-2002-02003) were of great importance for the development of electro-chemotherapy.68,24 In fact, they provided practical guidelines and standardization of the procedure. The clinical data evaluation demonstrated that the use of guidelines and a standardized protocol enabled to reach the same level of effectiveness also in the centers without previous experience with electrochemotherapy.17 The ESOPE study provided evidence for electrochemotherapy in the treatment of skin metastases of different histotypes.68 It included use of bleomycin or cisplatin as chemotherapeutics, different routes of administration (intravenous or intratumoral) and the use of either local or general anaesthesia. The pulse parameters (number, sequence and amplitudes) for different electrodes were however well defined within the Cliniporator project.45 A specific electric pulse generator has been consistently used with different electrodes, according to the size, depth and anatomical location of treated tumors.

As confidence with the procedure has developed, treatment indications have also widened. The first studies were based on patients with tumors less than 3 cm, however lesions greater than 3 cm are now routinely treated57,62,69, representing a natural development of the field based on success with smaller tumors for which the ESOPE guidelines were prepared. As such, there is a need to adapt and revise the SOP and this is already underway. Furthermore, new producers of electric pulse generators are coming to the market, and new electrodes with different design for different treatment settings are emerging. All these changes will make the clinical data evaluation even more challenging. First of all, to address this topic, future reports will need to state the type of anesthesia used (local or general; drugs and doses), the chemotherapeutic agent, drug concentration and dose used, which both depend on the route of administration. The duration of bolus injection, as well as time interval between the drug administration and application of electric pulses, should be specified. The type of electric pulse generator as well as the type of electrodes and their manufacturers should be reported. Additional information should include if the pulse generator is under software control and the specification of the version of that software. If new types of electrodes are used, a detailed description of the design and the sequence and amplitude of pulses is needed. It must be clearly stated whether applied electrodes are needle or plate, the distance between the electrodes, their shape and size, the amplitude of applied electric pulses, their duration, number and repetition frequency. Furthermore, the total number of pulse deliveries, as well as the time interval required for electrode applications after drug injection, should be specified. Additionally, the report of adequate or inadequate coverage of the tumor as well as the way the pulses are applied (e.g., from the margins to the tumor centre or if the pulses were applied in 4+4 (perpendicular) configuration each time) would also be advisable, when possible (Figure 7).70

Figure 7.

Electrochemotherapy has a high therapeutic index, therefore after successful treatment minimal damage is observed on normal surrounding tissues. During treatment, it is also possible for the treating physician to include a safety margin around the target tumor, depending on tumor size, biologic aggressiveness and propensity for developing satellite lesions such as in the case of malignant melanoma or soft tissue sarcomas. In order to improve reporting, the information about the safety margins and their extent should also be reported. In addition, electrochemotherapy can be repeated several times (however there is a ceiling for the total lifetime dose of bleomycin), according to tumor response and disease behavior.62,63 This fundamental aspect is not covered by the currently available SOP. For providing a more informative report, data regarding repetitive treatments should be included, along with the description on what basis the retreatment was performed and at what time interval.

Key elements on treatment information:

^* Legend: n = number; T = duration of pulses; U = voltage amplitude applied; I = measured current; f = pulse repetition frequency

The early studies on electrochemotherapy antitumor activity have carefully evaluated the response of treated tumors. Response assessment was initially performed by the bidimensional WHO criteria.71 According to these criteria, baseline and post-treatment tumor size is determined by bidimensional measurements e.g. the sum of the two longest diameters in the perpendicular dimensions. The tumor response to treatment is divided into four categories (complete response, partial response, stable disease, progressive disease, according to the change from baseline tumor measurement).

Indeed, most past studies were focused on tumor response and on patient early outcomes. Nevertheless, a number of reports indicate that the disease locally relapsed or progressed elsewhere, but only few reports indicated the value of electro-chemotherapy in the local management of patient symptoms. Hence, the clinical benefit for patients, especially in the palliative setting, where preservation of quality of life (QoL) and evaluation of patient reported outcome (PRO) are crucial, should be based on dedicated assessments and described.

The new RECIST (Response Evaluation Criteria in Solid Tumors) version 1.1 criteria, with some adaptations, have proven a suitable tool for response assessment of superficial tumors21,72, whereas for the setting of treatment of deep-seated tumors (i.e., electrochemotherapy application on liver metastases) the modified RECIST criteria represent the most appropriate and standardized method for the evaluation of tumor response.73 In general, for standard electrochemotherapy on superficial tumors, the RECIST 1.1 criteria, which are based on one-dimensional measurements, seem even more practical and offer highly concordant response assessment compared with the bidimensional WHO criteria.74

So far, most of the published papers do not report on any serious treatment related adverse event after electrochemotherapy. Nevertheless, the process surrounding the determination, recording and reporting of adverse events remains moderately challenging especially for the clinician who may not be involved in drug or device-related research. Nevertheless, it is important to understand the basic definitions of adverse events reporting in order to ensure that the proper information is collected in clinical protocols. Moreover, a comprehensive patient observation and a detailed report of all types and grades of toxicities are essential for providing a comprehensive report of treatment outcome, not only in the early, but also in the long-term follow-up. In this way, only large cohorts of patients will enable in-deep view of long-term toxicity and more detailed analyses of treatment-related adverse events according to different patients subgroup, as demonstrated by a recently published report on electrochemotherapy-related pain.43 For this purpose, Common Terminology Criteria for Adverse Events (CTCAE v4.0) is widely accepted throughout the oncology community as the standard classification and severity grading scale for adverse events. Unfortunately, most of the studies conducted so far do not report consistently on this crucial aspect.

Key elements of treatment outcome assessment:

A clear summary of the trial endpoints is essential. In fact, the field is moving beyond simply reporting on tumor control, as treatment now includes, in some instances, also primary tumors. Here it is important to report and discuss other parameters, such as time to local/systemic progression and, if possible, also the patient survival time and QoL as well. Such data will increase the evidence level of electrochemotherapy effectiveness, and consolidate a role for electrochemotherapy outside the palliative setting and into a confirmed primary treatment modality.

It has been clearly demonstrated that tumor size is the most reliable predictive factor for response in patients who underwent electrochemo-therapy.21,22,69 In future, detailed reports including data on previous local therapies (e.g., radiation) as well as on local (within electrochemotherapy field) tissue status (e.g., presence of lymphedema or fibrosis) and concomitant/adjuvant oncologic treatments would allow for the identification of other reliable predictive indicators for response.

Key elements for analysis and interpretation of the results:

Electrochemotherapy represents an effective treatment option for an increasing number of cancer patients with superficial tumors. Nevertheless, to further improve its evidence basis, it will be crucial to raise the quality of future reports.

In this study, we have highlighted some relevant aspects of clinical data reporting, with the aim of improving the quality of future studies in the field of electrochemotherapy. Although a large amount of data are published so far, clinical research needs to adopt detailed and accurate reporting as well as moving from small, non-comparative series to well-designed, possibly randomized, clinical trials. Despite the encouraging results indicated, the vast majority of included reports are case series from single institutions. Although there was a wide consensus to use previously published SOP for the treatment protocol, these studies often present a variety of designs and reporting methods, thus limiting the understanding of patient selection, treatment effect, toxicity and overall patient outcome. Of note, published studies often lack sufficient procedural as well as patient data. These shortcomings represent a major hurdle to performing systematic reviews or meta-analysis, which may provide a more robust evaluation of treatment effectiveness and, ultimately, encourage wider acceptance of electrochemotherapy in the clinical practice.

Our study has some limitations. We identified a set of manuscript quality criteria from available literature and we have expanded this list by including additional, procedure-specific criteria that were discussed and agreed among the authors. The list of 47 quality criteria that were used for reviewing published reports represents an arbitrary selection of criteria performed by a relatively small number of authors. There is potential for selection bias in the inclusion of papers for analysis, as the initial screen was based on broad, non-selective inclusion criteria. However, we feel that these were widely inclusive and fitting in order to develop the proposed recommendations. Nevertheless, we believe that our suggestions largely cover the most crucial aspects, which are required to improve the quality of clinical practice and future research: trial design and conduction, definition of study endpoints, patient selection, treatment delivery, patient management and follow-up, standardization of outcome assessment. Our recommendations are open to a broader discussion with the community users of electrochemotherapy and, possibly, to further improvements in line with other interventional oncology procedures.75,76 Electrochemotherapy requires standardization of terminology and reporting criteria to facilitate effective communication among researchers and appropriate comparison between different treatment technologies. As such, investigators involved in this field should be familiar with these recommendations and use them for future study design and conduction, treatment application as well as data reporting. We envision that the adoption of these recommendations will further improve the quality of future studies and allow more meaningful comparisons of outcome data of patients treated with electrochemotherapy (Supplementary file).