We searched MEDLINE via PubMed and SCOPUS databases for articles. We chose those databases because they provide the most thorough coverage in the health education field 12.

The selection criteria were as follows:

· articles;

· published between January 2000 to last searched date (May 2010);

· in French or English;

· addressed the concept of transferability defined, even implicitly, as the extent to which the measured effectiveness of an applicable intervention could be achieved in another setting 3;

· concerned health education interventions 13

We defined a list of keywords using semantic progressive steps, expanding the search to terms proposed in the Medical Subject Heading Terms’ (MESH) terminology framework: Translation, Diffusion, Dissemination, External validity, Adaptation, Generalization, Generalizability. We also searched for articles with the keywords [“transferability” OR “generalizability” OR “generalization” OR “translation” OR “diffusion” OR “dissemination” OR “external validity” OR “adaptation”] AND [“health promotion” OR “public health” OR “health education”] AND [intervention OR program].

We selected articles by reading the titles and abstracts and, if necessary, the full text.

The full text of the selected articles was analyzed using a specifically developed grid that explored the four research questions. The articles were independently analyzed by two of the authors (LC, FA). In cases of disagreement, the readers performed a third reading together.

We followed the PRISMA checklist 14 in carrying out the study and preparing the manuscript.

The search identified 3,143 abstracts. We excluded 3,100 abstracts because they:

· did not relate to a health education intervention (indeed, we chose “public health” and “health promotion” as keywords to ensure retrieval of all abstracts addressing health education) (1,139 articles)

· addressed the transfer of knowledge, skills, and practices, in particular in abstracts retrieved with the keywords “dissemination” and “diffusion” (797 articles).

· addressed applicability only, in particular in articles retrieved with the keywords “adaptation”, “dissemination”, “translation”, and “generalization” (1,164 articles).Finally, 43 abstracts met the selection criteria (i.e. Figure 1Flow Diagram). 3151617181920212223242526272829303132333435363738394041424344454647484950515253545556Of the 43 articles retained, we distinguished three types:

22 theoretical and methodological articles presenting analyses of the concept of transferability or related topics, such as the evaluation of interventions, the external validity of studies, or the process of adapting and implementing interventions within an evidence-based perspective;

14 describing one intervention, either a primary intervention or an adaptation of an experimental intervention in a different setting;

7 literature reviews that mainly addressed transferability in terms of generalizing an intervention.

Table 1 describes the articles (i.e. Table 1).

Only 13 articles 3151617181920212223245556 used the precise term “transferability” or a derivative of the term (“transferable”). Only one article 3 gave a detailed definition of transferability. However, some terms were used as synonyms for transferability (by order of frequency: “dissemination” [14 articles], “external validity” [13 articles], “generalization” [11 articles], “generalizability” [7 articles], “adaptation” [7 articles], “translation” [3 articles], “diffusion” [1 article], “translatability” [1 article], and “applicability” [1 article]).

Some articles referred to the notion of “pure” transferability (outcomes-focused) or did not discriminate between the concepts of transferability and applicability (i.e., the extent to which an intervention process could be implemented in another setting 3). The terms most often associated with transferability were “generalizability” and “external validity”, although they have different meanings. We will come back to the distinction between these terms and the concept of transferability later in this article.

Schematically, two levels of influence on transferability were described 25 : indirect (outcomes are not transferable because the terms and conditions for implementing the intervention are different) or direct (for the same implementation modalities, different outcomes are obtained) (i.e. Table 2).

The results of effectiveness studies performed in conditions closer to the "real world" were more transferable. In particular, Victora et al. 25 specified the parameters of dose-intervention variability and associated each of them with a specific type of efficacy or effectiveness study (i.e., clinical efficacy trial, public health regimen efficacy studies, public health delivery efficacy studies, public health program efficacy studies, and public health program effectiveness studies). Dzewaltowski 54 went even further, in modelling a drastic loss of effectiveness when modifying certain factors in a program on physical activity: the training of the professionals; the implementation of a routine with no required mobilization; adherence of practitioners; changes in competencies and in the implementation conditions. In this example, the effectiveness of the program, measured based on the participation of the beneficiaries, fell from the reference value of 100% in the initial program, to 0.4%.

Category 1 describes factors present in the target population that reduce the extent to which the intervention affects the outcome, defined as "antagonism." The factor may, for example, be about health education, or a passive event that generated mistrust, or a cognitive dissonance 57 of the beneficiary in relation to the intervention. Thus, specific interventions will have a positive impact on some subjects and a negative impact on others, depending on those people’s history, the representations they have of health issues, or even the method used in the intervention.

Category 2 describes factors present in the target population that enhance the extent to which the intervention affects the outcome, defined as "synergism". The factor may also be a passive but potentializing event, contrary to the previous example, that allows the beneficiary to pass, for example, from a Prochaska stage 58 to another behavioural change stage (i.e., the intervention will only work on subjects already sensitized, that is, ready to change).

Category 3 determines the beneficiaries’ actual need with respect to the intervention. This concept is based on the theory that the same dose will have less effect if there is less need for it, and is defined as a "curvilinear dose–response association." Health education practitioners in particular must pay special attention to emerging needs and representations before the intervention, either to adapt their action to them or to raise awareness of these sometimes unconscious needs and thus potentiate the effectiveness of the intervention.

Category 4 relates to the presence or absence of interventions that are antagonistic to the studied intervention, for example, the presence of messages dissonant from that conveyed by the intervention.

Category 5 relates to the absence of a necessary cofactor in the intervention’s causal chain. This category represents cases of important determinants of health-related behaviour, such as the inaccessibility of condoms despite information on the importance of their use.

Category 6 relates to the presence or absence of an external intervention that is synergistic with the objective of the intervention studied. One example would be a causal conflict generated by a nutritional intervention conducted in schools on pupils whose food balance at home is also undergoing change because their parents are on a diet. Determining what produces the outcomes—the school-based action, parental behaviour, or both—would be difficult.

Of the 43 articles, 18 specifically addressed the question of studies’ validity by emphasizing their internal and external validity; these included 12 theoretical and methodological articles 252627282930343637435455, 2 intervention-based articles 3233 and 4 literature reviews 31353839 (i.e. Table 3). Internal validity is what makes it possible to conclude there is a causal relationship between the intervention and the outcome 25. For internal validity of research, the randomized controlled trial is promoted as the standard. External validity, or generalizability, represents the measure of the extent to which the findings can be generalized to a wider population 59. It allows the researcher to draw conclusions about the generalizability of the intervention. For this reason, there has been increased focus on the issue of external validity and greater recognition of this issue in selecting articles for publication 60. The usual assumption is that the representativeness of the sample of individuals selected in the primary study normally ensures generalizability of the intervention to a larger population or, with some adaptation of the intervention, to a different setting 61, with the understanding that effective generalization is not always possible. This is the case only within the framework of a simple causal-chain intervention, for which the previously observed influence factors are not taken into account or are given little consideration. It might not be the case for health-related behaviours or, consequently, for health education 325293031323337395455. Thus, the external validity of a study allows for conclusions on its “potential transferability” (is the intervention potentially generalizable?) by means of a reporting logic. Transferability is different from external validity. It is a process performed by the readers of research—particularly those involved in public health—in a logical analysis related to a specific setting 62 (would the measured effectiveness be identical to the primary intervention in

this

This contrast of studies raises the question of the usefulness of the randomized controlled trial for producing transferable outcomes in health education. Moreover, of the 43 articles, 7 theoretical and methodological articles 3171921253041 and one intervention-based article 15, addressed the limitations of experimental frameworks for research, agreeing on two observations: at the level of proof, the randomized controlled trial is the highest-rated evaluation method in terms of demonstrating causality 19 in a given context but raises many questions when trials are used in health promotion. Indeed, the trial is not always applicable in the field of health education for technical or ethical reasons, because of difficulties associated with selecting individuals to implement the interventions and controlling all variables that influence the results, as we have seen previously. These variables are specific to the beneficiaries, to their environment, and to the collective interactions between individuals. For these reasons, some authors consider observational and quasi-experimental studies to be the most feasible, acceptable, and/or appropriate study designs for evaluating public health interventions 19. Furthermore, their experimental nature often limits interventions in terms of methodological aspects such as an oversimplified intervention context, being away from the real world, small sample size, and long-term outcomes not analyzed 2141. Finally, the principle of having a precise protocol for assessment and intervention appears to influence the outcomes 1525 by moderating the dose intervention or dose response. Elford’s article 21 highlighted, in the field of HIV, limitations to the generalization of interventions that had been shown to be effective in an experimental context, when it came to reproducing the same results after transfer. Roush 17 stressed that randomization allows for a balanced distribution of factors involved in the causal intervention/outcomes ratio. Therefore, it is a key element of the internal validity of studies and helps reduce the assessment of antagonistic or synergistic aspects of these factors, whose importance we highlighted earlier, and therefore, of the transferability.

On the question of the randomized controlled trial, two perspectives could be distinguished. Zubrick 15, Rychetnick19 and Wang 3 agreed that health promotion requires measuring effectiveness more than efficacy, and they called for reconsidering the methods, focusing more on experimental and quasi-experimental studies and observations. Victora et al. 25 meanwhile, moved away from discussions for or against controlled randomized trials, inviting researchers, instead, to consider choosing a study based on what they really want to obtain. Thus, the authors defined several study categories:

· Seeking an outcome that would be considered a probability assessment (i.e., did the program have an effect?) calls for a randomized controlled trial.

· Seeking an outcome that would be considered a plausibility assessment (i.e., did the program seem to have an effect above and beyond other external influences?) calls for observational studies with a control group (quasi-experimental).

· Seeking an outcome that would be considered an adequacy assessment (i.e., did the expected changes occur?) calls for an observational study.

Finally, 16 of the 43 articles highlighted the value of qualitative assessments that make it possible to explore and report on possible interactions among populations, interventions, and context and, therefore, to explain outcomes; these included 14 theoretical and methodological articles 317192021252627293031414254 and 3 intervention-based articles 143233. This is what is proposed in the realistic model 63. However, the authors acknowledge that these methods, complementary to the randomized controlled trial, make it possible to identify, but not to demonstrate, the influence of various factors on the outcomes. Therefore, once the factors are identified, their influence could be shown, if possible, with randomized controlled trials 17. Moreover, evaluation of the intervention’s implementation process is highlighted as providing necessary information to help explain "how it works" as well as to demonstrate "what works" 21313253. Indeed, for lay health worker programs, the wider inclusion of qualitative research with the trials would have allowed us to explore a number of factors that might have influenced program outcomes. These include factors associated with the program itself, such as how the lay health workers were selected and trained and their relationship with communities and with the professional health workers, but also with the broader context of the program, such as political, social, or cultural conditions 64.

From this analysis, we can see that the gold standard methods—in particular, the randomized controlled trial—are not useful for assessing the transferability of results in health education. Alternative methods, qualitative approaches, and process evaluations are required to produce transferable knowledge. Thus, the evidence-based health education and promotion approach should focus on different modes of complementary or integrative studies, as in mixed-method evaluations 65, combining qualitative and quantitative methods. It also requires not only describing the outcome of an intervention (what works?), but also how it came to be (how does it work?).

Of the 43 articles, 6 theoretical and methodological articles 272837435455 and one intervention-based article 33 discussed two tools for assessing the external validity of health promotion studies: RE-AIM (Reach, Effectiveness [or Efficacy, according to the study], Adoption, Implementation, and Maintenance) and the Practical, Robust Implementation and Sustainability Model (PRISM). No article proposed a framework or tool for assessing transferability.

The seven articles agreed that the criteria for internal validity may have been accurately reported in the studies, notably strengthened by the CONSORT (Consolidated Standards of Reporting Trials), but that this was not the case for criteria relating to external validity 262830313234353637383954. Nonetheless, the authors offered some frameworks for the analysis of external validity of health promotion studies.

The first of these frameworks is RE-AIM, which makes it possible to take into account, besides the efficacy or effectiveness assessment, the participation rate and representativeness of settings, the consistency with which different intervention components are delivered, the long-term outcomes on beneficiaries, and whether an innovation or program is retained or becomes institutionalized 2633. This model was promoted on the completion of studies and also in the production of a literature review to compare studies based on multiple and identical dimensions 262835383954. The literature reviews conducted using the RE-AIM model showed that very often data on all these dimensions was missing 3536383954. These authors highlighted that modulation of these variables considerably modified the impact of the intervention 55.

The second of these frameworks, based on implementation and thus referring more to applicability, is the PRISM model, which evaluates how health care programs or interventions interact with recipients to influence program adoption, implementation, maintenance, reach, and effectiveness. The model particularly facilitates 37 the diffusion of innovation by analyzing key factors for a program’s successful implementation and sustainability. Indeed, using key questions, this framework highlights elements associated with the success of an intervention’s implementation and sustainability in the RE-AIM key domains: the program (intervention), the external environment, the implementation and sustainability infrastructure, and the recipients. Assessing each key domain and its success factors early in the implementation effort is helpful to guide any necessary modifications. The authors believe further research is needed to determine whether the number of PRISM domains activated is an important predictor of success in other implementation and dissemination reports and which PRISM elements are most important for particular settings and clinical targets. Actually, the tool is intended more for translating research into practice than for assessing external validity.

Of the 43 articles, 20 explicitly provided, as criteria for external validity, evaluation, or processes to adapt existing interventions, elements that could be used to build a typology of transferability factors. A first cornerstone is based on the RE-AIM framework 26272835383954. A second cornerstone is based on a study of intervention processes and/or of the adaptation of interventions as sources for understanding the efficiency factors. A first group of authors 313244 described how the assessment process helps to explain applicability and/or transferability. These process elements become potential categories of transferability factors. A second group of authors 4045 examined not the intervention process, but the adaptation process. Unlike dose intervention, which modulates the intervention without fundamentally changing it, program adaptation is defined 46 by a process of change to reduce the dissonance between the characteristics and the new setting in which the program is implemented. This concept refers to the definition of adaptation criteria 40 and to the stages of this adaptation process that some authors have modeled 46. These criteria or adaptation factors could, again, be categories or potential transferability factors.

Six articles—4 conceptual articles 3192147 and 2 intervention-based articles 1518—give specific examples of criteria beyond the categories. From these elements, we have structured a potential list of transferability factors or categories (i.e. Table 4).

Even though the aim of this study was not to be comprehensive, it does have limitations related to the search strategy. In particular, articles were selected on the basis of abstracts. We may have missed articles that addressed the issue of transferability without it being mentioned in the abstract. Nonetheless, the consistency of the authors’ findings and the ease with which we were able ultimately to define a consensual list of factors among those debated by the authors argues that additional articles would not have contributed further to our findings.

As well, this review does not take into account other strategies to promote health—health public policy, supportive environments, health services reorientations—that pertain to other sectors of intervention. Indeed, it focuses on educational strategies for health promotion.

Finally, we relied particularly on the external validity criteria provided by the selected articles in the field of health promotion. However, there are other tools for assessing external validity in other intervention areas that contribute to evidence-based public health. These tools could be used, as we did with RE-AIM, to extrapolate transferability factors. However, we wanted to focus, as a first step, on an analysis of the concept in the specific field of health education. Undoubtedly, with further, more comprehensive work on the consolidation of a tool mentioned above, additional analysis of these tools would be necessary, as would the incorporation of this analysis of the transferability of planning frameworks (e.g., PRISM 37 PRECEDE/PROCEED 68).

Concept from the researcher’s point of view:

Generalizability: the extent to which the findings can potentially be generalized to an unspecified or wider population 59.

External validity: characteristic of the studies which provides the basis for generalizability to other populations, settings, and times 69.

Concept from the point of view of the readers/users of research:

Transferability: the extent to which the measured effectiveness of an applicable intervention could be achieved in another setting 3.

Applicability: the extent to which an intervention process could be implemented in another setting 3.

Secondly, with respect to knowledge development, the concept of transferability has been barely objectified. We were able to identify the presentation of some criteria, but the criteria we extracted and analyzed seemed more or less accurate, specific and structured, in that they were often produced by validity or process assessment or adapted from an intervention. How they were developed was not always shown and appeared to be the result of both common sense and exchange among practitioners involved in an intervention, rather than of any methodical and rigorous process. From the known elements of the issue, a transferability criteria tool could be developed that could be used to assess the transferability of interventions by comparing the settings of research studies with the setting in which the practitioner must implement an intervention. This guide could be used to incorporate transferability criteria into the reporting of studies, thus making the research more transferable and therefore more useful to health stakeholders. Such a tool would inform decision-makers and health stakeholders in choosing a specific intervention in a particular setting or in performing the necessary and possible adjustments to achieve real efficiency. This tool would thus contribute to the implementation of evidence-based health practice 11.

Finally, with respect to avenues for further research, this review highlighted the efforts that must be made to develop research in this field that will be more easily transferable and more useful for health stakeholders. Evaluation methods should also be developed that could take into account the dimension of transferability as well as internal validity. This will probably require the development, in research, of combinations of studies or approaches for assessing complex interventions 70.