• Diabetic retinopathy database: the diabetic retinopathy (DR) database contains retinal images of diabetic patients, with associated anonymous information on the pathology. Diabetes is a metabolic disorder characterized by sustained inappropriate high blood sugar levels. This progressively affects blood vessels in many organs, which may lead to serious renal, cardiovascular, cerebral and also retinal complications. Different lesions appear on the damaged vessels, which may lead to blindness. The database is made up of 63 patient files containing 1045 photographs altogether. Images have a definition of 1280 pixels/line for 1008 lines/image. They are lossless compressed images. Patients have been recruited at Brest University Hospital since June 2003 and images were acquired by experts using a Topcon Retinal Digital Camera (TRC-50IA) connected to a computer. An example of an image series is given in figure 1.
  The contextual information available is the patients’ age and sex and structured medical information (about the general clinical context, the diabetes context, eye symptoms and maculopathy). Thus, at most, patients records are made up of 10 images per eye (see figure 1) and of 13 contextual attributes; 12.1% of these images and 40.5% of these contextual attribute values are missing. The disease severity level, according to ICDRS classification [3], was determined by experts for each patient. The distribution of the disease severity among the above-mentioned 63 patients is given in table I.
• Digital Database for Screening Mammography (DDSM): the DDSM project [4] has built a mammographic image database for research on breast cancer screening. It is made up of 2277 patient files. Each one includes two images of each breast, associated with some patient information (age at time of study, rating for abnormalities, American College of Radiology breast density rating and keyword description of abnormalities) and imaging information. The following contextual attributes are taken into account in the system:
  • the age at time of study
  • breast density rating

In previous studies, we proposed to compute a signature for images from their wavelet transform (WT) [6]. These signatures model the distribution of the WT coefficients in each subband of the decomposition. The associated distance measure d [6] computes the divergence between these distributions. We used these signature and distance measure to cluster similar images.

Any clustering algorithm can be used, provided that the distance measure between feature vectors can be specified. We used FCM (Fuzzy C-Means) [7], one of the most common algorithms, and replaced the Euclidian distance by d.

• The fusion model: let θ = {θ₁, θ₂, …} be a set of hypotheses under consideration for a fusion problem; θ is called the frame of discernment. In DST, these hypotheses are assumed incompatibles (constrained model ℳ⁰(θ)), while in DSmT they are not (free model ℳ^f (θ)): if θ = {“blue”, “red”}, we may model objects that are both blue and red (magenta). Thus, in DSmT, a belief mass m(A) is assigned to each element A of the hyper-power set D(θ), i.e. the set of all composite propositions built from elements of θ with ∩ and ∪ operators, such that m(∅) = 0 and Σ_A∈D(θ)m(A) = 1; m is called a generalized basic belief assignment (gbba). Nevertheless, it is possible to introduce constraints in the model [2] (hybrid model ℳ(θ)): we can specify pairs of incompatible hypotheses (θ_a, θ_b), i.e. each subset A of θ_a ∩θ_b must have a null mass, noted A ∈ C(θ).
• The fusion operators and the decision functions: to fuse information in the DSmT framework, the user specifies a gbba m_j for each source of evidence S_j, j = 1..N. Then, these gbba are fused into a global gbba m_f, according to a given rule of combination. Several rules have been proposed to combine mass functions, including the hybrid rule of combination or the PCR (Proportional Conflict Redistribution) rules [2].
• The decision functions: once the fused gbba m_f has been computed, a decision function (such as the credibility, the plausibility or the pignistic probability) is used to evaluate the probability of each hypothesis. The pignistic probability BetP, a compromise between the other two functions, is used since it provides the best system performance:

  (1)

  where _ℳ(B_i) is the cardinality of B_i in the Venn diagram of ℳ(θ): in the examples of figure 3, _ℳ⁰(θ₁) = 1, _ℳ^f (θ₁) = 4 and _ℳ(θ₁) = 2.

• Outline of the method: let c_q be a case placed as a query by the user. We want to rank the cases in the database by decreasing order of relevance for c_q. In that purpose, for each case c_i in the database, i = 1..M, we estimate the degree of relevance of c_i for c_q, noted DR(c_i, c_q); then we rank the cases c_i by decreasing order of DR(c_i, c_q), and the top five results are returned to the user. To compute DR(c_i, c_q), we see each case descriptor as a source of evidence: the degrees of relevance are first estimated for each case feature F_j, j = 1..N, noted DR_j(c_i, c_q) (as described in section II-D.3), they are then translated into gbba (see section II-D.4) which are fused in the DSmT framework. Finally, DR(c_i, c_q) is estimated from the fused gbba by the pignistic probability (see equation 1). The procedure is summed up in figure 5.
• The model: the following frame of discernment is used for the fusion problem: θ = {C₁,C₂, …,C_M}, where C_i is the hypothesis “c_i is relevant for c_q”, i = 1..M. The cardinal of D(θ) is hyper-exponential in M (it is majored by 2^2M). As a consequence, from computational considerations, it is necessary to include constraints in the model. These constraints are also justified by logical considerations. Indeed, it is not possible for the case c_q to be similar to both cases c_a and c_b if
  • c_a and c_b are dissimilar, or
  • c_a and c_b have different severity levels
  In those cases, we set A ∈ C(θ) ∀ A ⊂ C_a ∩ C_b. To build the model ℳ(θ), we first define a non-oriented graph G_c = (V,E), that we call compatibility graph; each vertex v ∈ V represents an hypothesis and each edge e ∈ E a couple of compatible hypothesis. To build G_c, we link the hypothesis associated with each case c_i to the hypothesis associated with its l nearest neighbors at the same severity level. The distance measure used to find the nearest neighbors is simply a linear combination of heterogeneous distance functions (one for each case feature F_j - see section II-B for images), managing missing values [8]. We noticed that the complexity of the fusion operators mainly depends on the cardinality of the largest clique in G_c (a clique is a set of vertices V such that for every two vertices in V, there exists an edge connecting the two). The number l is closely related to the cardinality of the largest clique in G_c and consequently to the complexity of the fusion operation. Finally, the hybrid model ℳ(θ) is built from G_c by identifying all the cliques in G_c (see figure 4).
• The degree of relevance: to compute the degree of relevance DR_j(c_i, c_q) that a case c_i is relevant for c_q, given the feature F_j, we first define a finite number of states f_jk for F_j. Then we compare the membership degree of c_i (α_jk(c_i)) and of c_q (α_jk(c_q)) to each state f_jk. If F_j is a discrete variable, we associate a state with each possible value for F_j and α_jk is a Boolean function. If F_j is an image attribute, α_jk(c) is the membership degree of a case c to the k^th image cluster (see section II-B - the same clustering algorithm is used for simple continuous variables such as age). We assume that the state of the cases in the same class are predominantly in a subset of states for F_j. So, in order to estimate the conditional probabilities, we use a correlation measure S_jk1k2 between two feature states f_jk1 and f_jk2, regarding the class of the cases at these states. To compute S_jk1k2, we first compute the mean membership D_jk1c (resp. D_jk2c) of cases in a given class c to the state f_jk1 (resp. f_jk2) (equation 2):

  (2)

  where δ(x, c) = 1 if x is in class c, δ(x, c) = 0 otherwise, and β is a normalizing factor. S_jk1k2 is given by equation 3:

  (3)

  the expression of DR_j(c_i, c_q) is given by the following equation:

  (4)

• The generalized basic belief assignments (gbba): The gbba m_j is then defined as follows:
  • We identify the cases (c_i)_{i=1..M′≤M} such that DR_j(c_i, c_q) > τ_j,
  • We set ,
  • We set .

To find τ_j, we define the following test T_j=“if DR_j(c_i, c_q) > τ_j then C_i is true, otherwise C_i is false”. The sensitivity (resp. the specificity) of T_j represents the degree of confidence in a positive (resp. negative) answer to T_j. T_j is relevant if it is both sensitive and specific. As τ_j increases, sensitivity increases and specificity decreases. So, we set τ_j as the intersection of the two curves “sensitivity according to τ_j” and “specificity according to τ_j ”, using a dichotomic search. We set m_j0 as the sensitivity of T_j. is assigned the degree of uncertainty of T_j. Note that a single setup (τ_j, m_j0) is used for each feature, whatever c_i and c_q.