Diagnosis of Rare Diseases: a scoping review of clinical decision support systems

The author JAS performed an initial search in PubMed with a combination of the terms “Clinical Decision Support” and “Rare Diseases”. The terms were combined with a logical “AND”. The goal was to identify relevant keywords for a broad search. The results of the search were 165 publications. To obtain keywords, we checked titles and abstracts of the publications to determine whether they described an specific CDSS for RDs and identified five relevant publications [13, 14, 18‐20]. Afterwards, we extracted the keywords from these publications in a brainstorming session with all authors and decided which of them were relevant for the search (see Additional file 2 – Part A). Based on the identified keywords, we created a map to establish a relationship between them (see Additional file 2 – Part B-C).

In the next step, the identified keywords were tested and mapped to MeSH terms (Medical Subject Heading) by JAS, validated by MS and approved by all authors (described in Additional file 2 – Part D). We also added non-MeSH terms to cover articles which do not appear in the index of MeSH [21]. This led us to our final search terms (shown in Additional file 2 – Part E). We grouped the terms into four groups “A” to “D”. The groups were combined with a logical “AND”. Terms of group “A” and “B” (MeSH terms) and “C and D” (non-MeSH terms) were combined for the search. The search was conducted by JAS with the final query in Fig. 1.

Table 4 shows the summary of “CDSSs using machine learning”.

FindZebra [37] is a search engine and a fully developed system that allows clinicians to enter symptoms in a search field and find corresponding information in databases. The knowledge base of FindZebra is built on 33,144 documents covering approximately 90% of the RDs listed in the Orphanet database. FindZebra uses ten sources for their dataset, for instance OMIM (Online Mendelian Inheritance in Man) and GARD (Genetic and Rare Diseases Information Center) [37]. OMIM contains descriptions about human genes and their correlation with phenotypes, which are defined in genetics as a set of all visible characteristics of an organism [37]. GARD provides a database about RDs with symptoms, treatments and further research information [37].

To evaluate FindZebra, the authors compared FindZebra with other platforms like PubMed and Google, using 56 search queries with patient symptoms based on expert knowledge. The findings show that FindZebra outperforms Google and PubMed. Especially for queries with a long list of symptoms, FindZebra achieves better results. Google uses an algorithm based on how often a website is visited or linked to other websites. The authors concluded that this would lead to poor results for RDs [37]. However, an information about software updates, clinical usage or further data integration is not provided.

A further CDSS to identify relevant information in a database is the CDSS of Taboada et al. [38]. The authors described a fully developed system which can automatically capture relevant literature data based on phenotypes. Their CDSS uses so-called “text annotation”, a method from the field of Natural Language Processing (NLP), to identify relevant words in a text.

The evaluation of the CDSS was based on a disease “Cerebrotendinous xanthomatosis” (CTX), a rare disorder of bile acid metabolism. The authors extracted 223 abstracts of case reports from PubMed corresponding to CTX. Only the title and relevant parts of the texts were used for annotation. Those were annotated with the Human Phenotype Ontology (HPO) using the software Open Biological and Biomedical Ontologies (OBO) and Bioportal. The HPO describes the correlation between phenotypes and genetic diseases.

The evaluation was measured between the automatic annotation method and a manual annotation of two neurologists, who extracted the relevant phenotypes manually. The authors evaluated the capability to identify the relevant papers for both methods (F-measure). The CDSS achieved an F-measure of 74%, which is significantly lower than the result of the manual method with 88% [38]. The authors concluded that the annotation method could have a high impact on the quality of the results [38].

FindZebra [37] and Taboada et al. [38] both provide fully developed systems, using literature databases, but different ways to find relevant data about RDs in databases. As with FindZebra, no information about software update or clinical usage is available.

Garcelon et al. [36] developed a clinical prototype to find similar patients to an undiagnosed patient (index patient) in a clinical data warehouse containing about 400,000 patients. The data warehouse is a combination of different sources, e.g. electronic health records (EHR). The similarity is calculated using the Vector Space Model (VSM), by representing patient data in a mathematical vector. The similarity of two patients is measured as the presence or absence of words in the compared patient vectors [36].

Five different rare genetic diseases with 7 to 103 patient cases per disease were used for the evaluation of the CDSS. The authors evaluated its capability to find the patients who were most similar to an undiagnosed patient. Patients were considered to be similar (called true positive similar patients) when they were among the top 30 of the most similar patients and also appeared in the list of diagnosed patients, which was provided by a domain expert. The percentages of index patients, returning at least one true positive similar patient in the list of the top 30 similar patients, were reported as 94% for Lowe Syndrome, 97% for Epidermolysis Bulloas, 86% for Activated PI3K Delta Syndrome, 71% for Dowling Meara and 99% for Rett Syndrome. The average number of patients with the same disease among the top 30 similar patients was 51% [36]. Although the system achieved good results in diagnostics, it cannot be accessed and no information about a software update is available. However, data integration is described with ETL processes.

Shen et al. [18] developed a clinical prototype that uses not only clinical data. The authors merged clinical and literature data. They included clinical data from 13 million unstructured clinical notes on 700,000 patients’ electronic health records limited to described problems and diagnosis. Abstracts from research articles from the SemMedDB were extracted for the literature dataset. SemMedDb is a repository of semantic predications, extracted from titles and abstracts of all PubMed citations. The authors applied HPO and GARD terms to match the representation of both data types, followed by data fusion strategies to include the data into a collaborative filtering model to enable RD recommendation. Data fusion means that different types of data sets are combined into one dataset [18]. The authors used the following data fusion strategies: First, only the patient phenotype information was extracted from the EHR. For the second, the authors combined EHR data with phenotypes and literature. In the third fusion strategy, phenotype-rare disease associations were extracted from literature with the limitation that phenotypes of the literature data were deleted if they did not appear in the EHR data. The authors then evaluated the prediction output for each fusion strategy using a collaborative filtering model to determine which possible combinations provide the best results. This technique is used, for instance, in e-commerce to recommend products to customers based on similar buying preferences of other customers. That this scenario is similar to patients’ phenotypic information. If patients have similar phenotypes, their diseases might also be similar. The results are compared with the actual diagnosis of the patient [18]. The results show that the combination of EHR and literature data did not always lead to the best performance. The authors conclude that this may be due to different approaches and expressions in clinical notes varying from physician to physician [18]. Since the CDSS is a clinical prototype, the system cannot be accessed, no information for data entry and integration is available and neither there are any information about software updates and clinical usage.

Table 5 shows the summary of “CDSSs using information retrieval.”

Our review includes Phenopolis [31], GEMINI [24] and GenIO [29], which provide different tools for the investigation of genetic variants. All three of these CDSSs are fully developed systems and can be accessed by clinicians. We show further characteristics in Table 6.

Phenopolis [31] provides an open-source web server and different analysis tools like variant filtering and gene prioritization based on phenotypes of a patient using the HPO. Variant filtering allows to identify relevant variants for a diagnosis. With gene prioritization, potentially causative genes can be prioritized. Phenopolis contains 6048 exomes representing the 4,859,971 variants which comprise the data base [31].

GEMINI [24] is a software package which allows researchers to integrate the genetic variations in the Variant Call Format (VCF), a common format for the gene sequence variants. GEMINI provides variant analysis tools for the investigation of variants and a programming interface to customize the data analysis and exploration [24]. Koile et al. developed GenIO [29] a web interface for clinicians and researchers who do not have the necessary skills to annotate, classify and filter variants. GenIO uses the so-called “GenIO pipeline“, which consists of a variant annotation and phenotype processing [29]. At the end of the phenotyping process, a list of genes with matches to the patient’s phenotype is shown [43].

Table 6 shows the summary of “CDSSs using analysis of genetic and phenotypic data”.

The identification of similar patients and the sharing of patient cases is possible with GeneMatcher [27], GeneYenta [28], Phenotips [32], PhenomeCentral [34], MatchMaker Exchange [30], DECIPHER [33], PhenoDB [23] and GenomeConnect [25]. All of these CDSSs are fully developed systems and can be accessed in different ways. They allow clinicians and researchers to find similar patients in a database based on genetic or phenotypic data. We show further details for clinical usage in Table 7.

PhenomeCentral, DECIPHER and GeneYenta are connected to the Matchmaker Exchange Project (MME) which connects organizations and projects through a federate network of databases of genotypes and rare phenotypes using a REST API [30]. A REST (Representational State Transfer) API (Application Programming Interface) is a web architecture style and provides the opportunity to shift data to another software system over the internet [44]. The MME enables searches across multiple databases from different platforms by making requests to all databases, e.g. to find similar matches of patients. In order to use MME, a clinician must therefore be part of a participating MME project. MME itself does not provide a user interface, but only connects existing platforms via the MME API [30].

Another CDSS which uses the similarity of phenotypes by not comparing different patient cases is Phenomizer [22]. Phenomizer is a fully developed system and facilitates differential diagnoses by using the HPO for entering phenotypes. The software classifies all diseases listed in OMIM, Orphanet and DECIPHER and uses a semantic similarity metric to measure the similarity between phenotypes and genetic diseases. Several symptoms of a patient can be entered and combined to describe the entire spectrum of a patient’s symptoms. All related diagnoses with their statistical probability are shown to rank the candidate’s disease [22].

Table 7 shows the summary of “CDSSs using comparison of genetic and phenotypic data”. For interested developers, we provide links to the respective REST APIs showing which data can be integrated (Additional file 6).