Diagnostic Reasoning for Clinical Notes via Large Language Models (2024)

3.1 Clinical Notes

Clinical notes used in DiReCT are stored in the SOAP format [Weed, 1970]. A clinical note comprises four components: In the subjective section, the physician records the patient’s chief complaint, the history of present illness, and other subjective experiences reported by the patient. The objective section contains structural data obtained through examinations (inspection, auscultation, etc.) and other measurable means. The assessment section involves the physician’s analysis and evaluation of the patient’s condition. This may include a summary of current status, etc. Finally, the plan section outlines the physician’s proposed treatment and management plan. This may include prescribed medications, recommended therapies, and further investigations. A clinical note also includes a primary discharge diagnosis (PDD) in the assessment section.

DiReCT’s clinical notes are sourced from the MIMIC-IV dataset [Johnson etal., 2023] (PhysioNet Credentialed Health Data License 1.5.0), which encompasses over 40,000 patients admitted to the intensive care units. Each note contains clinical data for a patient. To construct DiReCT, we curated a subset of 511 notes whose PDDs fell within one of 25 disease categories $i$ in 5 medical domains.

In our task, a note $R=\{r\}$ is an excerpt of 6 clinical data in the subjective and objective sections (i.e., $|R|=6$): chief complaint, history of present illness, past medical history, family history, physical exam, and pertinent results.¹¹1We excluded data, such as review system and social history, because they are often missing in the original clinical notes and are less relevant to the diagnosis. We also identified the PDD $d^{\star}$ associated with $R$.²²2All clinical notes in DiReCT are related to only one PDD, and there is no secondary discharge diagnosis. The set of $d^{\star}$’s for all $R$’s collectively forms $\mathcal{D}^{\star}$. We manually removed any descriptions that disclose the PDD in $R$.

3.2 Diagnostic Knowledge Graph

Existing knowledge graphs for the medical domain, e.g., UMLS KG [Bodenreider, 2004], lack the ability to provide specific clinical decision support (e.g., diagnostic threshold, context-specific data, dosage information, etc.), which are critical for accurate diagnosis.

Our knowledge graphs $\mathcal{K}=\{\mathcal{K}_{i}\}_{i}$ is a collection of graph $\mathcal{K}_{i}$ for disease category $i$.$\mathcal{K}_{i}$ is based on the diagnosis criteria in existing guidelines (refer to supplementary material for details). $\mathcal{K}_{i}$’s nodes are either premise $p\in\mathcal{P}_{i}$ (medical statement, e.g., Headache is a symptom of) and diagnoses $d\in\mathcal{D}_{i}$ (e.g., Suspected Stroke). $\mathcal{K}_{i}$ consists of two different types of edges. One is premise-to-diagnosis edges $\mathcal{S}_{i}=\{(p,d)\}$, where $p\in\mathcal{P}_{i}$ and $d\in\mathcal{D}_{i}$; an edge is from $p$ to $d$. This edge represents the necessary premise $p$ to make a diagnosis $d$. We refer to them as supporting edges. The other is diagnosis-to-diagnosis edges $\mathcal{F}_{i}=\{(d,d^{\prime})\}$, where $d,d^{\prime}\in\mathcal{D}_{i}$ and the edge is from $d$ to $d^{\prime}$, which represents the diagnostic flow. These edges are referred to as procedural edges.

A disease category is defined according to an existing guideline, which starts from a certain diagnosis; therefore, a procedural graph $\mathcal{G}_{i}=(\mathcal{D}_{i},\mathcal{F}_{i})$ has only one root node and arbitrarily branches toward multiple leaf nodes that represent PDDs (i.e., the clinical notes in DiReCT are chosen to cover all leaf nodes of $\mathcal{G}_{i}$). Thus, $\mathcal{G}_{i}$ is a tree. We denote the set of the leaf nodes (or PDDs) as $\mathcal{D}^{\star}_{i}\subset\mathcal{D}_{i}$. The knowledge graph is denoted by $\mathcal{K}_{i}=(\mathcal{D}_{i},\mathcal{P}_{i},\mathcal{S}_{i},\mathcal{F}_{$.

Figure 2 shows a part of $\mathcal{K}_{i}$, where $i$ is Acute Coronary Syndromes (ACS). Premises in $\mathcal{P}_{i}$ and diagnoses in $\mathcal{D}_{i}$ are given in the blue and gray boxes, while PDDs in $\mathcal{D}^{\star}_{i}$ are ones without outgoing edges (i.e., STEMI-ACS and NSTEMI-ACS, and UA). The black and red arrows are edges in $\mathcal{S}$ and $\mathcal{F}$, respectively, where the black arrows indicate the supporting edges.

$\mathcal{K}$ serves two essential functions: (1) They serve as the gold standard for annotation, guiding doctors in the precise and uniform interpretation of clinical notes. (2) Our task also allows a model to use them to ensure the output from an LLM can be closely aligned with the reasoning processes of medical professionals.

3.3 Data Annotation

Let $d^{\star}\in\mathcal{D}^{\star}_{i}$ denote the PDD of disease category $i$ associated with $R$. We can find a subgraph $\mathcal{K}_{i}(d^{\star})$ of $\mathcal{K}_{i}$ that contains all ancestors of $d^{\star}$, including premises in $\mathcal{P}_{i}$. We also denote the set of supporting edges in $\mathcal{K}_{i}(d^{\star})$ as $\mathcal{S}_{i}(d^{\star})$. Our annotation process is, for each supporting edge $(p,d)\in\mathcal{S}_{i}(d^{\star})$, to extract observation $o\in\mathcal{O}$ in $R$ (highlighted text in the clinical note in Figure 3) and provide rationalization $z$ of this deduction why $o$ is a support for $d$ or corresponds to $p$.³³3All annotations strictly follow the procedural flow in $\mathcal{K}_{i}$, and each observation is only related to one diagnostic node. If $R$ does not provide sufficient observations for the PDD (which may happen when a certain test is omitted), the annotators were asked to add plausible observations to $R$. This choice compromises the fidelity of our dataset to the original clinical notes, but we chose it for the completeness of the dataset. They form the explanation $\mathcal{E}=\{(o,z,d)\}$ for $(R,d^{\star})$. This annotation process was carried out by 9 clinical physicians and subsequently verified for accuracy and completeness by three senior medical experts.

Table 2 summarizes statistics of our dataset. The second and third columns (“# cats.” and “# samples”) show the numbers of disease categories and samples in the respective medical domains. $|\mathcal{D}_{i}|$ and $|\mathcal{D}_{i}^{\star}|$ are the total numbers of diagnoses (diseases) and PDDs, summed over all diagnostic categories in the medical domain, respectively. $|\mathcal{O}|$ is the average number of annotated observations. “Length” is the average number of tokens in $R$.

3.4 Task Definition

3.5 Evaluation Metrics

We designed three metrics to quantify the predictive performance over our benchmark.

(1) Accuracy of diagnosis $\textit{Acc}^{\text{diag}}$ evaluates if a model can correctly identify the diagnosis. $\textit{Acc}^{\text{diag}}=1$ if $d^{\star}=\hat{d}$, and $\textit{Acc}^{\text{diag}}=0$ otherwise. The average is reported.

(2) Completeness of observations $\textit{Obs}^{\text{comp}}$ evaluates whether a model extracts all and only necessary observations for the prediction. Let $\mathcal{O}$ and $\hat{\mathcal{O}}$ denote the sets of observations in $\mathcal{E}$ and $\hat{\mathcal{E}}$, respectively. The metric is defined as$\textit{Obs}^{\text{comp}}=|\mathcal{O}\cap\hat{\mathcal{O}}|/|\mathcal{O}\cup$, where the numerator is the number of observations that are common in both $\mathcal{O}$ and $\hat{\mathcal{O}}$.⁴⁴4We find the common observations with an LLM (refer to the supplementary material for more detail). This metric simultaneously evaluates the correctness of each observation and the coverage. To supplement it, we also report the precision $\textit{Obs}^{\text{pre}}$ and recall $\textit{Obs}^{\text{rec}}$, given by$\textit{Obs}^{\text{pre}}=|\mathcal{O}\cap\hat{\mathcal{O}}|/|\hat{\mathcal{O}}|$ and $\textit{Obs}^{\text{rec}}=|\mathcal{O}\cap\hat{\mathcal{O}}|/|\mathcal{O}|$.

(3) Faithfulness of explanations Faith evaluates if the diagnostic flow toward the PDD is fully supported by observations with faithful rationalizations.This involves establishing a one-to-one correspondence between deductions in the prediction and the ground truth. We use the correspondences established for computing $\textit{Obs}^{\text{comp}}$. Let $o\in\mathcal{O}$ and $\hat{o}\in\hat{\mathcal{O}}$ denote corresponding observations. This correspondence is considered successful if $z$ and $\hat{z}$ as well as $d$ and $\hat{d}$ associated with $o$ and $\hat{o}$ matches. Let $m(\mathcal{E},\hat{\mathcal{E}})$ denote the number of successful matches. We use the ratio of $m(\mathcal{E},\hat{\mathcal{E}})$ to $|\mathcal{O}\cap\hat{\mathcal{O}}|$ and $|\mathcal{O}\cup\hat{\mathcal{O}}|$ as evaluation metrics $\textit{Exp}^{\text{com}}$ and $\textit{Exp}^{\text{all}}$, respectively, to see failures come from observations or explanations and diagnosis.

5.1 Experimental Setup

We assess the reasoning capabilities of 7 recent LLMs from diverse families and model sizes, including 5 instruction-tuned models that are openly accessible: LLama3 8B and 70B [AI@Meta, 2024], Zephyr 7B [Tunstall etal., 2023], Mistral 7B [Jiang etal., 2023], and Mixtral 8$\times$7B [Jiang etal., 2023]. We have also obtained access to private versions of the GPT-3.5 turbo [OpenAI, 2023b] and GPT-4 turbo [OpenAI, 2023a] ⁶⁶6These two models are housed on a HIPPA-compliant instance within Microsoft Azure AI Studio. No data is transferred to either Microsoft or OpenAI. This secure environment enables us to safely conduct experiments with the MIMIC-IV dataset, in compliance with the Data Use Agreement., which are high-performance closed-source models. Each LLM is utilized to implement our baseline’s narrowing-down, perception, and reasoning modules. The temperature is set to 0. For computing evaluation metrics, we use LLama3 8B with few-shot prompts to make correspondences between $\mathcal{O}$ and $\hat{\mathcal{O}}$ as well as to verify a match between predicted and ground-truth explanations (refer to the supplementary material for more details).

5.2 Results

Comparison among LLMs. Table 3 shows the performance of our baseline built on top of various LLMs. We first evaluate a variant of our task that takes graph $\mathcal{G}=\{\mathcal{G}_{i}\}$ consisting of only procedural flow as external knowledge instead of $\mathcal{K}$. Comparison between $\mathcal{G}$ and $\mathcal{K}$ demonstrates the importance of supplying premises with the model and LLMs’ capability to make use of extensive external knowledge that may be superficially different from statements in $R$. Subsequently, some models are evaluated with our task using $\mathcal{K}$. In addition to the metrics in Section 3.5, we also adopt the accuracy of disease category $\textit{Acc}^{\text{cat}}$, which gives 1 when $\hat{i}=i$, as our baseline’s performance depends on it.

With $\mathcal{G}$, we can see that GPT-4 achieves the best performance in most metrics, especially related to observations and explanations, surpassing LLama3 70B by a large margin. In terms of accuracy (in both category and diagnosis levels), LLama3 70B is comparable to GPT-4. LLama3 70B also has a higher $\textit{Obs}^{\text{rec}}$ but low $\textit{Obs}^{\text{pre}}$ and $\textit{Obs}^{\text{comp}}$, which means that this model tends to extract many observations. Models with high diagnostic accuracy are not necessarily excel in finding essential information in long text (i.e., observations) and generating reasons (i.e., explanations).

When $\mathcal{K}$ is given, all models show better diagnostic accuracy (except GPT-3.5) and explanations, while observations are slightly degraded. GPT-4 with $\mathcal{K}$ enhances Acc${}^{\text{diag}}$, $\textit{Exp}^{\text{com}}$, and $\textit{Exp}^{\text{all}}$ scores. This suggests that premises and supporting edges are beneficial for diagnosis and explanation. Lower observational performance may indicate that the models lack the ability to associate premises and text in $R$, which are often superficially different though semantically consistent.

LLMs may undergo inherent challenges for evaluation when no external knowledge is supplied. They may have the knowledge to diagnose but cannot make consistent observations and explanations that our task expects through $\mathcal{K}$. To explore this, we evaluate two settings: (1) giving $D^{\star}$ and (2) no knowledge is supplied to a model (shown in Table 4). The prompts used for this setup are detailed in the supplementary material. We do not evaluate the accuracy of disease category prediction as it is basically the same as Table 3. We can clearly see that with $\mathcal{D}^{\star}$, GPT-4’s diagnostic and observational scores are comparable to those of the task with $\mathcal{K}$, though explanatory performance is much worse. Without any external knowledge, the diagnostic accuracy is also inferior.⁷⁷7We understand this comparison is unfair, as the prompts differ. We intend to give a rough idea about the challenge without external knowledge. The deteriorated performance can be attributed to inconsistent wording of diagnosis names, which makes evaluation tough. High observational scores imply that observations in $R$ can be identified without relying on external knowledge. There can be some cues to spot them.

Performance in individual domains. Figure 5 summarizes the performance of LLama3 70B, GPT-3.5, and GPT-4 across different medical domains, evaluated using Acc${}^{\text{cat}}$, $\textit{Obs}^{\text{comp}}$, and $\textit{Exp}^{\text{all}}$. Neurology gives the best diagnostic accuracy, where GPT-4 achieved an accuracy of 0.806. LLama3 also performed well (0.786). In terms of $\textit{Obs}^{\text{comp}}$ and $\textit{Exp}^{\text{all}}$, GPT-4’s results were 0.458 and 0.340, respectively, with the smallest difference between the two scores among all domains. This smaller gap indicates that in Neurology, the common observations in prediction and ground truth lead to the correct diagnoses with faithful rationalizations. However, GPT-4 yields a higher diagnostic accuracy score while a lower explanatory score, suggesting that the observations captured by the model or their rationalizations differ from human doctors.

For Cardiology and Endocrinology, the diagnostic accuracy of the models is relatively low (GPT-4 achieved 0.458 and 0.468, respectively). Nevertheless, $\textit{Obs}^{\text{comp}}$ and $\textit{Exp}^{\text{all}}$ are relatively high. Endocrinology results in lower diagnostic accuracy and higher explanatory performance. A smaller gap may imply that in these two domains, successful predictions are associated with observations similar to those of human doctors, and the reasoning process may be analogous. Conversely, in Gastroenterology, higher Acc${}^{\text{cat}}$) is accompanied by lower $\textit{Obs}^{\text{comp}}$ and $\textit{Exp}^{\text{all}}$ (especially for LLama3), potentially indicating a significant divergence in the reasoning process from human doctors. Overall, DiReCT demonstrates that the degree of alignment between the model’s diagnostic reasoning ability and that of human doctors varies across different medical domains.

Reliability of automatic evaluation. We randomly pick out 100 samples from DiReCT and their prediction by GPT-4 over the task with $\mathcal{G}$ to assess the consistency of our automated metrics to evaluate the observational and explanatory performance in Section 3.3 to human judgments. Three physicians joined this experiment. For each prediction $\hat{o}\in\hat{\mathcal{O}}$, they are asked to find a similar observation in ground truth $\mathcal{O}$. For explanatory metrics, they verify if each prediction $\hat{z}\in\hat{\mathcal{E}}$ for $\hat{o}\in\hat{\mathcal{O}}$ align with ground-truth $z\in\mathcal{E}$ corresponding to $o$. A prediction and a ground truth are deemed aligned for both assessments if at least two specialists agree. We compare LLama3’s and GPT-4’s judgments to explore if there is a gap between these LLMs. As summarized in Table 5, GPT-4 achieves the best results, with LLama3 8B also displaying a similar performance. From these results, we argue that our automated evaluation metrics are consistent with human judgments, and LLama3 is sufficient for this evaluation, allowing the cost-efficient option.

A prediction example. Figure 6 shows a sample generated by GPT-4. The ground-truth PDD of the input clinical note is Hemorrhagic Stroke. In this figure, purple, orange, and red indicate explanations only in the ground truth, only in prediction, and common in both, respectively; therefore, red is a successful prediction of an explanation, while purple and orange are a false negative and false positive. GPT-4 treats the observation of aurosis fugax as the criteria for diagnosing Ischemic Stroke. However, this observation only supports Suspected Stroke. Conversely, observation thalamic hematoma, which is the key indicator of Hemorrhagic Stroke, is regarded as a less important clue. Such observation-diagnosis correspondence errors lead to the model’s misdiagnosis. More samples are available in the supplementary material.

A.1 Data Statistics

Table 6 provides a detailed breakdown of the disease categories included in DiReCT. The column labeled # samples indicates the number of data points. The symbols $|\mathcal{D}_{i}|$ and $|\mathcal{D}^{\star}_{i}|$ denote the total number of diagnoses (diseases) and PDDs, respectively. Existing guidelines for diagnosing diseases were used as References, forming the foundation for constructing the diagnostic knowledge graphs. As some premise may not included in the referred guidelines. During annotation, physicians will incorporate their own knowledge to complete the knowledge graph.

A.2 Structure of Knowledge Graph

The entire knowledge graph, denoted as $\mathcal{K}$, is stored in separate JSON files, each corresponding to a specific disease category $i$ as $\mathcal{K}_{i}$. Each $\mathcal{K}_{i}$ comprises a procedural graph $\mathcal{G}_{i}$ and the corresponding premise $p$ for each disease. As illustrated in Figure 7, the procedural graph $\mathcal{G}_{i}$ is stored under the key "Diagnostic" in a dictionary structure. A key with an empty list as its value indicates a leaf diagnostic node as $d^{\star}$. The premise for each disease is saved under the key of "Knowledge" with the corresponding disease name as an index. For all the root nodes (e.g., Suspected Heart Failure), we further divide the premise into "Risk Factors", "Symptoms", and "Signs". Note that each premise is separated by ";".

A.3 Annotation and Tools

We have developed proprietary software for annotation purposes. As depicted in Figure 8, annotators are presented with the original text as observations $o$ and are required to provide rationales ($z$) to explain why a particular observation $o$ supports a disease $d$. The left section of the figure, labeled Input1 to Input6, corresponds to different parts of the clinical note, specifically the chief complaint, history of present illness, past medical history, family history, physical exam, and pertinent results, respectively. Annotators will add the raw text into the first layer by left-clicking and dragging to select the original text, then right-clicking to add it. After each observation, a white box will be used to record the rationales. Finally, a connection will be made from each rationale to a disease, represented in a grey box. The annotation process strictly follow the knowledge graph. Both the final annotation and the raw clinical note will be saved in a JSON file. We provide the code to compile these annotations and detailed instructions for using our tool on GitHub.

A.4 Access to DiReCT

Implementation code and annotation tool are available through https://github.com/wbw520/DiReCT. Data will be released through PhysioNet due to safety issues according to the license of MIMIC-IV (PhysioNet Credentialed Health Data License 1.5.0). We will use the same license for DiReCT. The download link will be accessible via GitHub. We confirm that this GitHub link and data link are always accessible. We confirm that we will bear all responsibility in case of violation of rights.

B.1 Prompt Settings

In this section, we demonstrate the prompt we used for each module (From Table 7-9 for narrowing-down, perception, and reasoning module, respectively).

In Table 7, {disease_option} is the name for all disease categories, and {note} is the content for the whole clinical note. The response for the model is the name of a possible disease $\hat{i}$.

In Table 8, {disease} is the disease category name predicted in narrowing-down. The content marked blue is the premise, which is only provided during the $\mathcal{k}$ setting. In this module, {premise} is offered with all information in the knowledge graph. Different to narrowing-down, {note} is implemented for each clinical data $R=\{r\}$ and the outputs are combined together for $\hat{\mathcal{O}}$ and $\hat{\mathcal{E}}$.

In Table 9, {disease} is the disease category name and {disease_option} is consisted by the children nodes $\{d_{n}\}_{n}$. Similarly, the premise on the blue is only available for the $\mathcal{k}$ setting. It provides the premise that are criteria for the diagnosis of each children node. {observation} is the extracted $\hat{\mathcal{O}}$ in previous step. We provide all the prompts and the complete implementation code on GitHub.

B.2 Details of Automatic Evaluation

The automatic evaluation is realized by LLama3 8B. We demonstrate the prompt for this implement in Table 10 (for observation) and Table 11 (for rationalization). Note that we do not use few-shot samples for the evaluation of observation. In Table 10, {gt_observation} and {pred_observation} are from model prediction and ground-truth. As this is a simple similarity comparison task to discriminate whether the model finds similar observations to humans, LLama3 itself have such ability. We do not strict to exactly match due to the difference in length of extracted raw text (as long as the observation expresses the same description). In Table 11, {gt_reasoning} and {pred_reasoning} are from model prediction and ground-truth. We require the rationale to be complete (content of the expression can be understood from the rationale alone) and meaningful; therefore, we provide five samples for this evaluation. We also provide all the prompts and the complete implementation code on GitHub.

B.3 Prediction Samples

Figure 9 and 10 shows two sample generated by GPT-4. The ground-truth PDD of the input clinical note is Gastroesophageal Reflux Disease (GERD) and Heart Failure (HF). In these figure, purple, orange, and red indicate explanations only in the ground truth, only in prediction, and common in both, respectively; therefore, red is a successful prediction of an explanation, while purple and orange are a false negative and false positive.

In Figure 9, we can observe that GPT-4 can find the key observation for the diagnosis of GERD, which is consistent with human in both observation and rationale. However, it still lacks the ability to identify all observations and establish accurate relationships for diseases. In Figure 10, the model’s predictions do not align well with those of a human doctor. Key observations, such as the relationships between BNP and LVEF, are incorrectly identified, leading to a final misdiagnosis.

B.4 Experiments for No Extra Knowledge

We demonstrate the prompt used for $\mathcal{D}^{\star}$ and no knowledge settings in Table 12 and Table 13, respectively. {note} is the text of whole clinical note and {note} in Table 12 is the name of all leaf node $\mathcal{D}^{\star}$.

B.5 Experimental Settings

All experiments are implemented with a temperature value of 0. All close sourced models are implemented in a local server with 4 NVIDIA A100 GPU.