State of natural language processing in surgery: Part 2

Challenges of NLP

As with any technology, NLP comes with a set of challenges that may impact its usefulness.

Bias. Biases in AI are a concern because they can be built into the data used to develop AI. Less is known about biases specific to NLP, but it likely has similar issues.

“The model is only as good as the information on which it’s trained,” Dr Fischer says. “If a population is underrepresented or misrepresented in the training set, the algorithm won’t perform well on validation.” Providers’ conscious or unconscious attitudes may influence how information is entered into the EHR, resulting in bias, says J. Marc Overhage, MD, PhD, chief medical informatics lead for Elevance Health. Bias could occur in several areas, including sex, race, and socioeconomic status. “Whatever biases live in the data will live on in the algorithm,” Dr Overhage says.

“The issue of training [a machine] on an existing data set is that data set currently contains all the presuppositions that led to that circumstance,” says Yaa Kumah-Crystal, MD, MPH, clinical director for health IT in the biomedical informatics department at Vanderbilt University Medical Center in Nashville, Tennessee. “It’s not just ‘set it and forget it’: You have to continue to evaluate it so the model can continue to learn.”

For example, a 2021 study by Wong and colleagues found that the sepsis prediction model used in Epic’s EHR did not identify two-thirds of patients with sepsis. In its response to the study, Epic noted: “Each health system needs to set thresholds to balance false negatives against false positives for each type of user. When set to reduce false positives, it may miss some patients who will become septic. If set to reduce false negatives, it will catch more septic patients; however, it will require extra work from the health system because it will also catch some patients who are deteriorating, but not becoming septic.” This points to the need to validate whether a model used in one organization is effective in another.

Dr Kumah-Crystal, who also is a pediatric endocrinologist who works on improving the usability of EHR through NLP, says another way to address bias is to have an advisory panel with representation from clinicians, ethicists, social scientists, and patients who can help determine the types of questions that a computer should be asking. For example, social determinants of health such as homelessness should be incorporated if they are representative of the patient population being treated. “IT people are data optimistic—we see all the good things that can come from it,” she says. “But it’s important to have other people at the table who might have a different lens. [For example,] patients can give you different insights that you might not think about because you are walking a different path.”

Stigmatizing language plays a role in bias. For example, a nurse might write, ‘patient claims they stopped smoking a month before surgery but smoke odor noted on clothes,’ with ‘claim’ implying the nurse does not believe the patient. However, the odor might be because the patient lives with someone who smokes.

“Biases are propagated through stigmatizing language,” says Maxim Topaz, PhD, MA, RN, the Elizabeth Standish Gill associate professor of nursing at Columbia University Medical Center and Columbia University Data Science Institute in New York. Topaz adds that this type of language is often used with minority communities. He is working on a way to use NLP to alert clinicians to the use of stigmatizing language so they can avoid it. This is particularly important given that clinicians are required to share notes with patients when requested.

Cost. Basic NLP voice recognition is inexpensive and widely available, but the power of NLP lies in its ability to analyze and learn. Those capabilities require neural networks and deep learning techniques that are advanced technologies, and, therefore, expensive.

Privacy. Privacy is a concern, particularly when it comes to the large-scale use of EHRs that are needed by researchers to provide optimal results. “Hospitals are not able to share much information because of HIPAA restrictions, and there are few places where researchers can buy data for research,” says Dina Demner-Fushman, MD, PhD, an investigator at the National Library of Medicine in Bethesda, Maryland. She is also the chair of the American Medical Informatics Association NLP work group; her research includes the use of NLP to extract and use data for clinical decision support and education.

When data are shared for research purposes, steps must be taken to de-identify it. At the National COVID Cohort Collaborative, Gabriel says the team uses a variety of methods to ensure HIPAA compliance. One method, for example, is to use only 3-digit ZIP codes and to randomly shift dates in the record while still maintaining the sequence of events.

Technical and terminology barriers. The number of commercial products using NLP is increasing but still limited, with more information needed about their capabilities, Topaz says. “Not every organization has the IT ability to tweak the model,” he adds.

Another technical issue is that risk prediction models need to be specific to an organization because the terminology clinicians use and patient populations can vary widely between geographic locations and clinical settings. Dr Fischer notes that when dialects and terminology vary because of geographic region, it may influence how effectively an NLP-generated algorithm functions.

For example, what works for clinicians in Philadelphia may not work for clinicians in San Francisco, where there are different populations of people who use different terminology. “It really speaks to the importance of the NLP algorithm being a living thing,” he says. “Tools need to be tested to ensure performance is maintained in a new geographic setting.”

A new way of thinking. NLP enables predictive analytics, which requires a new way of thinking for clinicians, says Stephen Morgan, MD, senior vice president and chief medical information officer for Carilion Clinic in Roanoke, Virginia. “It has been more of a learning curve for clinicians to integrate these tools and data insights into their practice than we anticipated.”

Jennifer Martin, DNP, RN, NEA-BC, senior director of clinical informatics at Carilion, advises clinicians to consider NLP and other forms of AI as partners in delivering care. “It helps point out a little sooner than our natural brain process that there is an issue, but nurses and physicians still have to understand how to incorporate it into their practice,” she says. “The machine will put the data together logically, but it may not be accurate.”

For example, a nurse may not have documented an action that would impact what the machine is concluding. “You still have to use critical thinking and look at your patient,” says Martin, who uses the analogy of an ECG monitor alarming because of a flat line, but the patient is fine, indicating a mechanical, not physiological, problem. However, she adds, “[Information obtained through NLP and AI] can help us [clinicians] react faster and bring information together we have not considered.”

Dr Demner-Fushman says dashboards that incorporate results from NLP tools need to aggregate the results for clinicians, who do not have the time to do it themselves. She worked with a team to create a dashboard for interdisciplinary teams that included patient information and resources such as practice guidelines and current information on the patient’s diagnosis and medications. When Dr Demner-Fushman explored why usage did not grow as expected, most clinicians said they did not have time to read the information and wanted more of a summary. “We’re working on trying to accomplish that,” she says.