Public Datasets: A Foundation to Artificial Intelligence in Health Care

Health and Digital Literacy

AI in health care will affect everyone, requiring key stakeholders to understand how it works. Patients, providers, healthcare professionals, technology companies, and legislators all play a role in developing and accepting technology. There are high stakes in health care, resulting in a requirement to understand how AI works to ensure its safety. Mistakes in diagnoses or treatment can ultimately cost a patient’s life. To implement any new technology in health care, it is paramount that it is safe and effective. Understanding a new technology like AI is the first step in implementation.

Scientific articles on AI are jargon heavy, reducing the application to only those with specific informatics training. Healthcare professionals, particularly primary care providers, will not trust a system they cannot understand. Patients will also be hesitant to accept technology their provider does not. Legislators, working to guide technological advancement and protection of patients, need to understand AI’s impact to legislate effectively. With legislation moving at a significantly slower pace than technological innovation, it is more likely that legislative efforts will always be one step behind. Despite the speed of technological innovation, these key stakeholders need a working understanding to effectively participate in AI’s healthcare implementation.

Digital literacy is required in addition to a recent push for improving health literacy among patients. Health literacy is necessary to understand the impact of technology on health care, and digital literacy is required to understand the mechanisms of how AI affects patients. A general understanding of AI, combined with health and digital literacy, will ensure that providers and legislators make competent decisions to ensure patient safety.

AI in Health Care

While there are concerns in the public domain about AI in general and additional considerations for its application in health care, there is a strong case that it should be embraced. AI models have already shown significant improvements in health care. In emergency medicine, advancements include patient triage optimization, risk stratification, and patient outcome predictions.¹ AI in medical imaging provides interpretations that are at least equivalent to those of a radiologist.² While an all-purpose AI system is not available for widespread use, these examples show how specific implementations of AI in health care are incrementally improving patient care.

There are limitations to AI in health care that must be overcome. Transparency, or how an AI system is “thinking,” is a concern for validating new systems and ensuring quality control.¹ Patient data must be protected, which adds liability to technology companies.¹⁰ Ultimately, the quality of data used to train AI systems plays a foundational role in AI validation, reproducibility, and bias.¹⁰ Open data provides the foundation for solving these limitations. Despite patient safety and data anonymity concerns, AI systems are being built in all industries. To ensure safe and effective implementation in health care, AI innovations should be openly embraced to identify and resolve limitations.

AI Modalities and Training

There are several different AI training modalities and definitions for AI. For this review, AI will be referred to in general terms rather than as a specific type. Some common modalities include machine learning (ML), deep learning (DL), and large language models (LLMs). The functionality of a specific AI model may supersede others in health care, but the importance of open data for training remains central to effective implementation and patient safety.

ML is a common model for developing AI systems for specific tasks. The system is trained on a sufficient dataset, which allows the system to recognize patterns or perform regression and classification.¹¹ The human training component of ML and the complexity of the problem it must solve limit this model.¹¹

DL is a subset of ML that expands upon ML’s capabilities but introduces the black-box issue of transparency, or how the model makes predictions. This is a key issue in health care when considering patient safety. DL solves significantly more complex problems than ML by using a mathematical framework to automatically derive representations from given data.¹¹ This method’s limitations in health care are centered on the copious amounts of data required for training and the black-box nature of the decision-making.

LLM or natural language processing (NLP) are subsets of ML based on text generation.¹² These models also require a significant amount of training data but have seen promising use in the public with applications like ChatGPT. In health care, ChatGPT has also been effective at enhancing healthcare-associated infection surveillance.¹³ LLMs may also provide efficiencies in administrative tasks like clinical note-taking or patient chatbots that may provide medical advice. Consider a healthcare patient chatbot trained on existing medical knowledge with access to a patient’s entire medical record. The AI will surpass a doctor’s ability to correlate all the information. Additionally, depending on the training data, LLMs can already produce near authentic scientific articles.¹⁴ There are concerns about LLM AI being trained using copyrighted material like medical texts or scientific articles that are not public. Despite this concern, LLMs show great potential application in science and health care.

Regardless of the model or application, training plays a role in the success of AI. The training of AI systems can be supervised, unsupervised, or trained with reinforcement. Supervised and unsupervised learning is contingent on the data being labeled or unlabeled, respectively.² Reinforcement learning incorporates user input as feedback to the system.² Supervised and reinforcement learning requires additional human input, making these systems more trustworthy because of the human validation of the algorithm. Unsupervised training, as seen in DL, allows systems to identify patterns unrecognizable by humans. However, the black-box nature of this process is not acceptable in health care.

The type of data used for training depends on the AI model and function. The goal is to make the data open to the public for effective training of AI in health care. In health care, the data required can contain clinical notes, diagnoses, medical imaging, patient history, and patient demographics, such as age, sex, and race. Information about patients that may be used to identify them is termed protected health information (PHI). PHI used in training data is a concern for patient privacy. The training data will also include medical knowledge. This can come from various sources, such as textbooks, research, and expert opinions. Combined, the data form the knowledge base of the system being trained. To effectively train an AI system, the data are labeled by engineers.

Labeling data is important in supervised learning. The data label provides context to the system when training; this is primarily the training technique for ML.^2,15 For example, an AI system trained to detect pneumonia in chest x-rays will be trained using chest x-ray images labeled by a radiologist to indicate the presence of pneumonia. Additional data, such as patient demographics and medical history, aid the system in identifying patterns. Conversely, unlabeled data are used in DL models. DL models require copious amounts of data to build comprehensive AI systems in unsupervised learning. In DL models, the system is allowed to identify patterns without human intervention. Unlabeled data reduce the burden of human input required for ML but are also the impetus for the black-box decision-making of more complex AI systems.

Training data may be used in different ways, but regardless, the quality and quantity of data used for training will affect the outputs of the AI model. The adage, “garbage in, garbage out,” applies to the training of AI models. These are issues that open data can solve. Patient population bias and lack of interoperability are frequently seen in training sets. Consider an AI model programmed to assist with disease diagnosis. A dataset built by hospital X that contains information on healthy individuals and those with common chronic illnesses may not produce reliable results for uncommon diagnoses. Training data from a single healthcare facility in a rural setting, hospital X, will also not be interoperable with a large healthcare facility in an urban setting. These examples of homogeneous datasets show how the training data can hamper interoperability or create a bias toward the patient population contained within the training data. Open data solves this issue by pooling training data across many patient populations.

Homogenous data from a single source prohibit reproducibility and validation and increase the risk of bias.¹ A frequent issue with new AI models in health care is that a research facility or hospital will use internal data to train the system. The results of a system trained with specific internal data to a facility are not reproducible with different datasets or patient populations. This prevents the independent validation of a new model and significantly reduces the interoperability of AI models in varying health systems. AI models trained with open data will reflect better reproducibility.

Need for Open Data

AI training is time consuming and often restrictive for software companies or healthcare research teams starting from scratch.⁶ Copious amounts of data are required for an AI system to function properly. More data than what currently exists are required to train DL algorithms.⁶ The labeling of data is time consuming and labor intensive. Using open data removes the data collection step. If labeled data are shared with open data sets, then this removes an additional step. The sharing of trained AI models and curated data increases the quantity and quality of open data while accelerating the growth rate of AI in medicine.⁶

AI models trained with homogenous data may contain bias. Data from a single research facility or hospital will always be limited compared to open data pooled from diverse patient populations and sources. The patient population used to train AI will ultimately create a bias toward those patients. Using homogenous data rather than open data results in training data with similar patient demographics, chronic diseases, environments, and nutrition. These factors will affect the system’s predictions or pattern recognition.

New software or medical devices in health care must be validated and approved for use. Developers and research facilities may be reticent to share training data for additional facilities to use for validation due to their cost to curate the data or for proprietary concerns. If the data are shared, they may not represent a new healthcare facility’s patient population. AI models trained on open data are interoperable because the data are available for validation and, as a more extensive data pool with better diversity, are more likely to be accurate across differing patient populations.

Open Data Initiatives

Despite a historical lack of a data-sharing culture, open data is often used in research settings. Open science includes initiatives supporting the emergence of open data and open software.⁶ Funding also plays an important role in science and is significant in open data sharing.

Open science initiatives support open data in research. The F1000 Research model is an excellent example for future research. F1000 Research is an open research publishing platform that supports transparent and timely research publication in all study areas.¹⁶ The 3 tenets of F1000 Research are open access, open data, and open peer review. All articles published by F1000 Research are freely available.¹⁷ Researchers must include all data or, at minimum, list what data cannot be shared and why (eg, confidential patient identifiers).¹⁷ Finally, an open peer review process allows for faster review of articles and citation of peer reviews.

Research funding, in addition to the structure of research peer review and publishing, is available to support open data initiatives. In 2018, Plan S, launched by cOAlition S, mandates that any research using public grants must be published in compliant Open Access journals starting in 2021.¹⁸ Founders of Plan S include well-known organizations, such as the Bill and Melinda Gates Foundation, the Howard Hughes Medical Institute, and London-based funder Wellcome.⁷ In addition, the European Commission, the US government, and the Findability, Accessibility, Interoperability, and Reusability (FAIR) principles all leverage funding to drive the open science initiative.¹⁷

There are many initiatives to collect and promote open data. Both governments and private institutions support open datasets. Some examples of open data available include Germany’s Nationale Forschungsda-teninfrastruktur (NFDI); the United States’ data.cdc.gov, data.gov, and healthdata.gov; and Massachusetts Institute of Technology’s MIMIC program.^6,8,19

These large databases support the collection and storage of data for research use. The NFDI contains branches for many areas of study, but the NFDI 4 Health branch works explicitly to “build a comprehensive inventory of German epidemiological, public health, and clinical trial data.”²⁰ Data.cdc.gov, healthdata.gov, and data.gov provide US-specific data collected by government entities that may be used to inform the public, drive innovation, and support a transparent government.¹⁹ These examples of government databases provide secure collections of copious data that can be regulated by the departments that are gathering them.

Data.gov is less helpful for healthcare AI datasets because it focuses on broad population datasets published by US agencies, such as the Centers for Medicare and Medicaid Services (CMS) and the National Oceanic and Atmospheric Administration. Population and environmental data collected by these agencies may be important for some models, especially those used for public health tracking and prediction. Still, the open data provided by data.cdc.gov is better for AI training in health care. Data.cdc.gov includes open data sorted by categories, such as disease, injury, and disability. It also includes instruction for developers and integrates the Socrata Open Data API software to provide access to open data resources from governments, nonprofits, and global nongovernmental organizations.²¹

Open data is also available for specific diseases. During the COVID-19 pandemic, several datasets were developed specifically for the training of AI models to detect COVID-19 in chest x-rays: COVID-19 Image Data Collection, COVID-19 Chest X-ray Dataset Initiative, ActualMed COVID-19 Chest X-ray Dataset Initiative, and COVID-19 Radiography Database (Italian Society of Medical and Interventional Radiology).²² These types of highly focused datasets may be used or combined to train AI systems with the goal of highly accurate disease prediction. These 4 COVID-19 open datasets are frequently combined with pre–COVID-19 open data, such as the Radiological Society of North America Pneumonia Detection Challenge dataset. Combined, these make up an open data source termed COVIDx.²²

Google has provided a great example of the power of open data when compiled like the COVIDx data. In 2020, the Google DeepMind team released a DL system called AlphaFold. With the help of over 50 years of open data, this chemistry-based system predicted 98.5% of the human proteome. Previous years of research only managed to produce 17% of the human proteome.⁶ The success of AlphaFold supports the conclusion that open data infrastructures are the key to broader AI applications.⁶

Challenges

Open datasets are not without challenges. Patients, primary care providers, and legislators should know the challenges and the solutions for implementing AI in health care. Several issues to address include privacy, bias, confounding variables, and using datasets with errors.^17,22,23

Privacy is arguably the first concern for patients’ PHI. If not addressed appropriately, patients may refuse to share their data, which would negatively affect open data collection. Existing legislation, such as the Health Insurance Portability and Accountability Act of 1996 (HIPAA), protects patient’s privacy. However, despite the legal protection of PHI, data used to train AI models can require PHI. Cybercriminals can target the training data by tricking AI systems into divulging information about the training data.²³ These types of cyberattacks compromise patient data and undermine the trust of AI systems in health care.⁹

Differentially private ML is a possible technical solution to the issue of privacy.²³ Originally proposed by Dwork et al in 2006, this programming method adds “noise” to the data, obscuring targeted personal information.²⁴ This noise obscures PHI but also requires extra due diligence from the researchers to ensure the accuracy of the results. Another option is to ensure the use of anonymized patient data in training sets to prevent targeted attacks that leak PHI.²³

After considering the privacy of patient data, we must look at the quality of the data. Open data can lead to bias and propagation of errors or confounding variables if not used carefully.²² Confounding variables are presented when patient information is missing from datasets. This introduces variables that cannot be accounted for without that data. Open data with errors or confounding variables may be used repeatedly when not properly scrutinized, spreading bias into all models. Repeated publication of “successful” AI models trained with bad data may further reduce the trustworthiness of AI in health care.²² The repeated use of COVID-19 data containing errors demonstrated this.

The use of open data during the COVID-19 pandemic demonstrates how poor datasets can affect AI systems output and trust in AI in health care. One study analyzed the use of publicly available COVID-19 datasets to train AI models that detected COVID-19 from chest x-rays. Bias was demonstrated in COVID-19 prediction models trained on publicly available data.²² A significant issue with the training data was its labeling. Training data labeled for screening are not equivalent or appropriate for AI models intended for diagnosing.²² Chest x-rays labeled to indicate the presence of pneumonia will not help an AI system understand if the pneumonia is specifically due to COVID-19. Therefore, these data will help screen chest x-rays for pneumonia but will not be able to diagnose COVID-19 infection specifically. As a result, the interoperability of this COVID-19 AI model on external data was only 45%, highlighting the importance of standardization and quality review of data in any open database.²²

These hurdles to open data are not unsurmountable. Technology and legislation address privacy concerns. Process improvements related to data quality are essential to resolving issues with bias, confounding variables, and errors within datasets.

US Legislation

HIPAA protects PHI in several ways. The 3 parts of this legislation are commonly known as the Privacy Rule, the Security Rule, and the Breach Notification Rule. The Privacy Rule defines and protects PHI.⁹ The Security Rule addresses electronic PHI by requiring covered entities (healthcare providers, healthcare clearing houses, or health plans) to take steps to prevent cyberattacks and set limits to who can access PHI within their system. Finally, the Breach Notification Rule requires covered entities to notify patients or the public in the case of a data breach. However, HIPAA does not directly regulate business associates of covered entities, nor does it cover newer technologies, such as mobile health applications (eg, Apple Health or Samsung Health) or AI. These limitations of HIPAA are addressed in later legislation.

Subsequent applicable US legislation incentivizes the use of EHRs and promotes interoperability. The Patient Safety and Quality Improvement Act of 2005 (PSQIA) and the Health Information Technology for Economic and Clinical Health Act of 2009 (HITECH) are examples. PSQIA and HITECH have different goals but work together to build on HIPAA and set the foundation for heterogeneous open data.

PSQIA focuses on quality and patient safety, while HITECH supplements the legislation provided by HIPAA. PSQIA initiates collecting and storing deidentified patient data for research and quality assurance of patient safety and errors.⁹ This is not open data for research but is legislation that requires collecting and storing patient information. HITECH supports the framework of HIPAA by incentivizing the use of electronic health information technology (IT). Under HITECH, business associates of covered entities are responsible for HIPAA rules. It also enhances the risk assessment and breach notification portions of HIPAA, covering the gaps in the protection of PHI by business associates of covered entities.

CMS supports HIPAA and HITECH legislation with the Promoting Interoperability Programs. The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) and Merit-based Incentive Payment System provide financial incentives to providers for promoting interoperability of EHRs.⁹ Financial incentives may include higher reimbursement rates for services by providers meeting the higher level of care defined by this legislation and lower rates for providers who are not meeting expectations.²⁵

The CMS Information Blocking Rule took effect in 2020 under the 21st Century Cures Act. This rule removes barriers to information access by giving patients control over how, when, and with whom patient data are shared.⁹ It does this by creating new standards and mechanisms for healthcare providers, insurers, and software developers to enable patients access to their data.⁹ As of 2021, 88% of office-based physicians have adopted an EHR.²⁶ This represents a doubling of adoption since 2008.²⁶ The improved accessibility to information supports the quantity of data required for adequate AI training.

While this legislation supports EHR adoption and information sharing, AI in health care is not addressed. In December 2023, the Department of Health and Human Services (HHS) finalized a rule to advance health IT interoperability and algorithm transparency. This rule first establishes transparency requirements for AI and other predictive algorithms.^27,28 It also adopts the United States Core Data for Interoperability version 3. This data standard focuses on the accuracy and completeness of patient data with the goal of “promot[ing] equity, reduc[ing] disparities, and support[ing] public health data interoperability.”²⁸ The rule also enhances information blocking requirements to “encourage secure, efficient, standards-based exchange of electronic health information under the Trusted Exchange Framework and Common Agreement^SM (TEFCA^SM).”²⁸ Lastly, the rule implements the 21st century Cures Act’s “Insights Condition,” requiring focused reporting metrics by health IT program developers to further support interoperability. This timely legislation is the key to the structured collection of open data and regulation of AI transparency in health care to ensure patient safety.

International Legislation

Serving as a model for many other countries, the General Data Privacy Regulation (GDPR) passed by the European Union (EU) in 2016 is similar to the protections provided by HIPAA and HITECH in the United States. Organizations may use patient data for legitimate purposes, such as diagnoses or billing, and maintain it until that purpose is complete.⁹ Patients must be informed of how the data are being used and be able to amend them if there are errors or updates to their medical history.⁹ The GDPR requires security provisions but allows data sharing with third parties.⁹ As with HIPAA, GDPR does not cover the third party; the organization sharing the data is responsible for the third party’s use. This is demonstrated with healthcare organizations that outsource tasks like billing, requiring the sharing of PHI for the third party, the billing company, to bill the patient.

Canada protects PHI like the EU, with the addition that individual provinces maintain specific privacy laws. The Personal Information Protection and Electronic Documents Act requires any organization holding personal data to abide by 10 principles. The principles are accountability, identifying purposes, consent, limiting collection, limiting use, disclosure and retention, accuracy, safeguards, openness, individual access, and challenging compliance.⁹ These principles effectively provide the same safeguards seen in the United States and EU.

Health care is typically limited to a patient’s home country, but as globalization continues, the value of having open data that includes other countries is apparent. From a medical perspective, more data provide a more robust dataset. From the point of view of AI developers, systems trained on global datasets open the possibility of worldwide software interoperability.