Lesson 2 of 4 · AI for Doctors
The Healthcare AI Landscape
The CMO Who Couldn't See the Forest
Dr. Priya Narayanan had a problem that no diagnostic algorithm could solve. As the newly appointed Chief Medical Officer of a 340-bed community hospital in the Research Triangle, she had inherited a mandate from the board: modernize clinical operations with artificial intelligence. The board had seen the press releases -- Mayo Clinic deploying AI for cardiac risk prediction, Mount Sinai using deep learning for sepsis detection, Mass General demonstrating AI-assisted radiology reads that matched attending-level accuracy. They wanted their hospital to be "AI-forward." They gave Dr. Narayanan a budget, a twelve-month timeline, and a stack of vendor brochures that was, quite literally, fourteen inches tall.
Within six weeks, Dr. Narayanan had taken meetings with fifteen different AI vendors. Each one had a polished slide deck, a charismatic sales engineer, and a demo that looked flawless in a conference room. Each one promised transformative outcomes -- reduced documentation burden, faster diagnoses, fewer missed findings, lower readmission rates, happier physicians, better patient outcomes, and (of course) a positive return on investment within eighteen months.
The problem was not that none of them seemed good. The problem was that they all seemed good -- in their demos. But Dr. Narayanan had been practicing medicine for twenty-two years before moving into administration, and she had learned something in the ICU that served her well in the boardroom: the presentation that looks cleanest is often the one hiding the most assumptions.
She started asking harder questions. "What does your FDA clearance actually cover -- the algorithm itself, or just the workflow software around it?" One vendor paused for seven seconds before answering. "How does your ambient documentation tool handle a bilingual patient encounter where the patient switches between English and Tagalog mid-sentence?" Another vendor said they would "get back to her." They never did. "What happens to the clinical data my physicians generate while using your platform? Does it feed back into your model training? And if so, under what BAA terms?" A third vendor's compliance officer was brought in for a follow-up call that lasted ninety minutes and ended with more questions than answers.
By week eight, Dr. Narayanan realized she did not have a vendor selection problem. She had a landscape comprehension problem. She could not evaluate individual tools because she did not have a framework for understanding the categories of healthcare AI, what each category actually did well, what the realistic limitations were, and how the pieces fit together in a working clinical environment.
So she built one.
She spent a weekend at her kitchen table with a legal pad, her laptop, and two decades of clinical instinct. She drew a map of every AI tool she had evaluated, every one she had read about in peer-reviewed literature, and every one her physicians had asked her about. She organized them not by vendor name or marketing promise, but by what they actually did, what evidence supported them, and where they sat on the spectrum from proven to speculative.
The map she built that weekend became the foundation for her hospital's AI strategy. It also became the most-requested presentation at her state medical association's annual conference. Physicians did not want another vendor pitch. They wanted what Dr. Narayanan had built: an honest, category-by-category guide to what healthcare AI actually looks like in practice -- what works, what is promising, what is hype, and what you should spend your time learning.
This lesson gives you that map.
The Six Categories of Healthcare AI
Healthcare AI is not a single technology. It is an ecosystem of tools built on different technical foundations, designed for different clinical contexts, regulated under different frameworks, and sitting at very different stages of maturity. Treating "healthcare AI" as one thing is like treating "medication" as one thing -- technically accurate but practically useless for making clinical decisions.
The six categories below represent the major domains where AI tools are being deployed in clinical medicine as of early 2026. Understanding these categories -- and the boundaries between them -- is the first step toward making intelligent decisions about which tools deserve your attention, your institution's investment, and your patients' trust.
900+
FDA-Cleared AI Devices
As of early 2026, the FDA has authorized over 900 AI/ML-enabled medical devices, with radiology AI representing the largest subcategory by far.
Category 1: FDA-Cleared Diagnostic AI
This is the most mature, most regulated, and most evidence-backed category of healthcare AI. These are algorithms that have gone through the FDA's regulatory process and received clearance (typically via the 510(k) pathway or the newer De Novo classification) for specific clinical applications. The FDA maintains a public database of authorized AI/ML-enabled medical devices, and as of early 2026, it lists over 900 cleared devices.
What these tools actually do:
The key word is specific. FDA-cleared diagnostic AI tools are not general-purpose thinking machines. They are narrow algorithms trained on large datasets to perform one defined task -- detecting a particular finding in a particular type of image, flagging a particular pattern in a particular data stream, or quantifying a particular measurement in a particular clinical context.
Radiology AI represents the largest subcategory by far. Tools like Aidoc, Viz.ai, and Qure.ai analyze medical images -- CT scans, X-rays, MRIs -- to detect specific findings. Viz.ai's stroke detection platform, for example, analyzes CT angiography images to identify large vessel occlusions and automatically alerts the stroke team. It does not read the entire CT scan. It does not generate a radiology report. It does one thing -- detect LVO -- and it does it faster than the traditional workflow of waiting for a radiologist to open the study. In time-sensitive stroke care, where "time is brain," those minutes matter enormously.
Aidoc operates similarly but across a broader range of findings: pulmonary embolism on CT, cervical spine fractures, intracranial hemorrhage. The common thread is triage -- these tools do not replace the radiologist's read, but they re-prioritize the worklist so that critical findings get seen first.
Pathology AI is a rapidly growing subcategory. Paige AI received the first FDA clearance for an AI-based pathology tool -- its prostate cancer detection algorithm that assists pathologists in identifying areas suspicious for cancer on digitized prostate biopsy slides. PathAI is developing similar tools for other cancer types. These tools overlay heatmaps on digital pathology slides, highlighting regions of interest and flagging areas where the algorithm's confidence level suggests further review.
Cardiology AI includes tools for ECG interpretation (the AliveCor KardiaMobile system for atrial fibrillation detection was one of the early consumer-facing FDA-cleared AI devices), echocardiographic analysis (Caption Health's AI-guided ultrasound helps non-specialists obtain diagnostic-quality cardiac images), and continuous rhythm monitoring.
Ophthalmology AI is notable for containing the first fully autonomous FDA-cleared diagnostic AI system: IDx-DR (now Digital Diagnostics' LumineticsCore). Unlike most diagnostic AI tools that assist a human reader, IDx-DR was cleared to make a clinical decision independently -- it screens retinal images for diabetic retinopathy and provides a diagnosis without requiring interpretation by an ophthalmologist. A primary care clinic can use the device, obtain retinal images, and receive a diagnostic result. This remains one of the only truly autonomous AI diagnostic tools in clinical use.
What to actually expect:
If you are considering FDA-cleared diagnostic AI for your practice or institution, here is the honest assessment. These tools generally perform well on the specific task they are cleared for, within the specific patient population and imaging parameters they were validated on. They reduce time-to-detection for critical findings. They improve workflow efficiency by intelligently triaging work queues. And they provide a second set of "eyes" that does not get fatigued, distracted, or biased by anchoring.
However, they are narrow. An AI cleared for detecting pulmonary embolism on contrast-enhanced CT does not detect pneumonia, does not evaluate for pleural effusion, and does not notice the incidental thyroid nodule in the field of view. They also perform differently on populations that were underrepresented in their training data. The FDA has been increasingly focused on requiring demographic performance data in premarket submissions, but gaps persist.
Cost reality: These tools typically involve significant upfront integration costs (they must interface with your PACS, EHR, or clinical workflow systems), per-study fees or annual licensing costs ranging from tens of thousands to hundreds of thousands of dollars, and ongoing validation requirements. For large health systems processing high volumes, the ROI can be straightforward -- faster stroke alerts reduce disability and downstream costs. For smaller practices, the math is harder.
Category 2: Ambient Clinical Documentation
If FDA-cleared diagnostic AI is the most technically sophisticated category, ambient clinical documentation is arguably the one with the most immediate, tangible impact on physician daily life. These tools listen to patient encounters -- in-person or via telehealth -- and automatically generate clinical documentation.
The major players:
Microsoft's DAX Copilot (formerly Nuance DAX) is the market leader by deployment scale. Built on the foundation of Nuance's decades-long dominance in clinical speech recognition (Dragon Medical), DAX Copilot listens to the patient-physician conversation through a mobile app or ambient microphone, processes it through large language models, and generates a structured clinical note -- typically in SOAP or other standard formats. The physician reviews, edits, and signs the note. DAX is deeply integrated with Epic and other major EHR systems, which gives it a significant distribution advantage.
Abridge has gained rapid traction, particularly after its partnership with Epic and deployment at major health systems including UCSF, Yale, and the University of Kansas Health System. Abridge differentiates on its "linked evidence" feature -- each statement in the generated note is linked back to the specific moment in the conversation transcript, allowing the physician to verify any claim with one click. This auditability feature addresses one of the core trust concerns with AI-generated documentation.
Nabla targets outpatient and primary care settings with a lighter-weight, faster-to-deploy solution. It works across major EHR platforms and emphasizes ease of adoption -- physicians can start using it with minimal IT integration in some configurations.
Use Healthcare AI Landscape in a low-risk branch or scratch project first. That keeps the lesson concrete without making your first attempt carry production pressure.
Suki takes a slightly different approach, focusing on voice-powered assistance beyond just ambient documentation. Suki functions as a voice-activated clinical assistant that can help with note generation, information retrieval, and other tasks during the clinical encounter.
What they actually deliver:
The documentation time savings are real and well-documented. Multiple published studies and health system reports show reductions of 50-70% in after-hours documentation time. Physicians who were spending two to three hours per evening on "pajama time" documentation report cutting that to under an hour. The impact on burnout metrics, while harder to quantify definitively, is consistently positive in early studies.
The quality of the generated notes varies. For straightforward encounters -- a follow-up visit for a well-controlled chronic condition, a routine post-operative check -- the notes are often clinically acceptable with minimal editing. For complex encounters -- a patient with seven active problems presenting with a new complaint that requires nuanced clinical reasoning -- the notes require more significant physician review and editing. The AI captures what was said but does not always capture the clinical reasoning behind what was said.
The honest limitations:
Ambient documentation tools work best in English. Performance degrades with accented speech, multilingual encounters, heavy medical jargon in specialty contexts, and noisy clinical environments (a busy ED versus a quiet office). They also generate notes based on what was discussed in the encounter -- if you performed a physical exam finding but did not verbally narrate it, the AI has no way to document it. Some physicians have adapted by "narrating" their physical exam findings aloud ("Lungs are clear to auscultation bilaterally, no wheezes or crackles"), which feels unnatural at first but becomes routine.
Privacy considerations are significant. These tools are processing actual patient conversations. They require BAAs with the vendor, compliance with your institution's HIPAA policies, and patient notification (and in some cases consent) that the encounter is being recorded. Most institutions post signage and include notification in intake paperwork.
Cost reality: Ambient documentation tools typically run $100-300 per physician per month on enterprise contracts. For a physician whose time is valued at $150-400+ per hour, saving 1-2 hours per day makes the ROI calculation straightforward on paper. The real cost is in change management -- getting physicians to adopt a new workflow, trust the output, and integrate it into their existing patterns.
Category 3: General-Purpose AI for Clinical Work
This category includes the tools that most people think of when they hear "AI" -- the large language models (LLMs) like Claude, ChatGPT, Gemini, and their variants. These are not built specifically for healthcare, but they have enormous utility in clinical contexts when used appropriately.
If Healthcare AI Landscape becomes part of a recurring workflow, document the exact trigger, boundary, and verification step now. Future speed comes from clarity, not from memory.
What physicians are actually using them for:
The most common clinical applications fall into administrative and cognitive support categories:
Referral and prior authorization letters. This is where this track's chapter title -- "The 90-Second Referral Letter" -- comes from. A well-constructed prompt can transform bullet points about a patient's clinical situation into a polished, persuasive referral letter or prior authorization appeal in under two minutes. Physicians who previously spent 15-20 minutes per letter drafting and wordsmithing report that AI-assisted drafting cuts the task to 2-3 minutes of prompting plus review.
Patient education materials. Generating customized patient handouts at specific literacy levels. "Explain type 2 diabetes management to a patient with a 6th-grade reading level, in both English and Spanish, including dietary recommendations appropriate for a Caribbean diet." No stock patient education library covers that level of specificity.
Literature synthesis. "Summarize the last five years of evidence on SGLT2 inhibitors for heart failure with preserved ejection fraction, including key trial names, sample sizes, and primary endpoints." The output needs verification against the actual literature, but as a starting framework for a literature review, it saves hours.
Differential diagnosis brainstorming. Not as a diagnostic tool -- never as a diagnostic tool -- but as a cognitive aid. "A 34-year-old female presents with episodic flushing, diarrhea, and palpitations. CBC and CMP are normal. TSH is normal. What diagnoses should I consider beyond the obvious?" The AI generates a broad differential that can help combat anchoring bias and prompt consideration of diagnoses you might not have initially considered.
Administrative communication. Drafting emails to colleagues, writing up incident reports, composing committee meeting summaries, preparing slide presentations for grand rounds.
The critical boundaries:
General-purpose AI tools are not FDA-cleared for any clinical decision-making. They are not connected to your EHR (unless through a specific integration, which moves them into a different category). They have no access to your patient's records, labs, images, or history unless you manually provide that information -- which raises immediate HIPAA concerns if you are using a consumer-tier plan.
The appropriate mental model is: general-purpose AI is a brilliant medical scribe who graduated top of their class and has read every textbook, but who has never examined a patient, has no access to the chart, occasionally makes things up with complete confidence, and is not licensed to practice medicine. Use it for what a brilliant scribe is good for. Do not use it for what requires a clinician.
Measure the Healthcare AI Landscape Tradeoff
- Choose one task you repeat often.
- Run it with the model, cost, or performance setting discussed in this lesson.
- Record latency, quality, and cost so you can choose intentionally next time.
Cost reality: Consumer plans range from free to $20-30 per month. Professional and team plans run $25-60 per user per month. Enterprise plans with HIPAA compliance, BAA availability, and no-training guarantees typically start at $30-60 per seat per month with volume agreements. For administrative and cognitive support tasks, even consumer plans can be valuable -- provided you never put protected health information into them without proper de-identification or enterprise-grade contractual protections.
Category 4: EHR-Integrated AI
The major electronic health record vendors -- Epic, Oracle Health (formerly Cerner), MEDITECH, athenahealth -- are all integrating AI directly into their platforms. This category is distinct from standalone AI tools because the AI has access to the patient's chart, the institution's clinical data, and the EHR's workflow context.
Epic's AI ecosystem is the most expansive as of early 2026. Epic has integrated generative AI into multiple modules: in-basket message drafting (the AI reads an incoming patient message and drafts a clinician response), chart summarization (generating concise summaries of complex patient histories for handoffs or specialist referrals), clinical decision support alerts enhanced with AI reasoning, and coding suggestions. Epic has also built an open platform for third-party AI tools -- including Abridge and other ambient documentation vendors -- to integrate directly into the Epic workflow.
Oracle Health has been investing heavily in AI through its acquisition of Cerner and integration with Oracle's cloud AI capabilities. Their focus has been on predictive analytics -- identifying patients at risk for deterioration, readmission, or adverse events -- and on automating administrative workflows within the EHR.
The promise and the reality:
EHR-integrated AI has the highest ceiling of any category because it solves the context problem. Standalone AI tools do not know your patient. EHR-integrated AI does -- it can read the patient's problem list, medication list, lab trends, imaging history, and clinical notes. This contextual awareness enables capabilities that standalone tools cannot match: generating a referral letter that automatically pulls in relevant history, drafting a response to a patient portal message that references their recent labs, or flagging a new medication order that interacts with something already on the patient's medication list in a more sophisticated way than rules-based alerts.
The reality is that EHR-integrated AI is still early. The in-basket message drafting feature, for example, generates responses that physicians report accepting without significant editing roughly 50-60% of the time. The other 40-50% require modification -- sometimes minor, sometimes substantial. The chart summarization features work well for straightforward histories but can miss nuance in complex cases, sometimes omit relevant information, and occasionally hallucinate details that are not in the chart.
Optimize One Repeated Task
- Take one expensive or slow Claude workflow from your week.
- Apply the optimization idea from this lesson to it once.
- Keep the change only if quality stayed acceptable while speed or cost improved.
Cost reality: EHR-integrated AI features are typically bundled into EHR licensing or upgrade packages. For health systems already on Epic or Oracle Health, the marginal cost may be relatively low -- but the implementation cost (workflow redesign, physician training, IT support, validation) is substantial. Smaller practices on less feature-rich EHR platforms may not have access to equivalent AI capabilities.
Category 5: Clinical Decision Support Systems (CDSS)
Clinical decision support is not new -- CDSS has existed since the 1970s in various forms. But AI-powered CDSS represents a significant evolution from the rules-based alert systems that most physicians associate with the term (and often associate with alert fatigue).
Traditional CDSS operates on if-then rules: if potassium is below 3.0, fire an alert. If a patient is on warfarin and a new prescription for fluconazole is entered, fire an interaction alert. These rules-based systems are precise but rigid, and their proliferation has led to the well-documented problem of alert fatigue -- physicians receive so many alerts that they override them reflexively, including the ones that matter.
AI-powered CDSS aims to be smarter about what deserves an alert and how information is presented. Instead of firing an alert every time a lab value crosses a threshold, AI-powered systems can analyze patterns across multiple data points -- vital signs trending in a particular direction, lab values changing in combination, nursing notes mentioning specific symptoms -- to identify patients whose clinical trajectory suggests early deterioration before traditional triggers fire.
Notable tools in this space:
Sepsis prediction algorithms have been among the most studied and most controversial. Epic's sepsis prediction model, deployed across many health systems, has been the subject of published studies questioning its positive predictive value and clinical utility. The core tension: these models identify many patients who are not actually developing sepsis (false positives), leading to unnecessary workups and contributing to alert fatigue. The counter-argument: even a modest improvement in early sepsis detection saves lives, and the cost of missing sepsis is catastrophic.
UpToDate and DynaMed have incorporated AI-driven features that go beyond traditional search. Rather than requiring you to formulate a search query and browse articles, AI-enhanced versions can accept a clinical question in natural language and synthesize an evidence-based answer with citations.
Optimize One Repeated Task
- Take one expensive or slow Claude workflow from your week.
- Apply the optimization idea from this lesson to it once.
- Keep the change only if quality stayed acceptable while speed or cost improved.
Isabel Healthcare uses AI-driven differential diagnosis generation based on clinical features, acting as a clinical reasoning tool that can suggest diagnoses a clinician might not have considered.
VisualDx combines AI image analysis with clinical decision support for dermatologic conditions, allowing clinicians to photograph a skin lesion and receive a ranked differential diagnosis with linked evidence.
Cost reality: CDSS costs vary enormously. EHR-integrated CDSS features may be included in your EHR subscription. Standalone CDSS tools like UpToDate run $300-500+ per user per year for individual subscriptions (often covered by institutions). Specialty-specific AI CDSS tools may involve per-use fees, annual licenses, or integration costs that depend heavily on your existing infrastructure.
Category 6: Emerging and Experimental AI
This is the category where the hype-to-reality ratio is highest and where the most caution is warranted. These are AI applications that are in research stages, early pilot deployments, or pre-regulatory phases. They represent where healthcare AI is going -- but they are not where it is today for most practicing physicians.
AI-driven drug discovery is transforming pharmaceutical research. Companies like Recursion Pharmaceuticals, Insilico Medicine, and Isomorphic Labs (a DeepMind subsidiary) are using AI to identify drug candidates, predict molecular interactions, and accelerate clinical trial design. This matters for physicians indirectly -- it may bring new treatments to market faster -- but it is not a tool you will interact with in clinical practice.
AI-powered genomics is advancing rapidly. Tools that interpret genomic sequencing data, identify clinically significant variants, and match patients to targeted therapies are moving from research to clinical application. Companies like Tempus and Foundation Medicine are at the leading edge.
Multimodal diagnostic AI -- systems that combine imaging data with lab values, clinical notes, and genomic data to generate integrated diagnostic assessments -- is the next frontier beyond single-modality tools. Rather than an AI that reads a chest CT or an AI that analyzes lab trends, imagine a system that integrates the chest CT findings with the patient's lab trajectory, medication history, and clinical notes to generate a probability-ranked differential. This is actively being researched but is not in routine clinical use.
Large language models fine-tuned for clinical medicine -- models like Med-PaLM 2 (Google) and other medically specialized LLMs -- are demonstrating performance on medical board exams that matches or exceeds human physicians. The clinical significance of this is debatable -- passing a board exam is very different from practicing medicine -- but the trajectory suggests that medically specialized LLMs will become increasingly useful clinical tools.
Quick Check
What is the main benefit of using Healthcare AI Landscape well in Claude Code?
The honest assessment: Be aware of this category. Follow it in the literature. But do not make purchasing decisions or clinical workflow changes based on experimental AI. The gap between "performed well in a research study" and "works reliably in my clinical environment with my patient population" is enormous, and the history of medicine is littered with innovations that looked transformative in controlled settings but failed to deliver in the real world.
The Vendor Landscape: What They Will Not Tell You
Dr. Narayanan's framework was not just about categorizing tools -- it was about cutting through vendor presentations to find signal in the noise. Here is what she learned about evaluating AI vendors, distilled from fifteen sales meetings and dozens of follow-up calls.
Question 1: What is your regulatory status, exactly?
"FDA-cleared" has specific meaning. Some vendors will say "FDA-compliant" or "built on FDA-cleared infrastructure" or "designed to meet FDA requirements" -- none of these mean the same thing. Ask for the specific 510(k) number or De Novo authorization. If they cannot provide one, their core algorithm is not FDA-cleared, regardless of what the marketing materials suggest.
Also ask what the clearance covers specifically. An FDA clearance for detecting pulmonary embolism on contrast-enhanced chest CT does not extend to non-contrast studies, does not cover pediatric patients if the training data was adult-only, and does not mean the tool is validated for your specific CT scanner manufacturer and protocol.
Question 2: What is your evidence base?
There is a hierarchy of evidence in AI validation just as there is in clinical medicine:
- Retrospective studies on curated datasets: The weakest evidence. Every AI tool performs well on the dataset it was trained and tested on.
- Prospective studies in clinical settings: Stronger. How does the tool perform on real patient data in a real clinical workflow?
- Multi-site validation: Stronger still. Does performance hold across different institutions, different equipment, different patient populations?
- Published peer-reviewed studies: The gold standard. Has the evidence been scrutinized by independent reviewers?
- Real-world deployment data: The ultimate test. What are institutions that have deployed this tool actually reporting after six months, twelve months, two years?
Many vendors will present impressive accuracy numbers from retrospective studies. Ask for prospective, multi-site, peer-reviewed data. If it does not exist, that does not necessarily mean the tool is bad -- it may mean it is early. But you should price that uncertainty into your decision.
Quick Check
After reading this lesson, what should you validate when applying Healthcare AI Landscape?
Question 3: What does implementation actually look like?
The demo takes thirty minutes. The implementation takes six to twelve months. Ask about:
- Technical integration: How does this connect to your PACS, EHR, clinical workflow systems? What are the IT requirements? What is the expected downtime during integration?
- Workflow redesign: How does this tool change your physicians' daily workflow? What training is required? What is the learning curve?
- Change management: What support does the vendor provide for physician adoption? What is the typical adoption rate at other institutions?
- Ongoing maintenance: What happens when the algorithm is updated? How are performance monitoring and quality assurance handled? Who is responsible for validating that the tool continues to perform as expected?
Question 4: What are the total costs, honestly?
The per-study fee or annual license fee is only part of the cost. Ask about:
- Integration and implementation fees
- IT infrastructure requirements (server specifications, network bandwidth, storage)
- Training costs for physicians and staff
- Ongoing maintenance and support fees
- Contract terms, including minimum commitments and cancellation provisions
- Hidden costs: will you need to hire additional IT staff? Will you need to upgrade your PACS or network infrastructure?
Question 5: What happens to our data?
This question deserves its own section of the conversation. Where is the data processed? Where is it stored? Is it used to improve the vendor's algorithm? Under what terms? What happens to the data if you terminate the contract? Is there a BAA? What are the breach notification procedures? Can you get a copy of their most recent SOC 2 Type II report?
Hype vs. Reality: An Honest Scorecard
After all of her research, Dr. Narayanan created what she called her "Honest Scorecard" -- a blunt assessment of where healthcare AI delivers today, where it shows genuine promise, and where the marketing has outrun the evidence.
What works now (high confidence):
- Ambient clinical documentation for straightforward outpatient encounters
- Radiology AI triage for critical findings (stroke, PE, hemorrhage)
- AI-assisted drafting of administrative documents (referral letters, prior auth appeals, patient messages)
- AI-generated patient education materials
- Diabetic retinopathy screening in primary care settings
- ECG-based atrial fibrillation detection in consumer and clinical devices
What shows genuine promise (moderate confidence):
- Pathology AI for cancer detection assistance
- EHR-integrated chart summarization and in-basket management
- AI-enhanced clinical decision support that reduces alert fatigue
- Ambient documentation for complex, multi-problem encounters
- AI-driven predictive models for patient deterioration
Evaluating Healthcare AI Evidence
Demand published, peer-reviewed, multi-site validation data before deploying diagnostic AI in clinical settings
Accept vendor demo results or retrospective single-site studies as sufficient evidence for clinical deployment
Where marketing outpaces evidence (proceed with caution):
Quick Check
After reading this lesson, what should you validate when applying Healthcare AI Landscape?
- "AI will replace radiologists" -- No evidence supports this. AI augments radiologists' workflow. The radiologist shortage is real; AI helps manage it, not eliminate it.
- "AI diagnoses as well as physicians" -- In narrow, well-defined tasks on curated datasets, sometimes. In the messy reality of clinical medicine with incomplete data, complex comorbidities, and atypical presentations, no AI system approaches the adaptive reasoning of an experienced clinician.
- "AI will reduce healthcare costs by X%" -- The evidence for system-wide cost reduction is thin. AI may reduce costs in specific workflows (faster stroke treatment reducing disability costs, automated documentation reducing scribe costs) but the total cost of implementation, maintenance, and change management often offsets savings in the short term.
- "Our AI eliminates diagnostic errors" -- No AI eliminates errors. All AI introduces new error modes (false positives, false negatives, automation bias where clinicians over-trust the AI output). The question is whether the net error rate improves -- and that requires rigorous measurement.
Apply: Map AI to Your Practice
Build Your Personal Healthcare AI Map
Take thirty minutes this week to build your own version of Dr. Narayanan's landscape map. Here is the process:
Step 1: List your top 10 time-consuming tasks. Think about your typical week. Where do you spend time on activities that feel disproportionate to their clinical value? Common answers include: documentation, referral letters, prior authorizations, patient portal messages, literature review, patient education, coding and billing, quality reporting, peer-to-peer reviews, and committee administrative work.
Step 2: For each task, identify the AI category most likely to help.
- Documentation → Ambient documentation tools (Category 2) or general-purpose AI (Category 3)
- Referral letters → General-purpose AI (Category 3)
- Prior authorizations → General-purpose AI (Category 3) or EHR-integrated AI (Category 4)
- Patient messages → EHR-integrated AI (Category 4) or general-purpose AI (Category 3)
- Literature review → General-purpose AI (Category 3)
- Image interpretation → FDA-cleared diagnostic AI (Category 1)
Step 3: Assess your current access. For each category you identified, do you already have access to a tool? If you are in a health system, check whether your EHR has AI features you have not activated. Check whether your institution has enterprise AI licenses you are not using. Many physicians discover that their health system has already deployed tools they were not aware of.
Step 4: Identify one tool to try this month. Pick the lowest-friction, highest-impact option. For most physicians, this is either ambient documentation (if your institution offers it) or a general-purpose AI tool for administrative tasks (if it does not). Start with a single, well-defined use case -- not "use AI for everything," but "use AI to draft my referral letters this week and evaluate whether the output is worth the workflow change."
The Vendor Evaluation Exercise
If you are in a position to evaluate AI tools for your practice or department, use this framework for your next vendor conversation:
-
Before the meeting, write down the three most important clinical problems you want AI to solve. Not "improve efficiency" -- specific problems. "Reduce the time my radiologists spend on low-acuity studies so they can focus on complex cases." "Cut prior authorization letter drafting from 15 minutes to 3 minutes." "Identify early sepsis in my ED population 30 minutes faster than current protocols."
-
During the meeting, ask the five questions from this lesson (regulatory status, evidence base, implementation reality, total costs, data handling). Take notes on any question the vendor cannot answer immediately -- those gaps are the ones most likely to bite you later.
-
After the meeting, score the vendor on three dimensions:
- Problem-solution fit (1-10): Does this tool actually address your specific problem, or does it address an adjacent problem that the vendor is stretching to fit?
- Evidence maturity (1-10): Is there published, peer-reviewed, multi-site evidence? Or retrospective single-site data? Or just the vendor's internal metrics?
- Implementation realism (1-10): Given your IT infrastructure, physician culture, and budget, how realistic is successful deployment within twelve months?
Any vendor scoring below 5 on any dimension deserves serious skepticism, regardless of how compelling the demo was.
Reflect: The Map Is Not the Territory -- But It Helps
What Dr. Narayanan Decided
After eight weeks of evaluation, Dr. Narayanan did not choose one AI vendor. She chose three -- each from a different category, each addressing a different clinical problem, each with a different evidence profile and risk level.
First, she deployed an ambient documentation tool across her hospitalist group and primary care clinics. The evidence was strong, the physician pain point was acute, and the expected time savings were quantifiable. She started with a pilot of twelve physicians, measured documentation time before and after, surveyed physician satisfaction, and used the data to make the case for system-wide deployment.
Second, she implemented the hospital's EHR vendor's AI-powered in-basket management feature for patient portal message drafting. This required no new vendor relationship, minimal IT integration (it was already available in their EHR version), and addressed a growing source of physician burnout -- the rising volume of patient portal messages.
Third, she approved a one-year pilot of a radiology AI triage tool for stroke detection in the emergency department. This was the highest-cost, highest-complexity deployment, but the clinical case was compelling -- their average door-to-needle time for stroke had room for improvement, and the evidence for AI-driven LVO detection was strong enough to justify a pilot.
What she did not do was equally important. She declined the vendor promising AI-powered sepsis prediction, deciding that the evidence for their specific product was not yet strong enough to justify the implementation complexity and alert fatigue risk. She tabled the pathology AI evaluation until their pathology department completed its transition to digital slide scanning. And she told three vendors whose products were impressive but whose evidence was limited to retrospective, single-site studies: "Come back when you have prospective, multi-site data."
The Framework That Transfers
You do not need to be a CMO evaluating fifteen vendors to benefit from Dr. Narayanan's approach. Every physician navigating the healthcare AI landscape benefits from three principles she articulated:
Principle 1: Categorize before you evaluate. Understand which type of AI tool you are looking at before you assess its merits. A general-purpose LLM and an FDA-cleared diagnostic algorithm are not comparable technologies. They have different evidence standards, different regulatory frameworks, different risk profiles, and different use cases. Comparing them is like comparing a stethoscope to a CT scanner -- both are useful, but for entirely different clinical situations.
Principle 2: Match the tool to the problem, not the hype. Start with the clinical problem you want to solve. Then find the AI category that addresses it. Then evaluate specific tools within that category. The vendors who contact you are solving for their product's capabilities, not for your clinical needs. You must drive the evaluation from the problem side, not the solution side.
Principle 3: Demand evidence proportional to risk. An AI tool that drafts referral letters carries low clinical risk -- if the letter is imperfect, you will catch it during review. An AI tool that triages stroke patients carries high clinical risk -- a missed LVO has devastating consequences. The evidence bar should be proportionally higher for higher-risk applications. For administrative AI, a reasonable pilot with before-and-after measurement may be sufficient. For diagnostic AI, you should demand published, peer-reviewed, multi-site validation data.
The healthcare AI landscape will continue evolving rapidly. New tools will emerge. Existing tools will improve. Some will fail. Regulations will tighten. Evidence will accumulate. But the framework for evaluating them -- categorize, match to problems, demand appropriate evidence -- remains stable regardless of what specific tools come and go.
You do not need to become an AI expert. You need to become an informed consumer of AI -- someone who can see through the demos, ask the right questions, and make decisions grounded in clinical judgment rather than vendor enthusiasm. That is what being AI-ready means for a physician. Not adopting every new tool. Knowing which ones deserve your attention.
Key Takeaways
- Healthcare AI is not one technology -- it spans six distinct categories: FDA-cleared diagnostic AI, ambient clinical documentation, general-purpose LLMs, EHR-integrated AI, clinical decision support systems, and emerging experimental applications
- FDA-cleared diagnostic AI (Viz.ai, Aidoc, Paige AI, IDx-DR) is the most regulated category with over 900 cleared devices, but each tool is narrow -- cleared for one specific task on one specific data type, not general-purpose diagnostics
- Ambient documentation tools (DAX Copilot, Abridge, Nabla, Suki) deliver proven 50-70% reductions in after-hours documentation time, making them the highest-immediate-impact category for most physicians
- General-purpose AI (Claude, ChatGPT, Gemini) is most valuable for administrative tasks -- referral letters, prior authorization appeals, patient education materials, literature synthesis -- but must never be used as a diagnostic tool and requires de-identification of PHI on consumer plans
- EHR-integrated AI has the highest potential because it has access to patient context, but it is still early -- expect 50-60% acceptance rates for AI-drafted content with the remainder requiring physician editing
- When evaluating AI vendors, ask five critical questions: exact regulatory status, evidence base quality, implementation reality, total cost including hidden costs, and data handling practices
- The hype-to-reality gap is real -- AI will not replace radiologists, does not diagnose as well as physicians in real clinical settings, and does not guarantee cost reduction, but it does augment specific workflows with measurable benefit
- Start your AI adoption with one specific, well-defined use case rather than trying to transform your entire practice at once -- the lowest-friction entry points are typically administrative drafting or ambient documentation
Use ← → to navigate, Space to mark complete