The Beijing Eye Research Institute (BERI) implemented an experiment in which an artificial intelligence–based system was entrusted with operating an ophthalmology clinic. The project, called AI-TEC, retained human staff but was designed from the outset to place AI at the center of its workflows rather than retrofitting AI tools into existing procedures. Human oversight remained in place. The results were published in Nature Medicine and were reported by Science Alert.
Scope of the system and key performance findings
AI-TEC used AI across the patient pathway: consultations, image-based examinations, analysis and follow-up. Initially, the AI showed relatively low accuracy in identifying diseases on eye imaging—such as glaucoma and age-related macular degeneration. Researchers found that accuracy improved markedly when the model was trained on 1,426 high-quality, correctly labeled eye images provided by ophthalmologists. This smaller, higher-quality training set proved far more effective than an earlier dataset of 27,000 images that were lower quality and less reliably labeled.
Clinical uptake and the importance of integration
Although clinical use of the tools was high at the start, utilization fell over time. Five months after deployment, the AI was used in only 41 of 1,113 examinations (3.8%). After developers simplified the system, use rose the following month to 259 uses in 1,126 examinations (26%).
From these observations the team concluded that successful deployment of AI in healthcare depends less on raw model performance and more on how well the technology is integrated into clinical workflows and on the quality of the data used to train and run the system.
Additional observations and challenges
- The researchers emphasized the need for regular, rapid feedback from clinicians so the system can continue learning and improving in practice.
- They also noted a fundamental mismatch in focus: AI models tend to be outcome-oriented (for example, whether patient A has a diagnosable eye disease), while clinicians often focus first on symptoms (for example, whether patient A reports blurred vision). This divergence can mean that precise image analysis does not automatically translate into better patient care.
Implications and recommendations for future work
The study suggests future research should shift some attention away from narrow performance metrics—such as image classification accuracy—and toward issues of integration, workflow design, training data quality, and clinician engagement. According to the authors, these factors may be more decisive in determining whether AI meaningfully improves clinical outcomes.
Conclusion
The BERI AI-TEC experiment demonstrates that AI can be a valuable tool in medical imaging, but higher accuracy on imaging alone is not a guarantee of improved clinical care. High-quality labeled data, seamless workflow integration, and continuous clinician feedback are key to realizing the practical benefits of AI in patient care.
