Development and testing of feed a feedback expert system for EMS documentation /

by Saini, Devashish.

Emergency Medical Services (EMS) are delivered in a chaotic, dynamic, unpredictable, and therefore error-prone environment. Deviations from protocol are common and some may be severe enough to cause harm to the patient. EMS personnel, like all healthcare providers, can benefit from feedback, continued education and medical oversight. A Feedback Expert system for EMS Documentation (FEED) was developed for providing feedback to EMS personnel after the ambulance run is completed and documented electronically. FEED’s knowledge base was derived from Alabama State EMS Protocols and discussions with two EMS faculty members from UAB School of Medicine. FEED is rule-based and uses confidence factors to represent uncertainty. It obtains data input from a Microsoft SQL Server database built according to the specifications of the National Highway Transportation and Safety Administration. The inference engine analyzes this input using the rules in the knowledge base and produces output in the form of a checklist of recommended documentation for the completed run. FEED’s knowledge base and inference engine were developed iteratively using CLIPS (C Language Integrated Production System), an expert system development tool, and the user interface in Visual Basic .NET. FEED was developed, tested and refined over three iterations. Testing included local syntax checking, consistency checking, informal verification, and formal verification and validation using two expert focus groups. One group verified whether FEED met its requirements specifications, and another group validated its output ii using real-life patient care reports. The validation focus group revealed a sensitivity of 60% and a specificity of 85% when a cut-off value of 8.5 was chosen for FEED’s validity rating. The review of false positives and false negatives revealed several inaccuracies in the KB, and several potential expert errors. After appropriate corrections and revisions, FEED’s utility could be tested in the field using a larger group of EMS providers, leading to further refinement of all components of FEED. Ultimately, providing feedback to EMS personnel may improve patient care in the field and provide better quality data for EMS research. iii