Perform case review on a random sample of patients

Background and Strategy

An excellent complement to the gold standard review of aggregate results -- a "top-down" validation -- is a deep, record-by-record, "bottom-up" case review of a small number of randomly selected patients, which can identify problems in fields that aren't involved in the generation of the gold standard statistics and which therefore might otherwise slip through the cracks.

The Patients Natural object automatically generates a random integer between 1 and 1000 for each patient, which can be used to select a random sample. Generally, a small sample -- 5 to 10 patients -- will be sufficient. (The object also includes a Source ID field, which should be filtered on first to isolate patients associated with the appropriate source system.)

The Ursa Health Core Data Model contains a Synthetic object, Universal Patient Document Aggregator, that collects records from across several Natural objects (for example, institutional and professional claims and service line items, pharmacy claims, labs and other patient observations, etc.). This object provides a convenient compilation of a patient's full history of records.

Another valuable object for this kind of case review is the Patient Timelines Master for URSA-CORE Concepts. This object provides a read-out of each patient's status over time. Each record in the object represents a continuous period in a patient's life where their status -- defined by the dozens of fields in the objects -- remained constant, whether that period lasted a day, a week, or a year. (See How to track patient features over time for population health analytics for a deeper dive into the structure of "timelines" in Ursa Studio.)

The strategy for this review is to select a patient at random from the data source under scrutiny, generate a chronological history of patient documents using Universal Patient Document Aggregator and a history of the patient's changing status over time using Patient Timelines Master for URSA-CORE Concepts, and compare these to the story told by the raw data in the Source Data Layer objects. The comparison should not be limited to dollar amounts, ICD-10-CM codes, and other higher profile fields, but should also cover the less glamorous concepts -- provider NPIs, HCPCS modifiers, insurance membership coverage start and end dates, etc. -- that are often just as critical to analytic accuracy but which might have been overlooked.

To the extent any problems are identified, they should be corrected in this step, and the data model refreshed to confirm that the changes had the desired effect.

Detailed Implementation Guidance

1. Note that the Period End Date values in the Patient Timelines Master for URSA-CORE Concepts is exclusive. For example, a record with a Period Start Date of January 1, 2022 and a Period End Date of December 31, 2022 would only cover the first 364 days of 2022; a full year would require a Period End Date of January 1, 2023. (All interval end dates use this convention in Ursa Studio.)

2. Refreshing the data model will regenerate the random numbers assigned to each patient, so be sure to record the Patient ID values (which should remain consistent from run to run if the data mastering has been implemented properly) for the sample of cases. As usual, saving these as board elements on the appropriate objects is a convenient way to memorialize the analysis.