AI Ultrasound Module

1.0 Overview 

This module assesses whether a simulant receives an ultrasound during antenatal care, tracks the details of that care, and outputs a “believed” gestational age at the end of pregnancy for that simulant based on the simulant’s real gestational age at the end of pregnancy and the measurement error of the gestational age dating care they recieved. Notably, coverage and type of ultrasounds offered at ANC visits will be scenario-dependent.

2.0 Module Diagram and Data 

2.1 Module Diagram 

../../../../_images/ai_ultrasound_module_diagram.drawio.png

Todo

Update strategy to use all four categories of ANC attendance instead of treating it as a binary variable. Specifically, change ultrasound type to a 4-category variable by splitting the “standard ultrasound” category into 2 categories, “standard ultrasound in 1st trimester” and “standard ultrasound in later pregnancy”, as outlined in this PR comment, and use the data in this paper referenced below to have different standard deviations for GA estimation error based on ultrasound timing.

Note that making these changes will also require updating and recalibrating the facility choice model.

2.2 Module Inputs 

Module required inputs
Input	Source module	Application	Note
ANC attendance category	Antenatal care module	Decision node 1	4-category exposure variable. As described in the facility choice model document, ANC attendance is correlated with other model variables.
Gestational age at end of pregnancy	Pregnancy module	Action point V	Point value in days

2.3 Module Decision Nodes 

Module decision nodes
Decision node	Description	Information	Note
1	Attends ANC?	Decide “No” if ANC attendance category is none; decide “Yes” otherwise	ANC attendance has four exposure categories: The none category corresponds to no ANC attendance, while the other three categories indicate some ANC attendance
2	Receives ultrasound?	Scenario-dependent variable: see the pregnancy component scenario table for values (and baseline coverage section below for baseline coverage)	“Yes” if random propensity <= scenario-specific ultrasound coverage defined in table
3	Ultrasound type?	Scenario-dependent variable: pregnancy component scenario table for values (and baseline coverage section below for baseline coverage)	Possible values are “none,” “standard,” and “AI-assisted”
4	Is estimated gestational age < 37 weeks?	“Yes” or “No” depending on the estimated gestational age calculated in Action point V

2.3.1: Baseline coverage 

We assume 100% of ultrasounds are standard (and 0% are AI-assisted) at baseline. Baseline coverage of ultrasound among those who attend ANC:

Baseline ultrasound coverage values
Location	Value	Note
Ethiopia	60.7%	Ethiopia ultrasound rate
Nigeria	58.7%	Nigeria ultrasound rate
Pakistan	66.7%	India ultrasound rate (Table 8.12, averaged percentage of women attending ANC 1-3 times and 4+ times). we currently use ultrasound utilization rates derived from the India DHS 2015-2016 as an imperfect proxy that can hopefully be improved with further research

2.4 Module Action Points 

Module action point
Action point	Description	Information	Note
I	Record `none` for ultrasound type	Record to output
II	Record `standard` for ultrasound type	Record to output
III	Record `ai_assisted` for ultrasound type	Record to output
IV	Sample random error for estimated gestational age	GA error is normally distributed with mean 0 and standard deviation depending on ultrasound type, as specified in the table below	See instructions in section 2.4.1 Calculation of estimated gestational age below
V	Calculate estimated gestational age	Add the random GA error from Action point IV to the gestational age from the pregnancy module, and record to output	See instructions in section 2.4.1 Calculation of estimated gestational age below
VI	Record `believed_preterm` for believed preterm status	Record to output	corresponds to estimated gestational age < 37 weeks
VII	Record `believed_term` for believed preterm status	Record to output	corresponds to estimated gestational age 37+ weeks

2.4.1 Calculation of estimated gestational age 

Estimated gestational age should be calculated by adding a randomly sampled value from a normal distribution with a mean of zero and a standard deviation defined below to the simulant’s assigned gestational age at birth exposure (input from the pregnancy module).

Standard deviation values by ultrasound type
Ultrasound type	Standard deviation
None	10 days
Standard	6.7 days
AI-assisted ultrasound	5 days

Todo

Add references for these numbers. Here’s the notebook I used to get them, which includes the citations.

Note

BMGF sent us data on the error distribution of ultrasound accuracy based on gestational age so we could make this more accurate. (See first bullet in Limitations list below for more details.)

2.5: Module Outputs 

Module outputs
Output	Value	Note
Ultrasound type	“none” or “standard” or “AI-assisted”	Used for V&V
Estimated gestational age	Point values in days	Used for V&V, and determination of eligibility for antenatal corticosteroids
Believed preterm status	“believed preterm” or “believed term”	Used for V&V and for facility choice module in intrapartum component

3.0 Assumptions and limitations 

The timing of ANC visits impacts the ability to accurately estimate gestational age, but we use an average instead.
The current version of the model does not include any false positive rates for LBW. Since a false positive is unlikely to cause harm, only inclusion in higher level care, this seems sufficient.
Single cohort of pregnancies does not allow for cyclic effects such as improved ANC visit rates due to ultrasound presence
The data for baseline ultrasound utilization at the ANC is non-ideal for all of the locations. Our data for Ethiopia is most aligned with the value we are trying to find, as it comes from a paper that estimates ultrasound utilization at ANC, in a specific municipality of Jimma in Ethiopia. For Nigeria, our literature value is less trustworthy, coming from a paper that reports the percentage of study participants who had previously had an obstetric ultrasound. We were unable to find any value for Pakistan, instead using data from the India DHS 2015-2016 to inform our Pakistan ultrasound coverage. India is probably not a great proxy for Pakistan, as use of ultrasound technology in India is heavily regulated (see here).

Todo

If more suitable baseline coverage data for standard ultrasound utilization at ANCs for Nigeria or Pakistan, we should use that data instead and update this documentation accordingly.

Note

BMGF sent us data on the error distribution of ultrasound accuracy based on gestational age so we should be able to address the first limitation. We also found a paper that estimated uncertainty of GA dating by ultrasound was 6–7 days at 14 weeks’ gestation, 12–14 days at 26 weeks’ gestation and > 14 days in the third trimester.

From Nathaniel: I think the gestational age in the BMGF data and the gestational age in the paper are actually referring to two different things, and we may want to take both types of variation into account:

The BMGF microdata compares the gestational age at birth estimated by ultrasound (given at some unknown time during the pregnancy) with gestational age at birth estimated by last menstrual period (LMP). I think the paper compares the gestational age estimated by an ultrasound in late pregnancy at the time of the late ultrasound with the “true” gestational age at the time of the late ultrasound, determined from a combination of LMP and an ultrasound early in the pregnancy. From the BMGF data, I was interested in seeing whether there was bias (nonzero 1st moment) or skew (nonzero 3rd moment) in the error distribution depending on the gestational age at birth. It looks like there is: For babies born early, you’re more likely to overestimate their gestational age, whereas for babies born late, you’re more likely to underestimate their gestational age (that is, when using LMP vs. an ultrasound).

From the literature, I’m interested in how the size of the variance (2nd moment) of the error changes with the timing of when the ultrasound is administered. We know that the variance is higher when the ultrasound is given later in pregnancy, and the paper quantifies how much higher.

4.0 Verification and Validation Criteria 

Confirm ANC visit rate matches expectations
Confirm ultrasound rates matches inputs for all scenarios
Confirm gestational age estimate and real gestational age have the correct margin of error based on ultrasound type