Interpreting the real-time dynamic ‘sliding sign’ and predicting pouch of Douglas obliteration: an interobserver, intraobserver, diagnostic-accuracy and learning-curve study

OBJECTIVETo determine inter and intra-observer agreements, diagnostic accuracy and the learning curve required for interpreting ‘sliding sign’ and predicting Pouch of Douglas (POD) obliteration.METHODAn inter/ intra observer, diagnostic accuracy and learning curve study involving 6 observers who viewed 32 offline 'sliding sign' videos in two anatomical locations viz retro-cervix (RC) and posterior uterine fundus (PUF).Observer 1 was a medical student with no previous gynaecological ultrasound experience, observer 2 was a 2nd year obstetrics and gynaecology (O&G) trainee (50 scans), observer 3 was a 1st year sonographer trainee (50 scans), observer 4 was a 4th year O&G trainee (200 scans), observer 5 was a gynaecological ultrasound fellow (750 scans) and observer 6 was an expert sonologist (15,000 scans).Each observer interpreted the videos (Observation set 1) as positive or negative and predicted status of POD. The same observers reanalysed the same video sets albeit in a different order (Observation set 2), at least 7 days apart for intra-observer agreement. Observer 6 was reference standard for interpreting ‘sliding sign’ and gold standard laparoscopy was used for POD. Learning curve cumulative summation (LC-CUSUM) tests were conducted to assess if observer performances reached acceptable levels.RESULTSMultiple rater agreements for interpreting ‘sliding sign’ was moderate (Fleiss kappa (K) = 0.499). Observers were more consistent with observation set 2 vs. set 1 (K = 0.547 vs. 0.453) and for RC vs. PUF (K = 0.556 vs. 0.346). Intra-observer correlations were moderate (Cohen's kappa (k) = 0.447 - 0.496) to almost perfect (k = 0.678 – 0.953).Accuracy, sensitivity, specificity, positive and negative predictive value for predicting status of POD ranged from 65.4 - 96.2%, 80.0 - 100%, 64.7 - 100%, 50.0 - 100% and 94.7 - 100% respectively. Using LC-CUSUM score < −2.45, observer 4 reached acceptable levels for predicting POD obliteration and interpreting ‘sliding sign’ at both regions (RC and PUF) at 39, 54 and 28 and observer 5 at 56, 53 and 53 respectively.CONCLUSIONA minimum number of gynaecological ultrasound experiences are relevant for interpreting the ‘sliding sign’. Non-specialist observers with at least 200 prior TVS were more consistent in interpreting ‘sliding sign’ at RC vs. PUF regions.