Rater Variability of Testig Velocity Whe Assessig Lower Limb Spasticity,” Joural of Rehabilitatio Medicie 51（2019）。

另一项不涉及诊断的研究：Esther Y. Hsiag et al., “Associatio of Primary Care Cliic Appoitmet Time with Cliicia Orderig ad Patiet Completio of Breast ad Colorectal Cacer Screeig,” JAMA Network Ope 51（2019）。

还有一个例子也能说明临床医生会受到疲劳的影响：Hegche Dai et al., “The Impact of Time at Work ad Time Off from Work o Rule Compliace: The Case of Had Hygiee i Health Care,”Joural of Applied Psychology 100（2015）: 846。

对医学领域意义重大：Ali S. Raja, “The HEART Score Has Substatial Iterrater Reliability,”NEJM J Watch, December 5, 2018 [reviewig Coli A. Gersho et al., “Iter-rater Reliability of the HEART Score,” Academic Emergecy Medicie 26（2019）: 552]。

我们在前面提到，培训可以提高医生的技能：Jea-Pierre Zellweger et al., “Itra-observer ad Overall Agreemet i the Radiological Assessmet of Tuberculosis,” Iteratioal Joural of Tuberculosis & Lug Disease 10（2006）: 1123；Ibrahim Abubakar et al., “Diagostic Accuracy of Digital Chest Radiography for Pulmoary Tuberculosis i a UK Urba Populatio,” Europea Respiratory Joural 35（2010）: 689。

汇总多个专家的判断也能减少噪声：Michael L. Barett et al., Rater Variability of Testig Velocity Whe Assessig Lower Limb Spasticity,” Joural of Rehabilitatio Medicie 51（2019）。

另一项不涉及诊断的研究：Esther Y. Hsiag et al., “Associatio of Primary Care Cliic Appoitmet Time with Cliicia Orderig ad Patiet Completio of Breast ad Colorectal Cacer Screeig,” JAMA Network Ope 51（2019）。

还有一个例子也能说明临床医生会受到疲劳的影响：Hegche Dai et al., “The Impact of Time at Work ad Time Off from Work o Rule Compliace: The Case of Had Hygiee i Health Care,”Joural of Applied Psychology 100（2015）: 846。

对医学领域意义重大：Ali S. Raja, “The HEART Score Has Substatial Iterrater Reliability,”NEJM J Watch, December 5, 2018 [reviewig Coli A. Gersho et al., “Iter-rater Reliability of the HEART Score,” Academic Emergecy Medicie 26（2019）: 552]。

我们在前面提到，培训可以提高医生的技能：Jea-Pierre Zellweger et al., “Itra-observer ad Overall Agreemet i the Radiological Assessmet of Tuberculosis,” Iteratioal Joural of Tuberculosis & Lug Disease 10（2006）: 1123；Ibrahim Abubakar et al., “Diagostic Accuracy of Digital Chest Radiography for Pulmoary Tuberculosis i a UK Urba Populatio,” Europea Respiratory Joural 35（2010）: 689。

汇总多个专家的判断也能减少噪声：Michael L. Barett et al., “Comparative Accuracy of Diagosis by Collective Itelligece of Multiple Physicias vs Idividual Physicias,” JAMA Network Ope 2（2019）: e19009；Kimberly H. Alliso et al., “Uderstadig Diagostic Variability i Breast Pathology: Lessos Leared from a Expert Cosesus Review Pael,” Histopathology 65（2014）: 240。

最好的算法：Babak Ehteshami Bejordi et al., “Diagostic Assessmet of Deep Learig Algorithms for Detectio of Lymph Node Metastases i Wome with Breast Cacer,” JAMA 318（2017）: 2199。

深度学习算法：Varu Gulsha et al., “Developmet ad Validatio of a Deep Learig Algorithm for Detectio of Diabetic Retiopathy i Retial Fudus Photographs,” JAMA 316（2016）: 2402。

几乎和放射科医生一样出色：Mary Beth Massat, “A Promisig Future for AI i Breast Cacer Screeig”, Applied Radiology 47（2018）: 22；Alejadro Rodriguez-Ruiz et al., “Stad-Aloe Artificial Itelligece for Breast Cacer Detectio i Mammography: Compariso with 101 Radiologists,” Joural of the Natioal Cacer Istitute 111（2019）: 916。

表22-1，阿普加评分指南：Medlie Plus（last accessed February 4, 2020）。

阿普加评分产生的噪声很小：L. R. Foster et al., “The Iterrater “Comparative Accuracy of Diagosis by Collective Itelligece of Multiple Physicias vs Idividual Physicias,” JAMA Network Ope 2（2019）: e19009；Kimberly H. Alliso et al., “Uderstadig Diagostic Variability i Breast Pathology: Lessos Leared from a Expert Cosesus Review Pael,” Histopathology 65（2014）: 240。

最好的算法：Babak Ehteshami Bejordi et al., “Diagostic Assessmet of Deep Learig Algorithms for Detectio of Lymph Node Metastases i Wome with Breast Cacer,” JAMA 318（2017）: 2199。

深度学习算法：Varu Gulsha et al., “Developmet ad Validatio of a Deep Learig Algorithm for Detectio of Diabetic Retiopathy i Retial Fudus Photographs,” JAMA 316（2016）: 2402。

几乎和放射科医生一样出色：Mary Beth Massat, “A Promisig Future for AI i Breast Cacer Screeig”, Applied Radiology 47（2018）: 22；Alejadro Rodriguez-Ruiz et al., “Stad-Aloe Artificial Itelligece for Breast Cacer Detectio i Mammography: Compariso with 101 Radiologists,” Joural of the Natioal Cacer Istitute 111（2019）: 916。

表22-1，阿普加评分指南：Medlie Plus（last accessed February 4, 2020）。

阿普加评分产生的噪声很小：L. R. Foster et al., “The Iterrater Reliability of Apgar Scores at 1 ad 5 Miutes,” Joural of Ivestigative Medicie 54, o. 1（2006）: 293。

使用该量表进行评估和评分相对直接：Warre J. McIsaac et al., “Empirical Validatio of Guidelies for the Maagemet of Pharygitis i Childre ad Adults,” JAMA 291（2004）: 1587。

一项研究发现，BI-RADS提升了乳房X线片的评估者之间的一致性：Emilie A. Ooms et al., “Mammography: Iterobserver Variability i Breast Desity Assessmet,” Breast 16（2007）: 568。

在病理学领域：Fraces P. O’Malley et al., “Iterobserver Reproducibility i the Diagosis of Flat Epithelial Atypia of the Breast,” Moder Pathology 19（2006）: 172。

至少从20世纪40年代起，减少噪声就成为精神病学界的头等大事：Ahmed Aboraya et al., “The Reliability of Psychiatric Diagosis Revisited,” Psychiatry（Edgmot） 3（2006）: 41。有关概述，详见N. Kreitma, “The Reliability of Psychiatric Diagosis,” Joural of Metal Sciece 107（1961）: 876–886。

1964年，一项针对91名患者和10名有经验的精神科医生的研究：Aboraya et al., “Reliability of Psychiatric Diagosis Revisited,” 43。

为了初步揭示其中的原因：C. H. Ward et al., “The Psychiatric Nomeclature: Reasos for Diagostic Disagreemet,” Archives of Geeral Psychiatry 7（1962）: 198。Reliability of Apgar Scores at 1 ad 5 Miutes,” Joural of Ivestigative Medicie 54, o. 1（2006）: 293。

使用该量表进行评估和评分相对直接：Warre J. McIsaac et al., “Empirical Validatio of Guidelies for the Maagemet of Pharygitis i Childre ad Adults,” JAMA 291（2004）: 1587。

一项研究发现，BI-RADS提升了乳房X线片的评估者之间的一致性：Emilie A. Ooms et al., “Mammography: Iterobserver Variability i Breast Desity Assessmet,” Breast 16（2007）: 568。

在病理学领域：Fraces P. O’Malley et al., “Iterobserver Reproducibility i the Diagosis of Flat Epithelial Atypia of the Breast,” Moder Pathology 19（2006）: 172。

至少从20世纪40年代起，减少噪声就成为精神病学界的头等大事：Ahmed Aboraya et al., “The Reliability of Psychiatric Diagosis Revisited,” Psychiatry（Edgmot） 3（2006）: 41。有关概述，详见N. Kreitma, “The Reliability of Psychiatric Diagosis,” Joural of Metal Sciece 107（1961）: 876–886。

1964年，一项针对91名患者和10名有经验的精神科医生的研究：Aboraya et al., “Reliability of Psychiatric Diagosis Revisited,” 43。

为了初步揭示其中的原因：C. H. Ward et al., “The Psychiatric Nomeclature: Reasos for Diagostic Disagreemet,” Archives of Geeral Psychiatry 7（1962）: 198。具有生物医学背景的另外一位临床医生：Aboraya et al., “Reliability of Psychiatric Diagosis Revisited”。

DSM Ⅲ促使大量的研究关注诊断是否存在噪声：Samuel M. Lieblich, David J. Castle, Christos Patelis, Malcolm Hopwood, Alla Huter Youg, ad Ia P. Everall, “High Heterogeeity ad Low Reliability i the Diagosis of Major Depressio Will Impair the Developmet of New Drugs,” British Joural of Psychiatry Ope 1（2015）: e5–e7。

但这本手册远没有达到完美：Lieblich et al., “High Heterogeeity”。

即使在2000年对第4版——DSM Ⅳ进行了重大修订之后：Elie Cheiaux et al., “The Diagoses of Schizophreia, Schizoaffective Disorder, Bipolar Disorder ad Uipolar Depressio: Iterrater Reliability ad Cogruece Betwee DSM IV ad ICD 10,” Psychopathology 42（2009）: 296–298,特别是293页；ad Michael Chmielewski et al., “Method Matters:Uderstadig Diagostic Reliability i DSM IV ad DSM Ⅴ,” Joural of Abormal Psychology 124（2015）: 764, 768–769。

提高精神疾病诊断的可靠性：Aboraya et al., “Reliability of Psychiatric Diagosis Revisited,” 47。

存在着一个严重的风险：Aboraya et al., 47。

该手册的第5版：Chmielewski et al., “Method Matters”。

美国精神病学学会：Helea Chmura Kraemer et al., “DSM–5: How 具有生物医学背景的另外一位临床医生：Aboraya et al., “Reliability of Psychiatric Diagosis Revisited”。

DSM Ⅲ促使大量的研究关注诊断是否存在噪声：Samuel M. Lieblich, David J. Castle, Christos Patelis, Malcolm Hopwood, Alla Huter Youg, ad Ia P. Everall, “High Heterogeeity ad Low Reliability i the Diagosis of Major Depressio Will Impair the Developmet of New Drugs,” British Joural of Psychiatry Ope 1（2015）: e5–e7。

但这本手册远没有达到完美：Lieblich et al., “High Heterogeeity”。

即使在2000年对第4版——DSM Ⅳ进行了重大修订之后：Elie Cheiaux et al., “The Diagoses of Schizophreia, Schizoaffective Disorder, Bipolar Disorder ad Uipolar Depressio: Iterrater Reliability ad Cogruece Betwee DSM IV ad ICD 10,” Psychopathology 42（2009）: 296–298,特别是293页；ad Michael Chmielewski et al., “Method Matters:Uderstadig Diagostic Reliability i DSM IV ad DSM Ⅴ,” Joural of Abormal Psychology 124（2015）: 764, 768–769。

提高精神疾病诊断的可靠性：Aboraya et al., “Reliability of Psychiatric Diagosis Revisited,” 47。

存在着一个严重的风险：Aboraya et al., 47。

该手册的第5版：Chmielewski et al., “Method Matters”。

美国精神病学学会：Helea Chmura Kraemer et al., “DSM–5: How Reliable Is Reliable Eough?,”America Joural of Psychiatry 169（2012）: 13–15。

精神科医生的诊断仍然表现出明显的噪声：Lieblich et al., “High Heterogeeity”。

精神科医生就患者是否患有重度抑郁症很难达成一致：Lieblich et al., “High Heterogeeity,”e–5。

DSM Ⅴ的现场试验发现：Lieblich et al.,e–5。

另外一些现场试验表明：Lieblich et al., e–6。

使用指南之所以很难取得成功：Aboraya et al., “Reliability of Psychiatric Diagosis Revisited,”47。

明确诊断标准，舍弃模糊标准：Aboraya et al。

一位观察者曾说：Aboraya et al。

医学界需要更多的指南：一些有价值的注意事项参见Christopher Worsham, Aupam B. Jea, “The Art of Evidece-Based Medicie,” Harvard Busiess Review, Jauary 30, 2019。

第23章　绩效评估，用基于外部视角的共识框架做出量化判断

正如有家报纸的标题所示：Jea McGregor, “Study Fids That Basically Every Sigle Perso Hates Performace Reviews,” Reliable Is Reliable Eough?,”America Joural of Psychiatry 169（2012）: 13–15。

精神科医生的诊断仍然表现出明显的噪声：Lieblich et al., “High Heterogeeity”。

精神科医生就患者是否患有重度抑郁症很难达成一致：Lieblich et al., “High Heterogeeity,”e–5。

DSM Ⅴ的现场试验发现：Lieblich et al.,e–5。

另外一些现场试验表明：Lieblich et al., e–6。

使用指南之所以很难取得成功：Aboraya et al., “Reliability of Psychiatric Diagosis Revisited,”47。

明确诊断标准，舍弃模糊标准：Aboraya et al。

一位观察者曾说：Aboraya et al。

医学界需要更多的指南：一些有价值的注意事项参见Christopher Worsham, Aupam B. Jea, “The Art of Evidece-Based Medicie,” Harvard Busiess Review, Jauary 30, 2019。

第23章　绩效评估，用基于外部视角的共识框架做出量化判断

正如有家报纸的标题所示：Jea McGregor, “Study Fids That Basically Every Sigle Perso Hates Performace Reviews,”