|Year : 2020 | Volume
| Issue : 9 | Page : 5026-5034
Comparing the performance of two social risk screening tools in a vulnerable subpopulation
Cara C Lewis1, Robert Wellman1, Salene M W Jones2, Callie Walsh-Bailey3, Ella Thompson1, Alphonse Derus1, Andrea Paolino4, John Steiner4, Emilia H De Marchis5, Laura M Gottlieb5, Adam L Sharp6
1 Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave Suite 1600, Seattle, WA, USA
2 Hutchinson Institute for Cancer Outcomes Research, 1100 Fairview Ave. N., Seattle, WA, USA
3 Brown School, Washington University in St. Louis, 1 Brookings Dr, St. Louis, MO, USA
4 Kaiser Permanente Colorado Institute for Health Research, 2550 S. Parker Rd Suite 200 Aurora, CO, USA
5 University of California San Francisco, Department of Family and Community Medicine, 505 Parnassus Ave, San Francisco, CA, USA
6 Kaiser Permanent Southern California, Research and Evaluation Department, 100 S Los Robles Ave, Pasadena, CA 91101 Pasadena; Kaiser Permanente School of Medicine, Health Systems Science Department, 98 S. Los Robles Ave., Pasadena, CA, USA
|Date of Submission||17-Apr-2020|
|Date of Decision||10-Jun-2020|
|Date of Acceptance||03-Jul-2020|
|Date of Web Publication||30-Sep-2020|
Dr. Cara C Lewis
Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave Suite 1600, Seattle, WA 98101
Source of Support: None, Conflict of Interest: None
Background: Research shows the profound impact of social factors on health, lead many healths systems to incorporate social risk screening. To help healthcare systems select among various screening tools we compared two tools, the Your Current Life Situation (YCLS) and the Accountable Health Communities (AHC) Screening tools, on key psychometric properties. Method: Kaiser Permanente Southern California subsidized exchange members (n = 1008) were randomly invited to complete a survey containing either the YCLS or the AHC tool, as well as other measures related to care experience and health. Healthcare use was measured through the electronic health record. Agreement between the AHC and YCLS was assessed using adjusted kappas for six domains (food – worry, food – pay, insecure housing, housing quality, transportation, utilities). To assess predictive validity, items on the AHC and YCLS were compared to self-rated health and receipt of a flu shot. Results: Responders (n = 450) and non-responders (n = 558) significantly differed on sex, language, and depression (P < 0.05) but not anxiety, race/ethnicity, or healthcare use. Agreement between the AHC and YCLS tools was substantial on all items (kappas > 0.60) except for housing quality (kappa 0.52). Four out of six screening questions on the AHC tool and four out of seven on the YCLS tool were associated with self-rated health (P < 0.03). No social needs were associated with flu shot receipt except utilities on the AHC tool (P = 0.028). Conclusion: In this sample, the AHC and YCLS tools are similar in their ability to screen for social risks. Differences observed likely stem from the timeframe and wording of the questions, which can be used to guide selection in healthcare systems.
Keywords: Psychometric, social determinants of health, social risk, validity
|How to cite this article:|
Lewis CC, Wellman R, Jones SM, Walsh-Bailey C, Thompson E, Derus A, Paolino A, Steiner J, De Marchis EH, Gottlieb LM, Sharp AL. Comparing the performance of two social risk screening tools in a vulnerable subpopulation. J Family Med Prim Care 2020;9:5026-34
|How to cite this URL:|
Lewis CC, Wellman R, Jones SM, Walsh-Bailey C, Thompson E, Derus A, Paolino A, Steiner J, De Marchis EH, Gottlieb LM, Sharp AL. Comparing the performance of two social risk screening tools in a vulnerable subpopulation. J Family Med Prim Care [serial online] 2020 [cited 2020 Oct 29];9:5026-34. Available from: https://www.jfmpc.com/text.asp?2020/9/9/5026/296311
| Background|| |
The mounting literature documenting the profound impact of adverse social determinants on health,,,, has inspired many healthcare systems to focus on actionable, material factors referred to as social needs,, or social risk factors. Social risks include education, social, or community context (e.g., safety, isolation), neighborhood and physical environments, economic and housing stability, and food security., Findings, suggest that clinic-based screening and interventions targeting social risks can improve patient outcomes,,, and lower healthcare costs., The National Academy of Medicine recommends primary care physicians consider social risk-informed interventions, wherein clinical plans are tailored to reduce the impact of social risk on care access or adherence.,,
The effectiveness of social care interventions hinges on the ability to accurately detect social risks. Recent reviews concluded that although multiple low-cost, linguistically appropriate tools exist to screen for social risks, there is a dearth of psychometric evidence for their appropriate use.,,,, Without evidence of reliability and validity, it is unclear whether existing tools are adequately and accurately capturing patients' social information. This lack of psychometric information makes it difficult to select screening tools. In this study, we conducted a survey to examine the reliability and validity of two social risk screening tools, the Your Current Life Situation (YCLS) screening tool, developed by the Kaiser Permanente Care Management Institute, and the Accountable Health Communities (AHC) Screening tool, developed by the Centers for Medicare and Medicaid Services.
This study describes potential differences between responders to social risk screening and non-responders, and compares the AHC and YCLS screening tools with respect to prevalence of risk and association with self-reported health and receipt of preventive health care (immunizations). These results will help to improve understanding about patients who do and do not respond to social risk screening tools and inform primary care physicians and health systems attempting to decide among screening tools.
| Methods|| |
Kaiser Permanente's (KP) National Community Health funded SONNET—the Social Needs Network for Evaluation and Translation—to identify and evaluate existing programs within KP that address social risks to guide intervention design. SONNET's internal inventory of social risk-targeted programs across KP's eight regions demonstrated a dearth of social risk information. To inform initiatives, including a national social service resource locator, Thrive Local, SONNET sponsored a quality improvement survey1 to yield estimates of social risks and inform how best to identify patients' social risks. This survey focused on KP subsidized exchange members in Southern California. This patient population reflects a vulnerable, understudied group: income is typically too high for many government and/or community financial assistance programs, but low enough to cause financial strain. As of 2010, under the Affordable Care Act (ACA) individuals are eligible for subsidies to purchase insurance on exchange websites if they meet the following: (1) income between 100 and 400% of the federal poverty level, (2) legal US resident (including immigrants ineligible for Medicaid or CHIP), and (3) no access to employer-sponsored coverage (insurance premiums greater than 9.86% of individual income)., In 2018, nearly 8.9 million people enrolled in subsized exchange plans. Currently, about 3.8% of the over 4.6 million KP Southern California (KPSC) members are on subsidized exchange plans.
Inclusion and exclusion, stratification
Our study included 134,355 KPSC members on a subsidized exchange plan. Subsidies include premium subsidies, cost sharing reduction subsidies, or both. These members, as of September 1, 2018, were at least 18 years of age; continuously enrolled for at least the prior 9 months without gaps exceeding 90 days; had no record of death; and a geocoded address in the KPSC regional service area. Based on a 50% anticipated response rate, we randomly selected 1,008 members across language (English or Spanish), age, and sex strata to reach our target sample size of 500. Of the selected members, 33.5% had a cost-sharing reduction subsidy. Spanish speaking members (9.8% of the underlying population) were intentionally over-sampled at 20% to ensure even stratification between English and Spanish speaking participants by age (18–26, 27–44, 45–61, 62 and over) and sex.2, Block group randomization was completed for each stratum.
The survey included two new screening tools and previously validated scales (See [supplementary materials]). The survey included items selected and/or developed by a collaborating research team. The survey was revised with cognitive testing and expert input from the SONNET Steering Committee resulting in 48 questions that took respondents 10–15 min to complete, in either English or Spanish online or via telephone interview. The survey domains included: social risks, behavioral health (e.g., anxiety, drug/alcohol use), stigma, functioning, self-rated health, concerns about healthcare costs, and trust in healthcare providers. Social risks, self-rated health, and preventive care use are described below as they are central to this paper's psychometric assessment. For social risk screening questions, participants were randomized to either respond first to the AHC tool, followed by the YCLS, or vice versa. Randomization was achieved using simple random sampling of half of the members in each language, age, and sex stratum.
The AHC screening tool was developed by the Centers for Medicare & Medicaid Services Center for Medicare and Medicaid Innovation (CMMI). This tool was intended for use in the AHC Model, which aims to test whether systematically identifying and addressing social risks of Medicare and Medicaid recipients reduces health care utilization and costs, and improves quality of care and health outcomes., This tool includes 10 questions covering five social risks: current housing problems; food insecurity, transportation difficulties, or problems paying utilities over the prior 12 months; and interpersonal safety. The tool has been utilized in screening for a wide range of populations and settings. Currently, no psychometric evidence is available for this tool. The AHC screening tool is publicly available at no charge and has a reading level lower than the 9th grade, and is brief.,
The YCLS screening tool was developed by KP's Care Management Institute. This nine-item screening tool asks about patient risks over the previous 3 months related to the following domains: living situation, financial hardship, food insecurity, transportation, support for activities of daily living, stress, and desire for help. This screening tool was designed to be administered on paper, electronically, or over the phone. Currently, no psychometric evidence is available for this tool. The YCLS screening tool is publicly available at no charge, has a reading level below the 9th grade, and is brief.,
Housing. The study survey captured three housing issues: instability, poor housing quality, and difficulty paying for housing. For instability, the AHC assessed if respondents had (a) a steady place to live, (b) a steady place to live but are worried about losing it, or (c) do not have a steady place to live. A response to b) or c) indicated housing instability. The YCLS included one item that asked respondents to categorize their current living situation into one of seven response options3. Endorsement of either temporary housing, living in a shelter, or homelessness indicated housing instability. With respect to housing quality, the AHC included one item that asked if respondents had problems with up to seven issues (e.g., pests, mold, lead paint or pipes); endorsement of one or more problems reflected a housing quality concern. The YCLS housing quality item asked if the respondent had any concern about topics, including the condition of their current living situation. An affirmative response indicated risk and cascaded to five options describing housing concerns. For difficulties paying, the YCLS inquired about difficulties paying for housing (captured as “yes” or “no”) and an item from the Children's Health Watch (CHW) Housing VitalSigns, a measure with demonstrated evidence of validity in numerous settings, asked if in the last 12 months respondents had difficulty paying their rent or mortgage.
The AHC screening tool consists of two food insecurity questions, which are adapted from the Hunger Vital Sign. These two questions, based off of the USDA 18-item survey, have shown strong initial evidence of psychometric validity., One item asks about worry that food will run out, and the other about incidence of food running out, before the respondent has money to buy more. Responding “sometimes true” or “often true” reflected food insecurity. The YCLS food worry item asks about worry about running out of food but for 3 months and with different response options (”sometimes, “often,” or “very often” reflected worry related to food). The YCLS also included “food” as an option when asking respondents to indicate, “yes” or “no,” to trouble paying for items.
The AHC item inquires about lack of reliable transportation for “medical appointments, meetings, work or from getting things needed for daily living” (”yes” or “no”), similar to previously established screening measures. The YCLS transportation item has two parts that assess if lack of transportation kept respondents from a) medical appointments or getting medication, or b) from doing other things needed for daily living. An affirmative response to a) and/or b) indicated a transportation risk.
The AHC item asks respondents if their electric, gas, oil, or water company had threatened to shut off services in their home in the past 12 months (captured as “yes,” “no,” “already shut off”). The YCLS item asks if the respondent had trouble paying for heat or electricity in the prior 3 months as one of several options for a single question.
The survey contained a single item assessing self-rated health. The item asked respondents to rate their health on a 5-point scale ranging from excellent to poor. Self-rated health was dichotomized as poor/fair versus good/very good/excellent.
The survey was administered through the DatStat tool for online survey research. The survey was sent to participants via mail in two waves of 504 records per wave sent 2 weeks apart; the first wave fielded on November 5, 2018. The recruitment letter included an online web link and $2 pre-incentive. Participants received follow-up emails, phone calls, text messages and, if requested, a mailed paper survey. Reminder invitations were sent using multiple modes until a final survey disposition was obtained (e.g., complete, refused) or when the fielding window ended 2 months after the initial invitation. A $20 cash incentive was mailed to respondents who completed at least 50% of the survey items.
Administrative data from the electronic health record were obtained to characterize the sample and compare responders and non-responders across three categories of variables linked to social risks: (1) sociodemographic factors (age, sex, race/ethnicity, and language); (2) medical comorbidities (mental illness, Charlson Comorbidity); and (3) healthcare utilization including: receipt of a flu shot in 2018 (as a marker of preventive care), emergency department use, hospitalizations, rehospitalizations within 30 days of discharge, appointment no-shows, appointment cancellations and urgent care use.
To assess for potential bias related to survey non-response, we compared the demographics of responders to those invited who did not respond. We compared the two groups using administrative data on 28 variables. For binary and categorical variables, χ2 tests for two independent samples were used to assess the null hypothesis of no difference in distribution of variables in the samples. The Student's t-test and the Wilcoxon rank sum test were used to assess differences in continuous variables between responders and non-responders.
To examine the concurrent and predictive validity of the AHC and YCLS screening tools in the study sample, we conducted two analyses. Screening measures were divided into four social risk domains: (1) housing: insecurity, quality, difficulty paying; (2) food: worry and difficulty paying; (3) transportation issues; and (4) difficulty paying for utilities. First, to assess concurrent validity, we estimated agreement between the AHC, YCLS, and CHW (when available for housing) items within domains using unadjusted and adjusted Cohen's kappa coefficient with 95% confidence intervals.,, Adjusted kappa accounts for the influence of bias and prevalence. Kappas between 0.21 and 0.40 indicate fair agreement, 0.41 and 0.60 moderate, 0.61 and 0.80 substantial and over 0.80 excellent. Second, for predictive validity, we estimated the association between screening positive on the AHC and YCLS items and an individual's self-rated health and receipt of a flu shot using logistic regression. We hypothesized there would be a negative relationship between presence of social risks with lower self-rated health and lower incidence of flu shots. Regressions adjusted for age, sex, and race/ethnicity at the individual level, and median income and proportion with a high school education at the census tract level.
| Results|| |
Responders versus non-responders Examining bias
[Table 1] summarizes the distributions of demographic, medical comorbidities, and healthcare utilization among the overall sample, as well as for the responder (n = 450) and non-responder (n = 558) groups. Information on the percentage of the population within participants' census block with various social risks are included in [Table 1]. The response rate was 47% and cooperation rate (responders over total contacted) was 74%; see [Figure 1] for final survey disposition. The sample reflected the pre-specified sampling plan, containing an equal distribution of individuals within the four pre-selected age categories, sex, and language.
Compared to the non-responder group, responders had a higher proportion of women (P < 0.001), and a lower proportion of Spanish-speaking individuals (P = 0.001). Responders had a lower proportion of missing data on race/ethnicity (P < 0.0001), and a higher prevalence of depression (P = 0.040) compared to non-responders. The responders had a higher estimated prevalence of anxiety (11.3% vs. 8.2%), a higher proportion identified as white (46.6% vs. 41.0%), and a lower proportion identified as Hispanic (32.2% vs. 38.4%) than non-responders, but these differences were not statistically significant [Table 2]. Healthcare utilization and neighborhood level characteristics did not significantly differ between responders and non-responders [Table 3].
|Table 2: Insurance Characteristics and Medical Comorbidities of Responders versus Non-Responders|
Click here to view
Social risk screening tool comparisons Psychometric analysis
On the AHC tool, the percentage reporting social risks ranged from 7% to 29% [Table 4]. The percentage of patients reporting social risks on the YCLS tool ranged from 4% to 23% [Table 4]. Within social domains, percentages reporting a social risk tended to be higher by the AHC than the YCLS. Using unadjusted kappas, the AHC and YCLS items had substantial agreement for measures of food insecurity only. When examining the adjusted kappas that account for bias and prevalence, agreement between the AHC and YCLS items was substantial or higher (kappas > 0.60) for all social risks except housing quality (kappa = 0.52). The YCLS and CHW had substantial agreement (kappa 0.75) on housing.
|Table 4: Concurrent Validity of the AHC, YCLS, and Children's Health Watch|
Click here to view
Among survey respondents, 15.1% reported fair or poor health, and 42.7% had a flu shot. A reported social risk on the AHC and YCLS measures was strongly associated with having fair or poor self-rated health [Table 5]. Estimates of association between the AHC items and having had a flu shot were small, inconsistent in direction, and not statistically significant except with difficulty paying for utilities (OR = 2.48; 95% CI = 1.10–5.59). The YCLS measures were not associated with having had a flu shot and point estimates were small.
| Discussion|| |
This paper reports results of a survey of subsidized exchange members in an integrated healthcare system. The study compared two screening tools assessing patients' social risks: one from the AHC demonstration project and one from the KP system (YCLS). Responders did not substantially differ from non-responders except on sex, language and depressive disorder diagnosis. A higher percentage of participants screened positive for social risks by the AHC questions compared to the YCLS but agreement between the tools was substantial for most domains. Both tools were associated with self-rated health but not with preventive healthcare. These results are the first to suggest that both the AHC and YCLS have concurrent and predictive validity, supporting their use in healthcare settings, including by primary care physicians to engage in social risk-informed care.
Health care systems and primary care physicians will need to consider validity, reliability, and pragmatic aspects when selecting tools. For example, the higher proportion of respondents screening positive for housing problems on the AHC than the YCLS means the AHC may be more sensitive while the YCLS may have greater specificity. Our study suggests that if a primary care clinic is interested in casting a wide net with sensitive capture of patients at risk, then the AHC may be preferable. Differences in how the AHC and YCLS phrase the questions might drive differences in prevalence rates. For example, the housing insecurity item for the AHC includes the option to select worry about housing, whereas the YCLS tool only asks about the housing situation. The other domains (food, transportation, utilities) did not show as much disparity in prevalence. Associations with self-rated health suggest both screening tools have evidence of predictive validity. Although responses on thescreening tools and flu shots were unassociated, the lack of association could have been due to the ubiquity of flu shots being offered by KP.
Several study design limitations exist. Although responders did not substantially differ from non-responders, some demographic differences (i.e., language, gender, depression) were observed. The survey was cross-sectional and more psychometric testing is needed to evaluate whether the tools relate to worse health in a longitudinal study. Because this sample, by design, encompassed a broad array of members, we were only able to include one measure of preventative care relevant to all participants.
As work in this area advances, healthcare systems and primary care physicians will benefit from evidence to guide social risk screening and interventions. Future studies should examine whether the AHC and YCLS differ in patients' willingness to receive assistance with identified social risks.,,,,, One tool may better identify those in need who also want help, which could help focus intervention efforts. Although this study evaluated the psychometric properties of the AHC and YCLS, more information on the patient acceptability of these tools is needed., Comparison to other tools such as PRAPARE is warranted. Given the significant investment to undertake social risk screening, it is important to know if predictive analytics using administrative data can accurately identify those in need of additional screening, or if universal screening is needed. Although the AHC and YCLS both appear to be promising screening tools with evidence of key psychometric properties, answering the above questions is important to inform which tool should be used, with whom, when, and how often.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
- SONNET's social needs survey was conducted as part of a quality improvement initiative, which was deemed Not Human Subjects Research by KP Washington Research Institute's IRB.
- Due to people over 65 transitioning to Medicare, our sample was heavily weighted towards adults age 62 to 65. Although we stratified sampling using 18-26, 27-44, 45-61, and 62 plus categories, we elected to report the age breakdown using categories reflecting important differences in life stage including those affected by the ACA: able to be on parents insurance until age 26; age 45 reflecting midlife; and age 62 being the age at which people can retire early to start taking Social Security
- Open ended responses to the “other” option were recoded by one team member (CWB) to the most appropriate fixed response option and checked by the study analyst (AD) for accuracy.
| References|| |
Marmot M, Friel S, Bell R, Houweling TAJ, Taylor S. Closing the gap in a generation: Health equity through action on the social determinants of health. Lancet 2008;372:1661-9.
Braveman P, Egerter S, Williams DR. The social determinants of health: Coming of age. Annu Rev Public Health 2011;32:381-98.
Koch CG, Li L, Kaplan GA, Wachterman J, Shishehbor MH, Sabik J, et al
. Socioeconomic position, not race, is linked to death after cardiac surgery. Circulation 2010;3:267-76.
Lynch JW, Kaplan GA, Shema SJ. Cumulative impact of sustained economic hardship on physical, cognitive, psychological, and social functioning. New Engl J Med 1997;337:1889-95.
Woolf SH, Braveman P. Where health disparities begin: The role of social and economic determinants--and why current policies may make matters worse. Health Affair 2011;30:1852-9.
Blazer DG, Sachs-Ericsson N, Hybels CF. Perception of unmet basic needs as a predictor of depressive symptoms among community-dwelling older adults. J Gerontol A Biol Sci Med Sci 2007;62:191-5.
McMullen AM, Katz MH. Targeting unmet social needs-next steps toward improving chronic disease management. JAMA Intern Med 2017;177:252-3.
Shields-Zeeman L, Lewis C, Gottlieb L. Social and mental health care integration: The leading edge. JAMA Psychiatry 2019; In press. doi: 10.1001/jamapsychiatry. 2019.1148.
Gottlieb LM, Wing H, Adler NE. A systematic review of interventions on patients' social and economic needs. Am J Prev Med 2017;53:719-29.
Beck AF, Henize AW, Kahn RS, Reiber KL, Young JJ, Klein MD. Forging a pediatric primary care-community partnership to support food-insecure families. Pediatrics 2014;134:564-71.
Berkowitz SA, Hulberg AC, Hong C, Stowell BJ, Tirozzi KJ, Traore CY, et al
. Addressing basic resource needs to improve primary care quality: A community collaboration programme. BMJ Qual Saf 2016;25:164-72.
Gottlieb LM, Hessler D, Long D, Laves E, Burns AR, Amaya A, et al
. Effects of social needs screening and in-person service navigation on child health: A randomized clinical trial. JAMA Pediatr 2016;170:e162521.
Bachrach D, Pfister H, Wallis K, Lipson M. Addressing Patients' Social Needs: An Emerging Business Case for Provider Investment. New York, NY: The Commonwealth Fund; 2014.
Berwick DM, Nolan TW, Whittington J. The triple aim: Care, health, and cost. Health Aff (MIllwood) 2008;27:759-69.
Alderwick H, Gottlieb LM. Meanings and misunderstandings: A social determinants of health lexicon for health care systems. Milbank Q 2019;97:407-19.
National Academies of Sciences, Engineering, and Medicine; Health and Medicine Division; Board on Health Care Services; Committee on Integrating Social Needs Care into the Delivery of Health Care to Improve the Nation's Health. Integrating Social Care into the Delivery of Health Care: Moving Upstream to Improve the Nation's Health. Washington, D.C.: National Academies Press (US); 2019.
Cartier Y, Pantell M, De Marchis E, Gottlieb L. National Surveys Gauging Prevalence of Social Care-Related Activities in the Health Care Sector. SIREN Network; 2018.
Morone J. An integrative review of social determinants of health assessment and screening tools used in pediatrics. J Pediatr Nurs 2017;37:22-8.
Sokol R, Austin A, Chandler C, Byrum E, Bousquette J, Lancaster C, et al
. Screening children for social determinants of health: A systematic review. Pediatrics 2019;144:e20191622.
Henrikson NB, Blasi PR, Dorsey CN, Mettert KD, Nguyen MB, Walsh-Bailey C, et al
. Psychometric and pragmatic properties of social risk screening tools: A systematic review. Am J Prev Med 2019;57 (6 Suppl 1):S13-24.
Moen M, Storr D, German E, Friedman E, Johantgen M. A review of tools to screen for social determinants of health in the United States: A practice brief. Popul Health Manag 2020.
Oldfield BJ, Casey M, DeCew A, Morales SI, Olson DP. Screening for social determinants of health among children: Patients' preferences for receiving information to meet social needs and a comparison of screening instruments. Popul Health Manag 2020. doi: 10.1089/pop. 2019.0158.
LaForge K, Gold R, Cottrell E, Bunce AE, Proser M, Hollombe C, et al
. How 6 organizations developed tools and processes for social determinants of health screening in primary care: An overview. JACM 2018;41:2-14.
Social, IOM Capturing. “Behavioral Domains in Electronic Health Records, Phase 1.” Edited by Records CotRSaBDaMfEH. Washington DC: Institute of Medicine (2014).
Coleman-Jensen A, Rabbitt MP, Gregory CA, Singh A. Household Food Security in the United States in 2016, ERR-237. United States Department of Agriculture, Economic Research Service; 2017.
De Marchis EH, Hessler D, Fichtenberg C, Adler N, Byhoff E, Cohen AJ, et al
. Part I: A quantitative study of social risk screening acceptability in patients and caregivers. Am J Prev Med 2019;57 (6 Suppl 1):S25-37.
Lewis CC, Mettert KD, Dorsey CN, Martinez RG, Weiner BJ, Nolen E, et al
. An updated protocol for a systematic review of implementation-related measures. Syst Rev 2018;7:66.
Sandel M, Sheward R, Ettinger de Cuba S, Coleman SM, Frank DA, Chilton M, et al
. Unstable housing and caregiver and child health in renter families. Pediatrics 2018;141:e20172199.
Hager ER, Quigg AM, Black MM, Coleman SM, Heeren T, Rose-Jacobs R, et al
. Development and validity of a 2-item screen to identify families at risk for food insecurity. Pediatrics 2010;126:e26-32.
Gundersen C, Engelhard EE, Crumbaugh AS, Seligman HK. Brief assessment of food insecurity accurately identifies high-risk US adults. Public Health Nutr 2017;20:1367-71.
National Association of Community Health Centers I. PRAPARE: Protocol for Responding to and Assessing Patient Assets, Risks, and Experiences. 2016.
Lorig K, Stewart A, Ritter P, González V, Laurent D, Lynch J. Stanford Chronic Disease Self-Management Study. Outcome Measures for Health Education and other Health Care Interventions. Thousand Oaks, CA: Sage Publications; 1996. p. 25.
DatStat, Inc. [computer program]. SCI Solutions; 2019.
Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37-46.
Sim J, Wright CC. The Kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Phys Ther 2005;85:257-68.
Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol 1993;46:423-9.
Gold R, Bunce A, Cowburn S, Dambrun K, Dearing M, Middendorf M, et al
. Adoption of social determinants of health EHR tools by community health centers. Ann Fam Med 2018;16:399-407.
Bottino CJ, Rhodes ET, Kreatsoulas C, Cox JE, Fleegler EW. Food insecurity screening in pediatric primary care: Can offering referrals help identify families in need? Acad Pediatr 2017;17:497-503.
Wylie SA, Hassan A, Krull EG, Pikcilingis AB, Corliss HL, Woods ER, et al
. Assessing and referring adolescents' health-related social problems: Qualitative evaluation of a novel web-based approach. J Telemed Telecare 2012;18:392-8.
Hassan A, Scherer EA, Pikcilingis A, Krull E, McNickles L, Marmon G, et al
. Improving social determinants of health: Effectiveness of a web-based intervention. Am J Prev Med 2015;49:822-31.
Cullen D, Attridge M, Fein JA. Food for thought: A qualitative evaluation of caregiver preferences for food insecurity screening and resource referral. Acad Pediatr 2020:S1876-2859:30170-30174. doi: 10.1016/j.acap. 2020.04.006.
Rogers AJ, Hamity C, Sharp AL, Jackson AH, Schickedanz AB. Patients' attitudes and perceptions regarding social needs screening and navigation: Multi-site survey in a large integrated health system. J Gen Intern Med 2020;35:1389-95.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]