To be eligible for the sample, members need to have been a health plan member in the 4th quarter of the year prior to the survey, a current member at the time of the survey, be able to read in English, not be in a Skilled Nursing Facility or in hospice, and not have dementia, Alzheimer’s disease, or other types of cognitive disability.
Survey sample selection: 1993 through 2014/2015
Because the survey results were designed to be used to create profiles of KPNC’s 19 medical center service populations (MCSPs) as well as the region, separate age-gender stratified random samples were selected from among each MCSP. Members are administratively assigned to an MCSP based on where they receive or would be expected to receive most of their primary care. MCSPs are not the same as geographic catchment areas.
Most MCSP samples are comprised of 2,200 members, with set numbers of women and men selected from five age groups (20-44, 45-64, 65-74, and 75-79, and 80+). To ensure that adequate numbers of racial and ethnic groups are represented, we select slightly larger samples (approximately 2,500 members) from our Oakland, Hayward, Richmond, San Francisco, and Vallejo medical center service populations which have our most ethnically diverse service populations. Members aged 65 and over are also oversampled for each MCSP so that we can learn as much as possible about the characteristics of this group that is a major consumer of health care.
Survey sample selection in 2017
Because we needed to reduce our sample size, we decided to change our sampling design in order to obtain information from sufficient numbers of black, Latino, and Asian members to study race/ethnic disparities in sociodemographic, health, and health care characteristics. Taking advantage of the greater number of adult members with race/ethnicity data in our electronic health records, we will be selecting age-sex stratified random samples of English speaking members from our four largest race/ethnic groups: non-Hispanic white, black, Latino, and Asian. We restrict the sample to English speakers due to funding constraints and our past experiences of a very low response rate among members whose primary language is Spanish. We try to get information about our Spanish speaking members through other targeted surveys and by analysis of their electronic health data.
Data collection methods
The majority of respondents participate using a self-administered print questionnaires mailed to their home. We usually begin to mail out questionnaires in mid-April, followed by up to two more mailings four to six weeks apart to enhance the final response rate. Beginning in 2005, respondents have also been able to complete the survey online using a URL printed in the survey materials and/or a link sent to them in an email. The survey is also administered by phone interview upon request.
Separate questionnaires are used to collect data from women aged 20-64, men aged 20-64, and women and men aged 65 and over. The questionnaire for men aged 20-64 is virtually identical to that for similarly aged women but excludes questions or items specific to women and includes some items specific to men’s health. The questionnaire for men and women aged 65 and over is slightly longer than the women’s questionnaire, containing all of the core items plus additional items on functional health, use of health-related supportive services, and usual mode of transportation.
To encourage participation, respondents are automatically entered into a gift card drawing held toward the end of the survey year. There are no individual financial incentives offered for participation.
Survey response rate
Information about overall and age group-gender specific survey response rates for each survey year is found in the introductions of the region and medical center service population profile reports for each survey year. The overall response rate for the most recent (2014/2015) survey was 38 percent As with other similar surveys, the overall survey response rate has been declining over time (55 percent in 1993, 45 percent in 2005, 41 percent in 2008, and 39% in 2011). Response rates are lower for the 20-44 and 45-64 age groups than for members aged 65 and over, who through 2011 had response rates hovering around 70 percent but in 2014/2015 dropped to approximately 65 percent.
Assignment of post-stratification weighting factors
After we have the final respondent sample, we assign respondents a weighting factor based on the number of members of their sex and age group (5-year intervals) in their medical center service population. For example, data for 35 year old and 50 year old male respondents drawn from the Oakland medical center service population might be weighted to represent 241 and 183 men, respectively, whereas data for male respondents of these ages drawn from the Kaiser Permanente-Sacramento service population might be weighted to represent 344 and 386 men, respectively. We use the weighted data, which reflect the actual age, gender, and geographic composition of the membership, to estimate the demographic characteristics and prevalence of different health conditions and health risks in the region and service populations.
Linkage of survey data to other sources of information
Member Health Survey data are often linked at the individual level to Kaiser Permanente administrative and clinical information on health and health care use for research purposes. Survey data can also be linked to Census-derived and other geographic information data based on street address or zip code. The same health plan and geographic information can be obtained for non-respondents. All data linkage is done in a way to protect the confidentiality of people in the survey sample. We are pre-approved by our IRB to link certain data at the time of the survey as outlined in our study application. If we want to link additional data, we must request IRB approval to do so.