Frequently Asked Questions - Contracts
What is the web protocol for the MA & PDP CAHPS Survey?
The web component of the data collection protocol for the MA & PDP CAHPS Survey requires mailing of a pre-notification letter to all sampled enrollees, followed by an initial survey invitation. The survey invitation is sent via email to those enrollees with an available email address and via letter to those enrollees without an available email address. Enrollees with an email address will receive an email reminder to complete the survey. The pre-notification letter, initial survey invitation, and email reminder must contain the web survey URL and a PIN code unique to the enrollee. CMS describes the web protocol in detail in Chapter VI: Data Collection Protocol of the QAP&TS.
May my contract share enrollee email addresses with my CMS-approved MA & PDP CAHPS Survey vendor?
Contracts are encouraged to provide their MA & PDP CAHPS Survey vendor with email addresses for all enrollees to support email delivery of web survey invitations to enrollees sampled for the survey. Your contract should review the Business Associate Agreement (BAA) you execute with your survey vendor to make sure it includes sharing of your enrollee email address data.
How will my CMS-approved MA & PDP CAHPS Survey vendor manage the volume of emails to sampled enrollees?
Vendors approved by CMS to administer the MA & PDP CAHPS Survey have met the MA & PDP CAHPS Survey Minimum Business Requirements, including experience administering surveys via the web. CMS anticipates that vendors will employ common practices, such as use of multiple Internet Protocol (IP) addresses, reputation management of IP addresses, and email security protocols to manage the delivery of emails to sampled enrollees.
Are all MA and PDP contracts required to participate in the MA & PDP CAHPS Survey?
Medicare Advantage Organizations, 1876 Cost Contracts, Employer/union only contracts, Medicare-Medicaid Plans, and Part D sponsors with at least 600 enrollees are required to participate in the MA & PDP CAHPS Survey. Eligible enrollees must have 6 months of continuous enrollment in the MA or PDP contract beginning in July of the previous year and must still be enrolled in the contract at the time of the January sample draw. While the specific dates of data collection will vary from year to year, the survey is usually administered during March to June.
Who pays for survey administration?
MA and PDP contracts contract directly with approved MA & PDP CAHPS Survey vendors to conduct the survey. Contracts communicate their choice of vendor to CMS via a web-based Vendor Authorization and Oversample request tool.
How is my contract’s eligible sample determined?
In January, each contract’s sample is drawn from current enrollees with 6 months of continuous enrollment in the contract. Continuous enrollment in the contract is determined using CMS monthly enrollment data from July of the previous year to January of the current year. If a contract member has any gaps in the CMS monthly enrollment data, he or she is excluded from the sample. Continuous enrollment is one of several eligibility criteria. CMS excludes enrollees who are under age 18, known to be deceased, known to reside in institutional settings, or reside outside of the United States. Additionally, CMS sample procedures do not allow for the selection of more than one enrollee per household.
Where can a contract get information about Vendor Authorization?
Each November, CMS provides Medicare Compliance Officers with information about vendor authorization for the upcoming administration of MA & PDP CAHPS. Your contract’s Compliance Officer should contact mapdpcahps@rand.org or 1-866-690-1650 with any questions about vendor authorization for the upcoming MA & PDP CAHPS Survey cycle.
My contract would like to request an oversample; how do I do that?
For information about how to request an oversample, your contract’s Medicare Compliance Officer should contact mapdpcahps@rand.org or 1-866-690-1650. Oversample requests are usually due in early December for the upcoming administration of MA & PDP CAHPS.
Where can I find the annual timeline for the MA & PDP CAHPS survey cycle?
While specific dates will vary from year to year, the general timeline for the survey is:
-
HPMS memo publicizing key dates for contract actions is distributed in November
-
Medicare Compliance Officers for contracts required to report CAHPS receive instructions for vendor authorization and oversample requests in November
-
Contracts must authorize a vendor and request any oversample by early December
-
CMS begins sample selection in January, based on each contract’s current enrollment
-
Survey administration occurs from approximately March to June
-
Plan preview reports are emailed to Medicare Compliance Officers in August
-
Medicare Compliance Officers are emailed Detailed Plan Reports in late fall
What languages are available for MA & PDP CAHPS Survey administration?
The MA & PDP CAHPS Survey is available in English, Spanish, Chinese, Vietnamese, Korean, and Tagalog. All contracts are required to administer the survey in English and Spanish, and contracts have the option to administer the survey in Chinese, Vietnamese, Korean, and/or Tagalog depending on the needs of their enrollees. The Chinese translation of the survey is appropriate for Cantonese and Mandarin speakers.
Are supplemental questions allowed for MA & PDP CAHPS Survey administration? If so, are there any restrictions on the questions?
CMS allows contracts to add a maximum of 12 supplemental questions to the MA & PDP CAHPS Survey. Contracts submit supplemental items to CMS for review and approval for use via their authorized survey vendors.
Detailed guidance on the submission process and examples of questions that do not meet CMS guidelines for approved supplemental items can be found in Appendix P of the Quality Assurance Protocols & Technical Specifications (QAP&TS) manual.
As a resource for possible supplemental questions, CMS suggests the use of the CAHPS Supplemental Items for the Adult Health Plan Questionnaires posted on the AHRQ website. These items have been tested; however, please note that some of the items may not meet the protocols for MA & PDP CAHPS Survey supplemental items.
Are there any restrictions or limitations on using Medicare CAHPS to survey Medicare enrollees at other time points in the year outside the range of the official Medicare CAHPS data collection cycle?
CMS strongly encourages contracts and survey vendors NOT to ask Medicare enrollees any MA & PDP CAHPS Survey questions four weeks prior to, during, and four weeks after the MA & PDP CAHPS Survey administration period (that is, from February 1 to June 30 each year). Other CMS surveys are exempt from this guidance. CMS provides this recommendation to promote response rates to both MA & PDP CAHPS and additional data collection efforts.
Who will submit a Medicare Advantage contract’s CAHPS data to NCQA?
At the end of the annual MA & PDP CAHPS survey cycle CMS will submit the MA CAHPS survey data files to NCQA with calculated scores for accreditation. Note that NCQA uses a different scoring methodology from CMS in calculating results.
May contracts receive de-identified member-level response files from their CAHPS survey for quality improvement?
No, the CMS Data Use Agreement for MA & PDP CAHPS prohibits vendors from sharing enrollee-level data. These data could be used to identify an individual (either directly or indirectly), which would violate the guarantee of confidentiality that CMS provides all survey respondents.
When are MA & PDP CAHPS Survey Reports available each year?
The MA & PDP CAHPS Survey Plan Preview Reports are generally delivered to a contract’s Medicare Compliance Officer in August of each year by email. The MA & PDP CAHPS Survey Detailed Plan Reports are generally delivered to the contract’s Medicare Compliance Officer in late fall of each year by email. Contracts are encouraged to keep their Medicare Compliance Officer name and contact information up-to-date within HPMS.
How might CMS analysis of the survey data differ from the survey vendor’s analysis?
CMS-calculated results are official results. Every year some plans inquire about discrepancies between vendor and CMS results. Historically, discrepancies are due to factors such as vendor misapplication of forward-cleaning rules, vendor top-box scoring rather than linear mean scoring, and vendor errors in the determination of eligible surveys.
How are survey data edited and cleaned during data analysis by CMS?
A forward-cleaning approach is used for editing and cleaning survey data. This approach uses responses to the “screener” (or gate) items to control how subsequent items within the questionnaire are treated analytically, such as setting responses to a missing value or retaining the original response. Under this forward-cleaning approach, unanswered screener items are not updated or back-filled based on responses to subsequent items. The conventions and guidelines used to forward clean MA & PDP CAHPS data can be found in Chapter IX: Data Analysis and Public Reporting of the QAP&TS.
How is reliability calculated for measures?
Reliability is calculated for each contract’s score for each measure. This indicates how well the score for a single contract distinguishes its performance from that of other contracts, and it is scaled from 0 (indicating no information) to 1 (indicating perfect accuracy in comparing or ranking contracts). Interunit reliability (IUR) is calculated using the formula IUR = 1−V/(V+t2), where V is the variance of the estimate of the measure for that contract, t2 is the between-contract variance of the mean for that measure, and V and t2 are estimated from a linear random-effects model. IUR as defined here is related to Spearman-Brown reliability, but our formulation allows for variation in reliability across contracts due the varying rates and patterns of response.
Reliability of the estimated scores is affected by a number of factors, including:
• The number of a contract’s respondents who answer an item (or each item of a composite measure);
• The variability of responses within the contract;
• The amount by which contracts differ from each other nationally on that measure.
The reliability of a composite measure cannot be obtained by averaging the reliabilities of its constituent items. Measurement error decreases with multiple measurements (multiple items from the same enrollee in each composite), so the measurement error for a composite averaging multiple items tends to be less than the average measurement error for individual items. The total number of respondents providing information toward the composite might be greater than the number responding to any one item. Consequently, a composite often has higher reliability than most or sometimes any of the items that comprise it.
Low reliability scores are defined as those with at least 11 respondents and reliability ≥0.60 but <0.75 and also in the lowest 12% of contracts ordered by reliability. These scores are publicly reported and may affect Star Ratings. If reliability is very low (<0.60), the contract does not receive a Star Rating for that measure.
If two survey measures have low reliability in a composite based on the mean of scores of three survey measures, is the composite based only on one measure?
Reliabilities of individual survey measures or items are for internal quality improvement use only; item-level reliabilities do not have implications for scoring and items are not removed from a composite because of low reliability. Only reliability of the composite has implications for scoring or reporting.
Where can I find more information on how the survey results are calculated?
Information regarding scoring and analysis of MA & PDP CAHPS Survey results can be found in Chapter IX: Data Analysis and Public Reporting of the QAP&TS and on the Scoring and Star Ratings webpage.
Where can I find response rate information for the MA & PDP CAHPS survey?
CMS provides historic response rates on the MA & PDP CAHPS Survey website. The available information includes historic MA & PDP CAHPS response rates overall, by survey type, and by survey mode. Historic overall survey response rates for Medicare Advantage, Fee-for-Service, and Prescription Drug Plan coverage are also available. CMS does not make contract-level response rates publicly available.
If a contract decides to field the MA & PDP CAHPS Survey on an optional basis, will the results be used toward Star Ratings?
All contracts with fewer than 600 eligible enrollees are not required to field the survey; if the number of eligible enrollees is between 450 and 599, a contract may field the survey on an optional basis. Contracts that choose to participate will have their scores reported and used in Star Ratings.
Who can I contact with questions about my CAHPS scores and MA & PDP CAHPS Star Ratings?
Questions regarding your MA & PDP CAHPS scores should be directed to MP-CAHPS@cms.hhs.gov.
Questions regarding Part C and D Star Ratings calculations should be directed to PartCandDStarRatings@cms.hhs.gov.
Last modified on 10/24/2023