Sherlock Company

• Home
• SEER Forums
• Analyses
• Site Index
• Tools
• Contact Us

Description of our Quality Assurance Procedures

1. Strong Definitions. Ambiguity was minimized using both a top-down and bottom-up methodology of reporting guidelines. Functions and product lines were extensively defined in the survey instrument and Common Guidelines. Definitions frequently included lists of activities performed by the functional area. Additionally, examples of cost centers associated with various functions were provided so that, if a question arose as to what functional area a cost center should be included, the respondent might be able to locate where other similarly-described cost centers have been assigned. Because these definitions were provided electronically they were searchable.  Definitions were included as a separate Common Guidelines document, but were also included as “comments” on the electronic form of the survey.
2. Feedback, Reinforcing Definitions. If ambiguities were found in the survey, we communicated interpretations to all participants so that all respondents could adjust their responses accordingly.  Issues were addressed and disseminated through a SEER Forum, maintained on our website, www.sherlockco.com.
3. Participatory Protocol. The survey form was designed with the input of participants. This, we believe, helps to assure that respondents regard the benefit of information requested exceeds the cost of gathering it. Accordingly, we believe that Plans were earnestly committed to the integrity of the submission, rather than simply fulfilling the letter of the requirements. This approach also assures that the functions reflect the consensus of the actual practices of the respondents, reducing the need to reclassify to arbitrary standards. Finally, each of the participants receives a copy of the report, typically shared throughout the organization and with senior management, which will in part reflect the quality of their individual submission.
4. Reconciliation with Financial Statements. We requested that each of the Plans provide audited consolidated financial information to tie to expense information provided in the survey form.  To the extent that differences existed between the data submitted in survey form and in the audit, we asked for a reconciliation schedule. The Plans’ reconciliation of their submitted data with their financials was intended to assist in assuring the accuracy and completeness of their information.
5. Submissions Scanned for Anomalies. We use statistical models and visual screens to identify outliers in the submissions. Where outliers were discovered, we contacted the participants to determine whether variances stem from reporting errors or are true operational differences. Reporting errors were corrected for inclusion in this report. In certain cases, reported expenses were excluded from this report. Notably, most Medicare Advantage products that have been introduced during 2006 were excluded from this survey since to include them would have blended their start-up costs with normal operations of more mature plans. However, since all Medicare Part D products were new, all were included in this study.
6. Review by Participants.  Prior to final printing, a draft of the document was submitted to each of the participants. This draft was similar to the final SEER report in that it highlighted their own results in the context of the universe as a whole. This permitted them to identify any anomalies that we may have missed.
7. Practice Effect. 74% of the respondents participating in this year’s study participated with us last year, and 65% of this year’s participants have five or more years of experience participating in SEER. We believe that this enhances the accuracy of reporting to us.