Wiki.usask.ca

Association Between Licensure Examination Scores
and Practice in Primary Care
Robyn Tamblyn; Michal Abrahamowicz; W. Dale Dauphinee; et al. JAMA. 2002;288(23):3019-3026 (doi:10.1001/jama.288.23.3019) Primary Care/ Family Medicine; Quality of Care; Quality of Care, Other Association Between Licensure Examination
Scores and Practice in Primary Care
Robyn Tamblyn, PhD
Context Standards for licensure are designed to provide assurance to the public of a
physician’s competence to practice. However, there has been little assessment of therelationship between examination scores and subsequent practice performance.
Objective To determine if there is a sustained relationship between certification ex-
amination scores and practice performance and if licensing examinations taken at the end of medical school are predictive of future practice in primary care.
Design, Setting, and Participants A total of 912 family physicians, who passed
the Que´bec family medicine certification examination (QLEX) between 1990 and 1993
and entered practice. Linked databases were used to assess physicians’ practice per- formance for 3.4 million patients in the universal health care system in Que´bec, Canada.
Patients were seen during the follow-up period for the first 4 years (1993 cohort of physicians) to 7 years (1990 cohort of physicians) of practice from July 1 of the cer- tification examination to December 31, 1996.
sures the basic competence ofphysicians by requiring them to Main Outcome Measures Mammography screening rate, continuity of care in-
dex, disease-specific and symptom-relief prescribing rate, contraindicated prescribing
aminations.1 Although it is generally as-sumed that these examinations predict Results Physicians achieving higher scores on both examinations had higher rates (rate
increase per SD increase in score per 1000 persons per year) of mammography screening
(␤ for QLEX, 16.8 [95% confidence interval {CI}, 8.7-24.9]; ␤ for Medical Council of Canada ture,2 the data in support of this assump- Qualifying Examination [MCCQE], 17.4 [95% CI, 10.6-24.1]) and consultation (␤ for QLEX, 4.9 [95% CI, 2.1-7.8]; ␤ for MCCQE, 2.9 [95% CI, 0.4-5.4]). Higher subscores in diagnosis were predictive of higher rates in the difference between disease-specific and symptom-relief prescribing (␤ for QLEX, 3.9 [95% CI, 0.9-7.0]; ␤ for MCCQE, 3.8 [95% CI, 0.3-7.3]). Higher scores of drug knowledge were predictive of a lower rate (relative risk per SD increase in score) of contraindicated prescribing for MCCQE (relative risk, 0.88; 95% CI, 0.77-1.00). Relationships between examination scores and practice perfor- likely to adhere to evidence-based guide- mance were sustained through the first 4 to 7 years in practice.
lines in the delivery of care6,7 and achieve Conclusion Scores achieved on certification examinations and licensure examina-
tions taken at the end of medical school show a sustained relationship, over 4 to 7 status, which represents pass/fail status years, with indices of preventive care and acute and chronic disease management inprimary care practice.
on certification examinations, is an im-portant predictor of quality of care.8,9 Que´bec family medicine certification ex- tion predicted colleagues’ ratings of the quality of care delivered by internists5 to 8 years later. However, little is Author Affiliations: Departments of Medicine (Dr
University of Sherbrooke, Sherbrooke, Que´bec (Dr Tamblyn) and Epidemiology and Biostatistics (Drs Grand’Maison); and Centre d’e´valuation des sciences Tamblyn, Abrahamowicz, and Hanley, and Ms de la sante´, University of Laval, Ste-Foy, Que´bec (Dr Girard), McGill University, Montreal, Que´bec; Medi- cal Council of Canada, Ottawa, Ontario (Dr Dau- Corresponding Author and Reprints: Robyn Tam-
phinee); Foundation for Advancement of Interna- blyn, PhD, McGill University, Morrice House, 1140 Pine tional Medical Education and Research, Philadelphia, Ave W, Montre´al, Que´bec, Canada H3A 1A3 (e-mail: Pa (Dr Norcini); Department of Family Medicine, 2002 American Medical Association. All rights reserved.
(Reprinted) JAMA, December 18, 2002—Vol 288, No. 23 3019
EXAMINATION SCORES AND PRACTICE PERFORMANCE indicated drugs, and to refer more of their postgraduate training. It is generally taken patients for consultation.14 However, as- during the final year of medical school.
ited to the first 18 months of practice. We tices. For each cohort, the follow-up pe- required for unrestricted licensure in all used this opportunity to determine if the riod was between July 1 of the certifica- although most Que´bec graduates take the Family Medicine Certification
Examination
covers the costs of medical care for pro- vincial residents. In Que´bec, 14500 phy- ability scores were created for each sub- of services and 93% of physicians are paid tion.18 The overall score reliability was (Re´gie de lЈassurance maladie du Que´- 0.92, and subscore reliabilities were 0.71 Design and Study Population
cohort file by name, sex, and birth date.
Practice Assessment
tice in Que´bec, was followed up for the Data Sources and Retrieval. Four pre-
first 4 to 7 years of practice. Annual mea- sures of each physician’s practice per- eficiary identifiers, were used to assess adjust for differences in the difficulty of trant database provided patients’ age, sex, postal code, and date of death. The medi- curate way to identify all their patients, cation, diagnosis, treating and referring trained in another specialty. Potentially physician, and date of all services deliv- eligible physicians were identified by the ered on a fee-for-service basis. The pre- data on practice activity and linked this duration, prescribing physician, and date physician’s clinical behaviors were used Medical Council of Canada
pitalization database provided records of Qualifying Examination
tion scores and practice performance.
cian, for each year of practice, were re- test an individual’s competence to enter linked by 6-digit postal code to the reg- 3020 JAMA, December 18, 2002—Vol 288, No. 23 (Reprinted)
2002 American Medical Association. All rights reserved.
EXAMINATION SCORES AND PRACTICE PERFORMANCE thyroid medication). Symptom-relief medication was defined as drugs that re- lieve symptoms, but have little impact on the disease process (eg, nonsteroidal anti- cal services claims files were used to iden- epines, low-dose narcotic analgesics) us- Annual contraindicated prescribing rate Annual continuity of care was defined medical services, prescriptions, and hos- as the mean proportion of visits that were the year prior to the first contact with the updated expert review35,36 as 30 drugs that portion of all visits in the year for each Annual consultation rate was the pro- portion of all ambulatory patients in the Indicators of Practice Performance.
respective calendar year referred, at least patient’s proportion was weighted by the each follow-up year, selected on the basis of unexplained practice variation, and/or index for each physician’s practice was Time in Practice. Physicians’ pre-
tinuity of care were assessed only in the selected because of its importance in pre- ment,24-27 and to test the hypothesis that are predictive of better continuity.28,29 Case-Mix Assessment
Relevant data were retrieved for each eli- rate and contraindicated prescribing rate at least 1 outpatient, office practice, or preceding the first contact with the study were used as indicators of the quality of because variation in disease-specific rela- tive to symptom-relief prescribing rate30 just for between-physician differences in able on all prescriptions dispensed.
Annual difference between disease- scribing, which accounts for 20% of drug- specific and symptom-relief prescribing Finally, consultation rate was used as an sician to all elderly patients. Disease- sity for the use of health care services,37 specific medication was defined as drugs pitalization rates in the previous year.
(eg, anticoagulants, anticonvulsants, an- modeled as time-dependent covariates.
patients for specialty consultation.33,34 cause the prescription of relatively con- traindicated medication is rarely justified 2002 American Medical Association. All rights reserved.
(Reprinted) JAMA, December 18, 2002—Vol 288, No. 23 3021
EXAMINATION SCORES AND PRACTICE PERFORMANCE Statistical Analysis
lation in a given year was used as a weight tice experience were tested. For signifi- were evaluated by testing the statistical tions.39 Physicians were the unit of analy- assess rates of contraindicated prescrib- patients followed up over the first 5 years SEs were empirically estimated to account ence in rates corresponding to a 4-SD dif- excluded in years in which they had fewer than 5 patients, and the logarithm of the the result was multiplied by 5 to esti-mate the cumulative impact over 5 years.
PϽ.05 was used as the level of statistical Table 1. Physician Characteristics and Examination Scores for 912 Que´bec Family Physicians*
significance. We used SAS statistical soft- Certification Examination
(97.5%) started practice in Que´bec, 912 (TABLE 1). Overall, 85.8% of physi-
[0.84] vs 0.09 [0.92]; PϽ.001). Gradu- Mean (SD) [Range]
PϽ.001). Que´bec family physicians who −0.03 [1.02]; PϽ.001). Mean scores than in the reference group of first-time takers, with a typical range of 6 to 7 SDs *Examination scores are standardized to a mean of zero representing the average score for first-time takers of the examination from North American medical schools. MCCQE indicates Medical Council of Canada Qualifying Exami- nation; QLEX, Que´bec Certification Examination.
3022 JAMA, December 18, 2002—Vol 288, No. 23 (Reprinted)
2002 American Medical Association. All rights reserved.
EXAMINATION SCORES AND PRACTICE PERFORMANCE with higher rates of disease-specific rela- types of settings (TABLE 2). During the
score was the strongest predictor of dif- first 4 to 7 years of practice, an increas- ferences in the rates of disease-specific achieve statistical significance (Table 3).
private office practice. After the first 2 nificant predictor of contraindicated pre- tinued to increase, but at a slower rate.
increase in score. During the first 5 years, lion different patients (45.9% of the Que´- were in their primary care practice popu- Qualifying Examination
tients than a low-scoring physician.
tern of relationships as the certification examination (TABLE 4). For example, the
tained over the first 4 to 7 years in prac- tice (TABLE 3). The significant interac-
tion between certification examinationscore and practice experience indi-cated that the strength of the relation- Table 2. Practice Setting and Workload Characteristics for Family Physicians in the First 4 to
Physician Practice Year
years 5 to 7. The persistence of this re- Mean (SD)
their primary care practice population.
*Most physicians practiced in multiple practice settings (eg, in year 1-2, the average number of different practice set- tings was 2.6 per physician). For this reason, the number (percentage) of physicians practicing in each type of prac- over the first 5 years, in 58 more refer- tice setting adds to more than 100%, as most physicians were represented in more than 1 category.
†The residence of each patient in a physician’s practice population was categorized as urban (resided in the regions of Montre´al, Que´bec, Laval, or Monte´re´gie), intermediate (resided in the regions of Lanaudières, Estrie Saguenay-Lac-St-Jean, Laurentides, Mauricie-Bois-Francs, or Outaouais), or rural-remote (resided in the regions of Chaudières- Appalaches, Abitibi-Te´miscamingue, Gaspe´sie, Bas-Saint-Laurent, Côte-Nord, Nord-du-Que´bec, Kativik Terres- cries-de la Baie-James). For each physician, the proportion of patients from urban, intermediate, and rural-remoteregions was determined for the first 1 to 2, 3 to 4, and 5 to 7 years of practice. The mean represents the average proportion of patients in the practices of physicians in the cohort who resided in urban, intermediate, and rural-remote locations.
2002 American Medical Association. All rights reserved.
(Reprinted) JAMA, December 18, 2002—Vol 288, No. 23 3023
EXAMINATION SCORES AND PRACTICE PERFORMANCE Table 3. Association Between Family Medicine Certification Examination Scores and Practice Performance in the First 4 to 7 Years of Practice
Change in Outcome per SD Increase in Score
All Practice Years
Practice Year †
Mean (SD)
Rate per 1000
Type of Certification
P
P Value for
Patients
Examination Score
␤ (95% CI)*
Interaction‡
*The regression coefficient ␤ represents the estimated change in the rate or value of the practice outcome per SD increase in score in the first 4 to 7 years in practice with a 95% confidence interval (CI). In these overall models, the interaction term to test the potential modification of the magnitude of the effect between practice outcome and examinationscores in relationship with the number of months in practice is not included. In instances in which there was a significant interaction between examination score and months inpractice (ie, mammography screening rate), the estimates for each category of years in practice provide a more appropriate estimate of the effects. Each ␤ was estimated by amultivariate regression model within a generalized estimating equation framework, in which physician was the unit of analysis and annual assessment of outcome rates/values,were represented as repeated measurements for each physician. Observations were weighted by the logarithm of each physician’s annual practice size. The estimate of theexamination score, practice outcome relationship was adjusted for differences in annual practice case-mix including age and sex structure, socioeconomic status, geographicaccess to health care, comorbidity, and propensity to use health care services based on data for individual practice patients in the year prior to outcome assessment. The onlyexception was for contraindicated prescribing in which practice size was used to weight estimate regression coefficients and medical school was included, but practice case-mixcovariates were not included as these attributes of the practice population would rarely justify contraindicated prescribing in the elderly. When medical school was excluded fromthe regression models, the magnitude of the association between examination scores and practice outcomes increased because some medical schools had systematically lowerscores than others. If the analysis were based on usual practice, in which pass-fail decisions are made irrespective of medical school, the overall predictive relationship betweenexamination scores and outcomes would have been higher: mammography screening and overall score (␤, 19.3; 95% CI, 13.1-25.4), consultation rate and overall score (␤, 3.70;95% CI, 1.4-5.9), symptom relief prescribing and management score (␤, −7.30; 95% CI, −13.4 to −1.2), disease-specific minus symptom-relief prescribing rate and diagnosisscore (␤, 4.07; 95% CI, 1.2-7.0), and contraindicated prescribing and management score (relative risk, 0.89; 95% CI, 0.8-1.0).
†To facilitate interpretation of changes in the magnitude of the association between examination scores and practice outcomes over the first 4 to 7 years of practice, examination score outcome relationships were estimated for 3 time intervals of practice based on a categorization, for each physician, of the cumulative months in practice from the practiceentry month. The interaction effects presented by intervals of years in practice were produced by a separate analysis to facilitate easier interpretation, in which the interactionsbetween examination score and 2 dummy variables, representing practice years 3 to 4 and 5 to 7 relative to years 1 to 2 were estimated.
‡To test the hypothesis that the relationship between certification examination scores and practice outcomes would be attenuated with increasing time in practice, we tested the interaction between examination score and cumulative months in practice. Cumulative practice months, treated as a time-dependent covariate, were determined by countingeach month that the physician billed the Que´bec health insurance agency (Re´gie de I’assurance maladie due Que´bec; RAMQ) for fee-for-service or salaried care for Que´becmedical care beneficiaries. P values are reported for each of the interaction terms (examination score multiplied by cumulative months in practice) that were estimated for eachcombination of outcome and examination score.
§Included phenylbutazone, dipyridamole, reserpine, disopyramide, clofibrate, methylphenidate, chlordiazepoxide, diazepam, clorazepate, flurazepam, clonazepam, clobazam, primi- done, fluoxetine, phenelzine, tranylcypromine, moclobemide, amitriptyline, doxepin, imipramine, trimipramine, clomipramine, amoxapine, maprotiline, cyclobenzaprine, metho-carbamol, pentazocine, meperidine, triazolam, and theophylline. Data expressed as relative risk of contraindicated prescribing per 1 SD increase in score.
3024 JAMA, December 18, 2002—Vol 288, No. 23 (Reprinted)
2002 American Medical Association. All rights reserved.
EXAMINATION SCORES AND PRACTICE PERFORMANCE traindicated prescriptions for elderly pa- cians tend to report higher referral rates in clinical areas in which they felt more Table 4. Association Between Medical Council of Canada Licensing Examination Scores and Practice Performance in the First 4 to 7 Years of
Practice
Change in Outcome per SD Increase in Score
All Practice Years
Practice Year †
Mean (SD)
Rate per 1000
Type of Certification
P
P Value for
Patients
Examination Score
␤ (95% CI)*
Interaction‡
*The regression coefficient ␤ represents the estimated change in the rate or value of the practice outcome per SD increase in score in the first 4 to 7 years in practice with a 95% confidence interval (CI). In these overall models, the interaction term to test the potential modification of the magnitude of the effect between practice outcome and examinationscores in relationship with the number of months in practice is not included. In instances in which there was a significant interaction between examination score and months inpractice (ie, mammography screening rate), the estimates for each category of years in practice provide a more appropriate estimate of the effects. Each ␤ was estimated by amultivariate regression model within a generalized estimating equation framework, in which physician was the unit of analysis and annual assessment of outcome rates/values,were represented as repeated measurements for each physician. Observations were weighted by the logarithm of each physician’s annual practice size. The estimate of theexamination score, practice outcome relationship was adjusted for differences in annual practice case-mix including age and sex structure, socioeconomic status, geographicaccess to health care, comorbidity, and propensity to use health care services based on data for individual practice patients in the year prior to outcome assessment. The onlyexception was for contraindicated prescribing in which practice size was used to weight estimated regression coefficients and medical school was included, but practice case-mix covariates were not included as these attributes of the practice population would rarely justify contraindicated prescribing in the elderly. When medical school was excludedfrom the regression models, the magnitude of the association between examination scores and practice outcomes increased because some medical schools had systematicallylower scores than others. If the analysis were based on usual practice, in which pass-fail decisions are made irrespective of medical school, the overall predictive relationshipbetween examination scores and outcomes would have been higher: mammography screening and overall score (␤, 18.3; 95% CI, 10.3-26.3), consultation rate and overall score(␤, 5.13; 95% CI, 2.3-7.9), symptom relief prescribing and drug knowledge score (␤, −8.36; 95% CI, −15.8 to −0.9), disease-specific minus symptom-relief prescribing rate anddiagnosis score (␤, 3.39; 95% CI, −0.4 to 6.8), and contraindicated prescribing and drug knowledge score (relative risk, 0.85; 95% CI, 0.7-0.9).
†To facilitate interpretation of changes in the magnitude of the association between examination scores and practice outcomes over the first 4 to 7 years of practice, examination score outcome relationships were estimated for 3 time intervals of practice based on a categorization, for each physician, of the cumulative months in practice from the practiceentry month. The interaction effects presented by intervals of years in practice were produced by a separate analysis to facilitate easier interpretation, in which the interactionsbetween examination score and 2 dummy variables, representing practice years 3 to 4 and 5 to 7 relative to years 1 to 2 were estimated.
‡To test the hypothesis that the relationship between certification examination scores and practice outcomes would be attenuated with increasing time in practice, we tested the interaction between examination score and cumulative months in practice. Cumulative practice months, treated as a time-dependent covariate, were determined by countingeach month that the physician billed the Que´bec health insurance agency (Re´gie de I’assurance maladie due Que´bec; RAMQ) for fee-for-service or salaried care for Que´becmedical care beneficiaries. P values are reported for each of the interaction terms (examination score multiplied by cumulative months in practice) that were estimated for eachcombination of outcome and examination score.
§Included phenylbutazone, dipyridamole, reserpine, disopyramide, clofibrate, methylphenidate, chlordiazepoxide, diazepam, clorazepate, flurazepam, clonazepam, clobazam, primi- done, fluoxetine, phenelzine, tranylcypromine, moclobemide, amitriptyline, doxepin, imipramine, trimipramine, clomipramine, amoxapine, maprotiline, cyclobenzaprine, metho-carbamol, pentazocine, meperidine, triazolam, and theophylline. Data expressed as relative risk of contraindicated prescribing per 1 SD increase in score.
2002 American Medical Association. All rights reserved.
(Reprinted) JAMA, December 18, 2002—Vol 288, No. 23 3025
EXAMINATION SCORES AND PRACTICE PERFORMANCE bec for his assistance in retrieving the health service 21. Wilchesky M, Tamblyn RM. Predictive values of
data and responding to our many questions; and M.
diagnoses in medical services claims. J Clin Pharma- Jimmy Fragos, BSc, for his expert assistance in data- base management and variable creation.
22. Wilkins K. Use of postal codes and addresses in the
analysis of health data. Health Rep. 1993;5:157-177.
23. Salive ME, Guralnik JM, Brock D. Preventive ser-
REFERENCES
vices for breast and cervical cancer in US office- 1. Irvine D. The performance of doctors, I: profes-
based practices. Prev Med. 1996;25:561-568.
sionalism and self-regulation in a changing world. BMJ. 24. Bindman AB, Grumbach K, Osmond D, et al. Pre-
ventable hospitalizations and access to health care.
2. Tamblyn RM. Is the public being protected? pre-
vention of suboptimal medical practice through train- 25. Shea S, Misra D, Ehrlich MH, Field L, Francis CK.
ing programs and credentialing examinations. Eval Predisposing factors for severe, uncontrolled hyper- predictive of future practice may be use- Health Prof. 1994;17:198-221.
tension in an inner-city minority population. N Engl 3. Ayanian JZ, Hauptman PJ, Guadagnoli E, et al.
Knowledge and practices of generalist and specialist 26. Wasson JH, Sauvigne AE, Mogielnicki RP, et al.
physicians regarding drug therapy for acute myocar- Continuity of outpatient medical care in elderly men: dial infarction. N Engl J Med. 1994;331:1136-1142.
a randomized trial. JAMA. 1984;252:2413-2417.
4. Edep ME, Shah NB, Tateo IM, Massie BM. Differ-
27. Lambrew JM, DeFriese GH, Carey TS, Ricketts TC,
ences between primary care physicians and cardiolo- Biddle AK. The effects of having a regular doctor on ac- gists in management of congestive heart failure. J Am cess to primary care. Med Care. 1996;34:138-154.
Coll Cardiol. 1997;30:518-526.
28. Hulscher ME, van Drenth BB, Mokkink HG, et al.
5. Norcini J, Lipner RS, Benson JA Jr, Webster GD. An
Barriers to preventive care in general practice. Br J Gen analysis of the knowledge base of practicing inter- nists as measured by the 1980 recertification exami- 29. Safran DG, Montgomery JE, Chang H, Murphy
nation. Ann Intern Med. 1985;102:385-389.
J, Rogers H. Switching doctors: predictors of volun- 6. Jollis JG, DeLong ER, Peterson ED, et al. Outcome
tary disenrollment for primary care physician’s prac- of acute myocardial infarction according to the spe- tice. J Fam Pract. 2001;50:130-136.
cialty of the admitting physician. N Engl J Med. 1996; 30. McGavock H, Wilson-Davis K, Niblock RWF. Un-
suspected patterns of drug utilization revealed by in- 7. Weiner JP, Parente ST, Garnick DW, Fowles J,
terrogation of a regional general practitioner prescrib- Lawthers AG, Palmer HR. Variation in office-based qual- ing database. Pharmacoepidemiol Drug Saf. 1992;1: ity: a claims-based profile of care provided to Medicare patients with diabetes. JAMA. 1995;273:1503-1508.
31. Leape LL, Bates DW, Cullen DJ, et al. Systems analy-
8. Ramsey PG, Carline JD, Inui TS, Larson EB, Lo-
sis of adverse drug events. JAMA. 1995;274:35-43.
Gerfo JP, Wenrich MD. Predictive validity of certifi- 32. Franks P, Clancy CM. Gatekeeping revisited: pro-
cation by the American Board of Internal Medicine.
tecting patients from overtreatment. N Engl J Med. Ann Intern Med. 1989;110:719-726.
9. McAuley RG, Paul WM, Morrison GH, Beckett RF,
33. Reynolds GA, Chitnis JG, Roland MO. General prac-
Goldsmith CH. Five-year results of the peer assess- titioner outpatient referrals: do good doctors refer more ment program of the College of Physicians and Sur- patients to hospital? BMJ. 1991;302:1250-1252.
geons of Ontario. CMAJ. 1990;143:1193-1199.
34. Calman NS, Hyman RB, Licht W. Variability in con-
10. Murata PJ, Li J. Relationship between Pap smear
sultation rates and practitioner level of diagnostic cer- performance and physician ordering a mammogram.
tainty. J Fam Pract. 1992;35:31-38.
J Fam Pract. 1992;35:644-648.
35. Beers MH, Ouslander JG, Rollingher I, Reuben DB,
11. Nicolucci A, Cavaliere D, Scorpiglione N, et al. A
Brooks J, Beck JC. Explicit criteria for determining in- Author Contributions: Study concept and design:
comprehensive assessment of the avoidability of long- appropriate medication use in nursing home resi- term complications of diabetes. Diabetes Care. 1996; dents. Arch Intern Med. 1991;151:1825-1832.
A c q u i s i t i o n o f d a t a : T a m b l y n , D a u p h i n e e , 36. McLeod PJ, Huang AR, Tamblyn RM, Gayton DC.
12. Legorreta AP, Christian-Herman J, O’Connor RD,
Defining inappropriate practices in prescribing for el- Analysis and interpretation of data: Tamblyn, et al. Compliance with national asthma management derly people. CMAJ. 1997;156:385-391.
Abrahamowicz, Hanley, Norcini, Girard.
guidelines and specialty care. Arch Intern Med. 1998; 37. Starfield B, Weiner J, Mumford L, Steinwachs D.
Drafting of the manuscript: Tamblyn, Abrahamowicz, Ambulatory care groups: a categorization of diag- 13. Wilcox SM, Himmelstein DU, Woolhandler S. In-
noses for research and management. Health Serv Res. Critical revision of the manuscript for important in- appropriate drug prescribing for the community- tellectual content: Tamblyn, Abrahamowicz, dwelling elderly. JAMA. 1994;272:292-296.
38. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical
Dauphinee, Hanley, Norcini, Girard, Grand’Maison, 14. Tamblyn R, Abrahamowicz M, Brailovsky C, et al.
comorbidity index for use with ICD-9-CM adminstrative Association between licensing examination scores and databases. J Clin Epidemiol. 1992;45:613-619.
Statistical expertise: Tamblyn, Abrahamowicz, Hanley, resource use and quality of care in primary care prac- 39. Zeger SL, Liang KY. Longitudinal data analysis for
tice. JAMA. 1998;280:989-996.
discrete and continous outcomes. Biometrics. 1986; Obtained funding: Tamblyn, Abrahamowicz, 15. Statistiques Annuelles. Que´bec, Que´bec: Regie
de l’assurance-maladie du Que´bec; 1995.
40. Public trust and accountability for clinical perfor-
Administrative, technical, or material support: 16. Grand’Maison P, Lescop J, Rainsberry P, Brai-
mance: lessons from the national press reportage of the lovsky CA. Large-scale use of an objective, struc- Bristol hearing. J Eval Clin Pract. 1999;5:335-342.
tured clinical examination for licensing family physi- 41. Southgate L, Hays RB, Norcini J, et al. Setting per-
Funding/Support: Research support was provided by
cians. CMAJ. 1992;146:1735-1740.
formance standards for medical practice: a theoreti- the Canadian Institutes of Health Research, and the 17. Information Pamphlet on the Qualifying Exami-
cal framework. Med Educ. 2001;35:474-481.
Fonds de Recherche en Sante´ du Que´bec. In addi- nation. Ottawa, Ontario: Medical Council of Canada; 42. Norcini JJ Jr. Standards and reliability in evalua-
tion, Dr Tamblyn and Dr Abrahamowicz are medical tion. Acad Med. 1999;74:1088-1090.
scientists at the Canadian Institutes of Health Re- 18. Livingston SA, Zieky M. Passing Scores. Prince-
43. Schaffner W, Ray WA, Federspiel CF, Miller WO.
ton, NJ: Educational Testing Service; 1989.
Improving antibiotic prescribing in office practice.
Acknowledgment: We thank Tim Wood, PhD, and
19. Tamblyn RM, Lavoie G, Petrella L, Monette J. Use
Andre´ Phillipe Boulais, MSC, of the Medical Council of prescription claims databases in pharmacoepidemio- 44. Leape LL, Brennan TA, Laird N, et al. Nature of
of Canada for their expert assistance in data re- logical research. J Clin Epidemiol. 1995;48:999-1009.
adverse events in hospitalized patients. N Engl J Med. trieval, linkage, and score reclassification; Joelle Le- 20. Levy AR, Tamblyn RFD, McLeod P, Hanley J. Cod-
scop, MD, MPH, of the Que´bec College of Physi- ing accuracy of hospital discharge data for elderly sur- 45. Veloski JJ, Hojat M, Gonnella JS. The validity of
cians who made this study possible; M. Jacques Barry, vivors of myocardial infarction. Can J Cardiol. 1999; Part III of the National Board Examination. Proc Annu MBA, of the Re´gie de l’assurance maladie du Que´- Conf Res Med Educ. 1987;26:54-59.
3026 JAMA, December 18, 2002—Vol 288, No. 23 (Reprinted)
2002 American Medical Association. All rights reserved.

Source: https://wiki.usask.ca/download/attachments/3997711/LMCC+Pt+I_+QLEX+predicts+Tamblyn+et+al+JAMA+2002.pdf