CEHP Mon

Continuing Education in

the Health Professions

Proceedings of a Conference

Chaired by Suzanne W. Fletcher, M.D., M.Sc.

Edited by Mary Hager, Sue Russell,

and Suzanne W. Fletcher

, M.D., M.Sc.

Continui ng Educ ation in the Health Profess ions

Fl etcher

The Josiah Macy, Jr. Foundation is a private philanthropy dedicated to improving

the health of individuals and the public. Since its establishment in 1930, the

Foundation has focused its support principally on projects and conferences

designed to enhance the education of health professionals, especially physicians.

ISBN 0-914362-49-6

This monograph is in the public domain and may be reproduced or copied without

permission. Citation, however, is appreciated: Hager M, Russell S, Fletcher SW, editors.

Continuing Education in the Health Professions: Improving Healthcare Through

Lifelong Learning, Proceedings of a Conference Sponsored by the Josiah Macy, Jr.

Foundation; 2007 Nov 28 - Dec 1; Bermuda. New York: Josiah Macy, Jr. Foundation;

2008. Accessible at www.josiahmacyfoundation.org

A Conference Sponsored by the

Josiah Macy, Jr. Foundation

Chaired by Suzanne W. Fletcher, M.D., M.Sc.

Bermuda

November 2007

Edited by Mary Hager, Sue Russell,

and Suzanne W. Fletcher, M.D., M.Sc.

Published by the

Josiah Macy, Jr. Foundation

44 East 64th Street

New York, NY 10065

www.josiahmacyfoundation.org

2008

Continuing Education in

the Health Professions:

Improving Healthcare

Through Lifelong Learning

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 1

Preface …………………………………………………………………… 4

Introduction ……………………………………………………………… 7

Chairman’s Summary of the Conference ……………………………… 13

Conference Participants ………………………………………………… 24

Conference Images ……………………………………………………… 26

I. APPROACHES TO KNOWLEDGE DEVELOPMENT—

WHAT WORKS AND WHAT DOES NOT ………………………… 29

How Physicians Learn and How to Design Learning

Experiences for Them …………………………

—Donald E. Moore, Jr., Ph.D. …………………………………… 30

Transforming Continuing Medical Education Through

Maintainence of Certiﬁcation ………………

—F. Daniel Duffy, M.D. ………………………………………… 63

Discussion Highlights ………………………………………… 77

Internet Continuing Education ………………

—Denise Basow, M.D.…………………………………………… 82

Informatics Skills Needed! ………………………

—Donald A.B. Lindberg, M.D. ………………………………… 92

Remarks ……………………………………………

—David C. Slawson, M.D. ……………………………………… 94

Discussion Highlights ………………………………………… 98

II. FINANCING CONTINUING EDUCATION: ………………………

WHO, HOW, A ND WHY……………………………………………… 103

Financial Support for Continuing Education in the

Health Professions ……………………………

—Robert Steinbrook, M.D. ……………………………………… 104

Remarks ……………………………………………

—Jordan J. Cohen, M.D. ………………………………………… 127

Discussion Highlights ………………………………………… 132

III. DESIGNING SYSTEMS FOR LIFELONG LEARNING ………………

TO IMPROVE HEALTH …………………………………………… 141

Continuing Health Professional Education Delivery

in the United States ……………………………

—David A. Davis, M.D., and Trina Loofbourrow, B.A. ……… 142

Remarks ……………………………………………

—Pamela Mitchell, Ph.D., M.S., B.S. …………………………… 177

Discussion Highlights ………………………………………… 179

Table of Contents

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 2

The Macy Conference format provides ample time for discussion and, after

reading commissioned background papers and hearing remarks prepared

by the authors and several participants, the 36 participants seated around

the table offered countless informed observations, comments, and opinions.

The Discussion Highlights sections at the end of each session group these

comments according to themes that emerged, though many comments might

have been placed in several categories. The papers, remarks, and comments

have been lightly edited for brevity and clarity. On the ﬁnal day, partici-

pants developed the set of conclusions and recommendations found in the

Executive Summary and, again, beginning on page 219.

Learning to Work Together to Improve the

Quality of Healthcare …………………………

—Maryjoan D. Ladden, Ph.D., R.N. …………………………… 182

Remarks ……………………………………………

—Carol Havens, M.D. …………………………………………… 189

Discussion Highlights ………………………………………… 191

IV. REACTION FROM THE TRENCHES ……………………………… 195

Remarks ……………………………………………

—Grant S. Fletcher, M.D., M.P.H. ……………………………… 196

—James A. Clever, M.D. ………………………………………… 199

—Susan W. Wesmiller, M.S., R.N. ……………………………… 200

—Regina Benjamin, M.D., M.B.A. ……………………………… 203

Discussion Highlights ………………………………………… 206

V. MOVING TOWARDS THE FUTURE ………………………………… 209

Continuing Medical Education: Some Important ………

Odds and Ends …………………………………

—David C. Leach, M.D. ………………………………………… 210

Discussion Highlights ………………………………………… 216

Conference Conclusions and Recommendations …………………… 219

Additional References Suggested by Conference Participants ……… 225

Biographical Sketches and Statements of Potential Conflicts of ……

Interest of Conference Participants ……………………………… 230

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 3

Preface

June E. Osborn, M.D.

During my tenure as president of the Macy Foundation, we funded

efforts to improve undergraduate health professional and even

graduate medical education. However, it seemed difﬁcult, if not

impossible, to ﬁnd a useful entry point on continuing education,

even though it was a massive enterprise that was mandated very

broadly. What is more, I repeatedly bumped up against the fact that

much of the funding for continuing medical education (CME) was

being provided by pharmaceutical company sponsors. Decades

earlier that was true, of course, but not to anywhere near the extent

that had come to be the case. In fact, as I inquired of colleagues

closer to the issue than I was, I got estimates that 60-90 percent of

all CME was supported by industry. Indeed, so pervasive was that

mechanism of support that almost no alternative sources remained.

While there was nothing intrinsically wrong with that state of affairs,

it tended to be festooned by associated phenomena having little to

do with education per se, such as free lunches, gifting of other sorts,

or even more attractive subsidies such as opportunities to qualify for

CME credits by enrolling in courses offered in desirable spots.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 4

Clearly there was room for considerable concern about the planning

of offerings for life-long learning, not to mention possible distortion

of objectivity of CME content design and delivery.

I became increasingly troubled as, over time, requirements by state

licensing bodies to accumulate arbitrary hours of CME credit became

virtually universal, and yet they could be satisﬁed rather randomly,

or at least without any consistency or quality control as to format

and content. The well-established PowerPoint presentation format

had become nearly universal, and there was little evident impetus

or support for other forms of learning, especially at points of care.

All those worrisome trends in CME content, quality, and relevance

to practice were evolving at a time when bench-to-bedside transla-

tional research was being promoted and when biomedical advances

made the need for pertinent life-long learning experiences ever

more urgent and important.

By happy chance, I got into a discussion of these concerns with

Dr. Suzanne Fletcher early in 2006, at a time when I was trying to

formulate plans for the ﬁnal Macy Conference to be held prior to my

retirement at the end of 2007. When I asked if she — with her long

leadership of a large Harvard CME course — thought that would be

a timely and useful topic for a conference, she endorsed the idea

vigorously. Better yet when I began to plan, the ﬁrst step — choosing

a chairperson — fell neatly into place. Suzanne’s enthusiasm was

matched by her energy, and she agreed to chair the planning,

convening, and subsequent monograph preparation.

This volume, then, is the product of the resultant Macy Conference,

held in Southhampton, Bermuda from November 28 to December 1,

Osborn

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 5

2007. In preparation for the conference, an excellent and hard-working

planning committee delineated areas of need for newly commissioned

papers, set an agenda for the conference over two-and-one-half

days, and worked diligently to be sure that the professional and

experiential range of the participants invited to contribute to the

deliberations would provide as many pertinent voices as possible

to assure a rich discussion. Given that all the health professions are

facing similar or analogous problems to those of medicine, efforts

were made to include nursing, in particular, and other professions’

experience where possible. The resulting conference was remark-

ably stimulating and led to a broad consensus, expressed in the

conclusions and recommendations presented in this monograph.

As Dr. Fletcher says eloquently in her Introduction, the resultant

exploration of the facts, deﬁciencies, needs, and challenges for

continuing education of health professionals is certainly timely. I

hope it serves to move the health professions forward in their pur-

suit of optimal ways to provide and require continuing education

throughout the professional lifetimes of their members.

President Emerita, Josiah Macy, Jr. Foundation

Preface

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/12/08 11:07 AM Page 6

Introduction

Suzanne Fletcher, M.D., M.Sc.

Harvard Medical School

Chair

In November, 1908 the Board of Trustees of the Carnegie Foundation

authorized what many consider the seminal study of medical

education in this country. Two years later Abraham Flexner published

his report on medical education in the United States and Canada.

The report was so inﬂuential that most physicians in the United

States continue to be educated according to Flexner’s precepts.

One hundred years later, the Josiah Macy, Jr. Foundation convened

a Conference on Continuing Education in the Health Professions.

Our mandate was to examine continuing education in multiple

health professions, not undergraduate medical education. So why

bring up Flexner, who wrote his report many years even before the

birth of the Macy Foundation? When I perused Flexner’s report in

preparation for our gathering, I was struck that, even now, many

aspects of that nearly 100-year-old document remain remarkably

relevant to our deliberations at this conference.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 7

First, why is education in the health professions so important anyway?

The Carnegie Foundation undertook their study with two major

interests in mind, or as President Pritchett put it, “ﬁrst, the youths

who are to study medicine and to become the future practitioners,

and secondly, the general public, which is to live and die under

their ministrations.”

The planning committee for this conference

came to a remarkably similar conclusion. At our very ﬁrst meeting,

Marc Nivet asked about the view from the trenches, and this question

led to an important session in our conference. Later, when we were

deep into thinking about the complicated educational approaches,

ﬁnancing, and structures involved in continuing education, June

Osborn, who has stressed the public throughout her tenure as

President at Macy, and Jordan Cohen brought it all into focus by

reminding us that what makes these deliberations truly important

is the need to improve the quality of healthcare. This broader social

perspective separates education in the health professions from

many other worthy educational endeavors.

The second reason why Flexner’s report remains relevant is that he

addressed the type and mode of education. He visited every one

of the 150 medical schools in existence at that time to observe

precisely what subjects were taught and how they were taught. He

concluded that too often laboratory science and bedside teaching

were neglected in favor of didactic lectures. He summed up his

argument for the need for hands-on learning by quoting an earlier

article by Cabot and Locke:

Learning medicine is not fundamentally different from learning

anything else. If one had one hundred hours in which to learn

to ride a horse or to speak in public, one might proﬁtably spend

perhaps an hour (in divided doses) in being told how to do it,

four hours in watching a teacher do it, and the remaining ninety-

ﬁve hours in practice, at ﬁrst with close supervision, later under

general oversight.

Today, how best to conduct continuing education so as to positively

affect the health of patients and the public remains a challenge.

That we have a ways to go is clear from many studies demonstrating

large differences — a “chasm” according to the IOM report

—

between what should be done and what is done in practice. Don

Introduction

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 8

Fletcher

Moore presented a wonderful summary of what we know today

about how clinicians learn. He pointed out that most formal continuing

education programs continue to be made up of didactic lectures

and very few — Don estimates 5 percent — on assessing competence

and performance. What Flexner (and Cabot and Locke) said 100

years ago still applies.

Medical specialty boards are beginning to address this problem

with the requirement of maintenance of certification programs.

As Dan Duffy described in his presentation, these programs require

clinicians to review the care they actually deliver in their own prac-

tices, compare the results with standards of excellence, and create

a plan for improvement. This approach goes back to the 95-hour

rule Flexner advocated so long ago.

Perhaps the most important issue in continuing education today is

its ﬁnancing. Robert Steinbrook pointed out that commercial support

for accredited continuing medical education in the United States

has quadrupled in less than a decade, and last year it reached a total

of $1.5 billion—accounting for about 60 percent of the income for

all of the accredited continuing medical education programs in the

United States. No wonder we see a rising concern that commercial

interests may be distorting the education of practicing clinicians. The

U.S. Senate Finance Committee summed it up this way: “It seems

unlikely that [a] sophisticated industry would spend such large sums

on an enterprise but for the expectation that the expenditures will

be recouped by increased sales.”

The Senate Committee called for

better oversight to ensure that continuing education programs are

independent of drug company interests.

Flexner wrote his report 100 years ago, when very few effective

drugs existed, when Osler advocated little more than morphine,

nitroglycerin, iron, and quinine. How could Flexner have any rele-

vance to this vexing modern problem? I think he does. Flexner often

used the word “commercial” when describing the problems of the

worst medical schools he visited. He pointed out that laboratory

and patient-based education was expensive while didactic lectures

were cheap, and so the schools that survived and proﬁted only from

fees tended to emphasize the latter. He challenged the fundamental

assumption that medical education should be a business. He advo-

cated the need for standards that, if adopted, clearly would adversely

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 9

affect the ﬁnancial interests of some.

Today, it is increasingly clear that commercial ﬁnancing has inﬁltrated

the very fabric of continuing education. Last year JAMA published a

paper, coauthored by David Blumenthal and Jordan Cohen, calling

for academic medical centers to take the lead in eliminating conﬂicts

of interest between physicians and industry. The authors proposed

that manufacturers should not be permitted to provide support for

any ACCME-accredited continuing education program and, further,

that faculty should not serve on speakers’ bureaus sponsored by

drug companies.

Results from a recent survey of medical school department chairs,

published in JAMA, show just how difﬁcult it will be to move this

call from proposal to policy; 60 percent of the department chairs had

some form of ﬁnancial relationship with industry, as did two thirds

of their departments

Most chairs perceived no ill effects from these

relationships, and most viewed them as helpful to the educational

process. Our conference dealt squarely with this conundrum.

With regard to organizations that teach, Flexner recommended that

medical education be linked closely with universities. He was con-

vinced that such linkage would provide higher standards, better

teachers, and more resources for students. He also recommended that

teaching hospitals have close links to medical schools. Fundamentally,

he was suggesting that organizational aspects of education should

be set up to achieve the best possible education.

What about the situation today? Dave Davis has provided us with an

overview of the organizations involved in and accrediting continuing

education in medicine, pharmacy, and nursing; the landscape is

crowded, with many different types of organizations involved— far

more than for undergraduate and graduate education. Is this the best

way? We devoted a session of our conference to this topic.

One area barely mentioned by Flexner is interprofessional collabo-

ration, but today high-quality healthcare demands communication,

collaboration, and teamwork among heath professionals. Several IOM

reports on quality of healthcare in this country have stressed team-

work and systems changes. Maryjoan Ladden argued that continuing

education of various health professionals together promotes collabo-

Introduction

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 10

ration, decreases adverse health effects, and improves healthcare.

I found no comment about continuing education in Flexner’s report.

However, one of the giants of that age, William Osler, was acutely

aware of its importance. He thought physicians had an obligation to

keep learning about medicine from others, both through the literature

and through subjecting their practices to the scrutiny of colleagues.

He loved medical libraries and amassed a private library of approxi-

mately 8000 books. He summed up his feelings this way: “It is

astonishing with how little reading a doctor can practice medicine,

but it is not astonishing how badly he may do it

”

One hundred

years later, we still have a ways to go.

If Flexner had attended this conference, perhaps the most fascinat-

ing part for him would have been the discussion about the Internet

and its effect on the ability of doctors and nurses to keep up with

new information. Denise Basow and Don Lindberg both offered

presentations describing the ways in which this wonderful new

technology provides, really for the ﬁrst time, just-in-time learning

for all of us in the health professions, and, as Don pointed out, for

our patients.

Flexner made major recommendations for change. This conference

also has led to major recommendations, although we spent two and

a half days together as opposed to Flexner’s two years’ effort. Also,

David Leach reminded us about the efforts of other groups. What

can one more report from a Macy Conference contribute?

I think there are at least two possibilities. First, our perspective cuts

across health professions. Second, the participants in this conference

were invited as leaders in the medical, nursing, and education

professions, not as representatives of organizations involved in

continuing education. As such, we had the opportunity to consider

continuing education of the healthcare professions as it contributes

to our professions and to society at large. By adopting this approach,

our goal was that the description of a profession written many years

ago in a legal opinion, and echoing Justice Brandeis’ famous quote,

continues to apply today:

A profession is not a business. It is distinguished by the require-

ments of extensive formal training and learning, admission to

Fletcher

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 11

Introduction

practice by qualifying licensure, a code of ethics imposing stan-

dards qualitatively and extensively beyond those that prevail or

are tolerated in the marketplace, a system for discipline of its

members for violation of the code of ethics, a duty to subordi-

nate ﬁnancial reward to social responsibility, and notably, an

obligation on its members, even in non-professional matters, to

conduct themselves as members of a learned, disciplined, and

honorable occupation.

REFERENCES

1. Flexner A. Medical Education in the United States and Canada. A Report to the

Carnegie Foundation for the Advancement of Teaching. New York: The Carnegie

Foundation for the Advancement of Teaching; 1910.

2. Pritchett HS. Introduction. In Flexner A. Medical Education in the United States and

Canada. A Report to the Carnegie Foundation for the Advancement of Teaching. New

York: The Carnegie Foundation for the Advancement of Teaching; 1910. p. vi–xvii.

3. Cabot RC, Locke EA. The organization of a department of clinical medicine. Boston

Med Surg J. 1905;153:461–465.

4. Institute of Medicine Committee on Quality of Healthcare in America. Crossing the

Quality Chasm: A New Health System for the 21st Century. Washington DC: National

Academy Press; 2001.

5. Committee Staff Report to the Chairman and Ranking Member. Use of Educational

Grants by Pharmaceutical Manufacturers. Washington DC: United State Senate

Committee on Finance; April 2007.

6. Brennan TA, Rothman DJ, Blank L, et al. Health industry practices that create con-

ﬂicts of interest: a policy proposal for academic medical centers. JAMA.

2006;295:429–433.

7. Campbell EG, Weissman JS, Ehringhaus S, et al. Institutional academic-industry rela-

tionships. JAMA. 2007;298:1779–1786.

8. Osler W. Books and men. In: Osler W. Aequanimatas and Other Addresses.

Philadelphia: P. Blakeson’s Sons and Co; 1905. p. 22.

9. New York Court of Appeals Statement on Matter of Freeman, 311 N. E. 2d 480(1974).

Accessible at www.wsba.org/atj/committees/jurisprudence/jurisethics.htm (Access

date: February 15, 2008).

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 12

Chairman’s Summary

of the Conference

Continuing education (CE) of health professionals is essential to

the health of all Americans. With accelerating advances in health

information and technology, physicians, nurses, and other health

professionals must maintain and improve their knowledge and skills

throughout their careers in order to provide safe, effective, and high

quality healthcare for their patients.

Yet continuing education in the health professions is in disarray.

Over the past decade, both professional and lay reports have

identiﬁed multiple problems. CE, as currently practiced, does not

focus adequately on improving clinician performance and patient

health. There is too much emphasis on lectures and too little empha-

sis on helping health professionals enhance their competence and

performance in their daily practice. With Internet technology, health

professionals can ﬁnd answers to clinical questions even as they

care for patients, but CE does not encourage its use or emphasize

its importance. And, while studies show that inter-professional

collaboration, teamwork, and improved systems are key to high

quality care, accrediting organizations have not found ways to

promote teamwork or align CE with efforts to improve the quality

of health systems.

Another signiﬁcant problem is the growing link between continuing

education and commercial interests. In 2006, the total income for

accredited CE activities in medicine was $2.4 billion. Commercial

support from pharmaceutical and medical device manufacturers

accounted for more than 60 percent, about $1.45 billion, of the total.

Over the past two years, the Senate Finance Committee has investi-

gated pharmaceutical company support for continuing education

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 13

in medicine. Despite efforts to control improper inﬂuences, the

committee concluded that the organizations providing continuing

education could still accommodate commercial interests of sponsors

and sponsors could still target their funding for educational

programs likely to support sales of their products.

To address concerns about CE, the Josiah Macy, Jr. Foundation

convened a conference on “Continuing Education in the Health

Professions.” Suzanne W. Fletcher, M.D., M.Sc., Professor of

Ambulatory Care and Prevention, Emeritus, at Harvard Medical

School, served as chair. The two-and-one-half-day conference,

which was held in Bermuda in November of 2007, included 36

leaders in medicine, nursing, and education. Commissioned back-

ground papers covered a range of CE-related topics, including a

review of how physicians and other health professionals learn, the

role of information technology, ﬁnancing, and certiﬁcation.

Although much of the conference discussion was relevant to the

continuing education of all health professionals, participants focused

on accredited CE for medicine and nursing. They acknowledged

that much professional learning takes place informally and outside

accredited formats.

Conference themes were inter-related, for the methods used for

continuing education are inﬂuenced both by the means of ﬁnancial

support and by mechanisms for accreditation. Unfortunately,

participants found, current systems of CE do not meet the needs

of health professionals as well as they should:

— Too much CE relies on a lecture format and counts hours of

learning rather than improved knowledge, competence, and

performance.

— Too little attention is given to helping individual clinicians examine

and improve their own practices.

— Insufﬁcient emphasis is placed on individual learning driven by

the need to answer the questions that arise during patient care.

— CE does not promote inter-professional collaboration, feedback

from colleagues and patients, teamwork, or efforts to improve

systems of care, activities that are key to improved performance

Chairman’s Summary

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 14

by health professionals.

— CE does not make adequate or creative use of Internet technology,

which can help clinicians examine their own practice patterns,

bring medical information to them during patient care, and aid

them in learning new skills.

— There is too little high-quality scientiﬁc study of CE.

Participants warned that the health professions, especially medicine,

threaten the ethical underpinnings of professionalism by participat-

ing in a multi-billion dollar CE enterprise so heavily ﬁnanced by

commercial interests. This arrangement, which evolved over the

years, distorts continuing education. It places physicians and nurses

who teach CE activities in the untenable position of being paid,

directly or indirectly, by the manufacturers of healthcare products

about which they teach. At the same time, commercial support of

CE places learners in an obligatory position because they are often

given free meals and small gifts. Independent judgment of how

best to care for patients is compromised. Bias, either by appearance

or reality, has become woven into the very fabric of continuing

education. The professions, themselves, must right this wrong.

In a free-market system, commercial entities, such as drug and

device manufacturers, have a clear responsibility to shareholders to

gain market advantage and generate a proﬁt, while health profes-

sionals have a moral responsibility to provide safe, high quality

care for their patients, based on valid scientiﬁc ﬁndings. The two

responsibilities are fundamentally incompatible. Even if bias could

be avoided, the potential, and the perception, are ever-present.

Companies with billions of dollars at stake cannot be expected to

be neutral or objective when assessing the beneﬁts, harms, and

cost-effectiveness of their products, for they are in the legitimate

business of gaining market advantage and want clinicians to use

and prescribe their products.

Yet, an objective and neutral assessment of clinical management

options is precisely what is needed in continuing education.

Participants emphasized that, regardless of the ﬁnancial impact on

for-proﬁt companies, patient care must be based on scientiﬁc evidence

and commercial interests should not determine the topics or content

Fletcher

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 15

of CE. Because of these underlying ethical issues, participants

concluded that the commercial entities that manufacture and sell

healthcare products should not provide ﬁnancial support for the

continuing education of health professionals.

Participants acknowledged that many major advances in health-

care, especially in the development of new drugs and devices,

have come from careful collaboration between medical and

commercial investigators. Too, corporations have made valuable

donations to academic health centers to support professorships,

scholarships, programs, and buildings, all of which contribute to

the public good.

Despite recent changes in CE accreditation to reduce commercial

inﬂuence, the problem persists and organizations with little profes-

sional expertise in healthcare, and supported almost entirely by

commercial interests, provide accredited continuing education. At

the same time, accrediting groups require all organizations providing

CE to go through laborious, bureaucratic procedures to document

that no inappropriate inﬂuence has occurred.

Participants pinpointed another serious failure with current accredi-

tation mechanisms. At a time when inter-professional collaboration,

teamwork, and improvement of systems are key to high quality

healthcare, accrediting organizations for the various health profes-

sions still work in silos. Rather than promoting inter-professional

collaboration and education, regulations and procedures for

accreditation make inter-professional collaboration difﬁcult. And,

while systems of care have a major impact on the quality of health-

care delivered by clinicians, accrediting organizations have been

slow to align their CE activities with quality improvement efforts

by systems of care.

Participants identiﬁed a set of principles they believe should underlie

and guide continuing education of the health professions:

— Integrate continuing education into daily clinical practice.

— Base continuing education on the strongest available evidence

for practice.

Chairman’s Summary

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 16

— Minimize, to the greatest extent possible, both the reality and the

appearance of bias.

— Emphasize ﬂexibility and easy accessibility for clinicians.

— Stress innovation and evaluation of new educational methods.

— Address needs of clinicians across a wide spectrum, from

specialists in academic health centers to rural solo practitioners.

— Support inter-professional collaboration.

— Align continuing education efforts with quality improvement

initiatives at the level of health systems.

After two and a half days of discussion, participants agreed to the

following conclusions and recommendations:

CONCLUSIONS

Continuing Education and the Public

The quality of patient care is profoundly affected by the performance

of individual health professionals.

The fundamental purposes of continuing health professional

education (CE) are:

— To improve the quality of patient care by promoting improved

clinical knowledge, skills and attitudes, and by enhancing

practitioner performance.

— To assure the continued competency of clinicians and the

effectiveness and safety of patient care.

— To provide accountability to the public.

CE fulﬁlls a critically important, indeed essential, public purpose.

Given the accelerating pace of change in clinical information and

technology, CE has never been more important.

Fletcher

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 17

Responsibilities of Individual Professionals, Professional

Teams, and Health Systems

Maintaining professional competence is a core responsibility of

each health professional, regardless of discipline, specialty, or type

of practice.

The individual clinician has been the principal unit of accountability

for performance in the healthcare delivery system. Given that the

performance of health systems also profoundly affects patient care,

CE fails to take into account systems of care.

Effective patient care increasingly depends on well-functioning

teams of healthcare professionals. Therefore, CE must address the

special learning needs of collaborating teams.

Quality improvement efforts and CE activities overlap and ideally

are mutually reinforcing.

CE Methods

Traditional lecture-based CE has proven to be largely ineffective in

changing health professional performance and in improving patient

care. Lecture formats are employed excessively relative to their

demonstrated value.

Professional conferences play an important role in CE by promoting

socialization and collegiality among health professionals. Health

professionals have the responsibility to help one another practice

the best possible care. Meeting together provides opportunities for

cross-disciplinary and cross-generational learning and teaching.

Practice-based learning and improvement is a promising CE

approach for improving the quality of patient care. Maintenance

of certiﬁcation programs (in which clinicians review the care they

actually deliver in their own practices, compare the results with

standards of excellence, and create a plan for improvement) and

maintenance of licensure programs are moving CE in this direction.

Currently, most CE faculty are insufﬁciently prepared to teach

practice-based learning.

Information technology is essential for practice-based learning by:

Chairman’s Summary

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 18

— Providing access to information and answers to questions at the

time and place of clinical decision-making (point-of-care learning).

— Providing a database of clinician performance at the individual

and/or group practice level, which can be compared to best

practices and used to make plans for improvement.

— Providing automated reminder systems.

Interactive scenarios and simulations are promising approaches to

CE, particularly for skills development, whether the skill is a highly

technical procedure, history taking, or a physical examination

technique.

Insufﬁcient research is currently directed at improving and evaluating

CE. There is no national entity dedicated to advancing the science of

CE as there is for biomedical and clinical research.

Financing CE

The majority of ﬁnancial support for accredited CME, and increas-

ingly for CNE, derives directly or indirectly from commercial entities.

Pharmaceutical and medical device companies and healthcare pro-

fessionals have inherently conﬂicting interests in CE. Commercial

entities have a legitimate obligation to enhance shareholder value

by promoting sales of their products, whereas healthcare profession-

als have a moral obligation to improve patient/public health without

concern for the sale of products.

Commercial support for CE:

— Risks distorting the educational content and invites bias.

— Raises concerns about the vows of health professionals to place

patient interest uppermost.

— Endangers professional commitment to evidence-based decision

making.

— Validates and reinforces an entitlement mindset among health

professionals that CE should be paid for by others.

Fletcher

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 19

— Impedes the adoption of more effective modes of learning.

No amount of strengthening of the “ﬁrewall” between commercial

entities and the content and processes of CE can eliminate the

potential for bias.

Academic health centers and other healthcare delivery systems are

not sufﬁciently attentive, either to their roles in planning, providing,

and assessing CE or to their responsibilities in managing their own

conﬂicts of interest and those of individual faculty and administrators

when paid by commercial interests for CE teaching.

Accrediting CE

Current accreditation mechanisms for CE are unnecessarily complex

yet insufﬁciently rigorous. Compared to earlier, formal stages of

health professions education, the CE enterprise is fragmented, poorly

regulated, and uncoordinated; as a result, CE is highly variable in

quality and poorly aligned with efforts to improve quality and

enhance health outcomes.

With the increasing need for inter-professional collaboration,

accrediting bodies of the various health professionals need closer

working relationships.

RECOMMENDATIONS

CE Methods

The CE enterprise should shift as rapidly as possible from excessive

reliance on presentation/lecture-based formats to an emphasis on

practice-based learning.

New metrics are needed:

— To assess the quality of CE. These metrics should be based

on assessment of process improvement and enhanced patient

outcomes.

— To identify high-performing healthcare organizations. The

possibility of awarding CE credit to individual health professionals

who practice in such organizations should be explored.

Chairman’s Summary

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 20

— To automate credit procedures for point-of-care learning.

Federal and state policymakers should provide ﬁnancial support

for the further development of information technology tools that

facilitate practice-based learning and should strongly encourage all

clinicians to use these tools.

The responsibility for lifelong learning should be emphasized

throughout the early, formal stages of education in all health

professions. Students should be taught the attitudes and skills to

accomplish CE throughout their professional lifetimes.

A national inter-professional CE Institute should be created to

advance the science of CE. The Institute should:

— Promote the discovery and dissemination of more effective

methods of educating health professionals over their profession-

al lifetimes and foster the most effective and efﬁcient ways to

improve knowledge, skills, attitudes, practice, and teamwork.

— Be independent and composed of individuals from the various

health professions.

— Develop and run a research enterprise that encourages increased

and improved scientiﬁc study of CE.

— Promote and fund evaluation of policies and standards for CE.

— Identify gaps in the content and processes of CE activities.

— Develop mechanisms needed to assess and fund research

applications from health professional groups and individuals.

— Stimulate development and evaluation of new approaches to

both intra- and inter-professional CE, and determine how best

to disseminate those found to be effective and efﬁcient.

— Direct attention to the wide diversity and scope of practices

with special CE needs, ranging from highly technical specialties

on the one hand to solo and small group practices in remote

locations, on the other.

Fletcher

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 21

Chairman’s Summary

— Acquire ﬁnancial resources to support its work and provide

funding for research. Possible funding sources include the

Federal government, foundations, professional groups, and

corporations.

A concerted effort is needed to make the concept of a Continuing

Education Institute a reality. To achieve this, The Institute of Medicine

should convene a group to bring together interested parties to propose

detailed steps for developing a Continuing Education Institute.

CE Financing

Accredited organizations that provide CE should not accept any

commercial support from pharmaceutical or medical device compa-

nies, whether such support is provided directly or indirectly through

subsidiary agencies. Because many professional organizations and

institutions have become heavily dependent on commercial support

for current operations, an abrupt cessation of all such support would

impose unacceptable hardship. A ﬁve-year “phase out” period

should be allowed to meet this recommendation.

The ﬁnancial resources to support CE should derive entirely from

individual health professionals, their employers (including academic

health centers, healthcare organizations, and group practices), and/or

non-commercial sources.

Faculty of academic health centers should not serve on speakers’

bureaus or as paid spokespersons for pharmaceutical or device

manufacturers. They should be prohibited from publishing articles,

reviews, and editorials that have been ghostwritten by industry

employees.

CE Accreditation and Providers

Organizations authorized to provide CE should be limited to profes-

sional schools with programs accredited by national bodies, not-for-

proﬁt professional societies, healthcare organizations accredited by

the Joint Commission, multi-disciplinary practice groups, point-of-care

resources, and print and electronic professional journals.

Existing accrediting organizations for continuing education for

medicine (the Accreditation Council for Continuing Medical

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 22

Fletcher

Education) and nursing (the American Nurses Credentialing Center)

should meet and within two years develop a vision and plan for a

single accreditation organization for both nursing and medicine.

The new organization should incorporate the guiding principles for

CE and the recommendations laid out in this report where relevant.

The American Academy of Nursing and the Association of American

Medical Colleges should convene the two accrediting bodies for

this purpose.

Academic health centers should examine their missions to determine

how to strengthen their commitment to CE. They should help their

faculty gain expertise in teaching practice-based learning and incor-

porate information technology, simulations, and interactive

scenarios into their CE activities.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 23

Suzanne W. Fletcher, M.D., M.Sc.*

Harvard Medical School

Chair

________________________________

Barbara F. Atkinson, M.D.

University of Kansas Medical Center

Denise Basow, M.D.*

UpToDate

Waltham, MA

Regina Benjamin, M.D., M.B.A.

Bayou Clinic, Inc.

Bayou La Batre, AL

David Blumenthal, M.D., M.P.P.

MGH/Partners Healthcare System

James A. Clever, M.D.

Marin County, CA

Jordan J. Cohen, M.D.*

Association of American Medical Colleges

Ellen M. Cosgrove, M.D.*

University of New Mexico School of

Medicine

Linda Cronenwett, Ph.D., R.N.

University of North Carolina at Chapel Hill

David A. Davis, M.D.

Association of American Medical Colleges

Catherine D. DeAngelis, M.D., M.P.H.

JAMA and Archives

John Hopkins University

Lyn DeSilets, Ed.D., R.N.-B.C.

Villanova University

F. Daniel Duffy, M.D.*

American Board of Internal Medicine

Harvey Fineberg, M.D., Ph.D.

Institute of Medicine

Grant S. Fletcher, M.D., M.P.H.

University of Washington School

of Medicine

Melvin I. Freeman, M.D.

Virginia Mason Medical Center

Seattle, WA

Michael Green, M.D., M.Sc.

Yale University School of Medicine

Carol Havens, M.D.

Kaiser Permanente Medical Center

Sacramento, CA

Paul C. Hébert, M.D., M.H.Sc.

Canadian Medical Association Journal

Maryjoan D. Ladden, Ph.D., R.N.*

Harvard Medical School

Participants

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 24

David Leach, M.D.*

Accreditation Council for Graduate

Medical Education

Donald A. B. Lindberg, M.D.*

National Library of Medicine

Phil Manning, M.D.

Keck School of Medicine of the

University of Southern California

Paul Mazmanian, Ph.D.

Virginia Commonwealth University

Pamela Mitchell, Ph.D., M.S., B.S.

University of Washington

Donald E. Moore Jr., Ph.D.

Vanderbilt School of Medicine

Ajit K. Sachdeva, M.D.

American College of Surgeons

Marla E. Salmon, Sc.D., R.N.

Emory University

David C. Slawson, M.D.

University of Virginia

Robert Steinbrook, M.D.

New England Journal of Medicine

William Tierney, M.D.

Indiana University School of Medicine

Thomas R. Viggiano, M.D., M.Ed.

Mayo Clinic

Susan Wesmiller, M.S., R.N.

University of Pittsburgh Medical Center

Michael Wilkes, M.D.

University of California, Davis

Patricia S. Yoder-Wise, R.N., Ed.D.

Texas Tech University Health Sciences Center

MACY FOUNDATION

June E. Osborn, M.D.*

Marc A. Nivet, Ed.D.

Nicholas R. Romano, M.A.

Mary Hager, M.A.

Planning Commitee

Macy Conference participants are invited for their individual perspectives

and do not necessarily represent the views of any organization.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 25

Conference Images

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 26

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 27

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 28

Approaches to

Knowledge Development

—

What Works and

What Does Not

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 29

How Physicians Learn and How to

Design Learning Experiences for Them:

An Approach Based on an Interpretive

Review of Evidence

Donald E. Moore, Jr., Ph.D.

Vanderbilt School of Medicine

Researchers from multiple studies over the past several years have

reported that there are distressing gaps between the healthcare services

that patients receive and those that they could be receiving.

1,2

These

studies show that many patients do not receive the best possible

care, receive suboptimal care, or are victims of errors, despite the

fact that approaches to care are improving and demonstrating

enhanced outcomes. A variety of approaches have been suggested

to address this gap.

Continuing medical education (CME) has been

a longstanding suggestion. For many years, however, people have

expressed concerns about the effectiveness of CME. As a result,

conﬁdence in the ability of CME to address the identiﬁed gaps in

healthcare delivery was not high. But signiﬁcant work over the past

20 years has demonstrated the effectiveness of CME, if it is planned

and implemented according to approaches that have been shown

to work.

4–6

This interpretive essay reviews the evidence that describes how

physicians learn and proposes six principles from that evidence and

research from other ﬁelds that can be used to plan formal educational

activities designed to facilitate physician learning. Next, the essay

proposes an instructional design approach for designing effective

formal CME activities. Finally, the essay brieﬂy discusses assessment

of formal CME activities. See Appendix A for a brief review of the

evidence on how nurses and pharmacists learn.

How Do Physicians Learn?

At any given time, physicians are engaged simultaneously in several

different kinds of learning. Systematic reading, self-directed improve-

ment at work, participation in formal CME courses, and consultation

with colleagues are woven into the basic ﬁber of their professional

lives to create an approach to learning that is unique to each individual

physician. Studies on physician learning have revealed that the

learning process consists of several stages. In general, these stages

begin with a physician learner becoming aware of a problem or

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 30

ecognizing

n Opportunity

for Learning

earching for

esources

for Learning

ngaging

n Learning

rying Out

hat Was

Learned

ncorporating

hat Was

Learned

Priming

Reflecting-in-action

Reflecting-on-

action

Awareness

Priming

Pre-awareness

Awareness

Scanning

Awareness

Felt need

Articulate a

problem

Awareness

Pre-contemplation

Contemplation

Experiencing

Reflecting

Focusing

Decision to pursue

information

Actively seeking

a solution

Decision-making

Preparing to

make a change

Follow-up

Agreement

Evaluating

Obtain

knowledge about

an innovation

Search for

solutions

Information

seeking

Interest

Preparation

Conceptualization

Planning

Focusing

Develop

learning project

Making the

change

Follow-up

Learning

Obtain

knowledge about

an innovation

Preparation

Planning

Follow-up

Solidifying the

change

Follow-up

confirmation

Adoption

Gaining

experience

Favorable opinion

about the

innovation

Decision to adopt

or reject

Choice

Evaluation

Trial test

Action

Problem

resolution

Solidifying the

change

Adherence

Gaining

experience

Implementation

Confirmation

Application

of solution

Adoption

Integration

Maintenance

eertsma

et al

982

Schon

1983

Means

1984

Putnam,

Campbell

1989

Garcia,

Newsom

1996

Pathman

et al

1996

Slotnick

1999

21, 22

Rogers

1962

23, 24

Havelock

et al

1969

25, 26

Havelock

et al

1973

25, 26

Prochaska

1983

27-29

Kolb

1984

Table 1. Stages of Learning

tudies on Physician Learning

Studies on How People Learn

Moore

challenge and end when all stages are completed, with that physician

learner comfortably and conﬁdently applying newly learned knowl-

edge and/or skills (Table 1).

Stage theories are used commonly in the social and behavioral

sciences.

A stage theory describes a social or behavioral process

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 31

in terms of the collection of activities that an individual must pass

through in order to successfully complete that process. Descriptions

of learning as a process that includes several stages have been in

the educational literature at least since the early 1950s.

Tough,

9–11

Knowles,

and Knox

have reported stage learning projects of

adults who planned and directed the projects, consisting of multiple

stages, for themselves.

There is a danger of oversimpliﬁcation when describing a complicated

process like learning as a straight line that “ﬂows” from stage to stage.

In fact, the process of learning is extraordinarily complex, and it is

made even more complex because it is embedded in a social context.

It is more dynamic, with many interactions among the stages and

within the stages. The system dynamics model described by Hirsch

and his colleagues

is probably a more accurate way to depict the

process. For the purposes of this monograph, however, we use the

more static approach because the studies we are reviewing were

reported in that way, and while possibly oversimpliﬁed, this

approach helps us understand the process at a very macro level.

The ﬁndings of seven studies that examined physician learning and

ﬁve studies that examined how people learn are summarized in Table1.

These studies revealed that learning begins with an individual becoming

aware of a problem or challenge and ends when all stages are com-

pleted, with that learner comfortably and conﬁdently applying newly

learned knowledge and/or skills. We propose a five-stage model

that synthesizes the stages identiﬁed in the studies reviewed here. The

ﬁve stages are reﬂected as the headings of Table 1: 1) recognizing

an opportunity for learning; 2) searching for resources for learning;

3) engaging in learning to address an opportunity for improvement;

4) trying out what was learned; and 5) incorporating what was learned.

It is a precarious endeavor to synthesize the results of studies con-

ducted by others in an attempt to create a working theory. There

are dangers of over-interpreting some studies and under-interpreting

others. There is also a danger of making assumptions that the original

researchers did not intend. In addition, each of the models presented

has its own characteristics and features, which may be overemphasized

or underemphasized in the synthesis. Furthermore, the stages proposed

may contain part of a stage from one study and two stages from

another study. The following paragraphs describe these stages in

more detail as they relate to physician learning.

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 32

Stage 1: Recognizing an opportunity for learning

This is the initial stage of the process of physician learning. In this

stage, a physician begins to sense that there may be something in

his or her practice that is “not right” and may begin to consider

learning as a way to address it. The term “scanning” describes the

sometimes conscious, sometimes subconscious examination of a

physician’s practice and his or her professional environment that

results in a state of dissatisfaction with some aspect of the practice

or practice performance. Scanning may be passive, when a physician

reacts to surprises in practice or “feels” that something may not be

right, or it may be more active, when the physician is actively engaged

in examining areas that he or she has identiﬁed as opportunities for

improvement. The terms “reﬂecting-in-action” and “reﬂecting-on-

action” describe what a physician does when he or she becomes

“aware” that something is not right with the management of a patient

or group of patients, or if there might be a better way to manage a

patient. The product of reﬂection is an ongoing “articulation” of a

problem that describes the feeling that something is not right as a

difference between “what should be” and “what is”— what psychol-

ogists call “cognitive dissonance.”

For physicians in practice, “what

is” and “what should be” can be described in terms of performance

in practice or patient outcomes. Dissonance is associated with

discomfort that causes action to search for a solution to reduce the

discomfort. In a physician, cognitive dissonance will create discomfort,

which, in turn, will lead the physician to search for knowledge that will

make it possible to reduce the discrepancy and the discomfort.

17,32,33

Another way to describe what happens during this stage is to say

that a “teachable moment” emerges. A teachable moment is deﬁned

as the time when a learner’s psychological readiness for learning is

highest.

The strength and persistence of the teachable moment will

determine whether a physician will move to the second stage.

What causes a physician to move from recognizing that there is an

opportunity for learning to starting the process to pursue learning?

The strength and persistence of cognitive dissonance, or the teachable

moment, is important, but there are a variety of other issues and

considerations that affect the decision to pursue learning.

In the

general adult education literature, Cross

suggested that an indi-

vidual is more likely to participate in an educational activity if he or

she 1) possesses positive attitudes about education; 2) considers an

educational activity relevant to his or her educational need; 3) sees

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 33

more opportunities than barriers to participate; and 4) expects to be

successful in learning what he or she needs to learn to address the

educational need (cognitive dissonance) that initiated the search for

a learning opportunity. In the medical literature, Gorman suggests

that physicians will pursue learning if they believe that there is

something to be learned that might resolve the discrepancy at hand,

but they are not inclined to seek answers when they do not believe

that useful information exists.

Fox and his colleagues reported that

if a physician perceives learning and the results of learning to be

rational, relatively easy to achieve, and in the best interest of his or

her patients, the physician would be more likely to pursue learning.

If learning is coerced, however, by regulation or administrative

requirements, a physician’s participation would likely be minimal

and grudging. Moore and colleagues saw the decision to participate

in formal CME as part of information-seeking efforts and described

a complex transactional process in which the costs and beneﬁts of

participation were compared at several levels.

During this stage, questions may emerge for physicians like “Am I

treating patients (in this disease area) correctly?” “How are my

patients (in this disease area) doing?” “What are the acceptable

standards of care (in this disease area)?” “Is there anything “new”

in this disease area?” And, “What’s important in all this information

that I am hearing about this disease area?”

Stage 2: Searching for resources for learning

Practicing physicians seek to address problems they identify for

themselves by starting a search for resources for learning, framed

by these problems stated as questions, articulated at varying levels

of clarity. A question could focus on any number of the components

of patient management: pre-diagnostic evaluation, diagnosis, treat-

ment, or follow-up. A physician seeking information in any of these

areas might be concerned about declarative information (knowledge

base) but is more likely concerned about procedural information

(knowledge about how to use the knowledge base). In addition, a

physician seeking information may be seeking that information as

new knowledge to update his or her current knowledge or to rein-

force current knowledge.

During this stage, a physician prepares to make a change to address

the problem he or she has articulated by trying to understand it

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 34

and estimating what would have to be learned to address it. The

physician becomes aware of alternative approaches to address the

identiﬁed problem, decides which approach to use, and selects

resources to learn about that approach.

A physician begins to develop an image of the change in behavior

that is necessary to address the problem identiﬁed during stage 1

and how learning will help accomplish that change.

Slotnick found

that during this stage, a physician evaluates the problem or question

to determine whether pursuing learning will be beneﬁcial. Slotnick

suggests that a physician determines the beneﬁts by asking the fol-

lowing questions:

— Does the problem likely have a solution?

— Are resources available to learn the solution to the problem?

— Will learning the solution change my practice in

desirable ways?

If a physician responds to these questions with positive answers, the

next step will be to determine whether she needs to learn something

to make a change. With the image of change in mind,

the physi-

cian will determine the extent to which he or she is able to make

the change without learning (her capabilities are adequate) or the

extent to which she will have to pursue learning (she feels that her

capabilities are not adequate). A physician at this stage will assess

her capabilities in a number of ways, including personal reﬂection,

feedback, and interaction with colleagues, as well as personal,

community, and professional expectations.

In addition, a physician

may assess her capabilities while engaged in CME activities, such

as reading journals, attending a formal CME activity, and partici-

pating in a Web-based CME activity. Slotnick suggests that the

physician would ask the following questions:

— What’s important in all this information that I am hearing,

reading, or viewing?

— What experiences have other physicians had doing what I am

hearing about?

— What is the best way to learn?

At the end of this stage, a physician has evaluated the problem or

question that precipitated the search for learning resources, deter-

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 35

mined whether pursuing learning would be beneﬁcial, and decided

what type of learning to pursue.

Stage 3: Engaging in learning

During the entire ﬁve-stage process, a physician manages the pattern

of formal and informal resources that he or she ﬁnds most effective

to address the opportunity for learning that started the process.

Learning becomes more focused, intentional, and formal during this

stage. Physicians learn informally as well as in formal settings.

Informal learning consists of (but is not limited to) casual journal

reading, ad hoc conversations with colleagues, interactions with

industry representatives, and attendance at grand rounds and other

regularly scheduled conferences. Some informal CME focuses on

speciﬁc patient problems and is more structured, such as consultations

with colleagues and focused reading of journals or textbooks. Formal

CME is usually planned in some detail and consists of CME activities

planned by the physician learner or by someone else. Formal CME

activities planned by the physician learner would include preceptor-

ships in which educational activities are negotiated between the

learner and the preceptor and learning projects like the Maintenance

of Competence Program (MOCOMP) of the Royal College of Physi-

cians and Surgeons of Canada, in which a physician identifies an

opportunity for improvement, designs learning activities directed at

the improvement, and assesses accomplishments.

45–49

Other educa-

tional activities planned by the learner are those in the maintenance

of certiﬁcation programs of specialty societies that are members of

the American Board of Medical Specialties (ABMS).

CME activities

planned by others include formal CME courses and enduring materials.

Questions that emerge during this stage include but are not limited

to the following: “Is this educational activity addressing what I need

to learn?”; Is the content based on evidence from research?”; “What

do other participants think about what is being presented?”; How

does what is being presented relate to my patients?”; “Is this all I

have to learn?”; “Does what is being presented actually work?”; and

“How will what is being presented change my practice?”

Stage 4: Trying out what was learned

In this stage a physician begins to use newly learned skills and

knowledge to address the problem that precipitated the learning

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 36

process. During this stage a physician develops a favorable

opinion about what she has learned and makes a decision to

accept it or reject it. She experiments with what she has learned

in her practice setting but not before confirming the benefits of

what she has learned with colleagues. The stage begins with a

physician being less than comfortable with her new skills and

knowledge. As she progresses through the stage, however, she

will become more skillful and confident with what she is doing.

The stage is over when the physician is sufficiently comfortable

with the newly learned skills and knowledge and they become

second nature.

During this stage a physician is likely to rely more on her own

reﬂections and conversations with colleagues than the other resources

she used during the learning process.

51,52

Questions she might ask

during this stage include the following:

— How does what I have learned apply to my practice?

— What do I have to do in my practice to use what I have

learned?

— Am I doing it right?

— Does it work?

An important part of this trial and evaluation stage for a physician is

determining not only how to use what she has learned but also

whether it works in her setting. If the physician’s practice uses an

electronic medical record or other digital databases for monitoring

performance and patient outcomes, she will be able to determine

the effectiveness of the new learning. But it is more likely that she

will “reﬂect-on-her-practice” after several “experiments,”

and if the

new learning appears to be effective, will move on to the next stage.

This stage raises interesting ethical issues. One might ask what the

responsibility of a CME program is to ensure that every participant

has the opportunity to practice until conﬁdent in an environment in

which patients are not endangered by physician experimentation

“until he or she gets it right.”

Stage 5: Incorporating what was learned

During this ﬁnal stage, a physician integrates what he has learned

into his daily routines; it will become a part of what he does when

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 37

How Physicians Learn and How to Design Learning Experiences for Them

managing patients. Questions that emerge in this stage include but

are not limited to the following: What do I have to do differently

in my practice to use what I have learned? How do I make what I

have learned a part of my practice? What ofﬁce routines have to

be changed? What new procedures have to be introduced? What

training does staff need? And, what do I have to do for my patients?

If the physician has not done so already in the previous stage, during

this stage, a physician will have to make sure that ofﬁce routines

and procedures include not only what he has learned but also what

he will need to implement what he has learned. Most important, he

will need to train his staff in what he has learned.

Principles for Facilitating Physician Learning in Formal CME

As the research summarized here suggests, physician learning is

predominantly self-directed. Physicians engage in CME activities

because they want to learn something that will help them provide

the very best possible care to their patients, not because someone

has told them what to learn. In most cases, physicians proceed on

their own through the ﬁve stages of self-directed learning described

here, consulting a variety of resources, but essentially planning and

directing the learning project in which they are engaged on their

own. In many cases, physicians choose to enroll in a formal CME

activity that has been planned by someone else. The most effective

formal learning experience for physicians would take into account

where a physician is in his or her learning process (what stage) and

what will help him or her accomplish learning goals. The principles

summarized in the following paragraphs are drawn from the studies

reviewed in the previous section of this monograph as well as from

studies that have examined learning in general.

Principle 1: Planners of formal CME should consider physicians’

stages of learning.

Typically, CME planners asssume that all physicians who enroll in a

CME activity are at stage 3, engaging in learning, and need informa-

tion. In reality, a physician who enrolls in a formal CME activity

could be at any one of the ﬁve stages of learning and would have

questions related to his stage (Table 2). To make the learning expe-

riences of physicians more productive, CME planners should help

physicians recognize what stage they are in and plan activities to

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 38

Recognizing

an Opportunity

or Learning

Searching for

Resources

or Learning

Engaging

in Learning

Trying Out

What Was

earned

Incorporating

What Was

earned

Am I treating patients

(

in this disease area)

correctly?

ow are my patients

(in this disease area)

doing?

What are the

cceptable standards

f care (in this

disease area)?

Is there anything

“new” in this

disease area?

What’s important

in all this information

that I am hearing

about this disease

area?

Is this a problem

or me?

oes this problem

have a solution that

can be addressed

y learning?

Are there resources

vailable for learning

he solution?

Can I learn the

solution?

What’s important

in all this information

that I am hearing

about this disease

area?

What do I need to

learn?

What is the best way

to learn?

What experiences

have other

physicians had

with this problem?

How will learning the

solution change my

practice?

Is this educational

ctivity addressing

what I need to learn?

s the content based

on evidence?

What do the other

articipants think

bout what is being

presented?

How does what

is being presented

relate to my

patients?

Is this all I have

to learn?

Does what is being

presented work?

How will what is

being presented

change my practice?

How will what is

being presented

change my practice?

Am I doing what

am learning

correctly?

ow will I be able

to do what I am

learning in my

ractice?

Does what I am

earning to do work?

What do I have to

o differently to use

what I have learned

in my practice?

ow do I make what

I have learned a

part of my practice?

What office routines

ave to be changed;

hat new

procedures have

to be introduced?

What training does

staff need?

What do I have to

do for patients?

What do I have to

do for patients?

Table 2. Questions Physicians Have at Each Stage of Learning

Adapted from Slotnick

Moore

address the questions they have that are associated with that stage.

A physician would initiate the ﬁve-stage learning process for

problems related to speciﬁc patients and for more generalized

patient-related issues. The process that results for each is slightly

different. For problems related to speciﬁc patients (Why is Mrs.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 39

Jones not responding to her hypertension medication?), a physician

would likely use resources in his ofﬁce (journals, textbooks, online

resources) and local consultants in almost all stages. The physician

would typically enroll in a formal CME activity for learning initiated

by a more general patient-related problem (Why are the hemoglobin

A1c levels of my diabetes patients consistently high?).

With respect to a general patient problem, some physicians might

be at stage 1, recognizing an opportunity for learning, or not yet

at stage 1. These physicians would beneﬁt from information that

would help them recognize or create a teachable moment. Many

physicians who are at stage 2, searching for resources for learning,

may have difﬁculty articulating the opportunity for learning they

recognize into questions that can lead to a productive search for

information and an effective educational experience.

21,25,52–54

CME

planners should design experiences that allow physicians to translate

what they are feeling about their practice behavior into questions.

Some physicians may ﬁnd answers for the questions they develop

as part of the CME activity, while for others the questions may

remain. For the latter group, it is important for the CME activity

to help with that search.

Other physicians may be at stage 3, engaging in learning. At this

stage, a physician already has formulated questions. The CME activity

should be designed to provide the conditions in which this physician

can create the knowledge necessary to answer those questions.

Physicians who are at stage 4, trying out what was learned, will

beneﬁt from an opportunity to practice using what they have learned.

Practice in an educational setting will prepare them to transfer new

learning when they return to their practice setting. If physicians at

stage 5, incorporating what was learned, are enrolled in a CME

activity, it is important to provide them with information or resources

showing how to integrate new approaches into a practice.

Physicians may decide to discontinue activities within a stage or

stop the entire learning process itself when they feel they have

learned enough to pass on to the next stage or deal with the prob-

lem that precipitated the learning process, or they may decide that

the beneﬁts from learning are not worth the efforts.

Maintaining

motivation is important if physicians are to persist in learning through

all ﬁve stages and change their behavior based on that learning.

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 40

The physician was motivated to initiate the learning process because

of cognitive dissonance. Frequent reminders are an important part

of the educational activity to show that progress is being made in

closing the gap between “what is” and “what should be” that created

the cognitive dissonance. Linking feedback and motivation is the

key to a successful educational experience.

Principle 2: Formal CME activities should focus on clinical

problems and should provide information that physicians can

use in practice.

Practicing physicians are in a more or less continuous search for

information that they want to use to solve problems or address

opportunities for improvement in practice.

They may be at any one

of the ﬁve stages described earlier. Regardless of what stage they are

in, physicians are interested in clinical issues, not necessarily a detailed

description of the basic science or clinical research that led to the

ﬁnding that has clinical implications.

When physicians select learning resources in their self-directed

learning projects, one of the features that is important to them is a

focus on clinical issues.

27,51,57–60

This is useful to consider when

planning formal CME activities where physicians may be in any one

of the ﬁve stages of learning.

Principle 3: Formal CME activities should be organized to provide

opportunities to obtain knowledge and develop skills in settings

and applications that would normally involve the use of that

knowledge and/or those skills.

The ultimate goal of an effective formal CME activity is for the

physician to apply what he or she has learned in the practice, and

not just retain the facts. Like other learners, physicians are more

likely to learn and change their behavior if the learning experience

closely resembles the setting in which the learning will be used.

Physicians learn in relation to what else they know, believe, and

have experienced — the settings and applications that would nor-

mally involve that knowledge. They will transfer what they have

learned in an educational setting to a work setting more readily

when the educational setting is designed as an “authentic” work

setting.

Physicians will learn more effectively in this setting by

deriving insights from clinical practice with the help of CME faculty

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 41

who demonstrate how to reﬂect on a problem, identify the most

feasible solution, use the solution in a real-world setting, and assess

the effectiveness of the chosen solution. CME planners should work

with faculty to create authentic settings that will engage the learner

in complex, realistic, “messy,” problem-centered activities and create

a setting that will be familiar and in which physician learners can

recognize what they need to learn and begin the process of transfer-

ring learning to their practice settings.

Principle 4: Formal CME activities should provide opportunities

for active learning.

Like other learners, physicians learn best by doing.

Actively involv-

ing learners is more effective than simply disseminating concepts

and principles. Teachers should function more as facilitators of

learning than as disseminators of facts.

62,63

CME planners should not

only provide an authentic setting in formal CME activities but should

also provide physicians with opportunities to interact within that

setting. Because the goal is for physicians to apply what they have

learned in a formal CME activity in a dynamic social environment —

the practice setting — a formal CME activity will be more effective in

accomplishing this goal if it is planned as a social process dependent

on transactions with others within a context that resembles as closely

as possible the practice environment. Indeed, CME course design

and materials should permit and encourage physicians to explore

the new content, their own experience, and the experiences of

other physician learners. The choice of a particular method is not

as important as providing the opportunity for learners to practice

what they have learned and to receive feedback on their efforts.

Active learning is more effective than passive learning for several

reasons. First, it draws on the experience and previous knowledge

of the learners. Second, it recreates the social environment within

which the new information will be used. Third, it provides an

opportunity for practice and feedback, two activities essential for

learning and transferring that learning to action. Fourth, it permits

the physician learners to engage at their own stage of learning.

Learning is more effective if the experience and previous knowledge

of the learners is taken into account. Active learning formats in

formal CME activities facilitate the incorporation of a learner’s previous

knowledge and experience into learning activities. Learning is a

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 42

social activity. Learning is intimately associated with an individual’s

connection with other human beings, including colleagues, teachers,

peers, and family, as well as casual acquaintances, and physicians

attend CME activities because doing so provides opportunities to

talk with others sharing their interests.

Learning requires both practice and feedback. Again, like other

learners, physicians are more likely to learn if they are given an

opportunity to practice and receive feedback. For effective learning

to occur, there must be opportunities for practice as well as positive

feedback when something is performed correctly, and negative

feedback given constructively when something is performed incor-

rectly. In addition, practice provides opportunities to reﬂect on the

task at hand as well as past experience — to understand not only

what happened, but also why it happened.

In an active learning environment, physicians can feel vulnerable

and resist participation because they fear being exposed for lack

of knowledge or posing an incorrect solution to a problem encoun-

tered as part of an educational exercise. An individual must feel

safe before effective learning can occur.

An important part of the

andragogical approach developed by Malcolm Knowles is creating

a climate comfortable for learning and the learners.

Permitting

each participant to maintain his or her level of self-esteem is crucial

to creating conditions for learning.

Principle 5: Effective CME activities are more likely to consist of

multiple educational activities that are organized according to

an instructional design plan.

The results of several systematic reviews suggest that physicians are

more likely to learn and use what they have learned in their practices

if they participate in multiple learning activities on a single topic.

65–67

A recent systematic review suggests, however, that this may not

always be true.

The issue may not be so much how many activities,

but rather how the activities are organized. Instructional design

provides suggestions not only for what educational methods and

techniques are most likely to result in desired learning outcomes

but also how to organize them.

Conventional approaches to planning CME, loosely based on the

approach described by Malcolm Knowles,

68,69

focus only on the

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 43

ecognizing

an Opportunity

for Learning

earching for

Resources

for Learning

ngaging

in Learning

rying Out

What Was

Learned

ncorporating

What Was

Learned

Table 3. CME Planning that is Responsive to Questions that

Physician Have at Each Stage of Learning

PREDISPOSING

1. How are my patients

(in this problem area) doing?

(Stage 1)

2. Am I treating patients

(in this problem area) correctly?

(Stage 1)

3. What are the acceptable

standards of care (in this

problem area)? (Stage 1)

4. Is there anything “new”

in this problem area?

(Stage 1)

5. Does this problem have a

solution that can be addressed

by learning? (Stage 2)

6. What experiences have

other physicians had with

this problem? (Stage 2)

7. Are there resources available

for learning the solution?

(Stage 2)

8. Can I learn the solution?

(Stage 2)

9. What is the best way to learn?

(Stage 2)

10. What’s important in all this infor-

mation that I am hearing about this

problem area?(Stages1 and2)

ENABLING

1. What do I need to learn?

(Stage 2)

2. How will what is being

presented change my

practice? (Stage 2)

3. Is this educational activity

addressing what I need to learn?

(Stage 3)

4. Does what is being presented

work? (Stage 3)

5. Is the content based on

evidence? (Stage 3)

6. What do the other participants

think about what is being pre-

sented? (Stage 3)

7. How does what is being present-

ed relate to my patients?

(Stage 3)

8. How will learning the solution

change my practice? (Stage 2)

9. Am I learning?

(Stage 3)

10. Is this all I have to learn?

(Stage 3)

11. Am I doing what I am learning

correctly? (Stage 4)

12. What do I have to do differently to

use what I have learned in my

practice? (Stage 4)

13. What’s important in all this infor-

mation that I am hearing about

this problem area? (Stages 2, 3,

and 4)

REINFORCING

1. Does what I am learning

to do work? (Stage 4)

2. How do I make what I have

learned a part of my practice?

(Stage 5)

3. What new procedures have

to be introduced?

(Stage 5)

4. What office routines have to

be changed? (Stage 5)

5. What training does staff need?

(Stage 5)

6. What do I have to do differently

for patients? (Stage 5)

7. How will I be able to do what

I am learning in my practice?

(Stage 5)

8. What’s important in all this

information that I am hearing

about this problem area?

(Stages 4 and 5)

C M E P L A N N I N G

How Physicians Learn and How to Design Learning Experiences for Them

stage of the learning process when a physician is “engaged in learn-

ing” (stage 3). See Table 3. Davis and colleagues

0,71

observed,

however, that in effective CME, multiple educational activities were

organized as predisposing, enabling, and reinforcing, following

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 44

Moore

the PRECEDE-PROCEED framework developed by Green and

colleagues.

This observation has been reinforced by positive

ﬁndings in studies reported by Mann and colleagues,

White,

and

Cantillon.

The predisposing-enabling-reinforcing paradigm appears

to match well with the stage learning theories described by researchers

investigating how physicians learn. Predisposing activities relate

to stage 1 and most of stage 2; enabling activities relate to some

of stage 2, all of stage three, and some of stage 4; and reinforcing

activities relate to most of stage 4 and all of stage 5. Before engaging

in learning, most physicians feel a need to learn (cognitive disso-

nance); they become predisposed to learn. While engaging in learning,

physicians strive to learn what to do and how to do it so they

become enabled to do what they did not know how to do before.

And most physicians will seek reinforcing comments from peers

or local opinion leaders after learning something new and trying it

in practice.

Principle 6: Learning is enhanced if content is organized

in small units.

Learning tasks should be organized in such a way that what is to be

learned is presented in small amounts, so that each unit will be mas-

tered before the next one is considered. Cognitive scientists have

called this “chunking” and suggest that it improves the efﬁciency of

working memory.

External information (sensory input) must ﬁrst

be processed by short-term memory. When short-term memory is

overloaded, learning can be difﬁcult. Overload can be eliminated by

reducing or pacing sensory input. Designing formal CME activities in

15-minute modules would modulate sensory input and contribute to

improved learning conditions.

Designing Effective CME:

A Proposed Instructional Design Approach

No single tested theory exists that CME planners and others could

use as a guide for planning educational activities.

Several studies

have contributed to an emerging theory of how physicians learn,

some of which were reviewed in the ﬁrst section of this paper.

Other studies have suggested what contributes to effective CME,

and additional studies have reviewed the effectiveness of various

educational techniques in CME. However, the ﬁndings from these

studies do not appear to have been combined, synthesized, hypoth-

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 45

Presentation Techniques

• Lecture

• Panel

Interactive Techniques

• Teaching and learning from cases

Case presentation

Case discussion

• Small group discussion

Skill Development Techniques

• Presentation

• Demonstration

• Practice

• Feedback

Table 4. Educational Methods in Formal CME

How Physicians Learn and How to Design Learning Experiences for Them

esized, and tested in a way that would produce a theory on which

instructional design decisions in CME could be conﬁdently based.

This section represents an attempt to combine and synthesize the

material presented in previous sections into a proposed approach

for CME instructional design.

Educational methods used in formal CME activities traditionally fall

into three broad categories: presentation methods, interactive methods,

and skill development methods (Table 4). Presentation methods

are used when learning objectives call for knowledge acquisition.

Interactive methods are used when the intent is to increase under-

standing of educational content and improve the chances that it

will be applied in the practice setting. The most common interactive

method in medical education is teaching and learning from cases.

Two formats are useful in CME: case presentation and case discussion.

In a case presentation, information about the experience of managing

a patient with a speciﬁc disorder is reported to the audience in a

presentation format. Case discussion engages members of the audi-

ence in decision making about a case. Skill development methods

are used to help learners develop and use a skill. These methods

can be used to help learners develop cognitive skills, for example

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 46

Moore

deciding to prescribe an ACE inhibitor for patients with congestive

heart failure; interpersonal skills, like obtaining a sexual history

from an adolescent; or psychomotor skills, like performing ﬂexible

sigmoidoscopy or a new cardiac surgery technique. Merrill suggests

that there are four components of effective skill development, referred

to here as presentation, demonstration, practice, and feedback.

Although presentation methods, such as lectures, panels, and printed

materials, may not change physician behavior when used alone, they

continue to be the most prevalent method for CME. When presentation

methods are made more interactive through case presentations or

group discussion, moderately large changes in professional practice

can result. No systematic review of skill development methods has

been performed, but evidence from the literature suggests that multiple

CME activities focused on a single topic organized following the

predisposing-enabling-reinforcing model are effective in changing

physician behavior.

To make the learning experiences of physicians more effective,

CME planners should use the predisposing-enabling-reinforcing

framework to align learning activities with physician learning stages.

At each learning stage, physicians have speciﬁc questions, some

more conscious and articulated than others. In Table 3, the questions

listed in Table 2 under columns related to the stage of learning have

been resorted and placed under columns labeled “Predisposing,”

“Enabling,” and “Reinforcing.” The instructional design approach

described here involves identifying content to answer the questions

and determining the best way to help physicians learn that content,

drawing on what has been described about how physicians learn,

principles that describe how to facilitate physician learning, and

educational methods that have been effective in CME.

Planning predisposing CME activities

As mentioned earlier, predisposing activities appear to relate to all of

stage 1: recognizing an opportunity for learning, and part of stage 2,

searching for resources for learning. As a result, the questions that

need to be addressed to design predisposing learning activities are all

of the questions in stage one and some of the questions from stage

two. See Table 5 for a list of the questions, the content that could

address the questions, and suggested educational methods for each

question.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 47

Table 5. Planning Pre-disposing CME Activities

UESTION

. How are my patients

(in this problem area) doing?

2. Am I treating patients

(

in this problem area) correctly?

3. What are the acceptable

standards of care (in this

problem area)?

4. Is there anything “new”

in this problem area?

5. Does this problem have a

solution that can be addressed

by learning?

6. What experiences have

other physicians had with

this problem?

7. Are there resources available

for learning the solution?

8. Can I learn the solution?

9. What is the best way to learn?

10. What’s important in all this

information that I am hearing

about this problem area?

CONTENT

esults of audit that focus on

atient health status and outcomes

in the problem area and information

about those results.

esults of audit that focus on

hysician performance in the

problem area and information

about those results.

Local or national clinical practice

guidelines (CPGs) in the problem

area.

Findings from studies recently

published in peer reviewed journals

and vetted for level of evidence.

Separation of educational from

non-educational components

of CPGs.

Stories of other physicians.

Information about enabling

activities that have been or will

be designed to address this problem

area. Enabling activities can be

part of a single activity that follows

predisposing activities or part of

a series of activities that occur

at another time.

Information about the enabling

activities that have been developed

that demonstrates that they are

convenient, accessible, and will not

be an unreasonable burden.

Information that shows that the

enabling activities that have been

or will be designed to facilitate

physician learning, and that there

will be activities to reinforce it.

Regular reference to the gap

between current and actual and

how what is being presented

relates to reducing the gap.

EDUCATIONAL METHOD

ecture and group discussion

cademic detailing

Lecture and group discussion

Academic detailing

Lecture and group discussion

Academic detailing

Lecture and group discussion

Academic detailing

Lecture and group discussion

Academic detailing

Panel

Group discussion

White space on schedule for

informal interactions among

participants.

Recruitment material

Handout

Lecture (testimonial)

Recruitment material

Handout

Lecture (testimonial)

Recruitment material

Handout

Lecture (testimonial)

All methods

How Physicians Learn and How to Design Learning Experiences for Them

Planning enabling CME activities

Enabling CME activities appear to relate to all of stage three, engaging

in learning; parts of stage 2, searching for resources for learning;

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 48

Table 6. Planning Enabling CME Activities

UESTION

. What do I need to learn?

2. How will what is being

presented change my practice?

3. Is this educational activity

addressing what I need to learn?

4. Does what is being presented

work?

5. Is the content based on evidence?

6. What do the other participants

think about what is being

presented?

7. How does what is being presented

relate to my patients?

8. How will learning the solution

change my practice?

9. Am I learning?

10. Is this all I have to learn?

11. Am I doing what I am learning

correctly?

12. What do I have to do differently

to use what I have learned in my

practice?

13. What’s important in all this

information that I am hearing

about this problem area?

CONTENT

pecific behaviors that potential

participants would be expected

to perform after participating in

he enabling activity, e.g. learning

objectives.

escription (in general) of the

mplications of what will be learned

during the enabling activity for

practice organization and the people

that work within the practice.

Regular reference to the gap between

current and actual and how what is

being presented relates to reducing

the gap.

Summaries of CPGs or the findings of

studies that demonstrate the efficacy of

what is being presented, i.e., what is

being presented actually produces

desired results. In addition, summaries

should include the level of evidence.

Opinions of other participants.

Description of patients in the studies

reported. Comparison of study

patients with participant’s patients.

Description of the practice organization

in the studies and what the people in

those practices did as part of the study.

Summary of the criteria of

performance, highlighted during the

demonstration and coached during

practice. Information about how the

learner(s) performed the criterion

components of the skill and how to

improve to criterion.

Regular reference to the gap between

current and actual and how what is

being presented relates to reducing

the gap.

EDUCATIONAL METHOD

ecruitment materials

Handout

Lecture

Handout

Lecture

Group discussion

Teaching and learning from cases

All methods

Skill development (presentation)

• Panel

• Group discussion

White space on schedule for informal

interactions among participants.

Skill development (cases)

• Demonstration

• Practice

Skill development (cases)

• Demonstration

• Practice

• Action plan

Skill development (cases)

• Practice

• Feedback

• Action plan

All methods

Moore

and stage four, trying out what was learned. As a result, the questions

that need to be addressed to design enabling learning activities are

all of the questions in stage 3 and some of the questions from stages

2 and 4. (Table 6).

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 49

Table 7. Planning Reinforcing CME Activities

QUESTION

1. How will I be able to do what I

am learning in my practice?

2. Does what I am learning to

do work?

3. How do I make what I have

learned a part of my practice?

4. What new procedures have to

be introduced?

5. What office routines have to be

changed?

6. What training does staff need?

7. What do I have to do differently

for patients?

8. What’s important in all this infor-

mation that I am hearing about

this problem area?

CONTENT

Summaries of CPGs or the findings

of studies that demonstrate the

efficacy of what is being presented,

i.e., what is being presented actually

produces desired results. In addition,

summaries should include the level

of evidence.

Results of audit that focus on

physician performance in the

problem area and information about

those results.

• Summaries of CPGs or the

findings of studies that

demonstrate the efficacy of

what is being presented,

i.e., what is being presented

actually produces desired

results. In addition, summaries

should include the level of

evidence.

• Description of patients in the

studies reported. Comparison

of study patients with

participant’s patients.

• Description of the practice

organization in the studies

and what the people in

those practices did as part

of the study.

Regular reference to the gap between

current and actual and how what is

being presented relates to reducing

the gap.

EDUCATIONAL METHOD

Skill development

• Action plan

Reminders

Commitment to change

Lecture and group discussion

Academic detailing

Skill development (cases)

• Action plan

Group Discussion

Reminders

Commitment to change

All methods

How Physicians Learn and How to Design Learning Experiences for Them

Designing reinforcing CME activities

Reinforcing CME activities include part of stage 4, trying out what

was learned and all of stage 5, incorporating what was learned.

As a result, the questions that need to be addressed to design rein-

forcing learning activities are some of the questions in stage 4 and

all of the questions from stage 5 (Table 7).

Assessing CME Activities

At the beginning of this paper, the gap between the healthcare that

most Americans are receiving and the healthcare that they could be

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 50

Desired Results

Outcomes Impact

Competence Performance Patient

(Shows how) (Does)

health

status

Inputs

Current

Situation

Intervention

CME Activity

Outputs

• Learning

• Declarative

Knowledge

(knows)

• Procedural

Knowledge

(knows how)

Figure 1. Systems Overview of CME as an Intervention

Moore

receiving was identiﬁed as a signiﬁcant problem. CME, designed

according to the model presented here, is one approach to address

the gap. This section offers suggestions on how to determine the

effectiveness of CME activities planned according to the suggested

approach.

“Effective” is a term used to describe an activity that has accom-

plished an intended or desired result or outcome. Something that is

effective is considered to have value. Assessment is the act of judging

something in terms of its worth or value. Assessment can be done at

three levels: the individual physician learner; a single educational

activity; or a program of educational activities that might represent

all of the offerings of an institutional CME provider. At each of these

levels, an educational activity represents an intervention in a situation

to improve it by producing something (outputs) that, when introduced

into the situation, will result in outcomes (competence and perform-

ance) that represent improvements (the desired results), and an

impact (enhanced health status of a group or population of patients).

See Figure 1 for an overview.

A CME activity can produce two potential types of output. The ﬁrst

is declarative knowledge. An example of declarative knowledge

would be a physician knowing what to do, for example, to manage

the complications of type 2 diabetes. After a CME activity, the physi-

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 51

cian would also be able to state what to do, for example, to examine

the feet of all patients with diabetes (Figure 2). The second type

of output is procedural knowledge. An example of procedural

knowledge would be for the physician to know how to manage

the complications of type 2 diabetes. The physician could describe

how to do it. For example, after participating in a CME activity, the

physician should be able to describe how to examine the feet of

patients with diabetes. Furthermore, if a physician learns procedural

knowledge, four types of desired results are possible: two outcomes

(competence and performance) and two impacts (improved health

status of a group or population of patients). To demonstrate a com-

petence outcome, it would be necessary for a physician to show

how he or she would manage complications of type 2 diabetes, for

example, by examining the feet of a patient with diabetes in the

educational setting. To show a performance outcome, a physician

learner would then be expected to demonstrate what he or she has

learned to his or her associates in the job setting, for example by

examining the feet of all the patients with diabetes in the practice.

The impact of a CME activity or program can be ascertained by

measuring the health status of a group or population of patients, for

example decreased rates of amputations for patients with diabetes.

Figure 2. Levels of Physician Learning and Assessment

DOES

Performance

SHOWS HOW

Competence

KNOWS HOW

Procedural Knowledge

KNOWS

Declarative Knowledge

Adapted from Miller

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 52

Moore

When planning to assess a CME activity, it is important to remember

that measurement of competence outcomes should occur in the

educational setting, whereas performance outcomes should be

measured in the practice setting. Impact should be measured in the

practice or community setting. In many cases, it is difﬁcult for a

CME planner to gain access to data or develop instrumentation to

collect objective data. In these cases, data can be obtained through

self report. Self-report data are usually the opinions or perceptions

of individuals and may reﬂect their biases or “socially acceptable”

responses. Observational methods produce more objective data

from systematic observation and recording of those observations.

Data from self-report methods are not scientiﬁcally rigorous, but

they are useful for making judgments about outcomes, especially

if the alternative is no data.

Currently most assessment in CME focuses on outputs: self-reports of

knowledge gained in declarative or perhaps procedural knowledge.

It is difﬁcult to know precisely what percentage of CME assesses

outcomes at the level of competence or performance. The recently

released revised Accreditation Council for Continuing Medical

Education criteria will require assessment of competence, perform-

ance, and health status. Until then, a conservative guess would be less

than 5 percent. Many CME providers do not have access to practice

or community settings where the measurement of performance and

patient health status alluded to in the previous paragraph can be

accomplished. It may be, therefore, that the most realistic outcome

for most providers will be competence, where measurement can

occur in the educational setting.

In the approach described here, a CME activity that is designed to

be effective in accomplishing a competence outcome will provide

learners with an opportunity to practice the skill in which compe-

tence is desired and receive feedback on their level of competence.

While practice and feedback are considered “formative assessment”

for the purposes of planning opportunities for learning, they could

also be considered “summative assessment” for the purposes of

determining if desired results (outcomes) have been attained.

One approach that would create the most realistic setting for a

CME activity to accomplish competence outcomes would be to use

simulated and standardized patients. CME faculty could observe

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 53

physician learners practicing the skill (cognitive, interpersonal, or

psychomotor) with simulated patients and provide feedback to them

as formative assessment to help them understand the level of com-

petence they were functioning at and what they needed to do to

reach the target level. The faculty would then observe the learner

in a ﬁnal exercise with a standardized patient to conﬁrm that the

target level has been reached.

Not all CME programs have the resources for simulated and stan-

dardized patients. Planners associated with these programs could

consider observing physician learners in live CME activities working

through authentic case scenarios using an audience response system

(ARS) embedded at key decision or observation points in the case.

This would give CME planners a sense of a group’s level of compe-

tence. To determine an individual learner’s level of competence,

one-on-one sessions could be arranged with CME faculty after group

activities are completed. A similar approach could be used for

Webcasts and video or audio conferences. In a Web-based CME

activity, the interactivity and multimedia capability of the Web

provide an opportunity to develop truly authentic case scenarios.

Information technology associated with the Web can function in a

similar way to an ARS.

Many physicians participate in CME through print enduring materials,

like monographs or journal CME. It is challenging to design a CME

activity to achieve competence outcomes in the print enduring

material format. A blended approach might be one in which learn-

ers would read an article or monograph from which they would be

directed to access a site online, where there would be opportunities

for them to assess their competence level. A less desirable approach

would be to develop a paper-based scenario and multiple choice

test that learners could fax or mail to the CME provider.

The preceding paragraphs described how CME activities might be

designed to facilitate the development and assessment of competence.

If a CME activity designed to accomplish competence outcomes

is effective, two questions should be asked: 1) What should be

measured to determine if a physician is competent; and 2) How

should it be measured?

The example of foot care in diabetic patients will help us under-

stand what needs to be measured to demonstrate that physician

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 54

learners have accomplished the desired level of competence. The

competence outcome would be showing how to examine the feet

of patients with diabetes so that neuropathy and vasculopathy might

be detected and amputation prevented or delayed. According to the

American Diabetes Association, a “competent” provider would be

expected ﬁrst to demonstrate the correct techniques for the physical

examination of the feet. The following would be among the impor-

tant items to be checked: pulse; skin temperature; skin appearance

(ulcerations and/or infections); sensation to vibration or slight pain;

proprioception (sense of body position); muscle strength; and

presence of abnormalities (eg, bunions). Second, the learners would

be expected to detect, based on observations, if neuropathy or

vasculopathy is present and show how they would conﬁrm a diagnosis.

Finally, the learners would be expected to develop a management

plan that included follow-up.

An observation checklist, adapted

for each of the settings described here, might be the best way to

collect the data.

Concluding Thoughts

This paper has reviewed a wide range of literature on physician

learning, instructional design, and assessment. Physicians engage in

learning in response to problems and opportunities for improving

their practices. A five-stage learning process was developed to

describe the efforts of physicians to address problems encountered

in practice, understand what they need to learn to address those

problems, learn what they need to know and do, experiment with

it, and incorporate it into practice. An approach to instructional

design was proposed as a way to help physicians learn by determining

what content and which instructional methods are most appropriate

for the questions they have at each stage of their learning. The

predisposing-enabling-reinforcing model is one approach to organize

learning activities. An approach to assessment in educational activi-

ties designed to achieve competence outcomes was suggested in

which a summative assessment grew out of formative assessment

that was designed as part of the learning experience.

Recently published work recognizes that CME, no matter how well

planned, cannot produce changes in physician performance and

patient health status by itself.

Rather, as the research in CME and

other ﬁelds has matured, it has become clear that there are as many

approaches to deal with the many forces acting on physicians and

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 55

patients in and around the clinical encounter as there are forces,

and maybe more. The hope of many people in many ﬁelds is for

collaboration among the proponents and implementers of these

approaches. Richard Grol and his colleagues have made an important

start by developing a taxonomy in which they categorize these

approaches into two major groups: those that emphasize internal

processes and those that emphasize external inﬂuences, reﬂecting

assumptions about human behavior and how groups or organiza-

tions function.

Approaches that emphasize internal processes

include educational, epidemiological, and marketing approaches.

Approaches that emphasize the inﬂuence of external approaches

include external inﬂuences, social interaction, managerial approach,

and control and compulsion. In addition, theory-driven evaluation,

using the logic model,

82,83

and the newly emerging intervention

science may provide the tools to understand how the collaboration

will work (http://www.implementationscience.com/).

REFERENCES

1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the

twenty-ﬁrst Century. Washington, DC: The National Academy Press; 2001.

2. Institute of Medicine. Improving the Quality of Healthcare for Mental and

Substance-use Conditions. Washington, DC: The National Academy Press; 2006.

3. Grol R, Wensing M, Eccles M. Improving Patient Care: The Implementation of

Change in Clinical Practice. Edinburgh: Elsevier; 2005.

4. Mazmanian PE, Davis DA. Continuing medical education and the physician as a

learner: guide to the evidence. JAMA. 2002; 288:1057–1060.

5. Mansouri M, Lockyer J. A meta-analysis of CME effectiveness. J Contin Educ Health

Prof. 2007;27:6–15.

6. Marinopoulos SS, Dorman T, Ratanawongsa N, et al. Effectiveness of Continuing

Medical Education. Rockville MD: Agency for Healthcare Research and Quality; 2007.

7. Flavell JH. Stage related properties of cognitive development. Cognitive Psychology.

2007;2:421–453.

8. Lewin K. Field Theory in Social Science. New York: Harper and Row; 1951.

9. Tough A. The Adult’s Learning Projects. Toronto: Ontario Institute for the Study of

Education; 1971.

10. Tough A. Intentional Changes: A Fresh Approach to Helping People Change.

Chicago: Follett Publishing Company; 1982.

11. Tough A. Major learning efforts: recent research and future directions. Adult

Education Quarterly. 1978;28:250-63.

12. Knowles MS. The Adult Learner: A Neglected Species, 3rd ed. Houston: Gulf

Publishing; 1984.

13. Knox AB. How Adults Learn. San Francisco: Jossey-Bass; 1986.

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 56

14. Hirsch GB, Levine R, Miller RL. Using system dynamics modeling to understand the

impact of social change initiatives. Am J Community Psychol. 2007;39:239–253.

15. Geertsma RH, Parker RC, Whitbourne SK. How physicians view the process of

change in their practice behavior. J Med Educ. 1982;57:752–761.

16. Schon D. The Reﬂective Practitioner: How Professionals Think in Action. New York:

Basic Books; 1983.

17. Means RP. How family physicians use information sources. In Green JS, Grosswald

SJ, Suter E, Walthall DB, editors. Continuing Education for the Health Professions.

San Francisco: Jossey-Bass; 1984:72–86.

18. Putnam RW, Campbell D. Competence. In: Fox RD, Mazmanian PM, Putnam RW, edi-

tors. Changing and Learning in the Lives of Physicians.New York: Praeger;

1989:79–97.

19. Garcia R, Newsom R. Learning and change among emergency physicians. J Contin

Educ Health Prof. 1996;16:33–41.

20. Pathman DE, Konrad TR, Freed GL, Freeman VA, Koch GG. The awareness-to-adher-

ence model of the steps to clinical guideline compliance. The case of pediatric vac-

cine recommendations. Med Care. 1996;34:873–889.

21. Slotnick HB. How physicians learn: physicians’ self-directed learning episodes. Acad

Med. 1999;74:1106–1117.

22. Slotnick HB. Physicians’ learning strategies. Chest. 2000;118(2 Suppl):18S–23S.

23. Rogers EM. Diffusion of Innovations, 5th ed. New York: Free Press; 1995.

24. Lockyer J, Mazmanian PE, Moore DE Jr, Harrison A, Knox AB. The adoption of inno-

vation. In: Davis DA, editor, Physician as Learner: Linking Research to

Practice.Chicago: American Medical Association; 1994.

25. Havelock RG. Planning for Innovation through Dissemination and Utilization of

Knowledge, 6th ed. Ann Arbor: University of Michigan, Institute for Social Research;

1976.

26. Havelock RG. The Change Agent’s Guide to Innovation in Education. Englewood

Cliffs, NJ: Educational Technology Publications; 1973.

27. Prochaska JO, Velicer WF The transtheoretical model of health behavior change. Am

J Health Promot. 1997;12:38–48.

28. Parker K, Parikh SV. Applying Prochaska’s model of change to needs assessment,

programme planning and outcome measurement. J Eval Clin Pract. 2001; 7:365–371.

29. Cohen SJ, Halvorsen HW, Gosselink CA. Changing physician behavior to improve

disease prevention. Prev Med. 1994;23:284–291.

30. Kolb DA Experiential Learning: Experience as the Source of Learning and

Development. Englewood Cliffs, NJ: Prentice Hall; 1984.

31. Festinger L. A Theory of Cognitive Dissonance. Stanford CA: Stanford University

Press; 1957.

32. Knox AB. Proﬁciency theory of adult learning. Contemporary Educational

Psychology. 1980;5:378–404.

33. Fox RD. Discrepancy analysis in continuing medical education. Mobius.

1983;3:37–44.

34. Hunt D.E. Teachers’ adaptation: ‘Reading’ and ‘ﬂexing’ to students. Journal of

Teacher Education. 1976;27:268–275.

35. Cordes DL. Relationship of motivation to learning. In: Green JS, Grosswald SJ, Suter E,

Walthall DB III, editors. Continuing Education for the Health Professions; 1984:52–71.

36. Cross KP. Adults as Learners. San Francisco: Jossey-Bass; 1981.

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 57

37. Gorman PN, Helfand M. Information seeking in primary care: how physicians

choose which clinical questions to pursue and which to leave unanswered. Med

Decis Making. 1995;15:113–119.

38. Moore DE Jr, Knox AB, Bennett NL, Kristofco RE. Factors affecting physician deci-

sion-making to participate in formal CME activities. In: Davis DA, editor. Physician

Learner: Linking Research to Practice. Chicago: American Medical Association;

1994:219–240.

39. Slotnick HB. How doctors learn: physicians’ self-directed learning episodes. Acad

Med. 1999;74:1106–1117.

40. Florance V. Medical knowledge for clinical problem solving: a structural analysis of

clinical questions. Bull Med Libr Assoc. 1992;80:142–149.

41. Fox RD, Mazmanian PE, Putnam RW. Changing and Learning in the Lives of

Physicians. New York: Praeger; 1989.

42. Mann KV, Ribble J. The role of motivation in self-directed learning. In: Davis DA, editor.

Physician as Learner: Linking Research to Practice. Chicago: American Medical

Association; 1994:69–88.

43. Fox RD. New research agendas for CME: organizing principles for the study of self-

directed curricula for change. J Contin Educ Health Prof. 1991;11:155–167.

44. Jennett P, Jones D, Mast T, Egan KL, Hotvedt M. The characteristics of self-directed

learning. In: Davis DA, editor. Physician as Learner: Linking Research to Practice.

Chicago: American Medical Association; 1994:47–66.

45. Campbell CM, Parboosingh JP, Gondocz ST, et al. Study of physicians’ use of a software

program to create a portfolio of their self-directed learning. Acad Med.

1996;71:S49–S51.

46. Parboosingh JP. The maintenance of competence (MOCOMP) program. Can J

Cardiol. 1993;9:695–697.

47. Parboosingh JP. The maintenance of competence (MOCOMP) program. Can J Surg.

1993;36:29–32.

48. Wilkinson TJ, Challis M, Hobma SO, et al. The use of portfolios for assessment of the

competence and performance of doctors in practice. Med Educ. 2002;36:918–924.

49. Dornan T, Carroll C, Parboosingh JP. An electronic learning portfolio for reﬂective-

continuing professional development. Med Educ. 2002;36:767–769.

50. Miller SH. American Board of Medical Specialties and repositioning for excellence in

lifelong learning: maintenance of certiﬁcation. J Contin Educ Health Prof.

2005;25:151–156.

51. Slotnick HB, Harris TR, Antonenko DR. Changes in learning-resource use across

physicians’ learning episodes. Bull Med Libr Assoc. 2001;89:194–203.

52. Fox RD, Rankin R, Costie KA, Parboosingh JP, Smith E. Learning and the adoption of

innovations among Canadian radiologists. J Contin Educ Health Prof.

1997;17:173–187.

53. Slotnick HB. Physicians’ learning strategies. Chest. 2000;118(2 Suppl):18S–23S.

54. Slotnick HB. How doctors learn: education and learning across the medical-school-

to-practice trajectory. Acad Med. 2001;76:1013–1026.

55. Slotnick HB. How doctors know when to stop learning. Med Teach.

2000;22:189–196.

56. Slotnick HB, Razskowski RR, Lichtenauer D. Rethinking continuing medical educa-

tion. J Contin Educ Health Prof. 1995;15:8–22.

57. Gruppen LD. Physician information seeking: Improving relevance through research.

Bull Med Libr Assoc. 1990;78:165–172.

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 58

58. Haug JD. Physicians’ preferences for information sources: a meta-analytic study. Bull

Med Libr Assoc. 1997;85:223–232.

59. Verhoeven AA, Boerma EJ, Meyboom-de Jong B. Use of information sources by fami-

ly physicians: a literature survey. Bull Med Libr Assoc. 1995;83:85–90.

60. Thompson ML. Characteristics of information resources preferred by primary care

physicians. Bull Med Libr Assoc. 1997;85:187–192.

61. Brown JS, Collins A, Duguid P. Situated cognition and the culture of learning.

Educational Researcher. 1989;18:32–42.

62. Bransford JD, Brown AL, Cocking RR. How People Learn: Brain, Mind, Experience,

and School. Washington, DC: National Academy Press; 1999.

63. Haidet P, Morgan RO, O’Malley KO, Moran BJ, Richards BF. A controlled trial of

active versus passive learning strategies in a large group setting. Adv Health Sci

Educ. 2004;9:15–27.

64. Maslow AH. Toward a Psychology of Being, 3rd ed. New York: Wiley; 1998.

65. Davis DA, Thomson MA, Oxman AB, Haynes RB. Evidence for the effectiveness of

CME. A review of 50 randomized controlled trials. JAMA. 1992;268:1111–1117.

66. Davis D, Thomson M, Oxman A, Haynes R. Changing physician performance. a sys-

tematic review of the effect of continuing medical education strategies. JAMA. 1995;

274:700–705.

67. Thomson MA, Freemantle N, Wolf F, Davis DA, Oxman AB. Educational meetings,

workshops, preceptorships to improve the practice of health professionals and

healthcare outcomes. The Cochrane Library 2001;1.

68. Knowles MS. The Modern Practice of Adult Education: Andragogy versus Pedagogy.

New York: Association Press; 1970.

69. Knowles MS. The Modern Practice of Adult Education: From Pedagogy to

Andragogy. 2nd ed. New York: Cambridge Books; 1980.

70. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance:

a systematic review of the effect of continuing medical education strategies. JAMA.

1995; 274:700–705.

71. Davis DA, Thomson MA, Oxman AD, Haynes RB. Evidence for the effectiveness of

CME. A review of 50 randomized controlled trials. JAMA. 1992;268:1111–1117.

72. Green LW, Kreuter MW. Health Promotion Planning: An Educational and

Environmental Approach. Mountain View, CA: Mayﬁeld Publishing Company; 1991.

73. Mann KV, Lindsay EA, Putnam RW, Davis D.A. Increasing physician involvment in

cholesterol lowering practices: the role of knowledge, attitudes and perceptions.

Adv Health Sci Educ. 1997;2:237–253.

74. White MI. Toward an evidence-informed, theory-driven model for continuing med-

ical education. The University of British Columbia; 2003.

75. Cantillon P, Jones R. Does continuing medical education in general practice make a

difference? BMJ. 1999;318:1276–1279.

76. Gobet F, Lane PC, Croker S. Chunking mechanisms in human learning. Trends Cogn

Sci. 2001;5:236–243.

77. Mann KV. The role of educational theory in continuing medical education: Has it

helped us? J Contin Educ Health Prof. 2004;24:S22–S30.

78. Merrill DM. Instructional Design Theory. Englewood Cliffs, NJ: Educational

Technology Publishers; 1994.

79. Miller GE. The assessment of clinical skills/competence/performance. Acad Med.

1990; 65(9 Suppl):S63–S67.

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 59

80. Boike AM, Hall JO. A practical guide for examining and treating the diabetic foot.

Cleve Clin J Med. 2002;69:342–348.

81. Gilbody S, Whitty P, Grimshaw J, Thomas R. Educational and organizational interven-

tions to improve the management of depression in primary care: a systematic review.

JAMA. 2003;289:3145–3151.

82. Chen HT. Theory-drive Evaluations. Newbury Park. CA: Sage Publishers; 1990.

83. Chen HT. Practical Program Evaluation: Assessing and Improving Planning,

Implementation, and Effectiveness. Thousand Oaks, CA: Sage Publishing; 2005.

APPENDIX A:

How Nurses and Pharmacists Learn

At the time of this writing, I have not found an exact replica of the

model developed in this monograph for physicians in the nursing

or pharmacy literature. CME has the largest and most robust body

of evidence examining professional learning and effectiveness of

programming. However, I have found interesting and intriguing

parallels in the nursing literature.

Almost all the articles I reviewed mentioned that nurses function in

a complex healthcare environment where social, technological, and

medical changes present them with challenges and that self-directed

learning plays a vital role in ensuring that they respond to these

challenges in appropriate ways.

In a small qualitative study (n=24), Daley explored the different

ways that novice and expert nurses learn. The learning of expert

nurses was more “self-directed;” the learning of novice nurses was

more “other-directed.” Daley suggested that as nurses become more

“expert,” their meta-cognition skills become more developed, that

is, they begin to understand better how they learn.

While not stated

explicitly in the study that Daley reported, the need reduction and

integration into work approach described in this monograph is usually

associated with self-directed learning projects.

It may be that nurses

follow the same five stage process as physicians, but Daley’s study

provides an interesting perspective on the differences between

novice and expert practitioners.

Scanlon and Weir suggest that reflection and self-directed learning can

be facilitated by clinical supervisors who focus on the developmental

needs of nurses rather than disciplinary requirements established by

administration. This is necessary because, according to Scanlon and

How Physicians Learn and How to Design Learning Experiences for Them

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 60

Weir, the culture of nursing practice for the most part has become

routinized and automatic. A clinical supervisor, however, could develop

a more reflective nursing practice by helping nurses“reflect-on-practice,”

which they believe would lead eventually to “reflection-in-practice”

becoming part of how a nurse learns from work.

Johns suggested that clinical supervisors could help nurses frame

their learning through structured reflection using Carper’s “funda-

mental ways of knowing in nursing” as a heuristic device.

Carper’s

fundamental ways are empirical, personal, ethical, and aesthetic

knowing.

Empirical knowledge is systematically organized into

general laws and rules that describe, explain, and predict clinical

phenomena. Personal knowing is knowing oneself in the clinical

environment. Knowing of self involves three inter-related factors:

recognizing feelings and prejudices; managing feelings and prejudices

in order to respond properly; and managing anxiety and sustaining

the self. Ethical knowing is knowing what is right or wrong and

being committed to act on this basis. Aesthetic knowing involves

perceiving the nature of a clinical situation, interpreting this infor-

mation, envisioning desired outcomes, and subsequently determining

if the outcomes were effectively achieved. According to Carper, the

aesthetic way of knowing is the core way of knowing in practice,

informed by the empirical, personal, and ethical.

This framework resembles the first stage of the stage model described

in this monograph for physicians, recognizing the need to learn. Both

Scanlon and Weir and Johns state that the purpose of reflecting is

to identify, confront, and resolve contradictions between what practi-

tioners aim to achieve and actual results (cognitive dissonance), with

the intent to achieve more effective work and desirable outcomes. In

their review of literature on continuing nursing education, Furze and

Pearcey state that several studies have found that the primary reasons

for nursing participation in continuing nursing education are to

improve professional knowledge and skills and increase competence.

Because this reflecting occurs in the context of an interaction between

a clinical supervisor and a nurse, learning to reduce cognitive

dissonance can begin almost immediately.

In general, Daley found in her study that that novice nurses tend to

learn through more formal mechanisms, including review of policy

or procedures, attendance at continuing education programs, and

reading of journals. Experts seem to use more informal approaches,

Moore

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 61

such as consulting with peers and other healthcare professionals.

Novices appear to learn through concept formation while experts

seem to construct a knowledge base for themselves in the context

ßof their practice.

Studies in nursing education suggest that Kolb’s cycle of learning

(concrete experience, reflective observation, abstract conceptualiza-

tion, active experimentation) is a valid and useful model to

conceptualize learning in nurses.

I have not been able to locate articles at the time of this writing that

contain overwhelming evidence that nurses pursue learning in a

way that resembles the five-stage process described in this mono-

graph. I am convinced, however, by the information summarized

above, along with the information in the studies describing how

people in general learn (Table 1 in main text), that nurses generally

follow the same stages.

There is less explicit evidence in pharmacy. While there is a sense

that pharmacists follow a similar stage process, there are no studies

that specifically describe the steps.

Appendix A

REFERENCES

1. O’Shea E. Self-directed learning in nurse education: a review of the literature.

J Adv Nurs. 2003;42:62–70.

2. Daley BI. Novice to expert: an exploration of how professionals learn. Adult

Education Quarterly. 1999; 49:133–147.

3. Scanlon C, Weir WS. Learning from practice? Mental health nurses’ perceptions and

experiences of clinical supervision. J Adv Nurs. 1997;26:295–303.

4. Johns C. Framing lessons through reflection within Carper’s fundamental ways of

knowing in nursing. J Adv Nurs. 1995;22:226–234.

5. Carper P. Fundamental ways of knowing in nursing. Advances in Nurs Science.

1978;1:13–23.

6. Furze G, Pearcey P. Continuing education in nursing: a review of the literature.

J Adv Nurs. 1999;29:355–363.

7. Kolb DA, Boyatzis RE, Mainemelis C. Experiential learning theory: previous research

and new directions. In: Sternberg RJ, Zhang LF, editors. Perspectives on Cognitive,

Learning and Thinking Styles. NJ: Lawrence Erlbaum; 2000.

8. Fjortoft NF. Learning outcomes and behavioral changes with a pharmacy CPE

program. Am J Pharm Educ. 2006;70:1–6.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 62

Transforming Continuing Medical Education

Through Maintenance of Certiﬁcation

F. Daniel Duffy, M.D.

American Board of Internal Medicine

Competency-based Education and Evaluation

Over the past 15 years, ideas about the qualities and competencies

of physicians have changed. The Association of American Medical

Colleges’ Medical School Objectives Project (MSOP), the CanMEDS

2005 Physician Competency Framework reports, the American

Medical Association Council on Ethical and Judicial Affairs reports,

and the UK General Medical Council’s Good Medical Practice reﬂect

the medical profession’s search to describe essential attributes of

physicians.

Building upon these initiatives, the Accreditation Council for Graduate

Medical Education (ACGME) revolutionized residency education

and evaluation through its Outcomes Project in Graduate Medical

Education. Their aims were to improve GME by encouraging programs

to measure their educational outcomes based on the competencies

of their graduates and to accredit programs based on their use of

these measures.

To establish a framework for physician competence,

ACGME solicited the help of the larger medical community. On the

basis of this feedback, they identiﬁed six core physician competen-

cies: medical knowledge, professionalism, communications and

interpersonal skills, patient care, systems-based practice, and practice-

based learning and improvement.

At the same time, the American Board of Medical Specialists (ABMS)

revised its policy for member boards’ certiﬁcation and recertiﬁcation

through a new program called Maintenance of Certiﬁcation (MOC).

Adopting the ACGME framework, the ABMS boards issued time-

limited certiﬁcation following residency and required physicians to

maintain certiﬁcation through a four-part program. In MOC physi-

cians provide their boards with the following documentation: Part-1)

evidence of professional standing (unrestricted medical license and

appropriate staff credentials); Part-2) evidence of lifelong learning

and periodic self-assessment of knowledge and skills; Part-3) evi-

dence of cognitive expertise by passing a secure examination; and

Part-4) assessment of performance and improvement in practice.

ABMS initiated this program based on growing evidence that many

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 63

Transforming Continuing Medical Education Through Maintenance of Certiﬁcation

physicians fail to remain current in their practices.

Likewise, patients

expect their physicians to keep up with new developments in the

ﬁeld and would welcome more frequent evaluation by medical

boards and other peer groups to achieve this end.

The idea of demonstrating continuing competence has spread. The

Federation of State Medical Boards (FSMB) established the Physician

Accountability for Physician Competence Initiative to engage the

medical community in a dialogue about the future of healthcare in

the United States, with the ultimate goal of answering the following

question: How does the medical profession identify, measure, and

evaluate the ongoing competence of its members to assure the public

of its commitment to accountability?

One of the outcomes of this

initiative is the US version of Good Medical Practice, which is mod-

eled after a UK General Medical Council document and organized

according to the competencies framework. The FSMB, Osteopathic

Boards, National Board of Medical Examiners, American Medical

Association, Association of American Medical Colleges, ACGME,

ABMS, and other groups are contributing to this work in progress.

The Joint Commission issued hospital accreditation standards for

granting privileges and evaluating physician performance. This

evidence-based process goes beyond review of technical competence

and includes an ongoing evaluation of the six core competencies.

As with MOC, periodic evaluation of performance in practice is

incorporated into the process.

The adoption of a common framework for educating and evaluating

physician competencies evolved within a culture of intense public

scrutiny of the quality of medical care in the United States.

8,9

Many

authors have addressed problems with healthcare safety and

quality.

10,11

Others have proposed strategies for improvement in

these areas. Examples of the latter include the Six Sigma or Lean

Production methods adopted by other industries.

Improvement

science has been introduced into medical care and has begun to

transform the quality agenda from one of searching for and elimi-

nating poor performance into one for overall quality improvement.

MOC embraced this approach.

In the 1990s, the National League for Nursing established a competency

framework for education and evaluation that included outcome

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 64

Duffy

criteria.

A comprehensive nursing curriculum focused on Competency

Outcomes and Performance Assessment (COPA) was recommended

for development, implementation and assessment of initial and

continuing competence in nursing.

In addition, the American Board

of Nursing Specialties now certiﬁes nurses based on their knowledge,

skills, and experience in over 25 specialties assessed by examination

along with documented evidence of specialized training.

Pharmacists

have a national certifying board that provides specialty certiﬁcation

in nutrition support pharmacy, pharmacotherapy, psychiatric phar-

macy, nuclear pharmacy, and oncology pharmacy.

Most board-

certiﬁed pharmacists and pharmacies appreciate the value of this

specialty certiﬁcation; unlike board-certiﬁed physicians, however,

board-certiﬁed pharmacists are not widely recognized outside or

even within the pharmacy profession.

As of 2007, nothing compa-

rable to MOC existed for either nursing or pharmacy, although most

states require continuing education for license maintenance in both

professions.

The Significance of MOC’s Requirement for Lifelong

Learning, Self-Assessment, and Practice Improvement

The Maintenance of Certiﬁcation concept of lifelong learning and

practice improvement expands the deﬁnition of education from

passively learning new knowledge from experts to actively learning

from measurement and experimentation in one’s own practice.

We learn about new treatments and methods by reading, attending

lectures, participating in skills development workshops, and reﬂecting

on how they apply to us. Guided self-assessment helps physicians

determine their level of understanding of the new information or

skill in performing new techniques and protects us from the self

deception that convinces us we know more than we do.

These

elements are included in MOC Part-2: lifelong learning and periodic

self-assessment. But some of the most important learning occurs

when physicians examine the outcomes of their work and learn

how to change system processes to improve outcomes. This type

of learning is promoted in MOC Part-4: assessment of practice

performance and improvement.

Recognizing the role of active learning in the continuing develop-

ment of physicians, the American Medical Association modiﬁed its

AMA PRA Category 1 Credit

to permit accredited CME providers

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 65

Transforming Continuing Medical Education Through Maintenance of Certiﬁcation

to grant credit for Internet searching and learning, test item writing,

manuscript review, and performance improvement activities, in

addition to live activities (including some committee work), enduring

materials, and journal-based continuing medical education. The

Accreditation Council for Continuing Medical Education (ACCME)

now permits accredited CME providers to provide learning activities

based on performance improvement projects. In structured and

long-term projects, a physician or group of physicians can 1) learn

about speciﬁc performance measures (ie, a mechanism that enables

the learner to quantify the quality of a selected aspect of care by

comparing it to a criterion); 2) retrospectively assess their practice;

3) apply these measures prospectively over a useful interval; 4)

enhance their knowledge or competence as necessary; 5) modify

practice as warranted; and 6) re-evaluate performance.

The Practice

Improvement Module used in the American Board of Internal

Medicine’s MOC program is an example of how these principles can

be applied to learning from practice.

As our concept of physician competencies expanded to include sys-

tems-based practice, our ideas about competence in professionalism

were updated. The American Board of Internal Medicine (ABIM)

Foundation, the American College of Physicians (APC) Foundation,

and the European Federation of Internal Medicine created the

Physician Charter: Medical Professionalism in the New Millennium.

This document reafﬁrms the principles of putting patient welfare

first and respecting patient autonomy; however, it adds a new

principle of social justice. The charter afﬁrms traditional physician

commitments to update scientific knowledge and to maintain

trust by managing conﬂicts of interest, being honest with patients,

maintaining patient conﬁdentiality, and maintaining appropriate

relationships with patients. It adds the physician commitments to

maintain professional competence, improve the quality of care,

improve access to care, and manage ﬁnite resources.

Accountability for continuing competence is a newly articulated

commitment of medical professionalism; however, consumer and

employer groups also are urging physicians to report quality meas-

ures to the public with the expectation that the quality of practice

will improve. This is what happened when public reporting led to

improved quality of hospital care for patients with cardiovascular

diseases.

Many insurers, health plans, the Center for Medicare and

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 66

Medicaid Services (CMS), and self-insured employer groups (eg,

Bridges to Excellence) are enticing physicians to engage in quality

measurement, reporting, and improvement activities through pay-

for-performance programs. For example, the Bridges to Excellence

program rewards physicians who have achieved recognition from

the National Committee on Quality Assessment (NCQA) in diabetes

care and heart stroke prevention care and who have established

high performing medical practice systems.

CMS has launched the

ﬁrst phases of the Physician Quality Reporting Initiative, through

which physicians are encouraged to report additional codes on

claims forms from which performance measures can be calculated.

Physicians may receive additional payment if they report measures

on 80% of patients for whom the measures are applicable.

Integrated Model for Systems-based Practice,

Learning, and Improvement

How does all of this activity link MOC with CME and stimulate a

transformation in physician learning? Figure 1 shows a model for

understanding the interrelationship in the competencies of system-

based practice, practice-based learning and improvement, and

professionalism. It also shows how these qualities are measured and

how CME inﬂuences changes in practice.

Figure 1. Model of System-based Practice, Learning, & Improvement

Duffy

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 67

Transforming Continuing Medical Education Through Maintenance of Certiﬁcation

The model incorporates three interrelated processes: 1) creating

and disseminating new medical knowledge, methods, or technology

(shown in white); 2) translating innovations into practice to produce

measurable improvements in the outcomes of care (gray); and 3)

demonstrating accountability for producing valued medical care to

patients and the healthcare marketplace (black). Through this third

process the medical licensing boards and specialty certifying boards

stimulate improvement and provide a conduit for public accounta-

bility of physician quality.

The process for developing new medical knowledge, products, and

services begins with formulating new hypotheses based on meas-

urement of care. These hypotheses reshape theories of medicine,

which can be tested in scientiﬁc research or in local practice-based

experimentation. Authors report research results in scientiﬁc journals

and publish consensus guidelines that distill these ﬁndings.

Professional societies, educational institutions, and medical staffs

convert the scientiﬁc research and guidelines into “how-to” courses

that help physicians and practice managers redesign systems to

incorporate new knowledge, products, and methods into actual

practice. Empiric evidence demonstrates that the CME most effective

in changing practice outcomes involves “how-to”learning experiences.

Pharmaceutical and device companies are another source of infor-

mation through the materials and activities they offer to physicians

describing new medications, processes, and technologies.

The items in the black boxes in the diagram have little direct inﬂuence

on CME; they represent the objectives for physician accountability,

which can be provided directly to patients through public reporting

and indirectly by reporting performance measures as a part of

maintaining certiﬁcation or obtaining a medical license. Employers

demand that health plans provide evidence of the quality of the

healthcare they are purchasing. Through payment policies, health

plans inﬂuence the quality of care by providing capital needed for

implementing new technologies in practice. Some individual physi-

cians and medical groups actively manage the inner (gray) cycle of

practice-based learning and improvement to redesign systems in

order to eliminate waste, reduce variation, increase reliability, and

generally improve the quality of healthcare.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 68

Duffy

The gray cycle in the center of the model describes physician tasks

in practice-based learning and improvement. The ﬁrst task, acquir-

ing and assessing new knowledge, involves point-of-care learning,

traditional CME, learning from product detailing, and practice-based

learning. Point-of-service learning involves skill in retrieving informa-

tion to solve problems in delivery of care. Traditional CME involves

reading scientiﬁc publications and participating in continuing medical

education courses. Information delivered by pharmaceutical or

device marketing and education divisions provides knowledge

applicable to improving care; however, marketing techniques

unconsciously distort physician judgment when information is

associated with gifts or favors. Therefore physicians must carefully

avoid conﬂicts of interests.

Acquiring and assessing new knowledge

is both the ﬁrst step and the last step in the gray cycle of practice-

based learning.

Moving clockwise through the cycle, the second task is designing

practice processes to apply new knowledge. Physicians may know the

right thing to do but fail to reliably do what they intend. Learning

from practice changes that deﬁciency because it promotes frequent

experimentation and learning from mistakes. Industrial engineers such

as Demming and Juran have developed methods for translating new

knowledge and innovations into business, service, and manufacturing

operations.

The ACGME and ABMS competencies of systems-based

practice and practice-based learning and improvement adapted this

approach to medical care. The model shows the relationship among

medical societies, learning collaboratives, pharmaceutical and device

manufacturers, and local vendors in providing education and infor-

mation to help physicians re-design practice to incorporate new drugs,

products, and technologies. Examples of educational programs

are the ACP’s Closing the Gap project, the American Academy

of Family Practice TransforMed, and the Robert Wood Johnson

Foundation–funded Improving Performance in Practice project.

The third task in the gray cycle is executing/leading change in systems

for delivering care. This task requires the application of practice

management skills, negotiating with vendors, structuring personnel

policy, training staff, and implementing technology. This body of

knowledge and skills challenges traditional approaches to medical

education and evaluation, and it may seem more like business

school than medical education. For many physicians, the “system”

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 69

Transforming Continuing Medical Education Through Maintenance of Certiﬁcation

is an Orwellian organizational monster that can neither be under-

stood nor managed. Traditional ideas about physician autonomy

place value on rugged individualists who work around the system

for the beneﬁt of their patients. The new model of medical profes-

sionalism recognizes the disruption to systematic care of unbridled

physician autonomy and espouses physician competence in systems-

based practice, improvement of care, and just distribution of scarce

medical resources.

Today, physician competence includes capability

to manage or inﬂuence the management of clinical work processes;

develop teamwork habits, communication and interpersonal skills;

and understand the principles of human resources and business

management. Although not all physicians must achieve expertise in

system management, all must understand that system management

is essential for high-quality medical practice.

The fourth component of competence in systems-based learning and

practice improvement is measurement of practice performance.

Quantitative and qualitative measurement of the outcomes and

processes of care is the lynchpin that holds together systems-based

practice, practice-based learning and improvement, and the new

professionalism. Measurement is essential for improvement: “We

improve what we measure.” Through measurement we acquire

knowledge about how practice works and the outcomes it produces;

this knowledge informs re-design of work processes; and re-meas-

urement determines if the change improved care. Measurement leads

to creating new knowledge; by reﬂecting on the meaning of measured

results we generate new hypotheses about the theories of health,

medicine, and medical practice that can be tested in empiric

experiments to advance the theory and the evidence base for

improving patient care. Moreover, measurement is essential for

public accountability. Reporting performance measures to payers,

consumers, and purchasers gives them information they need to

purchase the highest-quality care. Measurement of practice perform-

ance is central to providing the evidence of continuing competence

for maintenance of board certiﬁcation, and the act of reporting is a

powerful motivator to change.

How Is MOC Changing CME?

MOC creates an obvious demand for CME board review courses

and durable study materials to prepare for the secure examination.

Because most boards require passing an examination demonstrating

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 70

Duffy

cognitive expertise every 6 to 10 years, a steady demand exists for

“brush-up” education before the test.

The desire to align the MOC requirement for lifelong learning and

periodic self-assessment with periodic hospital staff re-credentialing

and re-licensure has prompted many boards to build 2-year or 3-

year milestones for self-assessment of knowledge or skills focused

on recent advances in the ﬁeld. Commonly, these CME exercises

contain multiple choice questions appended to durable learning

materials on a particular topic. These modules have been developed

by boards and by medical societies, and they provide guided self-

assessment and focused learning on recent advances. Examples of

self-assessment programs are the ACP MKSAP (Medical Knowledge

Self-Assessment Program), ACCP-SAP, Nephrology-SAP, and American

Board of Family Medicine SAM (Self-Assessment Module).

The internal medicine CME community has developed group learning

sessions in which 10 to 200 physicians work with faculty on a self-

assessment module. Audience response technology makes it possible

to include a display of the participants’ answers to questions, and

the wide variation in answers permits the faculty to clarify the issues

contained in the question and answers. This type of course is fun,

challenging, and informative, and a popular addition to annual

meetings.

The MOC requirement for physicians to assess themselves and

make changes in their practice systems to improve their perform-

ance is arguably the most important innovation driving changes in

CME. As of 2004, few physicians had adopted formal learning and

improvement experiences in their practices.

For family physicians

and internists, completion of the MOC Part-4 (assessment of practice

performance and improvement) was the ﬁrst personal experience

with systematically reviewing medical records to calculate perform-

ance measures, receiving feedback from patient surveys about their

experience of care, and reﬂecting on the practice processes and

systems that produce the care. This was the ﬁrst objective of practice

performance measurement for most, and lower than expected

performance called for action to improve. Some physicians had

difficulty understanding how this activity could help them to learn

based on their more narrow view of learning and certiﬁcation being

a test of their medical knowledge. They could not imagine that

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 71

Transforming Continuing Medical Education Through Maintenance of Certiﬁcation

knowledge gained from examination and experimentation could

improve their practice. Others found the exercise tremendously

informative, were surprised by their performance, and felt motivated

to take remedial action but lacked the knowledge and skill to do so.

Others made simple changes in their practice, such as adding a ﬂow

sheet or changing the tasks medical assistants performed, and they

re-measured to determine if the change led to improvement. Although

they did not necessarily fully understand the theory behind the

“plan, do, study, act cycle,” of quality improvement, these physicians

learned that measurement is necessary for knowing the quality of

practice, and that in measuring quality, they could begin to change

the practice. These physicians acquired deeper knowledge about

their own practice than could be achieved through more traditional

learning experiences.

The ABIM Part-4 MOC exercises are called Practice Improvement

Modules (PIM).

Each PIM focuses on an aspect of practice, such

as diabetes, asthma, preventive cardiology, or comprehensive care.

One PIM allows physicians to use measures provided by health plans

or medical groups. Three PIMs focus on physician and practice

communication skills, and another uses the hospital measures

reported to CMS. All of the PIMs include a self-assessment of the

practice systems, and all guide physicians through reﬂection on the

measures, design, and testing of an idea for improving at least one

measure. Each PIM also provides the opportunity for physicians to

reﬂect on the ways in which learning from a small experiment can

lead to improvements in practice.

After completing their assessment of practice, physicians report

a variety of learning and improvement outcomes. Nearly all are

surprised to ﬁnd that their level of performance was lower than

expected. Many select a particular PIM to validate their anticipated

high performance and are surprised at the room for improvement.

Because the physicians themselves collect the data for calculating

the measures, they believe the results. Many connect the dots

between practice structure and processes and the outcomes of care;

they begin to develop a theory of systems-based practice. They see

the holes in their understanding and the skills they need to change

their systems of care. Some physicians learn the importance of team-

work and begin to understand that the roles and responsibilities of

practice staff can be changed to achieve more reliable care. Many

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 72

Duffy

begin to understand the value of information management, electron-

ic records, and prescribing and test tracking systems. Some commit

to adopting these technologies based on this exercise, and others

have been motivated to activate technologies they already possess

but have not used. For a minority of physicians, completing Part-4

MOC is a frustrating experience: Their conclusion after completing

it was that they must “learn more and work harder.” No response

could be further from the the mark. This type of self-assessment is

about the social learning that is essential for innovations and

improvement in medical practice.

Certifying boards now understand that their MOC programs’ Part-4

innovations are changing the learning experience of physicians.

The exercise engages physicians in measuring performance in patient

care, communications and interpersonal skills, systems-based practice,

and practice-based learning and improvement. But this is the barest

beginning. The next step is a robust CME program to meet the

urgent learning needs uncovered through the self-assessment of

practice performance. A theory of practice needs to grow, and new

learning activities that involve practice teams are needed. Some

early examples of learning collaboratives have been organized

by the Institute for Healthcare Improvement and projects like the

Robert Wood Johnson–funded AAFP, AAP, and ABMS Improving

Performance in Practice.

Although Part-4 MOC was developed to provide guided self-

assessment of practice performance and improvement, the data

obtained also enhance accountability for the quality of performance

to patients, payers, and purchasers of the care. ABMS boards have

developed agreements with multiple insurers and health plans to

use participation in MOC and completion of an assessment of practice

performance as measures of quality for pay-for-performance and

other recognition programs.

Boards who use the data in this way

must add an audit of a sample of physicians to assure the veracity

of the self-reported measures. In pursuit of the goal to reduce

redundancy in performance assessment, the ABMS boards are

working with the FSMB to align MOC with maintenance of licensure

and with the American Hospital Association to help hospitals use

MOC as evidence for meeting the joint requirements for evaluating

physician competencies in their credentialing processes.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 73

Conclusion

First, MOC provides physicians and the medical education community

with objectives and standardized measures of learning and practice

performance that can be used for guided self-assessment and sum-

mative evaluation of the effectiveness of learning outcomes. Moreover,

voluntary participation in MOC sets in motion self-imposed require-

ments for meeting 2-year to 3-year milestones for self-assessment of

knowledge about recent advances and assessment of the outcomes

and improvement of care delivered in the practice.

MOC is driving a new curriculum for physician education at all levels.

Systems-based practice calls for developing information and skills

in teamwork and people management, assuring consistent value

for patients through improving internal systems management, and

negotiating with purchasers and payers of medical care to assure

that we can meet the commitments of the new professionalism.

The self-assessment of practice provides a deeper understanding of

practice systems and the needs of the patients seeking care in the

system. This assessment will reveal the need for more training and

education in communication and interpersonal skills to help patients

from particular cultures and social groups. The performance assess-

ment will guide selection of education in how to change clinical

methods. Instead of focusing education on the curriculum devised

by experts, continuing education will be driven by patient demo-

graphics and patient care needs uncovered by the self-assessment

of individual practices.

Most importantly, MOC provides evidence of physician performance

that can be used to meet our social contract for accountability to the

public. Board certiﬁcation and the MOC milestones provide one way

for physicians to reclaim their role in assuring and improving the

quality of medical care.

Transforming Continuing Medical Education Through Maintenance of Certiﬁcation

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 74

REFERENCES

1. ACGME Outcome Project. Accessible at http://www.acgme.org/Outcome/ (Access

date: September 4, 2007).

2. Baltaden P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and

accreditation in graduate medical education. Health Affairs 2002;21:103–111.

3. Wasserman SI, Kimball, HR, Duffy FD, for the Task Force on Recertiﬁcation.

Recertiﬁcation in internal medicine: a program of continuous professional development.

Ann Intern Med. 2000;133:202–208.

4. Choudhry N, Fletcher R, Soumerai S. Systematic review: The relationship between

clinical experience and quality of healthcare. Ann Intern Med. 2005;142:260–273.

5. Brennan TA, Horowitz RI, Duffy FD, Cassel CK, Goode L, Lipner RS. The role of

physician specialty board certiﬁcation status in the quality movement. JAMA.

2004;292:1038–1043.

6. The Physician Accountability for Physician Competence Initiative Summits I-IV.

Accessible at http://www.innovationlabs.com/summit/. (Access date: September 4,

2007).

7. Joint Commission Requirements. Accessible at

http://www.jointcommission.org/NR/rdonlyres/8AB389E2-412D-49F0-BAC9-

996D7EF098B1/0/audio_conference_043007.pdf (Access date: September 4, 2007).

8. National Healthcare Quality Report 2005. Accessible at

http://www.ahrq.gov/qual/nhqr05/nhqr05.htm (Access date: September 4, 2007).

9. Blumenthal D. The role of physicians in the future of quality management. N Engl J

Med. 1996;335:1328–1331.

10. Institute of Medicine. To Err is Human: Building a Safer Healthcare System.

Washington, DC: The National Academy Press; 2000.

11. McGlynn EA, Asch SM, Adams J, et al. The quality of healthcare delivered to adults in

the United States. N Engl J Med. 2003;348:2635–2645.

12. Committee on Quality of Healthcare in America, Institute of Medicine. Crossing the

Quality Chasm: A New Health System for the 21st Century. Washington, DC:

National Academy Press; 2004.

13. Berwick DM, James B, Coye MJ. The connections between measurement and report-

ing. Med Care. 2002;41(suppl):I30–I38.

14. National League for Nursing (NLN). Criteria and guidelines for the evaluation of bac-

calaureate and higher degree programs in nursing. New York: NLN; 1992.

15. Lenburg CB. Redesigning expectations for initial and continuing competence for con-

temporary nursing practice. OJIN 1999 issues accessible at

http://www.nursingworld.org/ojin (Access date: February 19, 2007).

16. American Board of Nursing Specialties. Accessed at http://nursingcertiﬁcation.org/.

(Access date: September 13, 2007).

17. Gourley DR, Fitzgerald WL Jr, Davis RL. Competency, board certiﬁcation, credential-

ing, and specialization: who beneﬁts? Am J Manag Care. 1997; 3:795–801.

18. Pradel FG, Palumbo FB, Flowers L, Mullins CD, Haines ST, Roffman DS. White paper:

value of specialty certiﬁcation in pharmacy. J Am Pharm Assoc. 2004;44:612–620

19. Duffy FD, Holmboe ES. Self-assessment in lifelong learning and improving perform-

ance in practice: Physician know thyself. JAMA. 2006;296:1137–1139.

Duffy

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 75

20. AMA standards for the PRA. Accessible at

http://www.amaassn.org/ama1/pub/upload/mm/455/pra2006.pdf (Access date:

September 4, 2007).

21. ACCME standards for accredited CME providers. Accessible at

http://www.accme.org/ (Access date: September 4, 2007).

22. Project of the ABIM Foundation, ACP-ASIM Foundation, and European Federation of

Internal Medicine. Medical professionalism in the new millennium: a physician char-

ter. Ann Intern Med. 2002;136:243–246.

23. Steinbrook R. Public report cards—cardiac surgery and beyond. N Engl J Med.

2006;355:1847–1849.

24. Bridges to Excellence Physician Reward Programs. Accessible at

http://www.bridgestoexcellence.org/programs/ (Access date: September 4, 2007).

25. Center for Medicare and Medicaid Services Physician Quality Reporting Initiative.

Accessible at http://www.cms.hhs.gov/pqri/ (Access date: September 4, 2007).

26. Davis D, O’Brien MA, Freemantle N, et al. Impact of formal continuing medical edu-

cation: do conferences, workshops, rounds, and other traditional continuing educa-

tion activities change physician behavior or healthcare outcomes? JAMA.

1999;282:867–874.

27. Brennan TA, Rothman, DJ, Blank L, et al. Health industry practices that create con-

ﬂicts of interest: a policy proposal for academic medical centers. JAMA.

2006;295:429–433.

28. Demming WE. The New Economics for Industry, Government, Education, 2nd ed.

Cambridge, MA: MIT Press; 1994.

29. Cohen JJ, Cruess S, Davidson C. Alliance between society and medicine: the public’s

stake in medical professionalism. JAMA. 2007;298:670–673.

30. Audet AJ, Doty MM, Shamasdin J, Schoenbaum SC. Measure, learn, and improve:

physicians’ involvement in quality improvement: evidence that quality improvement

still has not permeated the professional culture of medicine, although progress is evi-

dent. Health Affairs. 2005;24:843–853.

31. Holmboe ES, Meehan TP, Lynn L, Doyle P, Sherwin T, Duffy FD. The ABIM diabetes

practice improvement module: a new method for self assessment. J Contin Educ

Health Prof. 2006;26:109–119.

32. American Board of Internal Medicine. Health Plans Recognizing Maintenance of

Certiﬁcation. Accessible at http://www.abim.org/moc/recognition_hp.shtm (Access

date: September 4, 2007).

33. Cassel CK, Holmboe ES. Credentialing and public accountability: a central role for

board certiﬁcation. JAMA. 2006;295:939–940.

Transforming Continuing Medical Education Through Maintenance of Certiﬁcation

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 76

DISCUSSION HIGHLIGHTS

Knowledge Development Approaches

CE must demonstrate effectiveness in terms of improving the per-

formance outcomes of health professionals. CE must help clinicians

accomplish practice-based learning and improvement. It should focus

on clinical problems and knowledge that can be used in practice,

using questions that clinicians have in practice to determine content.

If instruction really consists of presentation, example, practice, and

feedback, we do the presentation and example pretty well, but we

don’t give our attendees the opportunity to practice what we’ve told

them they should do, or offer expert consultation and feedback.

At the ABIM, almost 20,000 internists have completed self-assessment

of their practices and practice-based learning, and have begun to

demonstrate some improvement. The uniform comment that people

give back is “This was the best CME I ever took,” and they weren’t

in any CME environment doing it. But about 60 percent of them

also say, “Help. Help me. I don’t know how to improve my

performance.”

The ABMS Boards have provided as many options as possible for

recertiﬁcation in allowing self-directed learning to take place, but they

do not allow the avoidance of self-directed learning to take place.

When the ABIM had voluntary recertiﬁcation, it had a ﬁve percent

uptake. Regulation is important for keeping up. Continuing education

can and should be linked to recertiﬁcation.

An important area of continuing education, especially in the surgical

specialties, is learning and maintaining skills. The American College

of Surgeons has begun accrediting education institutes, which are

simulation centers to provide support for skill acquisition, often in

collaboration with other professionals working in the ORs, ICUs,

and so forth.

The ACCME has injected the phrase, “practice-based learning and

improvement” into our vocabulary. We are talking about a gigantic

culture change among physicians and CME providers, and all health-

care providers as a group, to make CME actually relate to practice.

It would not be out of the question to end every sentence with “and

we are doing this because it’s about improving patient care.” It may

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 77

Discussion Highlights

be obvious, but many CME providers never have made a conscious

association between CME and improving patient care, and a lot of

physicians haven’t made the association, either.

One of the strongest motives for improving patient care through

continuous professional development is the inherent desire of

physicians, indeed of all health professionals, to do a better job,

if only they knew what they were doing and how it compared with

benchmarks. Measuring performance and feeding it back to physi-

cians, irrespective of whatever else we may add on as continuing

education around that, can be a powerful agent for change.

The motivation for continuing education, from the perspective of

the public, has been a deﬁcit model — we must ﬁnd out what our

deﬁcits are and make sure they get plugged so we don’t harm patients.

But continuing education is also about professionalism and the

ability to grow in our professional development. We tend to lose

sight of that because of the pressing nature of clinical care problems.

The problem with our general concept of CME, which is conferences

and random reading, is that it depends on memory.

We need to get away from the idea that it is only about knowledge.

It really is about behavior and our ability to do complex tasks that

we refer to as competencies. These involve knowledge, skills,

attitudes, behaviors, and communication.

Medical practice has two components: doing what you know and

knowing what you do. Performance assessments shouldn’t be the

only thing; knowledge assessment also is essential. But too often,

performance is left out of CE.

Studies show doctors are more likely to change when learning is

initiated by a patient-based clinical question, not by a vague desire

to keep up. Continuing education ought to be learning from what

the patient brings.

The fundamental problem with CME is the total dominance of

courses. The use of hours is a misplaced metric. Also, there is an

over-reliance on numbers of attendees. People judge the effective-

ness of a CE offering by how many show up.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 78

Discussion Highlights

Continuing education is not only about expanding expertise, it’s also

about continuing to have your basic expertise intact.

We should not forget the importance of understanding in medicine.

Technical competence is not enough; plenty of technically competent

surgeons do the wrong operations on the wrong patients.

The current system has two ﬂaws: ﬁrst, lectures are ineffective,

except for passing on new information, and are opened to biased

information, or the perception that information could be biased;

and, second, their “entitlement mentality” signals to the public that

physicians are “for sale.”

The Need for Feedback and Evaluation

It is difﬁcult for physicians to provide feedback to other physicians.

In one example, a physician caring for a patient with HIV made ﬁve

mistakes, yet the physician who identiﬁed the mistakes wondered

if the physician who made the mistakes should be notiﬁed. The

physician was reluctant to say that evidence-based practices had

not been followed. Physicians are not trained early either to give or

receive this kind of feedback. In other professions feedback is part

of the practice. Architects don’t just design and build; they submit

designs to colleagues ﬁrst. How can physicians be trained to expect

this kind of feedback?

Lessons might be learned from other professions. For instance,

pilots are comfortable with the idea that someone will sit with them

on a regular basis to assess their performance. Pilots recognize this

lifelong learning as part of their professional responsibility while

physicians tend to think they are ﬁnished when they complete their

training.

The airlines, in addition to relying on another person for pilot

certification, also rely on computerized tests. Pilots sit in a simulator

with a real pilot and a third person dials in problems. Nothing in

medicine begins to compare. The only thing that even comes close

is simulation in surgery.

Perhaps there needs to be legal protection, an exemption from

“discovery” when mistakes are uncovered during evaluation and

feedback.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 79

Discussion Highlights

In the UK, general practitioners are encouraged to have a peer

mentor who meets with them every six months or so to review

charts. In Ontario, Canada, randomly selected physicians undergo

an audit by a peer physician every seven years. The peer reviewer

goes through 25 or 50 charts, and then writes a review that is shared

with the assessed physician. Based on that, some of the physicians

are then further assessed with low-tech simulations involving

standardized patients.

A group practice experimented with ways to give feedback so

physicians view it as an improvement opportunity, not as a punitive

exercise. With blinded data, everyone tended to migrate toward

the mean. When the data were unblinded, they tended to migrate

to people they respected, regardless of whether that had anything

to do with good quality of care. Now feedback is combined with

recommendations on what the evidence shows is the best practice,

followed with opportunities for the team to ﬁgure out how to do

better.

What group or party determines what outcomes are important?

The physician? The practice? Hospitals? Regulatory boards? There’s

not much opportunity for outside feedback. The ready reliance on

markers, or things that are easily measured, does not necessarily

relate to meaningful patient outcomes.

One way to measure or help competence is with a computer

and simulation, with a different problem each time. The process

depends on actually doing it, getting feedback, and reflective

thinking afterwards.

Sociability, companionship, and collegiality are pretty important in

education, and the opportunity to discuss actual practice data with

colleagues is quite valuable. Hospitals, group practices, specialty

and county societies should be able to offer these opportunities.

Two areas are involved: education and training, and certiﬁcation

and veriﬁcation of skills. The two are related but different. Skill

virtuosity requires thousands of hours of deliberative practice.

It is amazing that we give the practice, but little in the way of

feedback.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 80

Discussion Highlights

Lectures

While conferences and courses should not dominate, they still have

a place. Lectures play an important role in helping people keep up

with what’s new and what’s interesting.

Lectures can be made better, for example with interactivity, case

discussions, and small-group breakouts. Even groups of 5000 can

use these techniques.

Lectures may not be the best way to learn, but the reason so much

money is put into for-proﬁt lectures is that they are marvelous

propaganda and advertisement. In some continuing medical education

courses you get a real expert, someone highly respected, to talk

about the use, maybe off label, of a drug. That’s powerful, but it

may it not be good education. It is, however, good advertisement.

CME providers are allowed to focus on educational objectives,

such as knowledge updates, that can be accomplished by lectures.

There is no speciﬁc requirement that, to be accredited, they spend

a certain percentage of time providing and promoting educational

activities that focus on competence and performance in practice,

as well.

Think of a learning portfolio as a bucket into which you put things

like self-assessment of learning needs, performance measures that

are required, and audits done by peer assessors, as well as compe-

tency exams, maintenance of certiﬁcation and licensure. Where do

lectures ﬁt in?

Academic detailing creates an environment that permits people to

identify, on their own, opportunities to improve without the sense

that change is being imposed.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 81

Internet Continuing Education

Denise Basow, M.D.

UpToDate, Waltham, MA

An increasing number of physicians now use the Internet to obtain

medical information. The number of physicians using the Internet

speciﬁcally for continuing education is also rising. Potential beneﬁts of

continuing education delivered via the Internet include convenience,

ﬂexibility, reduced travel time and expense, multimedia format, and

possibly adaptability to different learning styles.

This review begins with a description of the current landscape for

continuing medical education and the role of the Internet within that

larger context. What follows is a discussion of the ways in which

continuing education delivered via the Internet relates to our current

understanding of adult learning. An additional topic is the current

state of research on the effectiveness of online continuing education

in reaching its goals of increased knowledge and improved patient

outcomes.

Most of the literature that addresses the inﬂuence of the Internet on

continuing education limits its focus to physicians, but various groups

of healthcare professionals have established their own terminology.

For this review the term “continuing education” (abbreviated hence-

forth as CE) refers to the educational activities of all of these groups.

CE Requirements

American Medical Association (AMA) Physician Recognition Award

(PRA) Category 1 credit is awarded by accredited providers to

physicians who participate in educational activities that are certiﬁed

as meeting educational goals set forth by the AMA.

Every US state

requires physicians to earn AMA PRA Category 1 credit for medical

licensure, although the amounts vary by state.

Most medical societies also recognize AMA PRA Category 1 credit

as evidence of an ongoing commitment to learning, although some

specialties have variations. As an example, the American Academy

of Family Physicians (AAFP) requires that their members earn both

“prescribed” and “elective” credit, but members can only use AMA

PRA Category 1 to cover elective credits. Similarly, the American

College of Obstetrics and Gynecology (ACOG) requires that its

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 82

Basow

Fellows earn a number of credits designated as “ACOG Cognates,”

and the American Academy of Pediatrics (AAP) requires speciﬁc

CE activities for members who want to be recognized as Fellows

of the Academy. Most non-physician medical providers, including

nurses, nurse practitioners, physicians’ assistants, and doctors of

osteopathy, also recognize AMA PRA Category 1 credit but require

additional types of CE for maintenance of licensure, society mem-

bership, or recertiﬁcation.

Outside the United States, with the exception of Canada, CE is gen-

erally less well structured, although that is changing. The European

Accreditation Council for Continuing Medical Education, for example,

only began operation in the year 2000. In some countries, CE is

essentially an ethical and moral obligation, while in others it is

required by the profession, insurance, or other bodies involved in

healthcare.

Italy has a well developed system regulated by a gov-

ernment body that encompasses all professions related to healthcare

and not physicians alone. The AMA recently established an agree-

ment with the European Union of Medical Specialties to mutually

recognize CE credit.

Internet Point-of-care CE

The AMA recognizes many activities as eligible for AMA PRA

Category 1 credit.

The newest activity to be recognized is Internet

point-of-care CE, deﬁned as structured, self-directed online learning

on topics relevant to a physician’s practice. To earn Internet point-of-

care credits, physicians must complete a “learning cycle” that includes

reviewing the original clinical question, identifying the relevant

sources of information, and describing the application of these ﬁnd-

ings to practice. The last component of the learning cycle, which has

also been termed “reﬂection,” is based on the idea that reﬂecting

upon one’s clinical practice is an essential component for the devel-

opment of expertise and therefore should be a component of the

educational process.

However, what constitutes reﬂection, and

whether this reﬂective process affects patient outcomes, is not clear.

Frequently, clinicians consult resources that award Internet point-of-

care CE to answer questions that arise during patient encounters.

This type of CE has the potential to be extremely vital, given results

from studies demonstrating that physicians often have important

questions (questions that could affect outcomes) during the patient

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 83

Internet Continuing Education

encounter that are left unanswered.

5–7

In one study, physicians only

pursued answers to 55 percent of questions that arose during the

patient encounter.

Internet point-of-care CE does not describe all CE done over the

Internet. “Live” Internet teleconferences, online case presentations

with questions and answers, and a variety of other online activities

might fall into different categories of CE. The wide variety of activi-

ties that fall under the umbrella of Internet CE makes it difﬁcult to

know what activities investigators are considering when they study

the utility of the Internet to provide CE.

The distinction between Internet point-of-care CE and other types

of CE delivered via the Internet has several implications:

1. In an effort to address the lack of quality standardization of the

plethora of CE activities available over the Internet, the AMA

and AAFP have limited the number of providers of Internet

point-of-care CE; currently this list includes ACP PIER, DynaMed,

UpToDate, and InfoRetriever. However, this list continues to

evolve.

2. The AAFP limits the number of credits that physicians can earn

from Internet point-of-care CE to 20 prescribed credits per year.

3. Although activities that fall into the realm of Internet point-of-

care CE are usually understood to be those that are used during

patient encounters, the documentation process mandated by

the AMA (completion of the learning cycle) has made it difﬁcult

to complete these activities during the course of a busy clini-

cian’s day. While a number of groups, including the AANP, the

AOA, and the AAP, offer credit for the same activities, only the

AMA and AAFP require the potentially limiting documentation

process.

It is too early to fully understand the implications of #3. Some

point-of-care providers have tried to simplify the documentation

process by “preﬁlling” a number of items on the required forms with

information that can be gleaned from the search itself. Although

physicians have not responded negatively to this process, it remains

unclear whether these activities fulﬁll the goals set out by the AMA

for the reﬂective component of the learning cycle.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 84

Basow

Availability and Usage of Internet CE

It is difﬁcult to ascertain the number of websites that deliver Internet

CE. According to one estimate, 300 sites offered more than 16,000

courses and more than 26,000 credits as of December 2006,

but this

is likely an underestimate. Sources for Internet point-of-care CE are

more scarce.

The Physicians’ Preferences in CME Study is the longest running

longitudinal study of its kind, albeit with a small sample size.

For

the 2006 survey, data were collected between September 28 and

November 2. Surveys were mailed to 1200 US-based physicians; 148

usable surveys were returned, for a response rate of 12.5 percent.

These were the key ﬁndings:

— About 26 percent of respondents said they planned to use

Internet point-of-care CE during the next 12 months.

— Of those who used the Internet to obtain CE, Internet point-of-

care CE was most popular (62 percent used Internet point of

care, versus 21 percent using enduring Internet activities).

— Respondents earned the largest percentage of their CE credits

by attending out-of-town meetings, but that amount was only

32 percent, the lowest in the 14-year history of the survey (with

the exception of 1998, when it was also 32 percent). Men were

more likely to travel than women (34 versus 21 percent).

— Internet point-of-care activities were more popular with women

than men (39 versus 20 percent) and with physicians over the

age of 55.

These ﬁndings suggest that Internet CE activities have become more

popular than attendance at out-of-town meetings, and that physi-

cians tend to use Internet point-of-care activities.

These results mirror trends in physicians’ overall use of the Internet

and in the availability of CE on the Internet. In a 2001 AMA survey

of 997 physicians in the United States, 78 percent said they used

the Internet in their practices, compared with 20 percent in 1997.

About half of the physicians said that the Internet had a major

impact on the way they practiced medicine. In a 2003 survey of

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 85

Internet Continuing Education

3347 US physicians, more than half reported daily (23 percent) or

weekly (37 percent) use of the Internet for clinical information; 28

percent reported accessing online CE several times a year.

Similarly, data from the Accreditation Council for Continuing Medical

Eduation suggest substantial growth in physician use of the Internet

for CE between 1998 and 2005. In 1998, 37,879 physicians (0.5 percent

of those receiving CE) accumulated 3436 hours of credit (0.5 percent

of total credits awarded) from Internet CE.

By comparison, in 2005,

over 1.3 million physicians (18 percent) earned almost 37,000 credits

(5 percent) via the internet.

Outcomes of Self-directed Learning

In consideration of how physicians learn and how this learning can

affect change, “optimal” CE has been described as “highly self-

directed with content, learning methods, and learning resources

selected speciﬁcally for the purpose of improving the knowledge,

skill, and attitudes that physicians require in their daily professional

lives that lead to improved patient outcomes”.

Malcolm Knowles,

15,16

a professor of adult education with an interest in how adults learn,

developed a theory of “andragogy,”which emphasizes the following

four points:

— Adults need to be involved in the planning and evaluation of

their learning.

— Experience provides the basis for learning.

— Adults are most interested in learning when it has immediate

relevance to their job or personal life.

— Adult learning is problem centered rather than content oriented.

These observations are consistent with results from CE studies.

The data demonstrate that exclusively didactic types of CE, such

as lectures, do not improve physician performance or patient care,

while interactive and sequenced learning has been associated with

a positive impact.

Internet CE, particularly point-of-care CE, takes advantage of this

concept of self-directed learning. A physician with a question that

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 86

Basow

arises in clinical practice who searches for an answer and then

applies the information to patient care has satisﬁed all four points

cited in the previous text. The search for that answer may occur in a

number of ways, but easy accessibility to online resources makes

the Internet an ideal source for this type of learning.

Although only a few randomized controlled trials have compared

Internet-based CE interventions with traditional CE, a number of

controlled trials have compared online CE with no intervention and

reported changes in nonbehavioral measures, such as knowledge,

attitudes, conﬁdence, and satisfaction, as summarized below:

18–23

— In one study, 643 practicing pathologists from around the world

took a pretutorial quiz on pathologic images of prostate carcinoma

specimens and then viewed an online tutorial with text describing

the Gleason grading system.

The tutorial signiﬁcantly improved

Gleason grading in 15 of 20 specimens; on average there was a

12 percent increase in assigning the correct Gleason score.

— Researchers in a study involving 354 physicians evaluated an

online CE program focusing on identiﬁcation of pigmented

lesions. They assessed the performance of the physicians using a

pretest, a tutorial, and an identical posttest.

Completing the CE

tutorial was associated with signiﬁcant improvements in physi-

cian conﬁdence, correct answers to a knowledge test (52 versus

85 percent correct), and correct answers to a clinical skills test

(81 versus 90 percent correct).

— A randomized crossover trial of physicians, pharmacists,

advanced practice nurses, and dietitians evaluated an Internet

case-based curriculum on herbs and dietary supplements.

Participants were randomly assigned to an immediate interven-

tion group or a waiting list and were surveyed about their

knowledge, conﬁdence, and communication related to herbs

and dietary supplements on enrollment in the study and after

both groups completed the curriculum. Scores improved signiﬁ-

cantly in all three areas for the immediate intervention group

compared with the waiting list group; after the latter group com-

pleted the program the scores in the two groups were similar

and signiﬁcantly improved from baseline.

— A randomized trial of primary care physicians evaluated the

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 87

Internet Continuing Education

effect of four CE modules released at 3-month intervals on

screening for Chlamydia infection.

Physicians completed an

average of 2.4 modules, with each module requiring an average

of 12 minutes to complete. Both groups demonstrated declines in

Chlamydia screening rates during the study period. Nevertheless,

despite this modest intervention, the physicians in the interven-

tion group had signiﬁcantly lower declines in screening

compared with the control group.

A number of studies have demonstrated that Internet-based CE

programs are at least as effective as traditional CE methods for

improving knowledge.

For example, a study of 358 physicians in

British Columbia compared four interventions to improve physicians’

knowledge about the management of workplace injuries.

Physicians

selected whether to receive intervention through an online learning

program, videoconferencing, traditional lectures, or face-to-face

small group outreach programs. The Internet and videoconferencing

programs were as effective as more traditional CE approaches in

terms of enhancing physician knowledge.

The literature examining whether Internet CE leads to behavioral

change or increased knowledge is limited. The only randomized

trial that looked at these outcomes compared an Internet-based CE

intervention that could be completed over multiple sessions over

2 weeks to a single “live,”small-group interactive CE workshop.

The results from the comparison group indicate that this type of

interactive CE affects performance, compared with purely didactic

CE.

The investigators made the following observations:

— Similar to other studies of Internet CE, both interventions

resulted in signiﬁcant and similar knowledge gains.

— The Internet-based CE resulted in objectively measured changes

in behavior and improved adherence to treatment guidelines,

compared with “live” CE.

The authors hypothesized that the behavior change associated with

Internet-based CE could be attributed to several factors, including

completing the activity over multiple sessions rather than one work-

shop and the ability of the physicians to structure their own learning

by moving freely throughout the website.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 88

Basow

Other data on the effect of Internet CE on patient outcomes are lim-

ited to results from observational studies and self-reported physician

behavior. In one study that evaluated a variety of online CE programs

to enhance physician knowledge, a retrospective survey of physicians

participating in online CE found that physicians reported changes in

practice based on CE participation.

Similarly, UpToDate, an online

resource designed to answer clinical questions at the point of care

and a provider of Internet point-of-care CE, has done multiple

subscriber surveys, with results suggesting that physicians change

behavior when they consult UpToDate to answer questions.

These

behaviors range from changes in diagnostic testing and management

decisions to avoiding specialist referrals. In addition, investigators

in an observational study of hospitals that subscribe to UpToDate

found that, compared with nonsubscribing hospitals, hospitals that

used UpToDate had signiﬁcantly fewer adverse events and shorter

duration of patient stays (personal communication). Although the

last report is limited by its observational nature, the results are

supported by the observation that these effects were greater with

increasing use of UpToDate at any given hospital; that is, there was

a “dose effect.”

Summary

Internet use, including use of Internet CE opportunities, is increas-

ing rapidly. Accreditation councils in the United States and

elsewhere and across all healthcare professions recognize this

method of CE as valid. Acquiring CE via the Internet makes sense

based on our understanding of the importance of self-directed learn-

ing. However, although a substantial amount of data suggest that

Internet CE activities enhance knowledge to an extent that is equiva-

lent to improvements with other types of CE, data on whether

Internet CE alters the behavior of clinicians and improves patient

outcomes are scarce. Thus, more studies are necessary to evaluate

the effect of Internet CE on patient outcomes.

Another question is whether Internet point-of care CE leads to

results that are equivalent to those from other types of Internet CE.

This question is particularly important given our knowledge that

physicians often do not answer questions during the patient

encounter despite the availability of point-of-care resources that can

provide the information they need. In addition, although the AMA

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 89

Internet Continuing Education

and such specialty societies as the AAFP have promoted point-of-

care CE opportunities over the past couple of years, potential

obstacles, including the documentation process, may actually limit

the routine use of these activities.

REFERENCES

1. The Physicians Recognition Award and credit system. Information for accredited

providers and physicians (2006 revision). Chicago: American Medical Association,

product number OP412806.

2. Braido F, Popov T, Ansotegui J, et al. Continuing medical education: an international

reality. Allergy. 2005;60:739–742.

3. AMA UEMS/EACCME Credit. Accessible at http://www.ama-

assn.org/ama/pub/category/14348.html (Access date: February 20, 2007).

4. Mamede S, Schmidt HG. The structure of reﬂective practice in medicine. Med Educ.

2004;38:1302–1308.

5. Bennett NL, Casebeer LL, Zheng S, Kristofco R. Information-seeking behaviors and

reﬂective practice. J Cont Med Educ. 2006;26:120–127.

6. Ely JW Osheroff JA, Chambliss ML, et al. Answering physicians’ clinical questions:

obstacles and potential solutions. J Am Med Inform Assoc. 2005;12:217–224.

7. Covell DG, Uman GC, Manning PR. Information needs in ofﬁce practice: Are they

being met? Ann Intern Med. 1985;103:596–599.

8. Sklar, BM. Online continuing medical education. http://www.cmelist.com/list.htm#M

(Access date: March 6,2007).

9. Hosansky, T. CME changes course. Medical Meetings. January 2007, p. 16.

10. American Medical Association. 2002 AMA Study of Physicians’ Use of the World

Wide Web. Chicago: AMA Press; 2002.

11. Bennett NL, Casebeer LL, Kristofco RE, Strasser SM. Physicians’ Internet information-

seeking behaviors. J Cont Educ Health Prof. 2004;24:31–38.

12. Accreditation Council for Continuing Medical Education. ACCME annual report data

1998. Accessible at http://www.accme.org/dir_docs/doc_upload/dc316660-2a48-

46d4-916f-60334f7527ba_uploaddocument.pdf (Access date: March 6, 2007).

13. Accreditation Council for Continuing Medical Education. ACCME annual report data

2005. Accessible at http://www.accme.org/dir_docs/doc_upload/9c795f02-c470-

4ba3-a491-d288be965eff_uploaddocument.pdf (Access date: March 6, 2007).

14. Bennett NL, Davis DA, Easterling WE, et al. Continuing medical education: a new

vision of the professional development of physicians. Acad Med. 2000;75:1167–1172.

15. Knowles, M. The Adult Learner: A Neglected Species, 3rd ed. Houston, TX: Gulf

Publishing; 1984.

16. Andragogy (M. Knowles). Accesible at http://tip.psychology.org/knowles.html

(Access date: March 6, 2007).

17. Davis D, O’Brien MA, Freemantle N, et al. Impact of formal continuing medical

education. Do conferences, workshops, rounds, and other traditional continuing

education activities change physician behavior or healthcare outcomes? JAMA.

1999;282:867–874.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 90

Basow

18. Kronz JD, Silberman MA, Allsbrook WC, Epstein JI. A web-based tutorial improves

practicing pathologists’ Gleason grading images of prostate carcinoma specimens

obtained by needle biopsy: validation of a new medical education paradigm.

Cancer. 2000;89:1818–1823.

19. Harris JM, Salasche SJ, Harris RB. Can internet-based continuing medical education

improve physicians’ skin cancer knowledge and skills? J Gen Intern Med.

2001;16:50–56.

20. Harris JM Jr, Kutob, RM, Surprenant ZJ, et al. Can internet-based education improve

physician conﬁdence in dealing with domestic violence? Fam Med. 2002; 34:287–292.

21. Kemper KJ, Amata-Kynvi A, Sanghavi D, et al. Randomized trial of an internet cur-

riculum on herbs and other dietary supplements for healthcare professionals. Acad

Med. 2002;77:882–889.

22. Allison JJ, Kiefe CI, Wall T, et al. Multicomponent Internet continuing medical educa-

tion to promote Chlamydia screening. Am J Prev Med. 2005;28:285–290.

23. Casebeer L, Andolsek K, Abdolrasulnia M, et al. Evaluation of an online bioterrorism

continuing medical education course. J Contin Educ Health Prof. 2006;26:137–144.

24. Wutoh R, Boren SA, Balas EA, et al. eLearning: A review of internet-based continuing

medical education. J Contin Educ Health Prof. 2004; 24:20–30.

25. Karlinsky H, Dunn C, Clifford B, et al. Workplace injury management: using new

technology to deliver and evaluate physician continuing medical education. J Occup

Rehabil. 2006; 16:719–730.

26. Fordis M, King JE, Ballantyne CM, et al. Comparison of the instructional efﬁcacy of

internet-based CME with live interactive CME workshops: a randomized controlled

trial. JAMA. 2005;294:1043–1051.

27. Curran V, Lockyer J, Sargeant J, Fleet L. Evaluation of learning outcomes in web-

based continuing medical education. Acad Med. 2006; 81:S30–S34.

28. UpToDate Caregroup physician survey, Massachusetts General Hospital (MGH)

and Brigham and Women’s Hospital physician survey, Harvard Vanguard physician

survey, and UpToDate subscriber survey. Accessible at

http://www.uptodate.com/research/index.asp (Access date: March 6, 2007).

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 91

Informatics Skills Needed!

Donald A. B. Lindberg, M.D.

National Library of Medicine

The information technology employed in continuing education

for health professionals cannot reasonably be less revolutionary than

the information technology that is quickly reshaping biomedical

research and clinical treatment. Information technology is also

revolutionizing the provision of scientiﬁc clinical guidance to

patients, families, and the public.

The best biomedical research increasingly shares scientiﬁc results in

publicly free electronic sources, publishes peer-reviewed analytical

results speedily and freely, and complies with governmental and

ethical obligations for voluntary clinical trials, including registration

of trials and public reporting of results. Physicians and healthcare

facilities are increasingly part of regional or national conﬁgurations.

Consequently, learning and continuing education, both for health

science students and for the best advanced practitioners, absolutely

demands understanding of and regular access to computer-based

systems that can provide the following four basic functions:

A. Fairly effortless access to traditional publications in the

peer-reviewed biomedical literature

B. Fairly effortless access to modern interactive biomedical

publications

C. Comprehension of elementary database structure of the

latest genome/phenome whole genome database systems,

along with understanding of the procedures to contribute

ﬁndings for one's patients, and understanding of the rules

for proper access to the accumulating combined wisdom

D. Technical capability for participation in medical informa-

tion systems for disaster management

This all may sound a bit like “overkill.” Please let me illustrate with

some examples of these existing systems:

1. Patients and the public access MedlinePlus.Gov about a million

times each day. They are urged to bring their questions to the

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 92

Lindberg

doctor, and they do! I have found practitioners to be full of praise,

especially for the tutorials that have been designed to increase

one’s understanding of medical diagnoses or procedures. About

25 percent of the use of MedlinePlus is now from physicians.

2. Over 2 percent of each year’s 900,000 new Medline articles are

interactive. That is, the message is contained in video attachments

and/or data tables with which the reader can interact to see “the

data behind the table,” or can learn online to do procedures —

even endoscopic surgical ones.

3. Db Gap is a new system from the National Institutes of Health for

storing and selectively providing genomic and phenomic data

from prospective population studies that include genome-wide

analyses (GWAS). These data will include (after appropriate

individual patient consent) populations such as the National

Heart Lung and Blood Institute Framingham studies and the

Genes and Environment study. Access to the summarized studies

and resulting data will be generally available. Individual patient

results will be available to research team participants and others

authenticated through Institutional Review Boards. These devel-

opments represent a new and powerfully important approach to

top quality medical practice — and to participation in new large

clinical research projects. These studies will supplement reports

from traditional clinical trials.

4. In interpreting both traditional clinical trials results—and espe-

cially the GWAS work — it is essential to have mastered the

rudiments of statistical, especially Bayesian, calculation of signif-

icance. Why? Because most traditional clinical trials are way too

small; that’s why authors present us with force-ﬁt meta-analyses

that combine multiple too-small studies.

The GWAS studies are exciting new probes into the genomic

unknown, but they will demand much care in interpretation.

Otherwise, there is the danger of almost hopelessly underspeci-

ﬁed statistical ﬁshing expeditions: 500,000 SNP (snapshot) ﬁles,

for example, and 400 cases!

5. The details of healthcare professional participation in disaster

management are not yet in their ﬁnal form, but it is already certain

that we must have a prominent role. The communication systems

of the past cry out for improvement, and local and regional

advanced planning, triaging, and team play are essential. So here,

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 93

Informatics Skills Needed

too, old fashioned continuing education aimed at individual

skills and test-passing abilities without care for the larger health-

care organizational framework will no longer be acceptable.

Remarks by David C. Slawson, M.D.

University of Virginia

We got involved in developing an “Information Mastery”

1-3

curriculum

for two reasons. The ﬁrst is that, basically, primary care doctors guess

at answers for about 70 to 80 percent of the information questions

that they have.

The second is that only three factors inﬂuence about 95 percent

of their decision-making.

Number one is what a patient asks for.

Forty percent of the time that patients ask for something by name,

they get it from their doctor. Number two is what the pharmaceutical

reps recommend. And Number three, and this inﬂuences them most

of all, is their local, expert-based continuing medical education. All

three of these factors are almost completely paid for by the pharma-

ceutical industry. So, basically you have a system where doctors are

guessing and what they do is based on what they get from drug

reps and their continuing education, both paid for mostly by industry.

If Oprah Winfrey knew what it was like right now, the public would

be in an uproar.

The last 100 years of medicine have been based on the biomedical

approach, basically breaking the body down into its components

and ﬁguring out how each works. Information Mastery is the exact

opposite. It is based on the probability approach to medicine: What

evidence out there has the highest probability of helping us do the

“right thing” for the patient? The “right thing” means doing more

good than harm, but knowing that you can’t always win. That is

probability. No matter how good you are, you can’t always win. Our

challenge is to ﬁnd the best, most likely probability.

The goal of Information Mastery is to answer 90 percent of physicians’

questions in 30 seconds to 60 seconds. John Ely’s work has shown

that doctors in busy clinical practice won’t look up a question if

they think it will take more than a minute to ﬁnd the answer.

That

means we’ve got to do it in less than a minute, and it must be the

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 94

Slawson

highest-quality information. Quality matters.

Unfortunately, the perception of quality matters more. Our job is to

teach physicians how to tell the difference between true quality and the

perception of quality. The way we do that is to use what’s called the

“Information Mastery Usefulness Equation.” That means the usefulness

of information depends upon three variables: How relevant is that

information, how valid is that information and, inversely, how much

work does it take to get that information. If the best answer takes

ten minutes, it’s too long. If I turn and ask someone who is practicing

right next to me what to do and what he gives me is neither relevant

nor valid, it is a useless answer because of the equation: zero times

anything is still zero and divided by anything is still zero.

We recognize four levels of information mastery proﬁciency. Level

zero is where most clinicians are today, where they do not base

their information decisions on the highest-quality information. Our

goal is to get every physician — and not just physicians, but dentists,

veterinarians, nurses, chiropractors, everyone — to Level 1, which is

where they make decisions based on the highest-quality information

that exists at that point in time. The way to do that, we believe, is

using what we call “specialty-speciﬁc comprehensive coordinated

hunting and forging tools.” A forging tool is basically a team of people

at Level 2, which involves less than one percent of those in each

specialty in each discipline. These people survey all of the journals

and ﬁnd the 10 to 20 important articles each month for their disci-

pline or specialty. We call these important articles POEMs, or

Patient Oriented Evidence that Matters, and we make sure that

everybody gets the POEMs each month. They say, “Hey, this is new

information that’s important. This is the kind of information that has

the potential to change what I do every day.”

The hunting tool basically takes all of that and puts it together in a

database where it’s accessible in less than one minute. Then, for

instance, I can say to somebody, “There’s a new study that says

Riboﬂavin 400 milligrams once a day works better than just about

everything else for migraine headaches at one-tenth the cost.” Five

months from now, a doctor may remember there is a vitamin but

won’t remember Riboﬂavin 400 milligrams. But with the hunting

tool and “MIG-go”— even if the doctor can’t remember whether the

i comes before the a or the a before the i with migraine — at least

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 95

the MIG ﬁnds that article for them. That’s the coordinated tool that

makes it work.

We’ve done work in the U.K., where doctors have been taught to

read the literature critically since the late 1980s. Their data show, for

example, that if there is an article about a particular drug, as time

goes on, use of that drug gets higher and higher. If, at a later time, a

major article in JAMA says that drug should NOT be used, their data

show that the use keeps going up and the article has no effect at all.

The bottom line is that practicing doctors do not have time to read

the literature. The approach must be to ﬁnd another way to be sure

that they get the information and they know it is important.

Figure 1. Inter-relationship Between Relevance of Medical Outcomes

and Validity of Medical Evidence

Everything that we use is based on what we call the X and Y-axis

(Figure 1). The X-axis is how valid the information is, starting with

expert opinion and ending with homogeneous systematic review of

well done randomized controlled trials. The Y-axis is how relevant

is this information. The lowest relevance is what effect does it have

on my blood sugar. The highest relevance is will this information,

or these outcomes, make me live longer or better. The only thing

patients care about, in the end, is am I going to live longer or better,

so the most useful medical information is both valid and relevant.

Remarks

Effect on Patient-

Oriented Outcomes

•

Symptom (drivers license)

•

Functioning (visual loss)

•

Quality of Life (Leg Ulcers,

Amputation)

•

Life Span

Effect on Disease Markers

•

Diabetes (GFR,

Photocoagulation rates)

•

Arthritis (Sed rates, X-rays)

•

Peptic Ulcer (endoscopic

ulcers, etc.)

Effect on Risk Factors

for Disease

•

Improvement in markers

(blood pressure cholesterol,

HBAIC, microalbuminuria)

Highly Controlled Research

•

Randomized Controlled Trials

•

Systematic Reviews

LOE 1b, 1a

Psysiological Research

•

Case reports

•

Observational Studies

LOE 2,3

Uncontrolled

Observations &

Conjecture LOE 4,5

Relevance of Outcome

Validity of Evidence

(LOE)

Disease Oriented Evidence

Valid Patient-

Oriented

Evidence

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 96

We need clinicians to make their decisions based on the highest rel-

evances that exist and use the lowest the least, but realize that over

time those positions may change. And over time we have to keep

them up to date with any changes. We want to get everybody at Level

1. We have training teams and we’re starting with cardiology and

with gastroenterology so that those specialties will have a team to

Level 2 to survey the literature, ﬁnd those important valid “POEMs”

for that specialty, and make sure that they are available to everyone.

As for point of care learning, Michigan State has developed a tool to

provide CME credit. Based on where the doctor looks for an answer

to a question, the prompt will ask, “Do you want CME for this,” and

if the answer is “yes,” then the prompt will ask, “Is this your clinical

question?” Based on whether the doctor looked in the diagnosis

section or the treatment section, the program can actually create the

question and ask, “Is this your question?” If the answer is “yes,” then

the next prompt asks, “What are you going to do now as a result of

this information?” The doctor may answer, “I’m going to change my

diagnosis,” or whatever. At the end it asks, “Do you want to send

CME?” If so, in less than ten seconds, it sends to their accrediting

agency something that says, “This was the question and this was the

reference that was used and this was the impact on practice,” and

they send that report automatically. The AMA has been very excited

about that work as this is the type of point of care CME that can

actually improve practice and patient outcomes.

REFERENCES

1. Shaughnessy AF, Slawson DC, Bennett JH. Becoming an information master: A

guidebook to the medical information jungle. J Fam Pract. 1994;39(5):489–499.

2. Slawson DC, Shaughnessy AF. Teaching evidence based medicine: Should we be

teaching information management instead? Acad Med. 2005;80:685–689.

3. Shaughnessy AF, Slawson DC. Are we providing doctors with the training

and tools for lifelong learning? BMJ. 1999: 319:1280. Accessible at

http://bmj.com/content/full//319/7220/1280 (Access date: February 25, 2008)

4. Prosser H, Almond S, Walley T. Inﬂuences on GPs’ decision to prescribe new

drugs — the importance of who says what. Fam Pract. 2003;20: 61-68.

5. Ely JE. Analysis of questions asked by family doctors regarding patient care. BMJ.

1999;319:358–361.

Slawson

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 97

DISCUSSION HIGHLIGHTS

Three of four components that should be included in practice-linked

CE involve IT: a practice database, so physicians can know what

they are doing; access to information at the time and place of

patient care (point of care); and a reminder system to help avoid

errors of omission. The fourth component of practice-linked CE is

the opportunity to discuss what has been done and why.

We should think in terms of the entire management of the patient,

not just the decision-making of the physician. We should develop a

database that includes every member of the team and that answers

the needs of every professional that is involved with the care of the

patients.

Technical competence is probably the core attribute of a profession.

One of the core capabilities that deﬁnes technical competence in

today’s world is the ability to search and use all resources at your

disposal. You can’t be a competent provider of care unless you use

electronic technologies in real time to manage your patients. You

can’t be a truly competent clinician unless you are fully interactive.

It is part of the practice and part of the delivery system, and the

delivery system owes patients a workforce that can use technology.

Why is it then, that in the U.S. only 10 to 20 percent of primary care

physicians has electronic records where in most of Western Europe

virtually 100 percent of physicians do?

There’s a clear link between the efﬁciency of our educational agenda

and our ability to take advantage of electronic technology. The

electronic revolution will happen in good time, but now about 40

percent of large medical groups and only six percent of solo practi-

tioners have an electronic record. The numbers are going up faster

among groups than individuals, but physicians working in areas like

the Mississippi delta need help to become part of a wired healthcare

system. Many states are doing better in supporting the electronic

infrastructure for continuing education than is the federal government.

The electronic health record is fundamental and whatever the

barriers have been are starting to erode, but too slowly.

Generational differences and cultural variance in learning and rela-

tionships to technology should be considered. I’m not sure people

in the age group 18 to 22 think about the Internet the way older

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 98

Discussion Highlights

people do. I just watch our faculty working with PDAs, as contrasted

with the ways our students work with PDAs night and day. Some

healthcare professionals are going to need more training in informa-

tion and other hands-on activities so it doesn’t seem like a big

project and they can see it can be done in real time.

Doctors should be excited about point of care learning because they

would be getting credit for what they are already doing, with a little

bureaucratic requirement. One payback would be a self-directed

learning report in which they could see all their questions, the topic

areas, the resources they used, what was helpful and what changed

their practice.

Healthcare providers also need to become informed consumers of

their resources. Clinicians must be able to appraise information

resources since they are predigested summaries.

Since physicians tend to consult multiple resources for their ques-

tions, maybe the on-line resources could be centralized for quicker

access. Clinicians don’t have the time or capacity to read multiple

studies and put all the information together. Centralization is going

to be critical because physicians practicing medicine don’t like using

lots of resources. They don’t have the time, so there’s got to be what

amounts to one-stop shopping. The question of which resources

physicians should be using is still up in the air. Also, more study is

needed to determine if the learning during point of care is surface

or deep.

Some physicians have said being able to ﬁnd answers to clinical

questions on the Internet has “put the fun back into medicine.” It is

fun to learn new things and the Internet and point-of-care resources

give them the answers that help them provide better medical care.

Studies show doctors are more likely to change when learning is

initiated by a patient-based clinical question, not by a vague desire

to keep up. Even though AMA now gives credit for that type of

learning, many doctors ﬁnd the documentation requirements too

burdensome. That is something that needs to be worked on.

One reason physicians won’t take the extra step to do point of care

documentation is they don’t need to. They can get credit a million

different ways and do no work at all.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 99

Discussion Highlights

100

It is good to improve the ways people use the Internet to obtain

information, regardless of whether credit is associated with it, but

the credit part could be less bureaucratic.

If you want to get information to people quickly, format is an issue.

Doctors have more access than they realize, and they are not aware

that many resources are free or relatively inexpensive. Look at what

the National Library of Medicine gives for free and there’s also the

possibility of partnering with academic centers.

IT offers a huge opportunity to do lifelong learning and continuing

education, as well as early education, across the professions. We all

have the opportunity to get the same information but we use it in a

different way. Physicians are primarily in private ofﬁces, nurses in

hospitals, but IT can bring them together.

The Internet is changing medical information for patients, their

families and the public. Thirty percent of searching on Medline

Plus is done by consumers. Now patients show up at the doctor’s

ofﬁce with a bunch of printouts, which is completely different from

the way we practiced medicine 40 years ago. Medline Plus has an

information prescription project in which the doctor is encouraged

to write a prescription for the patient to look at Medline Plus for

some information.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 100

101

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:36 PM Page 101

102

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 102

II.

Financing

Continuing Education:

Who, How, and Why

103

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 103

Financial Support of Continuing Education

in the Health Professions

Robert Steinbrook, M.D.

New England Journal of Medicine

Continuing education in the health professions is a multibillion-dollar-

a year industry. In 2006, the total income from continuing medical

education (CME) in the United States for activities presented by

providers accredited by the Accreditation Council for Continuing

Medical Education (ACCME) was $2.4 billion (Figure 1, Table 1).

A decade ago, the total income was less than $1 billion, and the major-

ity of support came from the registration fees of CME participants

or funds from the parent organization of a program’s sponsors. Since

2003, however, the majority of income has come from industry.

Does the commercial funding of continuing education distort medical

practice away from what is best for health professionals and patients?

Should changes be made to the funding and existing safeguards

against ﬁnancial bias and conﬂict of interest? This paper provides

background for the discussion of these and related questions. Although

all sources of ﬁnancial support are discussed, commercial support —

the focus of current controversy — is emphasized.

Figure 1. Income and Expenses for Accredited Continuing

Medical Education in the United States, 1998 –2006

MILLIONS OF DOLLARS

Total Income Total Expense Commercial Registration Fees Advertising and

Support and Other Income Exhibit Income

2500

2000

1500

1000

500

1998 1999 2000 2001 2002 2003 2004 2005 2006

Y E A R

104

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 104

Commercial support for CME quadrupled between 1998 and 2006—

from $302 million to $1.2 billion. Nearly all of this money was from

ﬁrms that manufacture products that are regulated by the US Food

and Drug Administration (FDA). Support doubled from other sources.

Advertising and exhibit income — the ACCME tabulates this sepa-

rately — increased from $126 million to $245 million. Registration

fees and other income increased from $458 million to $940 million.

In 1998, commercial support represented 34.0 percent of total CME

income. In 2003 it represented 54.7 percent. In 2006, it represented

50.3 percent, and the combined for-proﬁt support for CME — com-

Steinbrook

632 $888.5

million

$302

million

$125.9

million

$457.7

million

$842.1

million

5.5%

655 $1.11

billion

$388

million

$148.2

million

$574.6

million

$920.9

million

20.5%

680 $1.27

billion

$467

million

$168.9

million

$635.4

million

$1.05

billion

20.6%

674 $1.39

billion

$569

million

$160.0

million

$665.2

million

$1.18

billion

18.1%

686 $1.60

billion

$746

million

$187.3

million

$662.9

million

$1.33

billion

20.3%

697 $1.77

billion

$971

million

$183.3

million

$620.1

million

$1.54

billion

15.2%

716 $2.05

billion

$1.07

billion

$197.0

million

$784.5

million

$1.61

billion

27.4%

716 $2.25

billion

$1.12

billion

$235.7

million

$899.2

million

$1.72

billion

31.0%

729 $2.38

billion

$1.20

billion

$244.9

million

$940.3

million

$1.82

billion

31.0%

YEAR

1998

1999

2000

2001

2002

2003

2004

2005

2006

Table 1. Income and Expenses for Continuing Medical

Education in the United States, 1998 –2006

ACCME

ccredited

rganizations

Total Income Commercial

upport

Advertising

Exhibit

ncome

Registration

ees & Other

ncome

Total

xpense

Profit

argin

Data are from the 2006 annual report of the Accreditation Council for Continuing Medical Education. Of the 729 ACCME accredited

providers, 718 reported data on total income, 601 on commercial income, 446 on advertising and exhibits, 650 on registration fees and

other income, and 721 on total expense. Data on providers that are accredited by state medical societies are not included (see text).

105

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 105

Financial Support of Continuing Education in the Health Professions

106

mercial support, advertising, and exhibits income — represented

60.5 percent of total income. Although comparable data are not

available for continuing education in nursing and pharmacy, similar-

ities are likely, in part because some programs are accredited for

multiple professions. In 2006, an industry magazine noted in its

annual medical education report: “For continuing education to con-

tinue, commercial funding must remain healthy. The future of CME

depends on it.”

CME is also proﬁtable. Income has grown faster than expenses.

In 1998, the proﬁt margin for accredited providers was 5.5 percent.

In 2006, it was 31.0 percent. The three main types of providers are

physician member organizations, publishing/education companies,

and medical schools. In 2006, their overall CME proﬁt margins were

46.3 percent, 34.7 percent, and 20.7 percent, respectively (Table 2).

Of course, nonproﬁt entities, such as physician member organiza-

tions and medical schools, often use CME revenues to subsidize

other activities.

There is long-standing concern that continuing education is often a

marketing activity that promotes sales of newer and more lucrative

medical products, including their off-label use, notwithstanding the

various safeguards against conﬂict of interest and commercial bias

discussed below.

2,4–9

Patients and payers ultimately pay the bill,

through increased spending on prescription drugs and other prod-

ucts. In August 2002, an industry magazine asked the question,

“What’s [return on investment] got to do with CME?” Noting that

“in the commercial world, large investments of company resources

have to be justiﬁed; there has to be a tangible return,” the article

concluded that marketing professionals “must demand data that

demonstrates the worthiness of their [continuing education] invest-

ment, and they must invest in programs that change physician

practice patterns.”

There are multiple reasons for this concern about the commercial

aspects of CME. First, the public perceives that many gifts and pay-

ments to physicians and other health professionals from industry

amount to bribery or kickbacks, even if they are entirely legal.

Commercial support for CME is just one more physician-industry

relationship. Many of the physicians who are invited to speak at

CME events or to organize them have extensive ﬁnancial relation-

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 106

ships with industry, and drug companies have paid them to give

other presentations. The material presented may overlap with pro-

motional events and CME courses.

The off-label promotion of

gabapentin (Neurotonin), which was originally developed for the

treatment of epilepsy, is a well-documented example of commercial

inﬂuence on continuing education.

Steinbrook

6 $73.5

illion

.7% 0.5% 93.9% $74.4

illion

(

1.3%)

93 $110.6

million

52.4% 4.3% 43.3% $110.9

million

(-0.3%)

14 $2.8

million

9.2% 1.4% 89.4% $7.1

million

(-39.6%)

34 $122.8

million

40.3% 8.2% 51.5% $96.7

million

27.0%

267 $783

million

23.0% 25.5% 51.5% $535.2

million

46.3%

29 $54.2

million

51.5% 1.7% 47.0% $41.3

million

31.4%

154 $818.8

million

75.8% 1.9% 22.3% $608.0

million

34.7%

122 $418.9

million

61.9% 3.2% 34.9% $347.1

million

20.7%

729 $2.38

billion

50.3% 10.3% 39.4% $1.61

billion

27.4%

ORGANIZATION

TYPE

Government/

Military

ospital/Health-

care Delivery

System

Insurance

Company/

anaged Care

Non-profit

(Other)

Non-profit

(Physician

Member

Organization)

Not Classified

Publishing/

Education

Company

School of

Medicine

TOTAL

Table 2. Financial Support of Continuing Medical Education

for Different Types of Organizations, 2006

umber Total

Income

ommercial

Support

dvertising

& Exhibit

Income

egistration

Fees & Other

Income

otal

Expense

rofit

Margin

Data are from the 2006 annual report of the Accreditation Council for Continuing Medical Education. Publishing/Education

Company includes for-profit medical education and communication companies (MECCs). Non-profit (Other) includes foundations,

public health organizations, and voluntary health organizations, such as the American Cancer Society and the American Heart

Association. Non-profit (Physician Membership Organizations) includes specialty societies, the American Medical Association

and other non-specialty societies, and state medical societies. About 85 percent of the organizations in this category are specialty

societies. Financial data are available for overall categories, but not for sub-groups, such as MECCs or state medical societies.

Data on providers that are accredited by state medical societies are not included (see text). Totals may not sum to 100% due

to rounding.

107

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 107

Financial Support of Continuing Education in the Health Professions

108

Second, the drug company sponsorship of continuing education

may lead to an overemphasis on topics that are beneﬁcial to com-

mercial interests — such as programs about drug treatment or the

use of medical devices or diagnostic tests, thereby biasing “the over-

all ‘curriculum’ of topics.”

Too few programs relate to other aspects

of care, regardless of their importance to improving patient care.

Third is the accreditation of medical education and communication

companies (MECCs).

An array of such companies exists, and they

have various lines of business. For example, they may organize and

manage meetings, ﬁnd speakers, assess educational needs, develop

written educational materials, provide public relations services, or

prepare advertising campaigns, among other activities. Some are

primarily education companies; others are primarily communication

companies. In 2006, the 154 accredited publishing and education

companies, a category that includes MECCs, represented about a ﬁfth

of all accredited organizations. Publishing and education companies

receive about three quarters of their CME income from commercial

support, a higher percentage than any other type of organization. In

2006, publishing and education companies had a combined income

of $819 million, or 34.4 percent of all CME income. They had more

income than any other type of organization (Table 2).

Finally, extensive ﬁnancial relationships exist between some other

large providers of CME— professional societies, medical schools, and

academic medical centers — and industry.

5,9,15,16

Industry provides a

substantial part of the budget for many specialty societies; it pays for

national meetings, medical journals, and the development of clinical

practice guidelines.

If commercial support were to cease at a

medical society that was heavily supported by industry, programs

would be cut and staff would lose their jobs. Physicians or others

with ties to industry may run these organizations. Some of the ﬁnancial

relationships are signiﬁcant enough that it may be impossible for

these providers — and their personnel directly involved with CME —

to be independent of industry. Accreditation of MECCs raises the

same issue.

From 2005 to 2007, the U.S. Senate Finance Committee investigated

drug company grants to fund continuing education, including

allegations that grants were being used for improper purposes. The

Committee’s report, released in April 2007, concluded: “It seems

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 108

Steinbrook

109

unlikely that this sophisticated industry would spend such large

sums on an enterprise but for the expectation that the expenditures

will be recouped by increased sales. Press reports and documents

exposed in litigation and enforcement actions conﬁrm these suspi-

cions in some instances. There is also evidence from ACCME that

some accredited CME providers still allow commercial sponsors to

exert improper inﬂuence on educational activities that are supposed

to be independent from commercial interests.”

The report noted that although “major drug companies have adopt-

ed corporate policies that, on their face, do not allow educational

grants to be awarded for unlawful purposes,” the policies “still allow

this industry to walk a ﬁne line between violating rules prohibiting

off-label promotion and awarding grant money in a manner likely

to increase sales of their products, including sales for off-label uses.”

According to the Senate committee, much of the industry funding

for CME follows this pattern. A MECC submits a grant proposal. The

drug company funds “a program on a general topic (eg, treatment of

a speciﬁc condition— and the condition is one for which at least one

of the sponsoring drug company’s products is used), but the speciﬁcs

of the content are determined by the [MECC].” Pharmaceutical

company documents reveal no “explicit agreement that the CME

program will favorably discuss a company product. However, it is

possible that both parties reasonably expect that to be the result.”

Industry Activities and Expenditures on Promotion

Commercial support for continuing education occurs in the context

of spending for the promotion of drugs and other medical products.

Overall expenditures on pharmaceutical promotion in the United

States grew from $11.4 billion in 1996 to $29.9 billion in 2005,

accounting for 14.2 percent and 18.2 percent of sales, respectively.

In 2005, the spending included $4.2 billion for direct-to-consumer

advertising, $18.4 billion for free samples, $6.8 billion for detailing,

and $0.4 billion for journal advertising. For leading therapeutic classes

of drugs, between 7 percent and 24 percent of promotional spend-

ing was for professional meetings and events.

Based on information provided by drug companies, the Senate Finance

Committee estimated that “75 percent of the total educational grant

funding is used to support accredited educational grant programs.”

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 109

Financial Support of Continuing Education in the Health Professions

110

Many of these grants are awarded to MECCs.

A national survey conducted in 2003 and 2004 of physicians in six

specialties found that relationships between physicians and industry

are common and vary according to specialty, practice type, and

professional activities.

For example, 26 percent of respondents

reported receiving reimbursements for admission to CME meetings

(free or subsidized), and 15 percent reported reimbursements for

meeting expenses (eg, travel, food, and lodging, although the ques-

tion did not distinguish between CME events and other meetings);

18 percent reported payments for consulting, and 16 percent report-

ed payments for speaking engagements or for serving on a speakers’

bureau. Cardiologists were signiﬁcantly more likely to receive pay-

ments than anesthesiologists, family practitioners, pediatricians, and

surgeons, but not internists.

Another national survey, conducted in 2006, found that in the past

year 65 percent of clinical departments had received industry funds

to support CME “administered by your department” and that 62

percent of the chairs of these departments had one or more personal

relationships with industry, including 26 percent who had received

“personal compensation (such as honoraria) from industry for

participating as faculty/speaker in CME activities.”

More than half

of the chairs whose departments had relationships with industry felt

the support had “an overall positive effect on their ability to provide

educational offerings.” Few perceived negative effects of either

relationships between departments and industry or department

chairs and industry.

Verispan, an independent medical information company in Morristown,

NJ, collects data about physician meetings and events by surveying

a panel of about 3400 physicians in 19 specialties. Once a month,

the panel members report on all invitations they receive, regardless

of the sponsor or whether they attended the event. Physicians are

compensated for participating. Among other questions, panel mem-

bers are asked if a pharmaceutical company was associated with the

event and if CME credit was offered. Verispan codes events into

eight types: videoconferences, teleconferences, symposia (held in a

convention center for longer than 4 hours with more than 20 atten-

dees or at least 50 attendees), small meetings with a pharmaceutical

representatives (less than 21 attendees), small group meetings (held

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 110

in a restaurant or with less than 21 attendees or shorter than 3 hours

with unspeciﬁed number of attendees), third-party events (conducted

by a moderator), large group meetings (more than 20 attendees or

longer than 2 hours with unspeciﬁed number of attendees), and

other (primarily market research activities). The results are projected

to a “universe” of about 375,000 physicians in the 19 specialties.

To estimate expenditures, Verispan “benchmarks” the cost of different

Steinbrook

Year Events No CME Credit CME Credit Not Specified

2000 313,605 83.6% 13.7% 2.7%

Table 3. Panel A.

Continuing Medical Education Credit for Physician

Meetings and Events for 19 Specialities, 2000 –2006

2001 370,341 86.1% 12.3% 1.6%

2002 402,881 86.3% 12.5% 1.2%

2003 453,481 79.4% 11.2% 9.4%

2004 536,428 86.6% 12.7% 0.7%

2005 511,275 88.6% 10.0% 1.4%

2006 497,470 89.7% 8.8% 1.5%

Year Expenditures No CME Credit CME Credit Not Specified

2000 $1.87 billion 73.4% 24.5% 2.1%

Table 3. Panel B.

Estimated Expenditures in Physician Meetings and

Events for 19 Specialities, 2000 –2006

2001 $2.16 billion 76.7% 21.8% 1.5%

2002 $2.18 billion 75.4% 23.5% 1.1%

2003 $2.44 billion 67.6% 22.2% 10.2%

2004 $2.85 billion 76.0% 23.2% 0.8%

2005 $2.71 billion 78.4% 20.4% 1.2%

2006 $2.64 billion 79.5% 19.1% 1.5%

Data are from Verispan

111

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 111

Financial Support of Continuing Education in the Health Professions

112

activities; a symposium or a large group meeting is assigned a greater

cost ($50,000) than a small group meeting ($5000) or a teleconfer-

ence ($2500).

Between 2000 and 2006, CME credit was available for less than 15

percent of physician meetings and events; these activities accounted

for about one ﬁfth of expenditures (Table 3). In 2006, CME credit

was available at a smaller percentage of meetings and events — an

estimated 8.8 percent — than in prior years, and the activities for

which credit was offered accounted for a smaller percentage of

expenditures.

In 2006, an estimated 497,470 physician meetings and events took

place, with expenditures of $2.64 billion for the Verispan “universe.”

CME credit is most commonly available for symposia and large

group meetings. In 2006, this category accounted for an estimated

2.1 percent of events and 19.2 percent of expenditures. Credit was

available for about two thirds of symposia and large group meet-

ings. By comparison, credit was available for about 7 percent of

small group meetings. Such meetings represented an estimated 81.1

percent of all events and 70.6 percent of expenditures.

In 2006, pharmaceutical companies were associated with an estimated

89.1 percent of physician meetings and events. Of such meetings

and events, CME credit was offered for an estimated 8.9 percent,

not offered for 89.6 percent, and not speciﬁed for 1.5 percent. Of an

estimated 9449 symposia and large group meetings associated with

pharmaceutical companies, CME credit was offered for 67.9 percent,

not offered for 30.9 percent, and not speciﬁed for 1.2 percent. Of

an estimated 649 symposia and large group meetings not associated

with pharmaceutical companies, CME credit was offered for 72.6

percent, not offered for 18.5 percent, and not speciﬁed for 8.9 percent.

Verispan surveys a separate physician panel of about the same size

from the same specialties to gather information about electronic

promotional activities. These include online events (a self-guided

CME program, a symposium, a Web conference), “video detailing”

(real-time online communication with a pharmaceutical sales repre-

sentative), and “virtual detailing” (a self-guided online non-CME

activity).

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 112

Steinbrook

113

In 2006, there were an estimated 3.2 million electronic promotional

activities and expenditures of $329.4 million for the Verispan “universe.”

CME credit was available for about one tenth of electronic promo-

tional activities; all were online events. Online events accounted for

17.3 percent of the activities, and credit was available for about two

ﬁfths of them. Although activities with CME credit accounted for only

about 3 percent of expenditures on electronic promotion, they

accounted for about half of expenditures for online events. An

estimated 79.4 percent of CME events were associated with a phar-

maceutical company and 6.2 percent were not; for 14.4 percent of

events, this information was either not known or not speciﬁed.

Funding for CME

Each year, providers accredited by the ACCME submit information

about their CME programs, including the number and type of activi-

ties, and some ﬁnancial data, including total income, commercial

support, advertising and exhibit income, registration fees and other

income, and total expense. The data are self-reported, based on

terms and deﬁnitions provided by the Council. Although the infor-

mation is not audited, it is considered reliable. The ACCME deﬁnes

commercial support as “ﬁnancial, or in-kind, contributions given

by a commercial interest, which is used to pay all or part of the costs

of a CME activity. Advertising and exhibit income is not considered

commercial support” (emphasis in original source of quote).

“commercial interest’ is “any entity producing, marketing, re-selling,

or distributing healthcare goods or services consumed by, or used

on patients,” according to the expanded deﬁnition adopted in August

2007.

The ACCME compiles the information and publishes an

annual report, most recently for 2006.

Selected ﬁnancial data from recent ACCME annual reports are shown

in Tables 1, 2, and 4. In addition to the data highlighted previously

about overall income, commercial support, and proﬁt margins, and

the income and commercial support for publishing and education

companies, there is noteworthy information:

1. Between 1998 and 2001, income from commercial support

increased by about $90 million a year; from 2001 to 2003 it

increased by about $200 million a year, and from 2003 to 2006 it

increased by about $75 million a year (Table 1).

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 113

2. In 2006, following publishing and education companies, which

received 34.4 percent of total CME income, physician member

organizations received 32.1 percent of total income, and medical

schools received 17.6 percent. Thus, the other types of organiza-

tions combined accounted for only 15.1 percent of CME income

(Table 2).

3. In 2006, following publishing and education companies, which

received 75.8 percent of their CME income from commercial

support, medical schools received 61.9 percent and hospitals/

healthcare delivery systems received 52.4 percent. Physician

member organizations received 25.5 percent of their CME income

from advertising and exhibits, which represented about three

quarters of all advertising and exhibit income for CME. They

received 23.0 percent of their income from commercial support

and 51.5 percent from registration fees and other income. Total

for-proﬁt support for CME — commercial support and advertis-

ing and exhibits income combined — was 77.7 percent for

publishing and education companies, 65.1 percent for medical

schools, and 48.5 percent for physician member organizations.

4. Between 2004 and 2006, overall CME income increased by 16.2

percent to 12.0 percent for commercial support, 24.3 percent for

advertising and exhibits, and 19.9 percent for registration fees

Financial Support of Continuing Education in the Health Professions

Total

Income

Commerecial

Support

Advertising

and Exhibit

Income

Registration

Fees and

other Income

Total

Expense

Organization Type

Hospital/Healthcare

Delivery System

Non-profit (Other)

Non-profit (Physician

Member Organization)

Publishing/Education

Company

School of Medicine

All

+18.8% +10.1% +2.1% +33.8% +23.0%

Table 4. Changes in the Financial Support of Continuing Medical

Education for Five Types of Organizations, 2004 –2006

+24.7% +47.2% +63.2% +7.7% +16.0%

+11.2% +2.0% +13.4% +11.3% +12.8%

+17.3% +17.7% +64.0% +13.3% +15.2%

+7.2% +1.9% +31.5% +16.4% +8.2%

+16.2% +12.0% +24.3% +19.9% +12.9%

Data are from the 2004 and 2006 annual reports of the Accreditation Council for Continuing Medical Education.

Percentages are changes over two years.

114

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 114

Steinbrook

115

and other income (Table 4). Publishing and education companies

increased their income by 17.3 percent.

The ACCME also collects data on providers that are accredited by

state medical societies. In 2006, there were 1684 such providers.

These providers had a total income of $134.5 million and expenses

of $136.4 million, as compared with the $2.38 billion in income and

$1.82 billion in expenses for providers accredited by the ACCME. Of

the income earned by providers accredited by state medical societies,

$39.4 million (29.3 percent) was from commercial support, $10.2

million (7.6 percent) was from advertising and exhibits, and $84.9

million (63.1 percent) was from registration fees and other income.

Federal Laws and Regulations

The FDA issued guidelines for industry-supported scientiﬁc and

educational activities in 1997.

The guidelines speciﬁed that programs

and materials about therapeutic products regulated by the FDA that

are prepared by, or on behalf of, the companies that market the

products are subject to the labeling or advertising provisions of the

Food, Drug, and Cosmetic Act. However, “truly independent and

nonpromotional industry-supported activities” are not subject to

FDA regulation. According to the Senate Finance Committee, “it

is legal for independent third parties to run educational sessions

that recommend these products for off-label uses, so long as the

educational program is independent and the decision to favorably

discuss the off-label use cannot be attributed to the drug company”.

Factors that the FDA considers include the control of content and

selection of presenters and moderators, disclosures of company

funding and connections to speakers, the focus of the program,

the relationship between the CME provider and the supporting

company, the involvement, if any, of the CME provider in sales or

marketing of a company’s product, and a provider’s failure to meet

standards for other programs.

“Beyond this guidance, the FDA

does little to ensure that educational grants are used for bona ﬁde

educational purposes. Nor does the FDA have a system in place to

monitor educational programs.”

The US Departments of Justice and Health and Human Services,

through the Ofﬁce of the Inspector General, enforce the anti-kickback

statute and the False Claims Act.

Pharmaceutical manufacturers

have been prosecuted for illegal off-label promotion. One example

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 115

Financial Support of Continuing Education in the Health Professions

116

is the $430 million that Warner-Lambert paid in 2004 to settle claims

about the promotion of gabapentin. Another is the $704 million that

Serono paid in 2005 to settle claims about the promotion of somat-

ropin (Serostim), which is a human growth hormone.

The Ofﬁce of the Inspector General identiﬁed educational grants

from drug companies as a risk area for fraud and abuse. In 2003,

they offered guidance to reduce the risk.

Risk-reduction measures

include separating grant making from sales and marketing within

pharmaceutical companies, establishing objective criteria for awarding

grants that do not take into account recipients’ purchases, and

funding only bona ﬁde activities. The manufacturer should have no

control over the speaker or the content of educational activities.

Conference organizers should not improperly compensate physicians,

for example, by paying them to attend a conference. Many pharma-

ceutical manufacturers have voluntarily adopted this guidance and

implemented internal “ﬁrewalls” to separate research grants from

marketing and sales. For example, ﬁrms may have a formal policy,

a centralized support structure, and not fund individual physicians or

group practices. Sales and marketing representatives may no longer

have authority to solicit grant requests or award funds.

3,24,26,27

Of course, a pharmaceutical company decides what events to spon-

sor and chooses the CME provider. In addition, we do not know the

effectiveness of internal “ﬁrewalls” and other measures in preventing

problematic activities. The Senate Finance Committee found that major

drug companies “have limited the direct involvement of ﬁeld sales

representatives and sales and marketing departments in the educa-

tional grant-making process” and that “the overt use of educational

grants to provide kickbacks to physicians who attend educational

programs has decreased over time.”

However, the Committee

cautioned, “it is difﬁcult to quantify the risk of kickbacks related to

industry-sponsored education where companies overpay high-pre-

scribing physicians as “consultants” or “speakers” for minimal work

to develop educational material or teach at educational programs.”

Pharmaceutical Industry and

American Medical Association Policies

In 2002, the Pharmaceutical Research and Manufacturers of America

(PhRMA), the leading trade association representing drug companies,

issued a “Code on Interactions with Healthcare Professionals.”

The

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 116

Steinbrook

117

code is voluntary; it establishes no procedures to identify violations

or enforce policy.

The PhRMA code allows member companies to sponsor CME and

other educational, scientiﬁc, or professional events that are organized

by third-party providers. The sponsorship is for overall program

costs. Payments to non-faculty healthcare professionals, such as for

travel, lodging, or time spent attending the meeting, are not allowed.

Financial support for “modest” meals or receptions may be provided.

The event should “further [attendees’] knowledge on the topic(s)

being presented” and be “primarily dedicated, in both time and

effort, to promoting objective scientiﬁc and educational activities

and discourse.” When ﬁrms “underwrite medical conferences or

meetings other than their own, responsibility for and control over

the selection of content, faculty, educational methods, materials,and

venue belongs to the organizers of the conferences or meetings in

accordance with their guidelines.” Nothing should be provided or

offered “in exchange for prescribing products or for a commitment

to continue prescribing products” or “in a manner or on conditions

that would interfere with the independence of a healthcare profes-

sional’s prescribing practices.”

In May 2007, in response to public criticism about the inﬂuence of

drug company money on medical practice, Eli Lilly & Co. began

posting online (https://www.lillygrantofﬁce.com/grant_registry.jsp)

all of its educational grants and charitable contributions to health-

care-related organizations in the United States; the information is

updated quarterly.

29,30

Many of these grants are for CME activities.

In the ﬁrst 6 months of 2007, two of the largest recipients of Lilly’s

grants were the Pri-Med Institute, a Boston-based MECC ($1,676,978

for programs related to depression and pain, diabetes, and erectile

dysfunction), and the psychiatry department at Massachusetts General

Hospital ($1,650,000 for the 2007 Psychiatry Academy, a program

with multiple sponsors). Lilly is considering disclosing information

globally, and other drug companies are considering similar disclosures.

The ethical code of the American Medical Association (AMA) permits

“subsidies to underwrite the costs of continuing medical education

conferences or professional meetings.” According to the Council on

Ethical and Judicial Affairs of the AMA (Opinion E-8.061), “any sub-

sidy should be accepted by the conference’s sponsor who in turn

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 117

Financial Support of Continuing Education in the Health Professions

118

can use the money to reduce the conference’s registration fee.”

Payments “should not be accepted directly from the company by the

physicians attending the conference” and “subsidies from industry

should not be accepted directly or indirectly to pay for the costs of

travel, lodging, or other personal expenses of physicians attending

conferences or meetings, nor should subsidies be accepted to com-

pensate for the physicians’ time.”

ACCME Policy and Standards for Commercial Support

The ACCME has seven member organizations— the American Board

of Medical Specialties, the American Hospital Association, the AMA,

the Association for Hospital Medical Education, the Association of

American Medical Colleges, the Council of Medical Specialty Societies,

and the Federation of State Medical Boards of the United States.

It has a staff equivalent of 12 full-time employees and an annual

budget of about $3.5 million.

According to the ACCME, CME “is professional education created

for the medical profession by the medical profession or its agents.”

The Council’s “Standards for Commercial Support” were approved

in 2004.

32,33

The six standards are 1) independence; 2) resolution

of personal conﬂicts of interest; 3) appropriate use of commercial

support; 4) appropriate management of associated commercial

promotion; 5) content and format without commercial bias; and 6)

disclosures relevant to potential commercial bias. CME “must promote

improvements or quality in healthcare and not a speciﬁc proprietary

business interest of a commercial interest.” Presentations “must give

a balanced view of therapeutic options.”

A CME provider must ensure that the following six decisions are

made free of the control of a commercial interest.

They are 1)

identiﬁcation of CME needs; 2) determination of educational objec-

tives; 3) selection and presentation of content; 4) selection of all

persons and organizations that will be in a position to control the

content of the CME; 5) selection of educational methods; and 6)

evaluation of the activity. A commercial interest cannot specify how

these requirements are met. The standards notwithstanding, a com-

mercial sponsor can currently designate the topic or type of activity

and, if the provider requests, suggest speciﬁc topics or speakers,

discuss the content with the provider, and furnish data, including

information about off-label uses.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 118

Steinbrook

119

The ACCME will not accredit entities that are commercial interests or

that are “owned or controlled” by a commercial interest. Providers of

clinical service directly to patients, including physician organizations,

medical groups, and academic medical centers, are not considered

commercial interests. Previously, the ACCME has not considered

MECCs to be commercial interests. However, under the expanded

deﬁnition of the term ACCME adopted in August 2007, more entities,

including some accredited MECCS, are likely to be considered com-

mercial interests. For example, advertising agencies that do promotional

work for pharmaceutical companies and companies that resell drugs

or devices are now considered commercial interests. Some of these

ﬁrms have divisions that are accredited CME providers. Such ﬁrms

must spin their CME units off as separate companies by August 2009

or withdraw from accreditation. An accredited provider “can be

owned by a ﬁrm that is not a commercial interest” or have a ‘sister

company’ that is a commercial interest, if there is “an adequate

corporate ﬁrewall in place to prohibit any inﬂuence or control by

the “sister company” over the CME program.”

Under the standards, relevant financial relationships within the

past 12 months must be disclosed, and conﬂicts of interest must be

resolved before the educational activity occurs. The ACCME deﬁnes

ﬁnancial relationships as “those relationships in which the individual

beneﬁts by receiving a salary, royalty, intellectual property rights,

consulting fee, honoraria, ownership interest (eg, stocks, stock

options or other ownership interest, excluding diversiﬁed mutual

funds), or other financial benefit.”

The Council has not set “a

minimum dollar amount for relationships to be signiﬁcant” because

“inherent in any amount is the incentive to maintain or increase the

value of the relationship.”

The requirements apply to planners,

speakers, and authors with “both a current ﬁnancial relationship

with a commercial interest and the opportunity to affect content

relevant to products or services of that commercial interest.”

general, there are four ways to resolve conﬂicts of interest: 1) ﬁnd

another speaker, author or meeting planner; 2) assign the speaker or

author a different topic; 3) have an effective peer review of the content

of a presentation or written material so that unsubstantiated or

promotional content is eliminated; or 4) change the overall activity

so that there is no longer a conﬂict. Conﬂicts of interest are usually

resolved through peer review or by changing the overall activity.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 119

Financial Support of Continuing Education in the Health Professions

120

Compared to the CME industry, the ACCME has a small staff and a

small budget. It lacks “proactive and real time oversight” of adherence

to its standards for commercial support, for example by placing

monitors in the audience at meetings.

Based on information pro-

vided by the ACCME, companies have exerted improper inﬂuence

over the content of CME programs. The Senate Finance Committee

noted that 18 of 76 CME providers that were reviewed in 2005 and

2006 “did not comply with at least one of the standards meant to

ensure independence.”

The Senate Committee also noted that

“based on ACCME policies, it can take as long as 9 years from the

date of a non-compliant educational activity for an educational

provider to lose accreditation.” Moreover, systematic data are lacking

on the frequency of adherence. According to the Senate Finance

Committee: “Compliance with ACCME standards still allows CME

providers to accommodate the business interests of their commercial

sponsors and affords drug companies the ability to target their

grant funding at programs likely to support sales of their products.

The full extent to which drug companies inﬂuence the content of

putatively independent CME programs cannot be estimated from

the information we currently have.”

Financial Support for Continuing Nursing Education and

Continuing Pharmacy Education

The American Nurses Credentialing Center (ANCC), a separately

incorporated subsidiary of the American Nurses Association,

accredits continuing nursing education (CNE). About half the states

mandate CNE. The Accreditation Council for Pharmacy Education

(ACPE) accredits continuing pharmacy education (CPE). All states

mandate CPE.

The ACCME, the ANCC, and the ACPE have a uniﬁed application

that a continuing education provider can use to seek accreditation

from two or three organizations. ANCC has accredited 225 providers

of CNE, including about 34 organizations with triple accreditation.

The ACPE has accredited about 408 providers; about a third have

multiple accreditations. Both the ANCC and the ACPE have adopted

the ACCME standards for commercial support.

There are no overall statistics for the funding of CNE. Although the

ACPE does not systematically collect information on funding, some

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 120

Steinbrook

121

data are available. In April 2007, the ACPE conducted an anonymous

survey of commercial support for pharmacy continuing education

providers.

The Council received 214 responses (response rate, 53

percent). Of the respondents, 28 percent reported no commercial

funding in 2006, 59 percent reported that such funding provided

25 percent or less of their annual income, 19 percent reported that

it provided more than 75 percent, and 11 percent reported that it

provided 100 percent. Another study found that “approximately

86 percent of providers and 43 percent of programs received

commercial support” and that “although the [ACPE] requires that

providers review instructional content and materials for commercially

supported programs before delivery, only 43 percent always did

so.”

This study concluded, “Commercial support of continuing

education is widespread, affects continuing education programs,

and is perceived to have signiﬁcant educational and noneducational

consequences.”

Financial Support for Continuing Legal Education

In contrast to the situation in the health professions, tuition payments

by individual attorneys and their ﬁrms provide most of the ﬁnancial

support for continuing legal education (CLE). About 40 states require

CLE, typically an average of 10 to 15 hours annually; the require-

ments and accreditation are state-based and vary. Programs may be

accredited by multiple states. Conferences may receive income from

sponsors and exhibitors, such as vendors of legal software or legal

publishers, but such payments provide limited revenue. Typically,

the speakers are not paid, although their travel expenses may be

reimbursed. Speakers agree to such arrangements for various reasons.

For instance, they may become better known in their legal ﬁelds,

and the exposure may attract new clients and referrals.

Because there is no central source of information about the ﬁnancial

support of CLE, I spoke to ofﬁcials at three leading providers — the

American Bar Association Center for Continuing Legal Education in

Chicago, the American Law Institute-American Bar Association in

Philadelphia, and the Practicing Law Institute in New York City. All

of these groups provided the same information. CLE has grown

rapidly. The funding differs from that for the health professions.

There are no external sources of funds that are analogous to funding

in the pharmaceutical industry.Payments from sponsors and advertisers

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 121

Financial Support of Continuing Education in the Health Professions

122

are small, compared with tuition income. Members of a sponsoring

organization may receive tuition discounts.

Recent Developments

The Senate Finance Committee report on the use of educational

grants by pharmaceutical manufacturers focused attention on the

ACCME. In an April 2007 letter to the Council, the Senate Committee

chair and ranking member urged greater “oversight to better ensure

that the content of continuing education programs is independent

from the business interests of the drug companies who fund the

programs.”

36,37

In its August 2007 response, the ACCME stated its

commitment to “ensure the validity of the CME enterprise.”

The

Council identiﬁed ﬁve areas for attention, including enhanced data

collection, such as establishing a monitoring system that would

allow “independent decisions about compliance with [ACCME]

requirements,” changes in the administration of the standards, such

as procedures for more rapid responses to serious problems, and

expanded education and outreach programs.” The Council also

said that it would review “the management of commercial support

across the CME enterprise including funding models and the role

of industry in CME. Alternate funding models will be considered

(eg, pooled funding, limits, sources) including discussions on the

value, or impact, of no commercial support.” Also in August 2007,

the ACCME announced several new policies.

In June 2007, the Senate Special Committee on Aging held a hearing

on the relationship between doctors and the drug industry, entitled

“Paid to Prescribe?”

The Committee chair, Senator Herb Kohl

(D-Wisconsin), stated, “these gifts and payments can compromise

physicians’ medical judgment by putting their ﬁnancial interest ahead

of the welfare of their patients.”

Kohl proposed a national registry

that would require disclosure of payments and gifts. Separately,

Senator Charles Grassley (R-Iowa) proposed that drug manufacturers

make public any payments to doctors who bill the Medicare or

Medicaid programs, which includes almost all practicing physicians.

In September 2007, Senator Grassley, Senator Kohl, and four cospon-

sors introduced legislation to require manufacturers of drugs and

medical devices to report publicly nearly all payments and gifts to

physicians.

The federal government would make the information

available on “an Internet website that is easily searchable, down-

loadable, and understandable.” The information would include “a

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 122

Steinbrook

123

description of the nature of the payment,” such as “participation in a

medical conference, continuing medical education, or other educa-

tional or informational program or seminar, provision of materials

related to such a conference or educational or informational program or

seminar, or remuneration for promoting or participating in such a con-

ference or educational or informational program or seminar.” Minnesota

and Vermont already require similar but less comprehensive disclo-

sures. Maine and West Virginia have enacted legislation to do so.

Questions for Consideration

The conference should initially consider whether or not commercial

support distorts continuing education and medical practice in ways

that are bad for health professionals and patients. It should then

consider whether to recommend changes to the ﬁnancial support

of continuing education.

At least six approaches are possible: 1) no commercial support;

2) limiting the percentage of commercial support that is permitted

for a provider or activity; 3) withdrawing the accreditation of MECCs

or other providers that are dependent on commercial support; 4)

eliminating direct or indirect commercial support of ACCME-accredited

programs but allowing contributions to a central repository, which

would provide funds to approved programs; 5) tightening the

ACCME’s standards for commercial support and monitoring com-

pliance; and 6) increasing public information about the funding of

CME, for example by establishing a comprehensive searchable online

registry of commercial and other support for providers and activities.

A registry of CME funding could also include information on indus-

try’s gifts and payments to speakers, authors, event planners, and

ofﬁcials of provider organizations.

Pooling commercial support for continuing education has recently

been advocated.

Contributions might be made to a designated

ofﬁce at a medical school, academic medical center, or medical

society, which would in turn disburse the contribution to ACCME-

approved programs. Thus, the ultimate recipient of funds would not

be subject to the inﬂuence of any one donor. At present, a single

commercial entity supports many continuing education programs.

An analogy would be the support of medical journals by multiple

advertisers, with a ﬁrewall between advertising and editorial content.

Another analogy would be medical journal supplements; supplements

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 123

Financial Support of Continuing Education in the Health Professions

124

sponsored by a single pharmaceutical company often have promo-

tional attributes and may be of lower quality than other journal

content.

Many of these approaches are not mutually exclusive. Some, if not

all, might reduce funding for continuing education. All continuing

education providers have strong vested interests in maintaining

their income sources, as well as their eligibility for accreditation.

If commercial support were to decrease or stop, some continuing

education programs would be cancelled, some people would lose

their jobs, some providers would change their operations or go out

of business, and many of the remaining providers would be less

proﬁtable. The effects on educational offerings and quality are a

matter of conjecture. Health professionals or their employers, how-

ever, would pay more of the true costs of continuing education.

REFERENCES

1. Accreditation Council for Continuing Medical Education. Annual report data 2006.

(Accessible at

http://www.accme.org/index.cfm/fa/home.popular/popular_id/127a1c6f-462d-476b-

a33a-6b67e131ef1a.cfm (Access date: September 5, 2007).

2. Steinbrook R. Commercial support and continuing medical education. N Engl J Med.

2005;352:534–535.

3. Iskowitz M. CME’s new order. Medical Marketing Media. August 2006;37-7.

4. Angell M. The Truth About the Drug Companies: How They Deceive Us and What to

Do About It. New York: Random House; 2004.

5. Kassirer JP. On the Take: How Medicine’s Complicity with Big Business Can

Endanger Your Health. New York: Oxford University Press; 2004.

6. Relman AS. Separating continuing medical education from pharmaceutical market-

ing. JAMA. 2001;285:2009–2012.

7. Wazana A. Physicians and the pharmaceutical industry: is a gift ever just a gift? JAMA.

2000;283:373–380.

8. Elliott C. Pharma goes to the laundry: public relations and the business of medical

education. Hastings Cent Rep. 2004;34:18–23.

9. Harrison RV. The uncertain future of continuing medical education: commercialism

and shifts in funding. J Contin Educ Health Prof. 2003;23:198–209.

10. Casebeer LL, Shillman RS. What’s ROI got to do with CME? Medical Marketing

Media. August 2002.

11. Harris G. Lawmaker calls for registry of drug ﬁrms paying doctors. New York Times.

August 4, 2007;A9.

12. Steinman MA, Bero LA, Chren MM, Landefeld CS. Narrative review: the promotion of

gabapentin: an analysis of internal industry documents. Ann Intern Med.

2006;145:284–293.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 124

13. Katz HP, Goldﬁnger SE, Fletcher SW. Academic-industry collaboration in continuing

medical education: description of two approaches. J Contin Educ Health Prof.

2002;22:43–54.

14. Ross JS, Lurie P, Wolfe SM. Medical Education Services Suppliers: a threat to physician

education. Health Research Group Report July 19, 2000. Accessible at

http://www.citizen.org/publications/release.cfm?ID=7142 (Access date: September 5, 2007).

15. Kassirer JP. Professional societies and industry support: what is the quid pro quo?

Perspect Biol Med. 2007;50:7–17.

16. Steinbrook R. Haemoglobin concentrations in chronic kidney disease. Lancet.

2006;338:2191–2193.

17. Steinbrook R. Guidance for guidelines. N Engl J Med. 2007;356:331–333.

18. Accreditation Council for Continuing Medical Education. Policy updates August 2007.

Accessible at

http://www.accme.org/index.cfm/fa/news.detail/News/.cfm/news_id/3605f21a-

302a-40d1-ab4d-3ceb88087b1a.cfm (Access date: September 5, 2007).

19. Committee on Finance, United States Senate. Committee Staff Report to the

Chairman and Ranking Member, Use of Educational Grants by Pharmaceutical

Manufacturers. Washington, DC: Government Printing Ofﬁce; 2007.

20. Donohue JM, Cevasco M, Rosenthal MB. A decade of direct-to-consumer advertising

of prescription drugs. N Engl J Med. 2007;357:673–681.

21. Campbell EG, Gruen RL, Mountford J, et al. A national survey of physician-industry

relationships. N Engl J Med. 2007;356:1742–1750.

22. Campbell EG, Weissman JS, Ehringhaus S, et al. Institutional academic-industry rela-

tionships. JAMA. 2007;298:1779-1786.

23. Department of Health and Human Services, Food and Drug Administration.

Guidance for industry: industry-support scientiﬁc and educational activities. Fed

Regist. 1997;62:64093–64100.

24. Studdert DM, Mello MM, Brennan TA. Financial conﬂicts of interest in physicians’

relationships with the pharmaceutical industry—self-regulation in the shadow of

federal prosecution. N Engl J Med. 2004;351:1891–1899.

25. Department of Health and Human Services, Ofﬁce of Inspector General. OIG com-

pliance program for pharmaceutical manufacturers. Fed Regist. 2003;68:23731–743.

26. Liebman M. Drawing a line between education and promotion. Medical Marketing

Media. August 2003;44–49.

27. Iskowitz M. Beyond the ﬁrewalls. Medical Marketing Media. August 2005;34–46.

28. PhRMA code on interactions with healthcare professionals. Revised January 2004.

Accessible at

http://www.phrma.org/code_on_interactions_with_healthcare_professionals/

(Access date: September 5, 2007).

29. Johnson A. Under criticism, drug maker Lilly discloses funding. Wall Street Journal.

May 1, 2007;B1.

30. Iskowitz M. CME’s open gesture. Medical Marketing Media. August 2007;37–48.

31. 2004 Updated Standards for Commercial Support with Background Rationale and

Answers to Questions About Compliance. Chicago: Accreditation Council for

Continuing Medical Education; September 2004.

32. Standards for commercial support. Chicago: Accreditation Council for Continuing Medical

Education. Accessible at http://www.accme.org/index.cfm/fa/home.home/home.cfm

(Access date: September 5, 2007).

Steinbrook

125

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 125

Financial Support of Continuing Education in the Health Professions

126

33. Standards for commercial support including tools for implementation. Chicago:

Accreditation Council for Continuing Medical Education. Accessible at

http://www.accme.org/dir_docs/doc_upload/700a1624-d6f0-46d8-a4bc-

88e12a56eca2_uploaddocument.htm (Access date: September 5, 2007).

34. Travlos, DM, Sesti, AM. Status of commercialism in continuing pharmacy education.

Presentation at the ACPE 12th Conference on Continuing Pharmacy Education,

Denver, CO, October 7, 2007.

35. Smith JL, Cervero RM, Valentine T. Impact of commercial support on continuing

pharmacy education. J Contin Educ Health Prof. 2006;26:302–312.

36. Baucus M, Grassley C. Letter to Murray Kopelow, Accreditation Council for

Continuing Medical Education, Washington, DC: Committee on Finance, United

States Senate, April 25, 2007. Accessible at

http://www.accme.org/index.cfm/fa/news.detail/news_id/9a586937-430d-4384-b31a-

1e557d49f9d3.cfm (Access date: September 5, 2007).

37. Baucus, Grassley continue work for independence of continuing medical education.

Press release of the Senate Finance Committee. Washington DC; May 1, 2007.

38. Kopelow M. Letter to Max Baucus and Charles E. Grassley, Committee on Finance,

United States Senate. Chicago: Accreditation Council for Continuing Medical

Education, August 3, 2007. Accessible at

http://www.accme.org/index.cfm/fa/news.detail/News/.cfm/news_id/edf1572d-

bb76-416c-9b41-86fe1f7c1055.cfm (Access date: September 5, 2007).

39. Harris G. Senators to push for registry of drug makers’ gifts to doctors. New York

Times. June 28, 2007;A16.

40. Kohl examines drug industry practice of providing payments to doctors, inﬂuencing

prescribing behaviors. Press release of the Senate Special Committee on Aging.

Washington, DC: June 27, 2007.

41. Physician Payments Sunshine Act of 2007. S. 2029, 110th Cong., 1st Sess.

42. Harris G. Senators seek public listing of payments to doctors. New York Times.

September 7, 2007;A18.

43. Brennan TA, Rothman DJ, Blank, L et al. Health industry practices that create con-

ﬂicts of interest: a policy proposal for academic medical centers. JAMA.

2006;295:429–433.

44. Bero LA, Galbraith A, Rennie D. The publication of sponsored symposiums in med-

ical journals. N Engl J Med. 1992;327:1135–1140.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 126

Remarks by Jordan J. Cohen, M.D.

Association of American Medical Colleges

I think the issue that we’re dealing with is really a key issue in terms

of how we articulate our values. This conference is occurring at a very

pregnant moment in the history of our profession. I think we have

an opportunity to change the paradigm, given all the things we’ve

heard and what we know is happening already in the world of contin-

uing medical education. I think we are on the verge of something

signiﬁcantly transformative in terms of how we are going to be doing

this in the future. If we take the opportunity to come up with some

well-targeted and strong recommendations, they will be received in

a way that could have some signiﬁcant impact.

I want to frame my editorial comments about the ﬁnancing in a broader

context. First of all, I think we all acknowledge that maintaining one’s

professional competency throughout a lifetime of work is really a

key element of professionalism. It’s highlighted in a physician charter

which speaks to physicians, but I think it is true for all healthcare

professionals that this is an absolutely core component of what we

do as professionals. We maintain our competence and are account-

able for that to the public.

It is also clear from what we know that the fundamental purpose

of continuing education, be it for medical or other health profes-

sionals, is ultimately to improve the quality of patient care by

changing clinician behavior. The whole notion is that we are trying

to modify behavior that is documented to be insufﬁcient to meet

whatever standard or whatever benchmark one is seeking, and

we’re trying to change that behavior in the direction of improving

the ultimate outcome.

What does that tell us about the current mode of continuing education?

It seems to me it’s got two fundamental ﬂaws that are not unrelated.

One is that what has traditionally been thought of as continuing

education, the presentations, the courses, the large lectures and

so forth, is largely ineffective. There is some utility to the lectures,

certainly in passing on new information and keeping people abreast

of new developments, so I wouldn’t entirely discount the value of

large lectures and presentations, but in terms of improving quality,

it has not proven to be an effective modality. So that’s the ﬁrst ﬂaw:

It is an ineffective way of achieving the end that we’re looking for.

127

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 127

Remarks

128

The second major ﬂaw is that it is funded predominantly by com-

mercial entities with self-interested motives. Commercial entities are

deﬁned by the ACCME as entities that produce, market, resell, or

distribute healthcare goods or services consumed by or used on

patients. The fact that we have allowed the system to become so

dependent on the funding from sources that have an ulterior motive,

that is clearly related to their self interest and not in the ultimate

interest of the continuing education enterprise, is what we are

talking about.

How did we get to a point where we’ve allowed so much of this

core responsibility of ours to be funded by external forces that have

other interests in mind? It reminds us of the parable of the frog in

boiling water: It has happened gradually. Obviously the ramping up

in recent years has been quite dramatic, but over the past several

decades this has been a gradual process. We’ve been immersed in

this gradually warming boiling water to the point where we haven’t

really sensed the changes that have occurred or sensed how close

we are to the fatal boiling point.

On the other hand, if you put a frog in boiling water immediately,

it dies instantly. And that’s what our students perceive when they

come into our system. They get it. It’s not a surprise, and not difﬁcult

for the students to recognize, that they are in a system that really

doesn’t make any sense. One bit of evidence to support that is what

the American Medical Student Association (AMSA) has done with

“no free lunch.” They actually request students to sign a pledge that

they will not accept any gifts or any support from any commercial

entities. If we tap into their perceptions of what is really correct and

right and appropriate, we will have done a good deal of our job.

The next question I would ask is why should we be concerned

about the commercial entity supporting so much of our education?

I think that tolerating commercial intrusion into one of our core

responsibilities, namely educating ourselves as professionals, is

opening ourselves to biased information. I’m not suggesting that

all of the information that emanates from commercial sources is,

ipso facto, biased, but certainly there is the perception that it could

be and there is no doubt that there are instances where it ﬂagrantly

is. So, there is no question about the fact that the information that

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 128

Cohen

129

is ultimately supported by these sources has that very unwanted

consequence.

That raises the question about how our vow to place patients’

interests uppermost is really being maintained. It belies our commit-

ment to evidenced-based information, undermines our commitment

to cost-effective prescribing, and validates and reinforces an entitle-

ment mentality. One thing that we have to be cognizant of is how

our current system so generates the sense of entitlement on the

part of our graduates and practicing physicians that they feel that

they have an entitlement to other people helping them do their job.

It also signals to the public that we are, in fact, for sale. Any sort

of knowledgeable public person looking at what we’re doing certainly

would raise the question of why we are willing to sell ourselves

for a price.

There are opportunity costs here as well. The extent to which we

devote energy and administrative activities to garnering and manag-

ing this commercial support takes time and effort away from doing

the core mission of education. Another reason for being concerned

is that these commercial courses and commercially supported

education are so seductive that it impedes the adoption of more

effective continuing educational activity. To the extent that this is

seen as the easy, the very seductive road it diverts people’s attention

from actually doing what we now are recognizing as more effective

kinds of continuing education.

The next question, and I think it is an obvious one, is why does the

industry support medical education? In theory, one could argue, it is

in gratitude for all of the hard work that doctors and nurses do. This

is a sustainable hypothesis, but, of course, the fact is that industry’s

support is clearly in the hope of garnering a favor for its products.

This is an openly avowed objective by the industry. They make

no bones about it. This is no secret among the industry. They admit

it openly.

Then the question is, if the industry is supporting education for

reasons of trying to garner a favor, does it in fact work? Again, the

answer is obvious. Prescribing behavior and other behaviors are clearly

inﬂuenced. The documentation for that is overwhelming, so I don’t

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 129

Remarks

130

think we need to debate whether or not what the industry or com-

mercial entities do in supporting education is effective. One doesn’t

need any other evidence than the fact that they are spending billions

of dollars doing it. Obviously they are not spending money because

they don’t think it’s effective. Clearly we are in a situation where we

are opening ourselves to the biased inﬂuences of the commercial

entities that have these other negative consequences as well.

If you accept all that, could we still enjoy the ﬁnancial support if we

could erect a sufﬁciently impermeable ﬁrewall between the industry

support and the educational activity? This, it seems to me, is what

the Accrediting Council for Continuing Medical Education, ACCME,

has been trying to do. I give them credit. I think they have tried hard

to develop policies that would ensure that commercial inﬂuences

are minimized in the educational activities that they accredit. The

most recent modiﬁcation of their guideline has strengthened that

ﬁrewall considerably. But, given the strong and legitimate motivation

of industry to market its goods to doctors, there is no ﬁrewall that

we could conceivably construct that they wouldn’t be able to ﬁnd

a way around and still have the inﬂuence that they desire.

I think there are a lot of things we can criticize about the way the

industry tries to inﬂuence physician behavior, but, in the ﬁnal analysis,

it seems to me that we can’t quarrel with their fundamental motivation.

This is a capitalist society. They are obligated to return rewards to

their shareholders, so they market their activities to those individuals

who have most inﬂuence over the ﬁnancial success of their operations.

To put the entire blame on commercial entities, I think, is totally

wrong. The burden is really on us, on the profession. We have, in

my judgment, failed to construct sufﬁciently robust policies and

procedures to prevent the unwanted inﬂuence of commercial entities

that are simply doing what any of us would do were we in their

place, and that is trying to inﬂuence our colleagues.

We know that doctors rationalize their acceptance of industry sup-

port. When you ask doctors whether they are inﬂuenced by the gifts

and the support that they receive, they say of course not. “I’m not

inﬂuenced, but she is for sure inﬂuenced.” So there’s an acknowl-

edgment that the inﬂuence is there, it just isn’t touching them

personally. They argue that other business people get perks, too,

giving rise to the notion that medicine is just another business, which

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 130

Cohen

131

again is one of the major consequences of the commercialization

of what goes on in medicine today. Medicine is not just another

business. It is a calling. It is a profession. It has different values.

It seems to me that what we need to do is to regain control of our

medical education from these commercial entities and deposit it

where it belongs, namely ﬁrmly within the profession itself. That

raises the question of whether physicians can afford to pay for their

own continuing education. Of course they can. No other profession

depends upon vendors of their activities to support their continuing

education. There is a fundamental obligation of professionals to

support their own continuing professional development and to

pay whatever is required to do that.

My ﬁnal thoughts are these: We recognize that we are witnessing

a paradigm shift in continuing professional development and we

recognize that the paradigm we have lived with, with all of this

commercial support, is not effective. It is certainly not sufﬁciently

effective to warrant the kind of investments that we’re making

presently. We have new modes of continuing professional develop-

ment, particularly practice-based learning, that are more effective,

more efﬁcient, and a much more targeted way of ensuring that we

achieve the goals of continuing professional development.

Given the fact that we’ve got this new mode that is much more

effective and is going to enjoy the beneﬁts of the information tech-

nologies and is on the threshold of becoming the dominant mode,

and we’ve got this archaic, commercially supported mode, it seems

to me that we are in a very good position to say let’s get rid of

that old mode. We’re not going to give up much of anything by

abandoning the commercial support for this outmoded, ineffective

way of continuing education.

But I would argue that we need to be very careful if we move towards

a more practice-based learning mode of continuing education. We

have to be very cognizant that commercial entities are going to want

to get into that area, just as they have invaded the traditional area

of CME, so we’ve got to protect it. We can’t repeat the mistakes of

the past and let the new mode of continuing education also become

dominated in a decade or two by commercial intrusion. How we go

about doing that, I don’t know. But I think caution is essential, that

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 131

Remarks

132

we need to be careful not to repeat history as we move to this new

mode of continuing education.

My recommendations would be that, ﬁrst of all, we strongly endorse

the obligation of health professionals to provide for their own

continuing professional development. This is a variation on the

theme of “doctor, heal thyself.” Secondly, we ought to prohibit all

commercial interests, as deﬁned by the ACCME, from making any

contribution to continuing professional development. Perhaps we

would want to put a timeline on that, by 2025 or some deﬁnite period

of time. We need to recognize we simply cannot ignore the fact that

we have a heavy dependence on the ﬁnancial support that’s coming

from these sources. We can’t ignore that reality and say that we

ought to stop this cold in its tracks while institutions need an

opportunity to somehow modify their activities to accommodate

this. It seems to me that it would make sense to recognize that some

kind of phase out period may be necessary, and maybe, in the interim,

some mechanism of pooling. I think the likelihood of commercial

entities being willing to pool their support at a level that would really

make sense, namely at the institutional level, is unlikely to yield any

signiﬁcant ﬁnancial support, so I don’t think we ought to be lulled

into a sense of security about that. If we wanted to have an interim

position to allow for some continued commercial support, it seems

to me the only way in which we could do that, and be dutiful to

the professional values, would be to ensure that it is pooled in such

a way that it is clearly more than an arm’s length away from the

actual distribution of those funds.

DISCUSSION HIGHLIGHTS

Although there is a great deal of commercial support of accredited

continuing education programs (estimated at about $1.5 billion in

2006), this pales in comparison to commercial support of unaccred-

ited continuing education, which is estimated to be anywhere from

$20 to $60 billion annually.

An entire enterprise outside of the ACCME umbrella offers physi-

cians and surgeons courses in the use of new devices, yet many are

not accredited by anybody. They are run by industry over weekends

and physicians return home to try out these devices on their own or

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 132

Discussion Highlights

133

with minimal help. That’s a big problem.

Effects of Commercial Support

A lot of concern is about the conﬂict of interest commercial support

may be causing. The International Committee of Medical Journal

Editors points out that conﬂict of interest doesn’t have to exist to

cause a problem; it just has to be clearly possible.

It is not only clinicians who proﬁt from commercial support.

Continuing education providers enhance their bottom line by work-

ing with commercial providers. They don’t have to do a lot of work

because industry is happy to do the entire planning of an activity.

When we banned commercial support for planners, providers sud-

denly had to start planning their own activities but didn’t have a

clue how do that because, even though they had been “planning”

for years, they didn’t pick the topics or the speakers, and they didn’t

set objectives or measure outcomes. All they did was show up and

put their names on it.

I cannot understand how we can call ourselves professionals if we

sell our educational souls for a commercial pot of porridge, which is

usually not speciﬁc to our patients’ needs or our educational needs

and is dictated by commercial self-interest needs.

As primary care doctors, and probably this is true for others, we

guess 70-80 percent of the time and 95 percent of the decisions are

directly or indirectly inﬂuenced by the pharmaceutical industry, not

by best practices.

Most pharmaceutical inﬂuence is below the radar and physicians

don’t even realize it’s there. The evidence says the more that people

think they’re not inﬂuenced, the more they actually are. As educa-

tors we have to recognize that and work hard to overcome it,

realizing we have to overcome what’s below the radar and help

people to become more aware of it.

While a number of companies, based on guidance from the Inspector

General of the Department of Health and Human Services, have

divorced their continuing education operations from marketing, they

are still positioned to promote the ﬁelds of medical inquiry and the

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 133

Discussion Highlights

134

areas of work that the company’s line of products naturally come

upon. As long as doctors are writing prescriptions and companies

are selling for proﬁt products that are the object of those prescrip-

tions, the companies will ﬁnd legal ways to seek to inﬂuence the

judgments and choices of those who are writing the prescriptions.

We are not going to eliminate that reality. But there should be serious

recommitment to developing a ﬁre walled entity that would be an

intermediary between the companies and recipients of the direct

support for education.

Any medical school will say they are training for life-long learning,

but they need to recognize that just training doctors how to critically

read the primary literature is not going to work for them. We have

to train them how to evaluate secondary sources, have speciﬁc tools

to evaluate secondary sources, and make sure those sources are put

together carefully without either commercial or non-commercial bias.

And as much as we try to stop students or residents from undue

inﬂuence from pharmaceutical reps during their training, when they

go into practice they will be exposed to them. We have to train future

practitioners how to deal with pharmaceutical reps.

At least one study has shown that commercial support of continuing

education narrows the focus of courses primarily to new therapies,

often therapies that include recent FDA approval of a product man-

ufactured by the commercial supporter. Whole areas of practice are

ignored. Bias may not be so much in individual talks as in the over-

all spectrum of what is covered.

When there is a gift, even a small gift, there’s an inescapable sense

of obligation that comes with it. This has been well documented.

Sadly, commercialism has developed within our specialties, which

see their knowledge and skills as proprietary, not to be shared with

other members of the profession. We need to minimize the conﬂict

of interest which doesn’t come only from pharmaceutical compa-

nies. Academic conﬂicts may be more pervasive and difﬁcult to

measure than ﬁnancial conﬂicts of interest.

The physicians who teach continuing medical education are the

same ones who teach residents and medical students and are doing

research. Several studies show the extent to which those at the

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 134

Discussion Highlights

135

highest levels and in the greatest position of inﬂuence in academic

medicine personally proﬁt from their relationships, and the extent to

which solvency and discretionary dollars within a departmental

budget come from these sources. These relationships affect the con-

tent and topics of research, as well as whether research ﬁndings can

even be disseminated. These relationships affect teaching at every

level, not just in continuing education.

The issue of fraud and abuse is important for people at the front

lines. As an example, you organize a conference, you recruit partici-

pants who pay a registration fee, and then you send a letter to the

referring physicians who are invited to attend for nothing. For

instance, you have a new gamma knife and get a CME provider to

put on a conference. Then you invite the physicians who could refer

patients for the procedure to tell them about the gamma knife, and

they get the whole thing for free.

Industry funding is less of an issue in nursing, but it is growing.

This is particularly true for advanced practice nurse continuing

education, when nurse practitioners can prescribe.

The Roles and Responsibilities of Industry

The industry has to be responsible for ethically responsible dissemi-

nation of its information in ways physicians will understand. Industry

has to be more responsible for post-marketing surveillance and for

comparative effectiveness studies.

A huge public good comes from the interaction between academia

and industry in terms of promoting the transfer of new discoveries

into useful products and services, but that produces a different set

of conﬂicts, such as protecting patients who are subjects of human

investigation. I would draw the line in terms of involvement with

education, not with research.

Not all pharmaceutical inﬂuences are bad. A lot of great things have

come through research grants with money that was at arm’s length.

The pharmaceutical industries provide new drugs and new cures

and, because of them, we’ve had substantial advances in medicine.

It would be a mistake to exclude commercial exhibits from continuing

education, particularly in the nursing arena with national conferences.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 135

Discussion Highlights

136

The exposure during exhibit times is a form of learning.

I’m against commercial support but I have to say that the pharma-

ceutical companies have certainly changed the lives of people with

asthma, hypertension, coronary artery disease, osteoporosis and

infectious disease, plus many others. Those have been the bulk of

what a number of the conferences have been around and that’s

been a good thing.

Not all commercial funding is bad. Samples save patients time and

money. Free pads and pens save money that can go to patient care.

A free clinic had to close when it was told it could take nothing.

Who Should Finance Continuing Education

Why should we seek support of any kind in any model from com-

mercial entities to educate ourselves to maintain our competence?

That is part of our core professional responsibility.

Funding for continuing education should come from us, or our

employers, so we can control the destiny of our education. Once

commercial groups, governments, or foundations become involved,

the patient is no longer the focus.

We should pay for it ourselves. Education is a personal responsibility.

If it is system-based responsibility, other elements will come in to

support it.

While philosophically it makes sense for the profession to pay for

continuing education, one could make a case that it would be in the

enlightened self-interest of the government, through what it already

pays directly and indirectly, to promote better care. That would be

a net savings in the long run, if you look at a big enough picture.

When the government gets involved, ideology and politics, especially

if somebody is being adversely affected, often do too. An example

about a decade ago was the AHRQ-sponsored recommendation

about back pain, and the effort of politicians supporting the spinal

surgeons to zero-budget the agency. Currently, there is a big lobby-

ing effort regarding reimbursement for erythropoietin treatments in

oncology and nephrology patients.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 136

Discussion Highlights

137

If we pursue the purity of elimination, how would we set a bound-

ary so that any organization in healthcare would be able to provide

continuing education? If a school has a corporate gift, does that make

it ineligible? If you are a professor whose chair has been funded by

a company, does that make you ineligible? What the goal ought to

be is to eliminate the distortion by neutralizing and pooling, and by

public disclosures.

Though some would love to paint the industry as being responsible,

it has been with the willing collusion of physicians and continuing

medical education providers who have created an atmosphere where

physicians have come to expect that continuing medical education

will be free or low cost, that they will not have to do any work, that

they are just going to sit in a dark room and listen, that it will

include food, and will mainly be updating their knowledge.

Many doctors have a sense of entitlement, the idea that someone

else should pay for their continuing education. They see it as a

bureaucratic burden rather than a moral obligation. This is a vastly

neglected aspect of undergraduate and graduate medical education.

We have to start early and change the culture so doctors view their

continuing education as a professional obligation.

Do you think of professionalism as lettuce or potato? Lettuce rots from

the outside in, potatoes rot from the inside out. So is commercial

contamination, contamination of education, and promotion, a slimy

brown leaf of lettuce that can be removed, or is there a deeper

lesion? I think it’s an interesting question.

Continuing education and lifetime learning is a form of higher edu-

cation. This is a public good. Two funding sources for continuing

education have been eroded. One is federal and state support,

which funded AHECs that played a phenomenally important role in

educating community clinicians. The other is the ability to use dol-

lars that come into academic health centers for continuing

education.

We need to model what the future would look like under revised

rules. What would happen if commercial support went away com-

pletely, 50 percent, 25 percent? We also need to create the vision of

an alternative form of continuing education that might be smaller.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 137

Discussion Highlights

138

The enterprise might be smaller in its measurable size, fewer lec-

tures, etc., but the amount of education might be the same or larger,

using new technology.

If you take away commercial support, you are going to eliminate a

vast majority of the people who now provide continuing education.

It would be appropriate to look at what might be the unintended

consequences, and trying to address those in advance instead of

trying to pick up the pieces later.

If you look at hours of instruction by provider category, state

medical societies and schools of medicine are at the top, but state

medical societies get very little commercial support.

Without industry, someone has to play a larger role, maybe state

professional associations, or groups like Kaiser and the VA system

that are interested in quality of care. That applies to medicine, nursing

and everybody else. Kaiser has already stopped commercial support

of its continuing education programs.

It is possible that physicians would under invest in their own edu-

cation because they don’t think they get sufﬁcient return on that

investment. One problem in medicine is that it is an imperfect

system for rewarding good performance, for there is no good way of

measuring quality at the individual physician level. But if physicians

under invest, there must be a societal investment that might come

from many sources, from corporations, from insurance companies,

from individual health organizations.

Top journals take ads from many companies to provide a pooling

of resources. They also build a ﬁrewall so that the advertising staff

does not know what is happening editorially. This may be a model

for funding CE.

We always talk about the cost when we should be talking about the

investment, but until we can demonstrate the return on investment,

we may not be able to make the business case.

If you separate everything immediately, there will be a huge vacu-

um and lack of support for continuing medical education. If there’s

no support, what happens to physicians in remote areas, who work

hard and have no time off? Perhaps the profession should ﬁnd a

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 138

Discussion Highlights

139

way to help those members.

A common plea in community practice is that physicians don’t have

access to resources, that they’re too expensive. It is almost seen as

justiﬁcation for getting their continuing medical education from

drug reps who visit their ofﬁces. This is a real issue that needs to be

addressed. If we pay for our stethoscopes, we should pay for infor-

mation we need. It is the job of the profession at large to help

doctors who don’t have the access.

A lot of department chairs will be absolutely petriﬁed at the idea

that the slim proﬁt margins they get from their continuing medical

education courses might go away. Institutions that are doing this in

a very proﬁtable way are not going to want to give up that proﬁt if

they can hang on to it.

I don’t have any problem with commercial people putting funding

into a pool and the only recognition is that they be listed along with

all the foundations and the government and other sources of revenue.

One approach could be to establish an independent commission

or some other entity that would be empowered to receive money

from any source and to promote improved patient care through

education and improvement in practice, that would reach across

professions, capitalize on the electronic capabilities, and represent

an enterprise for learning and improving the quality of care through

lifelong practice.

The government ought to have a major role in funding for research in

continuing education and funding for the development and acquisition

of IT capability by providers, The funding of continuing education,

per se, ought to reside with the profession in some form, individually,

by employers, or by professional organizations.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 139

140

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 140

III.

Designing Systems for

Lifelong Learning

to Improve Health

141

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 141

Continuing Health Professional Education

Delivery in the United States

David A. Davis, M.D. and Trina Loofbourrow, B.A.

Association of American Medical Colleges

This report focuses on the scope and delivery of continuing medical

education (CME) for physicians with respect to regulatory policy,

research, and data collection. There is less information available

for continuing education for nursing and pharmacy professionals;

continuing education for these professions is addressed in Appendix

A and Appendix B, respectively.

Background

The History of CME

The history of CME has been marked by two distinct areas of devel-

opment. The ﬁrst is the development, production, and delivery of

CME activities and events themselves; the second is the CME credit

movement, leading to certiﬁcation, membership in specialty organi-

zations, and licensure.

Developing and Producing CME Programs and Activities

. How old

is CME? Although the self-education of physicians has probably

existed for as long as the profession itself, the ﬁrst formal reference

to continuing education for physicians came in the early 1900s,

when Sir William Osler articulated the importance of physician

education and urged the profession toward a position that recognized

clinical advances and sought methods to deliver them to practicing

physicians. Along with other advice, he urged the profession toward

a position in which it recognized clinical advances and sought

methods to deliver them to its member physicians.

The ensuing century brought with it many changes, including the

following:

— Changes in the model or delivery mode of CME from early out-

rider programs to large conferences and medical update sessions

— A clear understanding of adult learner theory and its application

to the CME process

— Adaptation of Tyler’s model of program design and evaluation

to CME

142

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 142

Davis & Loofbourrow

143

— A rapid increase in the number of courses and conferences

following World War II, brought about by new advances in the

biomedical sciences and physicians returning to practice

The last half of the twentieth century has been marked by the

following developments:

— The establishment and application of accreditation principles

to CME program design and delivery

— The use of information and distance education technology

— A focus on research and outcomes, leading to a stronger

emphasis on clinical performance and healthcare outcomes

— The growing presence of commercial interests in CME

Requiring CME for Licensure and/or Recertiﬁcation

. Almost concur-

rently, there were movements in the licensure and expectations

of physicians about the “amount” of CME they might need in order

to maintain their competence. The ﬁrst to reify CME “hours,” the

American Academy of Family Physicians (AAFP; then General

Practice), stipulated a 150-hour/3-year membership requirement

in the late 1940s, using the terms “formal” and “informal” CME. The

AAFP replaced this nomenclature in 1955 with the terms Category 1

and Category 2 to recognize these two forms of learning. Subsequently,

the American Medical Association (AMA) Physician’s Recognition

Award (PRA) was created, requiring a similar number of hours of

credit and consolidating, in 1985, several categories of learning into

two, following the lead of the AAFP.

Several changes in the credit system have evolved over the subse-

quent years. First, while formal CME can be conveniently measured

in hours, informal and self-directed activities cannot. Thus, “credit,” as

opposed to “credit hours,” has become the preferred term. Second,

given the self-reporting and immeasurable qualities of self-directed

learning, credit regulators have chosen to place caps on these latter

activities. Although it was not initially intended for this purpose, CME

credit has been used to meet the need for establishing or guaranteeing

practice competence. As a consequence, a robust movement has

arisen over the past three decades to link these credits to other

regulatory and membership requirements.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 143

Continuing Health Professional Education Delivery in the United States

144

First, licensure is strongly linked to CME participation. In 1971, the

state of New Mexico adopted a policy of relicensure based on

CME participation.

Many states have followed suit. Currently, CME

participation is mandated by the majority of state medical boards.

Within these policies, however, there are wide variations: Some of

the boards require that these credits be all or partly from the AMA

Physician’s Recognition Award system, and the number of hours

required ranges from 20 hours a year to 50 hours a year.

Second, the role of the medical specialty boards in assuring the

competence of their members is a long and familiar story. Physicians

who have met the (re)certiﬁcation requirements of an American

Board of Medical Specialties (ABMS) member board receive an AMA

Physician’s Recognition Award certiﬁcate. Following its efforts to

standardize methods to ensure competence in the trainees of

graduate programs, the ABMS has undertaken the creation of an

extensive Maintenance of Certiﬁcation program, with several ele-

ments, including 1) professional standing; 2) lifelong learning and

self-assessment; 3) demonstration of cognitive expertise; and 4)

practice performance assessment.

Third, the AMA Physician’s Recognition Award certiﬁcate meets the

CME requirements of the Joint Commission on Accreditation of

Healthcare Organizations (JCAHO) related to hospital accreditation.

Accreditation Bodies

Because most of the data in this report are derived from CME

accreditation sources, and these sources are important in the

delivery of CME, we outline some of the history and background

related to the accreditation processes.

The Accreditation Council for Continuing Medical Education

(ACCME) is the accrediting body for institutions and organizations

administering and accrediting CME programs used by physicians.

ACCME was established in 1981, taking over the accreditation

process from the AMA. Accreditation of CME providers in medicine

has been the subject of consideration and action for nearly four

decades. In the 1960s, under the rubric of a “coordinating commit-

tee” for medical education, initiatives were launched that led to the

creation of “liaison committees” for undergraduate, graduate, and

continuing medical education. In CME, the Liaison Committee on

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 144

Davis & Loofbourrow

145

Continuing Medical Education (LCCME) operated until 1978, when

it disbanded. Subsequently, the ﬁve major parents groups (AAMC,

AMA, Federation of State Medical Boards, American Hospital Associ-

ation, and American Board of Medical Specialties) joined with the

Council on Medical Specialty Societies and the Association for

Hospital Medical Education to form ACCME. Initially ACCME was

a small and voluntary organization with support from the AMA, but

ACCME has evolved to more robust and independent status, with

clear regulatory authority for the accreditation of CME providers in

the United States. In particular, its position on commercial support

and CME is widely regarded as the clearest statement about the

permissible nature and extent of such support in the United States.

ACCME is the most complex and detailed of the accrediting bodies,

and it is important to understand it fully. All providers, according to

ACCME standards, must have a clear mission, purpose, and scope;

have an organizational structure and business process appropriate

for its purpose; engage in a planning process that identiﬁes learning

needs, states objectives, and presents CME in a way that complies

with clear commercial support standards; and evaluate the effective-

ness of programming for individual activities and for the program

as a whole.

Deﬁnitions

Shaped in large part by these historical and regulatory forces, CME

can be seen as a huge “enterprise,” comprising the production of

large numbers of courses, conferences, rounds, and other activities,

and led by the need to acquire credit hours. In this report, we use

the acronym CME to refer to the last and longest phase of education

undertaken by physicians. CME consists of any activity undertaken

to enhance competence, increase learning, and provide better care

for patients. It does not include master’s or doctoral level work

leading to an additional degree, and further, does not encompass

non-clinical learning, for example, education in the areas of business

management. More deﬁnitions are included in Appendix C. Other

definitions commonly used throughout this report include the

following:

— CPD, Continuing Professional Development: term used to

describe advancement by members of professional associations

to maintain, improve, and broaden the knowledge and skills

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 145

Continuing Health Professional Education Delivery in the United States

146

required by their profession; CPD may be seen as broader than

“CME” in the sense that it might encompass all forms of learning

and personal development, not just clinical education

— CEU, Continuing Education Unit: measurement used in continu-

ing education programs to deﬁne hours of participation in

qualiﬁed instruction

Methods

First, we looked for data reported in the United States produced

by the accreditation bodies of continuing education providers and

by other providers or provider organizations. Second, we made a

concerted effort to contact key informants in each discipline to

determine and validate our search strategies and to indicate other

data sources where available. Third, we searched and reviewed the

literature, using the key words “accreditation of continuing education

providers”and similar terms, to round out our ﬁndings and to provide

a perspective that would be helpful in generating comments for the

paper’s conclusions. We augmented this process with a Google

search on relevant key words and on the websites of professional

associations and authorities in ﬁve countries. Key words used to

elicit results were accreditation, CME/CNE/CPE accrediting body,

continuing medical education, continuing nursing education, and

continuing pharmacy education, as well as their respective acronyms.

In each discipline, we developed a series of questions and compiled

the responses we found. For results dealing with continuing educa-

tion in the nursing profession see Appendix A, and for results

dealing with the pharmacy profession, see Appendix B.

Results

I. Using the most recent data available, what are the major cate-

gories of CME providers? A wide variety of CME providers exists

in the United States, and these providers are categorized various-

ly across the disciplines. The ACCME employs the following

categories of CME providers:

1. Government/military (e.g., Centers for Disease Control and

Prevention; US Food and Drug Administration Center for

Drug Evaluation and Research; and National Institutes of

Health Foundation for Advanced Education in the Sciences)

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 146

Davis & Loofbourrow

147

2. Hospital/healthcare delivery system, including

consortium/alliance (e.g., Advocate Healthcare; Children’s

National Medical Center; and Georgetown University

Hospital)

3. Insurance company/managed care company (e.g.,

American Physicians Assurance; America’s Health

Insurance Plans; and Medical Mutual Group)

4. Nonproﬁt, including not-for-proﬁt foundation, voluntary

health association (e.g., American Cancer Society;

BioSymposia, Inc.; and Brain Trauma Foundation)

5. Nonproﬁt, including physician membership organization

and specialty physician member organization (e.g.,

Academy of Psychosomatic Medicine; Alliance for

Continuing Medical Education; and American Academy of

Allergy Asthma and Immunology)

6. Not classiﬁed/other (e.g., American Academy of Medical

Management; American Board of Quality Assurance and

Utilization Review Physicians; and American Society of

Health-System Pharmacists)

7. Publishing/education company, including communications

company, education company, and publishing company

(e.g., The Academy for Continued Healthcare Learning;

Academy for Healthcare Education, Inc.; and Advanced

Health Education Center)

8. School of medicine (e.g., Albany Medical College; Baylor

College of Medicine; and Boston University School of

Medicine)

9. State medical societies

The ACCME provides two avenues of accreditation: accreditation as

a provider and accreditation of an organization as an approver. The

ﬁrst eight categories listed above are accredited by the ACCME as

providers of CME. In contrast, state medical societies are accredited

as approvers of CME. The state medical societies then accredit

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 147

Continuing Health Professional Education Delivery in the United States

148

organizations within their own states as providers of CME. In 2005,

the AACME reported 2322 separate providers, only a slight increase

from 2003, in which they reported 2295 providers (no data available

prior to 2003). State medical society–approved providers comprised

69 percent of providers offering CME.

The publishing/education company category includes medical

education and communication companies (MECCs). This is a hetero-

geneous category, comprising entities that are public or private,

proﬁt or not for proﬁt, and focused on publications or other forms

of CME that provide medical education and information to health-

care professionals. These bodies provide CME mostly in the form of

courses, enduring materials, Internet activities, and journals. Courses

comprise almost half (48 percent) of the CME activities provided by

the publishing/education companies; they contribute 12 percent

of all CME course activities offered by CME providers. The Alliance

for Continuing Medical Education (ACME) and North American

Association of Medical Education and Communication Companies

(NAAMECC) are two organizations advocating for these companies;

the former group provides membership through a section (the

Medical Education/Communication Company Alliance, MECCA).

The NAAMECC provides advocacy and education for its member

companies. It represents 87 member organizations, of which 47

percent are accredited by the ACCME as providers of CME.

In addition to those ﬁndings recorded under “medicine” are the

activities reported by the American Osteopathic Association and its

accrediting body. In 2003, 162 provider organizations were accredited

by the American Osteopathic Association. In this instance, provider

organizations included afﬁliates (nonpractice and practice) (1),

colleges (19), foundations (6), hospitals (53), military (1), profes-

sional associations (1), and societies (81). In 2004 through 2006,

a total of 14.1 million credit hours were recorded by the American

Osteopathic Association over a 3-year cycle. This number is a slight

increase from the previous 2001 to 2003 cycle with a value of 13.1

million total credit hours.

The American Osteopathic Association

establishes the accreditation policy for sponsors of osteopathic CME,

but the Council on Continuing Medical Education has been given

the authority to monitor the osteopathic CME by the American

Osteopathic Association Board of Trustees.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 148

The role of the states in accrediting CME providers seems, if not

particular to medicine, then certainly a striking feature in that disci-

pline: it is by far the largest accrediting category. Finally, a number

of educational companies and pharmaceutical companies or com-

mercial interests produce accredited CME, often for proﬁt.

II.

What are their “activities”? To what extent do they list and describe

the courses and other learning formats (e.g., self-assessment

programs, quality assurance/improvement, etc.). The following

activities are listed as major categories of CME activities from

the ACCME:

1. Courses

2. Regularly scheduled conferences

3. Test item writing

4. Committee work

5. Performance improvement

6. Internet

Davis & Loofbourrow

12000

11500

11000

10500

10000

9500

9000

8500

8000

7500

7000

6500

6000

5500

5000

4500

4000

3500

3000

2500

2000

1500

1000

500

Courses Regularly Test Item Committee Performance Internet Other Journal CME &

Scheduled Writing Work Improvement Enduring Journal-Based

Conferences Materials

Government/ Hospital/ Insurance Non-profit Non-profit Not Publishing/ School of State Medical

Military Healthcare Company/ (Other) (Physician Classified Education Medicine Society

Delivery Managed Care Membership Company

System Company Organization)

P R O V I D E R C A T E G O R Y

N U M B E R O F A C T I V I T I E S

Figure 1.

Accreditation Council for Continuing Medical Education 2005

Annual Report Data Depicting the Breakdown of Number of

Activities by Provider Category

149

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 149

7. Other enduring materials

8. Journal CME and journal-based manuscript review

Overall, CME activities are focused on courses, regularly scheduled

conferences (the current name for rounds and other regular confer-

ence such as tumor boards), and activities through the Internet.

Courses comprise 57 percent of the activities offered by providers,

followed by regularly scheduled conferences at 19 percent and

Internet activities at 15 percent. Figure 1 shows the number of CME

activities reported in the Council’s 2005 annual report. Here again,

we see that the largest number of accredited providers were state

medical societies, and as expected, these societies delivered a large

portion of the activities. The state medical societies provided all of

the activities listed under the major categories. Sixty-seven percent

of their approved offerings were described as courses; regularly

scheduled conferences generated 26 percent of their activity.

The state medical society category includes all data by all providers

approved by these societies. These accredited providers produced

41 percent of the CME activities for medical education in 2005. In

2005, state society–accredited providers produced courses, regularly

scheduled conferences, test item writing, committee work, perform-

Continuing Health Professional Education Delivery in the United States

460000

440000

420000

400000

380000

360000

340000

320000

300000

280000

260000

240000

220000

200000

180000

160000

140000

120000

100000

80000

60000

40000

20000

Figure 2.

Accreditation Council for Continuing Medical Education 2001

through 2005 Annual Report: Accredited Hours of Instruction

by Provider Category

2001 2002 2003 2004 2005

Government/ Hospital/ Insurance Non-profit Non-profit Not Publishing/ School of *State Medical

Military Healthcare Company/ (Other) (Physician Classified Education Medicine Society

Delivery Managed Care Membership Company

System Company Organization)

P R O V I D E R C A T E G O R Y

N U M B E R O F H O U R S

150

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 150

ance improvement, Internet activities, enduring material, and journal-

based activities with courses and regularly scheduled conferences

comprising 94 percent of the activities delivered.

III. What trends can be observed over time?

The data reveal several trends over the 5-year period covered by this

report, primarily from ACCME data from 2001 to 2006, augmented

by data from the Society for Academic CME (SACME) in 2006.

General trends

. Figure 2 shows accredited hours of instruction from

2001 through 2005, obtained from the ACCME annual report. Hours

of instruction have increased gradually (along with an increase in

the number of accredited providers, from 674 [2001] to 2322 [2005]).

Because data were not available for 2001 and 2002 for state medical

societies, we used data from 2003 through 2005. A slight downward

trend can be observed in the reporting of the number of hours of

state medical society accredited provider activities from 2003 to

2005. In contrast, and with the additional exception of the non-proﬁt

category, all providers appear to have increased production of their

activities. Several of these providers report a peak in 2004 and

slightly less production in 2005, although overall the trends are

upwards. Clearly the production of 360,000 hours of accredited CME

Davis & Loofbourrow

Figure 3.

Number of Internet Activities by Provider Category, 2001–2005

2001 2002 2003 2004 2005

Government/ Hospital/ Insurance Non-profit Non-profit Not Publishing/ School of *State Medical

Military Healthcare Company/ (Other) (Physician Classified Education Medicine Society

Delivery Managed Care Membership Company

System Company Organization)

P R O V I D E R C A T E G O R Y

NUMBER OF INTERNET ACTIVITIES COMPLETED

10000

9500

9000

8500

8000

7500

7000

6500

6000

5500

5000

4500

4000

3500

3000

2500

2000

1500

1000

500

151

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 151

Continuing Health Professional Education Delivery in the United States

152

activity by medical schools continues to lead the ﬁeld among single-

provider entities.

Self-study Activities (Internet and Media-based Programs)

. Internet

activities, including live activities, searching and learning, online,

and Web-based enduring materials, have increased dramatically over

the past 3 years, a trend we observed especially among the publish-

ing/education companies and schools of medicine (Figure 3).

Since 2001, the ACCME data show a rise in the use of online courses,

online use of enduring materials, and accessing information using

the Internet for most of the providers. The largest increase occurred

for publishing/education companies beginning in 2003, and for

school of medicine providers, who significantly increased their

Internet-enabled activities in 2004. Harris et al.

studied the use of

online programs for CME and discovered that younger physicians

and women physicians appear to be using online programs at a

faster rate than their male, older counterparts. In 2003, state medical

society–approved providers ﬁrst reported data on Internet-based

CME to the ACCME. The ﬁrst year there was a spike of Internet

activities followed by a dramatic decrease in the number of activities

for 2004 and 2005. Explanations for this spike include the reporting

of a backlog of activities launched prior to 2003 and adjustments to

the ACCME activity category titles.

In contrast, the use of the Internet for conferences or self-study

activities, as recorded by a biennial survey of the Society for

Academic CME (SACME), has not increased. In 2006, the SACME

noted that only 9 percent of the schools reported using the Internet

to broadcast conferences, a slight increase from 1998 at 5 percent.

Internet self-study activities reported for 2004 and 2005 remained at

68 percent, essentially unchanged from other comparative data

(2002–2006) but much greater than the 25 percent reported in 1998.

Self-study CME in some format (Internet, written, audio, and video)

has increased slightly from 74 percent in 1998 to 80 percent in 2006.

Based on data from the SACME, four of ﬁve medical school CME

divisions offer self-study activities: over two thirds offer Internet

courses, 56 percent offer print-based materials, 51 percent provide

computer-based learning, and about a quarter offer video- or audio-

based resources.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 152

Davis & Loofbourrow

153

Courses and Regularly Scheduled Conferences. Courses continue to

be the main area of CME activity for all ACCME-approved sponsors

and state medical societies. Whereas the percentage of courses

produced has declined (from 57 percent of all activities in 2001 to

49 percent in 2005), their total number has risen.

Over a 5-year period, we noted a clear downward trend in the num-

ber of regularly scheduled conferences, from 40,000 to less than

15,000 since 2003. This phenomenon may be accounted for in part

by the rise in the number of Internet activities, course activities,

audio and video activities, and Web-based learning.

Conferences conducted by hospitals, healthcare delivery systems,

and state medical societies have dropped in number by 50 percent

since 2003. It appears that these providers have shifted to course

activity. In contrast, accredited regularly scheduled conferences

conducted by schools of medicine remain a large and rather ﬁxed

component of their activity, based on data from the SACME.

Courses remain a high percentage of the CME reported by the

SACME. In the 2006 survey data, 36 percent of the medical schools

reported more than 100 courses being offered. This is an increase

from 1998, when only 21 percent of the medical schools reported

more than 100 courses.

Performance and Quality Improvement

. In 2004 the AMA and AAFP

approved performance improvement as a CME activity, following

calls for such activity on the part of other specialty societies, the

Accreditation Council for Graduate Education,

and clear directions

from the research literature about the efﬁcacy of more direct, point-

of-care educational interventions based on performance and other

objective data. The response, however, has been lukewarm. In their

annual survey, the SACME reported in 2006 that only one quarter

of the responding medical schools provided between one and 20

performance improvement activities. A further 5 percent offered

Internet point-of-care learning activities. In addition, while over 80

percent of schools had full afﬁliations with hospitals, less than a

third were “somewhat” or “a lot” linked with hospital programs

geared toward education or quality improvement.

ACCME began reporting on performance improvement activities in

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 153

2005. In 2005, performance improvement activities constituted less

than 1% of the total number of activities provided by CME sponsors

and state medical societies. These numbers were unchanged in 2006.

Commercial Support

. In 2005, accredited providers reported a

total income of $2,250,468,669 to the ACCME. Of this amount,

$1,115,597,071 was income from commercial support, or slightly

under half of the total income. This overall ﬁgure is identical to that

reported by the SACME in 2006 and similar data from the same year

developed by the ACCME; however, the ACCME data indicate about

60 percent of commercial support. In contrast, in 2001, 41 percent of

the income was reported as emanating from commercial support. In

2005, the hospital/healthcare delivery systems, publishing/education

companies, and schools of medicine received more than 50 percent

commercial income. Commercial income reported by the state med-

ical society providers is reﬂected in the commercial total in 2003

and 2004 and does not include the income total for comparison. The

total commercial income reported by state medical society providers

has declined from $51,775,737 to $37,588,680 since the ﬁrst year of

reporting in 2003.

The categories of publishing/education companies, non-proﬁt

(physician membership organization), and schools of medicine

Continuing Health Professional Education Delivery in the United States

Figure 4.

Accreditation Council for Continuing Medical Education

2001 through 2005 Annual Commercial Income by

Provider Category

2001 2002 2003 2004 2005

Government/ Hospital/ Insurance Non-profit Non-profit Not Publishing/ School of *State Medical

Military Healthcare Company/ (Other) (Physician Classified Education Medicine Society

Delivery Managed Care Membership Company

System Company Organization)

P R O V I D E R C A T E G O R Y

INCOME DOLLARS / MILLIONS

600

550

500

450

400

350

300

250

200

150

100

154

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 154

Davis & Loofbourrow

155

exhibit an increase over time in the amount of commercial support

for CME activities (Figure 4). In 2005, all categories declined in the

amount of commercial income except the publishing/education

companies, which increased their commercial support by almost

$70 million. Commercial income for publishing/education companies

also increased in 2006.

Based on the 2006 biannual report of the SACME, more than 60

percent of medical schools reported that commercial support for

their CME activities was increased by more than a third. These

ﬁgures showed an increase from 1994 to 1995, when only 6 percent

of medical schools reported more than 60 percent of commercial

revenue. In addition, 15 percent of the medical schools indicated

that more than 20 percent of courses were supported by only one

commercial company. This is an increase from 1994 and 1995, when

15 percent of the medical schools reported support from one

commercial company.

The SACME summarizes this issue by indicating that commercial

support forms the largest source of income for their medical school

members. A useful way to express this is to describe the “typical”

(or median) medical school. Based on the same report, the “typical”

medical school produces 83 courses per year, generating over 700

hours of credit and attracting nearly 3000 physicians and 1500 other

professionals. Of these courses, slightly over half, or 45 courses,

were sponsored by commercial interests. This support is sufﬁciently

robust that an average of 11 courses/year would be cancelled if

it were not for the ﬁnancial aid provided by commercial entities.

Further, of these 45 courses, two, on average, were supported by

only one company.

Forces Affecting the CME Enterprise

Apart from the need to maintain competence, we see several major

forces, challenges. or directions which form integral parts of the

picture of CME in the United States.

Accreditation of CME Providers

Along with the credit system, another movement in CME has led

to the accreditation of CME/CPD providers. ACCME was the ﬁrst

organization in the United States, and perhaps the world, to develop

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 155

standards for CME. Providers seeking this accreditation status must

demonstrate that their programs have met the standards, referred to

as the Essential Areas and Elements (including the Standards for

Commercial Support) and Accreditation Policies, developed by

ACCME. Accreditation is awarded for a period of 4 to 6 years. CME

providers maintain compliance during the term by voluntarily

continuing to meet the ACCME standards and are required to submit

annual reports to maintain accreditation. Recognition as an accreditor

is awarded to state medical societies, which subsequently accredit

providers within their state.

There is a strong dependence on formal CME, such as courses,

conferences, and similar live, face-to-face meetings. None require

content-speciﬁc learning objectives to be met, though two groups

specify discipline-based content in family medicine (AAFP) and

osteopathic medicine (American Osteopathic Association). In all

these programs,

— “Other” categories form an eclectic mix. They can include teach-

ing or preceptoring activities; attaining credit at the master’s or

graduate level or subspecialty certiﬁcation; writing/publishing

in the medical literature; poster presentations; and advanced

degrees, self assessments, and research (American Medical

Association CME Physician resources for CME, 2005).

— There exists a clear movement away from exclusively formal CME

to acknowledging participation in a wide range of educational

activities, now including Web-based learning, although these

activities are still considered voluntary.

— While clearly linked to quality in the description of categories,

there is no or little speciﬁc mention of quality improvement

or peer-reviewed programs in the materials produced by the

appropriate organizations, for example, the AMA Physician’s

Recognition Award materials. Pilot programs have been docu-

mented,

but where such initiatives are mentioned, they are

considered optional.

We identiﬁed some unique elements among the programs, including

the following:

Continuing Health Professional Education Delivery in the United States

156

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 156

Davis & Loofbourrow

157

— Recognition of excellence: the American Osteopathic Association

offers a certiﬁcate of excellence, and fellowship in the College of

Family Physicians of Canada is awarded based on extra work of

a reﬂective, or performance-based nature, leading to extra credits

in a program called Mainpro-C.

— Speciﬁc content: certain associations (American Academy of

Family Physicians, American Osteopathic Association) distinguish

between those programs designed for and by their members, or

with their membership in mind, and those accredited programs

accredited or developed by other organizations.

— The Royal College of General Practitioners in the UK is the ﬁrst

program to remove the requirement for “credit” entirely. The

requirement in this program — the formal annual peer appraisal

process — is unique among its peer programs in medicine.

In medicine, parallel to the accreditation process for CME, the

Accreditation Council for Graduate (residency) Medical Education

(ACGME) has also made a sizable impact on the training of residents

and the expectations of their programs. In particular, we note the

advent of the ACGME competencies, which articulate new dimen-

sions of care and learning. These competencies are instructive not

only at this level of training but also at the CME level. They include

systems-based practice, which focuses on the healthcare system; the

judicious use of resources and other matters related to the practice

environment and its constraints and resources; and practice-based

learning and improvement.

A listing of continuing education requirements for medicine by state

is available at thewebsite of theAmerican College of Emergency

Physicians.

Professionalization and Training for CME Providers

A further force for change in the context of CME/CPD has been the

drive toward certiﬁcation and increased training on the part of CME

providers. Coming from a wide variety of backgrounds — confer-

ence management and planning, adult education, educational

psychology, medicine and allied ﬁelds, business management —

such providers have formed two professional associations in North

America. The ﬁrst of these is the Alliance for CME (ACME), an

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 157

organization with over 2000 member groups comprising CME/CPD

professionals in medical schools, hospitals, private organizations,

communication companies, and other commercial interests. Much

smaller, at a tenth the size, the SACME has its roots primarily among

medical school deans and directors of CME and focuses less on

the technology and broad scope of CME (the primary interest of the

Alliance) and more on its research and development interests. Both

organizations, now in their third decade, have grown in size and

importance in the number and type of training programs offered

to expand the professional repertoire of CME providers.

In addition, the Journal of Continuing Education in the Health

Professions and the Journal of Continuing Nursing Education

provide scholarly work to drive this provision, based in adult

education, informatics, quality improvement, and other disciplines.

Commercialization of CME

We note the strong inﬂuence of pharmaceutical (and increasingly)

technology-based companies, evident in the reports from the

ACCME and in the survey data from the SACME. Concern about the

extent to which such inﬂuence might drive the CME/CPD “agenda”

or its curriculum has given rise to clear statements from ACCME

and other organizations about creating ﬁrewalls and reducing bias

in presentations or content. Nonetheless, concerns remain, and

increasing pressures (as recently as this summer, from the Senate

Finance Committee about the question of oversight in this area) may

well affect this picture. The picture of commercialization of CME has

also been marked by an increased presence of medical education

and communication companies, originally constituted as a section

within the ACCME, and now represented by NAAMEC, the North

American Association of Medical and Education Companies. Initially

construed as publishing companies, these organizations now provide

a wide range of educational activities, are of a proﬁt or not-for-proﬁt

nature, and are heavily subsidized in many instances by commercial

interests. There has been some debate as to whether these companies

should be considered legitimate purveyors of CME.

The Methods of Continuing Education: Outcomes and Options

Increasing attention has been paid to the “outcomes” of CME, with

examples including competency-based assessment, performance, and

Continuing Health Professional Education Delivery in the United States

158

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 158

Davis & Loofbourrow

159

healthcare outcomes. Competency-based assessments of skills or

knowledge are commonplace in CME. Examples include self-assess-

ment programs and tests of knowledge frequently mounted by the

specialty societies. Increasing emphasis on simulation techniques

(post-course testing of resuscitation skills on mannequins, for example)

has made this process simpler and more ubiquitous and may drive

some of the CME agenda in future years. Further, emphasis has

also grown on the performance of physicians in the work or practice

setting, giving rise to consideration of performance measures and

their utility to measure such outcomes. Finally, patient outcomes—

though somewhat distal to the CME or CPD activity, are also key to

any discussion about the future of CME. It is clear that the “clinical

practice gap,” expressed by the Institute of Medicine

and Rand

reports,

is a product of some failures on the part of the CME enter-

prise to transfer knowledge in a way that is timely and useful.

Results from studies of the effectiveness of CME, especially in medi-

cine, shape the discussion of the future of CME. While not the focus

of this paper, these studies lend themselves to an observation of the

mismatch between “what is” (a situation in which less than effective

didactic lectures and conferences dominate the CME delivery picture)

and “what should be” (using methods such as academic detailing,

reminders, or feedback, which have a larger effect, relative at least

to physician performance improvement). This is not to say that

formal conference-based CME should cease to exist, but rather that

it has a purpose (the dissemination of new information, for example)

that would be useful at least to some clinicians but that must be

carefully tailored and matched to learning and course objectives

and the practical and clinical learning needs of all professionals

considered the target of an educational intervention.

Commensurate with our understanding of the delivery of educational

interventions and their effect — or lack of it—has grown an increased

awareness of the lack of self-assessment abilities. As summarized by

Davis and colleagues,

it appears as though in many situations, and

across many disciplines, professionals have difﬁculty judging their

past performance, current learning needs, or future competencies

without some form of benchmarking or feedback.

Finally, while the research in this area is still descriptive, it appears

that the technology of CME/CPD is changing rapidly. New vehicles

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 159

to “deliver” educational content include information and communi-

cation technology, represented by personal digital assistants (PDAs),

audio- and video-casting, Web-casting, and computer-based programs.

Despite its wide variety of modalities, such technology holds both

the promise and the reality of bringing informational resources to

the practice setting, addressing the need for “just in time” learning.

To the extent that such learning is therefore more useful and can

directly affect patient care, and that it might provide the clinician

with more feedback or benchmarking, we imagine that these tech-

nologies will be more effective and widespread.

Social, Contextual, and Practical Issues Shaping CME

Many forces for change arise from the current practice environment.

First among these is the nature of the environment itself, which is

marked by increased demands on time, the pressures of generalism,

the advent of new biopsychosocial pathologies, and the social

implications of disease and its management. Second and added to

this complex picture is the heavy information burden that challenges

all health professionals, described as the “drinking from a ﬁre hose”

phenomenon. Compelling reasons for changing the system also come

from a heightened awareness of gaps in the delivery of healthcare,

with a gap between what is known in prevention, disease manage-

ment, or treatment and what is actually done.

13,14

Finally, we note

the issue of social accountability for physicians and the need to

assure the public of the safety of their practices. In this process of

advocacy, perhaps the most compelling voice is that of the patient.

Norcini

suggests that patients believe that revalidation is necessary,

focusing on performance in practice.

Summary: Certain Challenges, Possible Solutions

This report focuses on the numbers of clinicians participating or the

numbers of courses. Little if any attention is paid— either in the report

or the data from which it is derived — to the content or method of

educational delivery, the site of this “delivery,” or its effect on the

patient or the healthcare system. Thus, it is evident to us that what

we portray here — the numbers and types of courses and other

offerings, the history and processes of accrediting bodies, and the

processes of producing education — operates at a level removed

from the actual delivery of care or even the delivery of education.

Further, these elements operate in ways that appear to be at odds

with what is currently known about physician learning and change.

Continuing Health Professional Education Delivery in the United States

160

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 160

Davis & Loofbourrow

161

Examination of this complex structure reveals a heavy dependence

on at least two major premises on which these systems are built,

and on which their linkages to competence, performance, and

healthcare outcomes are maintained. The ﬁrst is the dependence on

the physician’s ability to self-assess learning needs and to direct his

or her attention to meeting them; the second is the heavy depend-

ence — despite attempts to augment this category of learning or

add new independent learning methods to it — on formal CME or

similar educational activities, such as clinical meetings, rounds, and

conferences.

We do not wish to say that any aspects of this system are broken or

irreparable, or that the system has not been based on a strong and

expressed intent to monitor quality. Instead, we end this report with

a summary of each of the areas we have looked at along with sug-

gestions for consideration of possible solutions.

The Physician

Among the major issues this report identiﬁes is the extent and perva-

sive nature of commercial interests in the production of CME. While

some have called for a complete disengagement of commercial

funding from accredited CME, it is clear that such inﬂuences will

remain—whether in accredited formats or otherwise. Further, it

appears from the literature that self-assessment of professionals is

limited — despite its use in determining learning needs — and is at

least to some extent contingent on benchmarking and feedback.

A more coherent solution to the issues of bias in content and less-

than-adequate self-determination of learning needs may involve the

training and assurance of competency for physicians in the domains of

evidence-based healthcare and in appraising their own performance.

Including these measures as requirements of CME professionals may

ensure that these groups are better equipped to critically appraise

the research literature, judging for themselves the effect of med-

ications, weighing management strategies, and making rational,

evidence-based decisions. Further, training in self-assessment and

critical reﬂection based on objective feedback appear to be important

elements in training from undergraduate to graduate education and

certainly beyond. We note the introduction of “required” instruction

in certain elements mandated by some states and suggest that courses on

evidence-based medicine also be considered in this manner.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 161

Continuing Health Professional Education Delivery in the United States

162

The Educational System: Methods and Means

Despite a large and growing body of literature indicating that didactic

conferences appear to be less than optimally effective in changing

provider performance, the CME enterprise continues to concentrate

its efforts in the production of courses and regularly scheduled

conferences. This is not to say that there are no examples of other

interventions, such as academic detailing or reminders at the point

of care, but these appear not to be generated by CME providers, or

at least not documented in the activity proﬁles of CME providers.

In this light, we suggest that CME providers, led by an active and

evidence-based accreditation process, move to a more comprehensive

understanding of the literature of CME and to incorporation of its

results into the production and delivery of educational interventions.

Further, we urge providers to document these changes in a way that

makes tracking more readily achievable.Such a process may be enabled

by stronger and more explicit accreditation criteria in these areas.

Accrediting Educational Systems

Our investigation has shown a wide variety in accreditation and

reporting customs in the United States. Medicine has developed

a robust and detailed process leading to the accreditation of its

providers. Other developed nations appear to have much less need

for such accounting and rarely report these activities. These models

permit us to imagine a world with several possibilities:

— No CME accreditation system except for that provided by the

health systems and health professionals themselves to monitor

their own competence and performance. Such systems appear to

be much more commonplace outside the United States. We would

argue against such “deregulation” for two reasons. First, accredit-

ing educational competency appears to be a tall order for the

more clinical accrediting bodies to undertake. Second, such a

process might permit serious irregularities in the delivery of

educational interventions and puts too much weight on the

consumer-clinician-learner to judge the value of all educational

materials. Finally, the lack of an accrediting body leaves a void,

allowing no one to act as spokesperson or to lead the profes-

sions toward a posture of best educational practice.

— An accreditation system that is less individualized and less focused

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 162

on the speciﬁcs of the programs and content, allowing for more

focus on establishing standards or structural criteria to show a

provider the right path for developing a CME program. Here for

example, ACCME or an equivalent body would set high standards

or guidelines that organizations could adopt and implement in

their set of accreditation standards/processes. To some extent,

this has already occurred at the state medical society level.

— A more robust and rigorous accreditation system, including the

following features:

– Increased attention to making essentials and standards match

the principles of effective CME delivery.

– Heightened capacity to measure and provide feedback to

CME providers at a more detailed practice-based level to

ensure compliance with standards and essentials. Such feed-

back would not preclude the use of accreditation processes

as educational.

– An increase in measuring performance or outcomes rather

than and in addition to numbers of activities, hours, and

providers.

– Increased attention to the content of CME interventions,

stressing adherence to best evidence (and downplaying the

more negative approach to reducing commercial interests).

Such a process would allow for encouragement and tracking

of the incorporation of new competencies, such as those

proposed by the Accreditation Council for Graduate Medical

Education.

– The possibility of co-locating CME, CNE, and CPE data, allow-

ing for common terminology, agreed upon across systems, as

well as certiﬁcation, transportability, clarity, and observability.

Strong movements toward more focus on accreditation provide an

argument for increased measurement of performance or outcomes.

First, an international commitment to improving quality medical

education began in 2004 when the World Health Organization

(WHO) and the World Federation for Medical Education (WFME)

developed the WHO/WFME Guidelines for Accreditation of Basic

Medical Education. These guidelines or recommendations provide

Davis & Loofbourrow

163

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 163

Continuing Health Professional Education Delivery in the United States

164

assistance for providers and institutions in developing best practices

and an accreditation system. The Guidelines, which are accessible at

http://www.wfme.org/, consist of fundamental requirements of an

accreditation system, the legal framework, organizational structure,

standards and criteria, the process of accreditation, main elements

in the process of accreditation, decisions on accreditation, public

announcement of decisions, and beneﬁts of accreditation. Second, in

healthcare and business, examples of this movement toward quality

and accreditation include the Baldridge Awards, Toyota’s lean prin-

ciples, and the International Organization of Standardization.

We urge support for a stronger and more evidence-based accredita-

tion system of CME. Such a process would meet the goals we have

outlined and help to close the gap in clinical care.

Certifying the Competence of the Learner-Clinician

and the Healthcare System

In the clinical realm, two major regulatory forces exist: the accreditation

of hospitals and healthcare systems and the licensing of physicians

by state medical boards and their certiﬁcation by specialty boards. The

Joint Commission (until 2007 called the Joint Commission on Accredi-

tation of Healthcare Organizations [JCAHO]) develops standards and

provides evaluation and accreditation services for healthcare organi-

zations and programs in the United States.

The Joint Commission

uses surveyors who conduct onsite surveys at healthcare organiza-

tions to evaluate their operational practices and facilities.

Here we note the clear disconnect between the degrees to which

clinician competencies are determined in undergraduate and gradu-

ate health professional training versus CME. In CME, competency is

determined by the proxy measure of total number of hours spent in

lectures, the equivalent of awarding a medical degree based on class

attendance.

In this area, we note several ﬁndings from other jurisdictions and

studies of relevance to a discussion about improving the CME

“system” and its role in healthcare:

— Appraisal and the learning portfolio: The UK has developed a

system of peer appraisal using a learning portfolio for its general

practitioners, who comprise 50 percent of the physician popula-

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 164

Davis & Loofbourrow

165

tion. Here a senior, trained physician meets on a yearly basis

with a colleague and reviews a ﬁle of performance and compe-

tency data, along with a list of CME activities. Such a process

recognizes the complex layering and dimensionality of physician

competence and performance and “triangulates” self-learning

and self-identiﬁed needs, competency, and performance. In such

systems, strict adherence to the science (ie, reliability, validity,

and generalizability) of measurement should be incorporated

along with principles of adult learning.

— The concept of relative value, as described by Davis and Willis.

— Applied principles of quality improvement, such as METRIC

(Measure, Evaluate, Translate Research into Care), developed by

the AAFP.

— Finally we note the advent of 360-degree assessments of clini-

cian performance adapted from the business world. Here

measurements of performance are made by questionnaire or

survey of colleagues and patients.

We urge a more complete understanding of CME requirements and

their role in the assurance of competency and quality, both individu-

ally and collectively in the healthcare system. Such an understanding

may come from convening separate meetings to address this ques-

tion across disciplines, or in providing support for the work of one

or more national groups already engaged in this process.

Acknowledgments

The data collection efforts of this report were the responsibility of

Trina Loofbourrow, employed as an intern by the Association of

American Medical Colleges in 2007. In addition, we recognize the

signiﬁcant help and input of Dr. Murray Kopelow, Laure Perrier,

Paul Mazmanian, Anne-Marie Sesti, and Marian Taliaferro.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 165

REFERENCES

1. Tyler RW. New trends in education. Am J Psychiatry. 1966;122:1394–1398.

2. Davis NL, Willis CE. A new metric for CME Credit. J Contin Educ Health Prof.

2004;24:129–144.

3. Derbyshire RC. Relicensure for continued practice. J Med Licens Discipline.

2003;89:16–22.

4. Johnson DA, Austin DL, Thompson JN. The role of state medical boards in CME.

J Contin Educ Health Prof. 2005;25:183–189.

5. Miller SH: ABMS and reposition for excellence in lifelong learning: maintenance of

certiﬁcation. J Contin Educ Health Prof. 2005;25:151–156.

6. Accreditation Council for Continuing Medical Education. Accessible at

http://www.accme.org (Access date: March 15, 2008).

7. Rodgers DJ. Osteopathic continuing medical education. J Am Osteopath Assoc.

2007; 107:67–81.

8. DO-Online Accreditation and Program Approval. Accessible at https://www.do-

online.org/index.cfm?PageID=acc_main. (Access date: March 10, 2008).

9. Harris JM Jr, Novalis-Marine C, Harris RB. Women physicians are early adopters of

on-line continuing medical education. J Contin Educ Health Prof. 2003;23:221–228.

10. Accreditation Council for Graduate Education. Accessible at

http://www.acgme.org/acWebsite/home/home.asp (Access date: March 10, 2008).

11. American Medical Association Physician Recognition Award. Accessible at

http://www.ama-assn.org/ama/pub/category/15889.html (Access date: March 10, 2008).

12. American College of Emergency Physicians. CME Requirements by State. Accessible

at http://www.acep.org/cme.aspx?id=22366 (Access date: March 15, 2008)

13. Committee on Quality of Healthcare in America, Institute of Medicine. Crossing the

Quality Chasm: A New Health System for the 21st Century. Washington, DC:

National Academy Press; 2004.

14. McGlynn EA, Asch SM, Adams J, et al. The quality of healthcare delivered to adults in

the United States. N Engl J Med. 2003; 348:2635–2645.

15. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy

of physician self-assessment compared with observed measures of competence.

JAMA. 2006;296:1094–1102.

16. Norcini JJ. Where next with revalidation. BMJ. 2005;330:1458–1459.

17. The Joint Commission, About Us. Accessible at http://www.jointcommission.org/

(Access date: March 10, 2008).

18. Lipner RS, Blank LL, Leas BF, Fortna GS. The value of patient and peer ratings in

recertiﬁcation. Acad Med. 2002;77(Suppl):S64–S66.

Continuing Health Professional Education Delivery in the United States

166

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 166

APPENDIX A:

Continuing Education in the Nursing Profession

The earliest continuing nursing educational activities date back to

the late 1800s, when alumnae associations sponsored continuing

education (CE) activities for their members. In 1899, the Teachers

College at Columbia University, Department of Domestic Science,

offered a course in Hospital Economics—the first that would corre-

spond to what we currently consider CE.

At the time, nursing

education took place through hospital apprenticeship, and physician

education was moving toward the university. Nursing education,

including postgraduate work, grew substantially in the 1900s, but

nurses’ salaries remained very low. The concept of a “free” course

evolved, wherein nurses would trade work in order to attend a

course. In the 1920s, the American Nurses Association (ANA) and

the National League for Nursing Education (NLNE) began offering

“institutes” or short courses, and the Teachers College at Columbia

University began offering free CE.

In the 1960s, federal legislation (Title VIII in 1964 and the Medicare

Act in 1965), along with the establishment of the first coronary care

unit, laid the groundwork for federally funded CE efforts, in turn

leading to increased support at the university and college level.

The growing movement toward nursing specialization led to further

CE opportunities to establish certification in specialty areas. In 1976,

the American Nurses Association established the first accreditation

system for providers of nursing CE, and hospitals began sponsoring

CE programs for staff and non-employees through colleges, univer-

sities, and outside consultants. Professionalism in nursing remains

an important topic. Decreased government funding for CE and a call

to reduce non–bachelor degree nurses will reduce the number of

nurses requiring CE. One emerging area for nurses is in home care,

which will require a new form of CE focusing on home care tech-

niques and technologies.

State boards of nursing were established over 100 years ago to ensure

safe nursing practices. Requirements for maintaining licensure vary by

state. A summary of nursing requirements organized by state is avail-

able online at http://www.allnursingschools.com/faqs/boards.php.

The renewal period may be yearly or bi-yearly and varies from state

167

Davis & Loofbourrow

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 167

to state, as do contact hour requirements, which may be specific for

one topic, such as a 4-hour course on infection control every 4 years

(New York State Board of Nursing), or constitute a general require-

ment, such as 30 contact hours per renewal period (California State

Board of Nursing).

Methods

First, we looked for data reported in the United States produced by

the relevant accreditation bodies. Second, we made a concerted effort

to contact key informants to determine and validate our search

strategies and to indicate other data sources where available. Third,

we searched and reviewed the literature, using the key words “accredi-

tation of CE providers” and similar terms to round out our findings

and to provide perspective helpful in generating comments for the

paper’s conclusions.

Accrediting Bodies

The American Nurses Credentialing Center (ANCC) is a subsidiary

of the ANA tasked with managing its accreditation and certification

programs. The ANA established a voluntary accreditation system in

1974 for continuing nursing education. The governing organizations

of the accreditation system formed a National Accreditation Board,

and its five regional accrediting committees (RACs) accredited the

first program in 1976. Since that time, the governing bodies have

been consolidated into three regional committees, and in 1984,

the National Accrediting Board changed its name to the Board on

Accreditation. In 1988, the governance permitted the designations

of provider and approver accreditation. The new system provided

two common pathways for organizations seeking accreditation:

through the Board on Accreditation or through an accredited

approver. The ANA Center for Credentialing Services incorporated

itself as a separate entity called American Nurses Credentialing

Center (ANCC) in 1991. The ANCC takes on the additional roles of

examinations and certification renewals through the Certification

Program. The Center is accredited by the American Board of Nursing

Specialties and the National Commission for Certifying Agencies.

Categories of CE Providers

Similar to its colleagues in medicine, The ANCC employs the following

categories of CE providers:

168

Appendix A

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 168

169

Figure 1.

The Number of American Nurses Credentialing Center

Accredited Providers of Continuing Nursing Education in

Each Provider Category for the Year 2005

100

Government/ Hospital/ Specialty Private Publishing/ School State

Military Healthcare Education of Nursing Association

System Company

PROVIDER CATEGORY

NUMBER OF PROVIDERS

Davis & Loofbourrow

1. Government/Military (e.g., Centers for Disease Control and

Prevention; Naval Medical Education and Training Command;

and Joint Commission Department of Education)

2. Hospital/Healthcare Delivery System (e.g., Albany Medical

Center Hospital; Allina Hospitals and Clinics; and Children’s

Hospitals and Clinics)

3. Private (e.g., American College of Cardiology Foundation;

Annenberg Center for Health Sciences at Eisenhower; and

Audio-Digest Foundation)

4. Publishing/Education Company (e.g., Allegra Learning Solutions,

LLC; American Academy of CME, Inc.; and Anderson Continuing

Education)

5. School of Nursing (e.g., Georgetown University School of Nursing;

Indiana State University School of Nursing; and Indiana University

School of Nursing)

6. Specialty (e.g., American Association. of Occupational Health

Nurses; American Association of Critical Care Nurses; and American

Association of Diabetes Educators)

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 169

170

7. State Association (e.g., Alabama State Nurses Association;

American Holistic Nurses Association; and Arizona Nurses

Association)

In order to draw a comparison with physician data, we assigned a

provider category to each of the 224 providers based on the ACCME

categories. A large proportion of accredited providers for nursing

CE are publishing/education companies, at 30 percent, and schools

of nursing, at 22 percent (Figure 1). These percentages differ from

those for CME, in which the state medical societies comprise most of

the accredited providers.

Activities and Commercial Support

Searching the professional organizations’ websites and contacting

key individuals failed to yield data or statistics for recent or compar-

ative years on continuing nurse education activities and commercial

support. We learned through correspondence with the ANA that

they do not collect data from accredited providers, and previously

collected data were distributed only to those who were accredited.

Accreditation of Continuing Education Providers

Providers seeking ANCC accreditation must demonstrate that their

programs have met the Commission on Accreditation (COA) criteria

developed by the ANA. Continuing nursing education programs are

either accredited as providers for a 6-year term or by an Approver,

called a Constituent Member Association (CMA), for a 3-year term.

CMA-approved recognition is only available for ANA constituent

member associations, nursing specialty organizations, and federal

nursing services. CE activities approved by a CMA are only valid

for 2 years.

REFERENCES

1. Stein AM. History of continuing nursing education in the United States. J Contin Educ

Nurs. 1998;29:245–252.

Appendix A

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 170

APPENDIX B:

Continuing Education in the Pharmacy Profession

In the 1940s, about 35 schools of pharmacy began offering some form

of continuing pharmacy education.

After World War II, workers return-

ing to the pharmacy profession found a need for refresher courses,

and by the 1970s healthcare workers were encouraged at the federal

level to continue education in their field. Florida and Kansas were

the first states to establish regulations of minimum CE for renewal of

the pharmacist license. These regulations stimulated new programs

and concern for development of standards and assessing results.

Currently, all states have legislation mandating pharmacy CE for re-licen-

sure. These requirements are available in summary form online at

(http://www.acpe-accredit.org/pdf/2006StateBOPRequirements.pdf.)

Methods

First, we looked for data reported in the United States produced

by the relevant accreditation bodies. Second, we made a concerted

effort to contact key informants to determine and validate our

search strategies and to indicate other data sources where available.

Third, we searched and reviewed the literature, using the key words

“accreditation of CE providers” and similar terms to round out our

findings and to provide perspective helpful in generating comments

for the paper’s conclusions.

Accrediting Bodies

The Accreditation Council of Pharmacy Education is a national

agency accrediting professional degree programs and providers

of continuing pharmaceutical education. In 1932, the American

Association of Colleges of Pharmacy helped establish the American

Council on Pharmaceutical Education as an accrediting body setting

standards initially for pharmacy baccalaureate degree and doctor

of pharmacy programs. The formal accreditation process replaced

the earlier practice of school visits. The Accreditation Council for

Pharmacy Education expanded in 1975 to establish standards for

CE providers, and in 2003 the American Council on Pharmaceutical

Education changed its name to the Accreditation Council of

Pharmacy Education.

171

Davis & Loofbourrow

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 171

172

100

Colleges or Educational Local, State Publishers, Hospitals,

Schools Companies and or National Government Healthcare

Pharmaceutical Associations Agencies, etc. Networks

Manufacturers

PROVIDER CATEGORY

Figure 1.

The Number of Accreditation Council for Pharmacy Education

Accredited Providers of Continuing Pharmacy Education in

Each Provider Category for the Year 2005

NUMBER OF PROVIDERS

Appendix B

Categories of Continuing Education Providers

The Accreditation Council for Pharmacy Education (ACPE) employs

the following categories of CE providers in its reports.

1. Colleges and Schools

2. Education Companies and Pharmaceutical Manufacturers

(Pharmaceutical manufacturers were no longer accredited as of

July 2005 and did not supply data)

3. Local, State or National Associations

4. Publishers, Government Agencies, etc.

5. Hospitals, Healthcare Networks

Figure 1 shows the breakdown of number of providers by provider

category for the year 2005 year. The distribution of the 388 providers

is spread evenly among all the categories. Educational companies

and pharmaceutical manufacturers; colleges or schools; and local,

state, and national societies show the highest number of CE

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 172

173

Davis & Loofbourrow

providers.

We compared the ACPE 2005 data to data they published in 1985.

Colleges, schools, and associations remain the largest providers,

although, as is the case for continuing medical education, the

number of medical education companies has increased.

Activities

The following activities are listed as major categories from

the ACPE:

1. Live Activities

2. Home Study Activities

3. Combined Activities (activities that contain both live and home

study or mediated components)

4. Computer/Internet Activities

The total number of activities accredited in 2005 was 22,876. The

number has remained relatively stable since 1985. Live CE activities

composed almost two thirds (59 percent) of the total for 2005,

a percentage unchanged over time (in 1985 this figure was 54

percent). Home study CE activities comprised 39 percent, and

Internet activities 16 percent.

Accreditation of Continuing Education Providers

Providers seeking ACPE accreditation must demonstrate that their

programs have met a published set of standards, and accredited

programs can apply to the ACPE if they wish to grant ACPE certifi-

cation to their participants.

REFERENCES

1. Sonnedecker G. Kremers and Urdang's History of Pharmacy. American Institute for

the History of Pharmacy; 1986:246–248.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 173

APPENDIX C:

Definitions and Acronyms

DEFINITIONS

Categories of credit:

— Category 1: Activities such as publishing articles, poster presenta-

tions, medically related advanced degrees, and independent

learning. Examples include the American Medical Association

Physician’s Recognition Award Category 1 (AMA PRA Category

1), developed by the AMA as a method of assigning credit for

CME activities provided by approved CME providers and American

Osteopathic Association (AOA) method of assigning credit for

formal (A) and less formal (B) osteopathic CME activities.

— Category 2: Activities such as consultation with peers and medical

experts, small group discussion, self assessment activities, med-

ical writing and teaching residents. Examples include American

Medical Association Physician’s Recognition Award Category

2 (AMA PRA Category 2), developed by the AMA as a method

of assigning credit for CME activities provided by approved CME

providers and American Osteopathic Association (AOA) method

of assigning credit for formal (A) and less formal (B) non-osteo-

pathic CME activities.

• Committee Work: involvement in formal committees of hospitals

or professional organizations

• Computer-based, Instructional Program: accessed on a CD-ROM

or other computerized modality

• Conference: live, audio- or video-mediated, face-to-face or

web-based meeting of people planned in order that learners

acquire knowledge and/or discuss a topic

• Course: generally a live instructor-led session planned on a

one-by-one basis with learner participation in person

• Credit: the designation used to assign a value to continuing

education classes or time performed by the learner; in general,

174

Appendix C

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 174

credit is referred to in hours, as in hours of credit obtained

• Enduring Materials: learning resources used by clinicians,

generally printed, recorded or computer-based

• Internet or Web-based Activity: a CE experience available online

(e.g., live, enduring material, searching and learning)

• Journal CME and Journal-based: reading of an article, a

provider stipulated/learner directed phase (include reflection,

discussion, or debate about the material contained in the article)

and requirement for the completion by learner of a pre-deter-

mined set of questions or tasks relating to the content of the

material as part of the learning process

• Manuscript Review: participation in pre-publication review

process of a journal article

• Performance Improvement: physician identifies an educational

need through a measure of performance in practice, engages in

educational experience, integrates into patient care and then re-

evaluates performance

• Regularly Scheduled Conferences (RSC): planned, ongoing series

of sessions, generally involving members of a staff or organiza-

tion (e.g., hospital rounds)

• Regularly Scheduled Series (RSS): previously described as RSCs,

these have a series with multiple sessions occurring on an ongo-

ing basis, primarily planned and presented by an accredited

organization (e.g., Grand Rounds, Tumor Boards)

• Self Study: educational materials provided in paper, video, or

web-based to be completed at the learner’s convenience, often

accompanied by self-assessment examinations or other tests of

knowledge and skill acquisition

• Test Item Writing: participation in pre-publication development

and review of test items (e.g., multiple choice questions)

• Video and Audio: media used to assist with learning activities,

often used to assess clinical-decision making

175

Davis & Loofbourrow

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 175

• Workshop: live face-to-face meeting marked by the instructor-led

presentations of concepts and principles followed by real-time

application, such as the presentation of cases, to generate dis-

cussion, analysis, and demonstration of knowledge

ACRONYMS

• AAFP, American Academy of Family Physicians

• ACCME, Accreditation Council for Continuing Medical Education

• ACPE, Accreditation Council for Pharmacy Education

• AMA, American Medical Association

• ANA, American Nurses Association

• ANCC, American Nurses Credentialing Center

• AOA, American Osteopathic Association

• CACME, Committee on Accreditation of CME

• CFPC, College of Family Physicians of Canada

• MOC, Maintenance of Certification

• RCPSC, The Royal College of Physicians and Surgeons of Canada

• SACME, Society for Academic Continuing Medical Education.

REFERENCES

1. Accreditation Council for Continuing Medical Education. Accessible at

http://www.accme.org/index.cfm/fa/index.cfm/fa/home.home/home.cfm

(Access date: April 4, 2008).

176

Appendix 3

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 176

177

Remarks by Pamela H. Mitchell, Ph.D, M.S., B.S.

University of Washington School of Nursing

I appreciate David Davis’s paper because when I was asked to

provide some information about CE from nursing, I wondered where

I was going to get that information. His paper conﬁrmed that we

don’t have the same kind of information in nursing as is available in

medicine. However, we do have a substantial amount of continuing

education in much the same formats and courses as medicine, most

of which are accredited by The American Nurses Credentialing

Center (ANCC). It is useful to spend a bit of time talking about the

providers in nursing because most of our discussions have been

speciﬁc to continuing medical education.

As in medicine, several groups provide continuing nursing education.

Historically, university programs in nursing have been involved

in continuing education. In the 70s and 80s, specialization began

through the HRSA programs. (Nursing has Title 8 and medicine has

Title 7.) The two together, were part of what started specialization.

I would group the six categories in the Davis paper into four for

nursing. First are the schools of nursing. These include university

based schools of nursing that target primarily people who have

been prepared at the bachelors level or the advanced practice level,

such as nurse practitioners who are now doctors of nursing practice.

The community college programs are aimed primarily at the begin-

ning level and provide basic refreshers. Practice-based education in

hospitals is driven largely by requirements of the Joint Commission

and is aimed at point-of-care certiﬁcation, maintenance of skills, and

development of new skills, with some team-based work. The nurs-

ing specialty organizations are another major force, and specialized

nursing programs are analogous to medicine. Finally, some state

nursing organizations are involved in CNE.

For the specialty organizations, maintaining and upgrading skills

in a particular nursing specialty certiﬁcation that doesn’t require an

advanced degree often drives continuing education. That is probably

the area where commercialization is most evident in continuing

nursing education. If you go to the critical care nurses’ annual

meeting, with the big exhibits and all of the monitoring devices,

many of the exhibitors are also providing CE independent of the

CE that is being provided in the courses.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 177

I don’t think the state organizations for nursing are as prominent as

they are in medicine, although I am from a state that does not have

mandatory continuing education, so our state nurses association

does relatively little. Those states with mandatory continuing educa-

tion have more active state nursing associations. Whether you have

mandatory continuing education or not drives the CE effort. In my

state we brieﬂy had mandated CE, but then the state board of nursing

realized that it didn’t have the money to monitor a program, so it

cancelled it. So we are basically on our own as to whether or not

to do continuing education.

All the accreditation of providers is through ANCC. ANCC doesn’t

collect data about the number and categories of courses, but it does

accredit providers on the basis of whether the planning committee

knows something about the topic, the same sort of thing as in medical

education. We have not gone as far as medicine has in terms of the

competencies and practice based provision and assessment, and

those are areas where we can learn a great deal.

In nursing, everybody and nobody is responsible for CE. Everyone

has some stake as an individual, namely his or her own accreditation.

But the accreditation authority does not set the standards for what

CE should look like. There are simply standards for whether you are

documenting appropriately, who planned it, how long each session

is, and whether it is a lecture, but not whether it is interactive.

There has been some work on outcomes and whether CE really

changes practice, but not to the extent that we see in continuing

medical education.

My guess is that commercialization is probably about the 10 percent

of what was shown in Robert Steinbrook’s graph, but it is mostly the

vendors who help support the conference. Some of the vendors also

provide some CE but most don’t. If we ask is there a model for what

it would look like if we had to pay for our own continuing educa-

tion, CNE is one model because such a small percentage is paid for

commercially. That is not because we are purer than everybody

else. It is an accident of history that we haven’t been prescribing as

long and only a small part of the nursing workforce prescribes.

However, in hospitals and in some of the specialty organizations,

nurses do inﬂuence what gets bought in terms of specialized beds

178

Remarks

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 178

and monitoring equipment and the like, so that is where you see the

opportunity for commercialization to bias the education. In states

where CNE is mandatory, nurses themselves and their employers

pay for CE and we have good continuing education, so I think it is

entirely possible to do it without 60 percent of it being funded by

commercial sources.

Another question that I would ask is whether we should have a

more centralized set of expectations about what is appropriate CE.

That gets to the topic of lifelong learning, which is not the same thing

as delivering CE. My personal belief is that we ought to start with

the expectation that we must learn throughout our lifetime of pro-

fessional practice from the very date we start professional education.

We should be educating ourselves together around those things

where we have common interests with the expectation that we will

continue to do it and not simply with a set of CE offerings. It should

be a habit of professionalism that we should expect of ourselves,

and of our students. I think that we should be doing a great deal

more of this together across the professions. Nursing and medicine

have much more in common than we have separately. In particular,

when we are talking about team-based performance, we should be

having some team-based learning, and that means that we should

have common sets of expectations for the lifelong learning and for

that which is accredited.

Formalized credit does drive what becomes part of continuing

education. In nursing what is counted has been far more the confer-

ence, the continuing education hours of lecture because it is easier

to count; self-learning doesn’t count as much. We need to rethink

that, and this is a prime opportunity to rethink it together. I think

we could blow it up and try all over again.

DISCUSSION HIGHLIGHTS

The evidence base for what works or is needed for continuing edu-

cation must be fairly ﬂimsy, given the range of requirements state by

state. For instance, osteopaths in Vermont must be three and a half

times more effective than physicians in New Hampshire because

they have to take that many more hours.

179

Mitchell

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 179

It makes no sense for states to have such different requirements.

The continuing medical education community needs to deﬁne what

is important to deliver the best care and then go to the Federation

of State Medical Boards.

The status quo means that marketing products is the major incentive

of CME, not improving the quality of healthcare. It is time to develop

a system that cleans up the wildness of the status quo. Both under-

graduate and graduate medical education are structured and regulated

and produce a standard product, with a guarantee that most

graduates have a comparable set of competencies. It is time for

continuing medical education to have a system with some kind of

institutional oversight, curriculum design, a focus on quality and

outcome measurements, and no conﬂict of interest. We need a faculty

that can do that, using modern methods, and practice-based learn-

ing based on patient needs and outcome evaluation.

The incompetencies of physicians to teach are striking. Will new

competencies be required to provide the kind of help people are

asking for, or will excellent models be lost because of the absence

of education for the educators in this new kind of education?

Already many institutions have established teaching academies

utilizing excellent faculty educators to help faculty improve the way

they teach.

The purpose of continuing education is to assure competence and

provide accountability. We should be open to radically different

systems that can provide that assurance. One way might be a system

that rewards high performing organizations. Why should individuals

who are part of such organizations have to undergo continuous

competency assessment? The organization has demonstrated it

produces a good product and takes responsibility for the competency

of people who work there. Or, if individuals maintained certiﬁcation

through a demanding and intense process, that should substitute

for continuing education requirements. What we need is to accredit

systems and groups that perform on a reliable basis and produce

high quality healthcare, but because we lack measures for that kind

of performance, we have to look at the individual.

In hospitals, it is amazing how much linking reimbursement and

publicly provided quality information has focused attention on qual-

180

Discussion Highlights

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 180

ity improvement. For example, at our institution, inﬂuenza vaccina-

tion rates were especially low; suddenly, groups of professionals

came together to address this issue and came up with solutions.

Mobilization of the system became much easier and vaccination

rates went up. Contrast that with the more traditional approach in

which you test a clinician on the criteria for vaccination — what age

group, comorbidities, etc.

How much responsibility should academic medical centers have

for CE? We clearly are responsible for undergraduate education,

nursing, allied health, medicine, public health, etc. Every school

of medicine has infrastructure for GME and graduate nursing educa-

tion, but that is not necessarily true for CME. System CE is beyond

most academic medical centers. Maybe we can provide the support

for the educational part, but we cannot reformat whole health systems.

Our role should be to support the educational piece.

In North Carolina, practice-based networks link pediatric practices to

the medical school, so that the practices can continuously improve

their performance. That is one way academic medical centers can

support healthcare systems.

We need a consistent approach to collect CE information across pro-

fessions. Sixty percent of states now require CNE, but we have few

details about it. Also, CNE is too often inﬂuenced by “disease-of-the-

month” state legislation.

Regularly scheduled accredited conferences are decreasing in fre-

quency because it is too difﬁcult to get accreditation, especially now

with the new requirements about conﬂict of interest. Medical educa-

tion and communication companies do not offer regularly scheduled

conferences because they don’t ﬁt their business model.

We’ve had conversations, at least among continuing medical educa-

tion providers, that sounded like this for 30 years, but nothing ever

came of them, For the ﬁrst time in three decades, we have the

opportunity to take these ideas and work them into something tan-

gible and workable.

181

Discussion Highlights

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 181

182

Learning to Work Together

to Improve the Quality of Healthcare

Maryjoan D. Ladden, Ph.D., R.N.

Harvard Medical School

Reports from the US Institute of Medicine have highlighted the fact

that almost 100,000 people die each year due to medical errors.

These reports have focused national attention on the quality and

safety of healthcare. In addition, they have generated a considerable

amount of research on the causes of medical errors, strategies for

improving the quality and safety of care, and redesigning the health-

care system.

According to the Joint Commission,

communication failure was the

cause of over 70 percent of the errors or sentinel events reported to

them in the 10 years between 1995 and 2005. Several factors contribute

to communication failures. These include early training that teaches

nurses and physicians to communicate differently, historical power

differentials between the health professions that prevent patients

from speaking up, and lack of standardized communication proce-

dures to help create a shared mental model.

Many of the Joint

Commission’s 2008 Patient Safety Standards are aimed at structuring

and improving communication between health professionals.

The Institute of Medicine reports led researchers to develop and test

strategies for improving communication, collaboration, and teamwork.

Researchers have also explored links between effective communica-

tion, teamwork, and improved quality and safety. Jain et al.

found

that implementing four key system changes in the intensive care unit

(ICU) — including interdisciplinary rounds and a focus on team

decision making — had a signiﬁcant impact on nosocomial infection

rates, the number of daily adverse events in the ICU, average length

of stay, and average cost of an ICU episode. Between baseline and

re-measurement periods, nosocomial infection rates declined for

ventilator-associated pneumonia (from 7.5 to 3.2/1000 ventilator

days, P = 0.04) and bloodstream infections (from 5.9 to 3.1/1000 line

days, P = 0.03), with a downward trend in the rate of urinary tract

infections (from 3.8 to 2.4/1000 catheter days, P = 0.17). The

researchers also observed a strong downward trend in the rates of

adverse events in the ICU and in the cost of an average length of

stay per episode (from $3406 to $2973). The authors concluded that

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 182

a systematic approach, including implementation of multidiscipli-

nary teams and the use of evidence-based bundles, improved

communication and patient outcomes.

The US Department of Veteran’s Affairs (VA) has also focused on

enhancing communication, interprofessional collaboration, and

teamwork to decrease adverse events and improve patient outcomes

within the VA system. From 2003 to 2007 they implemented a VA

Medical Team Training program in 43 VA medical centers across the

United States.

The program, which is based on aviation principles

of crew resource management, includes a full-day interactive learning

session, administration of pre-and post-intervention safety attitudes

questionnaires, and follow-up semi-structured interviews.

The Reality of Collaboration and Teamwork in Practice

Every day in practice, health professionals must work together to

address a patient care issue or solve a systems problem. In some

situations (ICU, operating room), an explicit structured interprofes-

sional approach is necessary. In others, health professionals may

come together once or twice around a speciﬁc patient or clinical

issue and then return to their independent work. Both of these

examples require a team, deﬁned as two or more individuals who

have speciﬁc roles, perform interdependent tasks, are adaptable,

and share a common goal.

Most clinicians learn about teamwork on the job. Health professionals

are educated on improving relationships between patients and

healthcare providers. However, enhancing communication, collabo-

ration, and teamwork among health professionals is rarely included

in educational curricula. Effective communication and teamwork are

not a consequence of grouping health professionals together in the

same location or designating them as a team. Teamwork requires

knowledge of and respect for the skills and contributions of others

as well as positive attitudes toward collaboration.

In its 2003 report on education in the health professions, the Institute

of Medicine included “work in interdisciplinary teams” as one of the

ﬁve core competencies to be required by all healthcare professionals

to meet the needs of the 21st century healthcare system.

These

recommendations have been embraced by healthcare organizations,

the health professions, and accreditors.

3,10–13

183

Ladden

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 183

Preparing Health Professionals for Teamwork

and Collaboration

Despite the recommendations and the evidence that effective team-

work is necessary for high-quality, safe care, health professionals

have few real opportunities to learn how to work together. Most

health professional education in the formative years remains in

discipline-speciﬁc silos. Didactic instruction occurs in separate

buildings, and even though medical and nursing students may share

the same site for their clinical training, they (and their faculty) do

not know each other or work together — even if they are caring for

the same patient.

Education in the Formative Years

Many individuals have suggested that exposure to other disciplines

early in the professional education continuum will foster respect

for one another’s skills and positive attitudes toward collaboration,

leading to improved communication and teamwork. Since the 1960s,

interprofessional education, deﬁned as when healthcare professionals

learn together, learn from each other, and learn about each others’

roles, has been promoted as a way to stimulate collaboration.

However, in the formative years of professional training, interpro-

fessional education is often stymied by such factors as scheduling

logistics, mismatch in learner’s age and educational level, and

academic policies.

The results of early interprofessional education efforts have also

been mixed. Anecdotally, educators report positive changes in

attitudes toward collaboration. However, in 2001, Zwarenstein et al.

concluded in a Cochrane meta-analysis that evaluation studies

lacked the methodological rigor to show the impact of interprofes-

sional education in the formative years on professional practice

and/or healthcare outcomes.

Some might still argue that, because the formative years in profes-

sional education are an impressionable time, this period represents

the best opportunity to instill the values and skills required for

effective teamwork and collaboration. Interprofessional teaching

and learning efforts during these years are necessary, but they are

not sufﬁcient to assure successful collaboration and teamwork after

graduation. Once in practice, health professionals experience a

184

Learning to Work Together to Improve the Quality of Healthcare

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 184

healthcare system that supports a discipline-speciﬁc focus and

upholds the norms and attitudes that deter collaboration.

They also

often see their colleagues modeling behaviors and attitudes that do

not support collaboration.

17–19

Education in the Practice Years: Using Continuing Education to

Improve Collaboration and Teamwork

The early training years are just the beginning of a long continuum

in professional education. In most states, continuing education (CE)

is required for re-licensure. CE is also one component of professional

certiﬁcation. Medicine has developed a professional competency

framework, with its Maintenance of Certiﬁcation program, that

encourages life-long learning and practice improvement. Nursing

is not far behind in the competency movement.

Over the course of their career, health professionals spend more years

in continuing professional education, both formal and informal, than

they do in their early professional training. The audience for most

CE programs, from small ofﬁce-based activities to large auditorium

conferences, is already often interdisciplinary. Physicians and nurses

learn side by side about best practices in clinical care management.

How can we use CE to foster collaboration and teamwork values

and skills and improve the quality of the care we deliver? In their

re-analysis of the Zwarenstein et al.

ﬁndings on interprofessional

education, Koppel et al.

identiﬁed three factors that make CE an

ideal venue for teaching and learning interprofessional skills and

values. These factors are the duration of the program, the location of

the program, and the developmental stage of the learner. Koppel et

al.

found that programs that engage interprofressional learners

over a longer time period are more likely to generate behavior change

in the individual (provider or patient) or organization. Many CE

programs are held over several days, with content presented in

a variety of learning formats. In these programs, interprofessional

concepts and skills could be (and should be) integrated with the

clinical content to demonstrate interprofessional management of

acute and chronic conditions.

Teamwork and collaboration could be presented in a didactic case-

based format and then reinforced over the next few days in smaller

185

Ladden

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 185

discussion groups, active exercises using real-life patient scenarios,

and skill-based workshops with feedback. Likewise, simulations of

common team management situations or resource management

principles would be useful interprofessional additions to a clinical

CE program. Because many clinicians attend the same professional

meeting each year, a curriculum could be developed to reinforce

and then expand interprofessional concepts, teamwork, and collab-

oration skills over time.

In terms of location, Koppel et al.

found that CE programs at

the practice site are more likely to produce behavior change in the

clinician, patient, and organization. Small CE programs at a practice

site or hospital draw clinicians who are already working together.

Interprofessional concepts and skills could easily be integrated into

clinical case presentations, as well as into other venues, such as

team meetings and quality and safety rounds. Because the clinician

audience is generally stable, new knowledge and skills can also be

tested and reinforced over time.

To be successful, interprofessional education and practice efforts

need strong administrative and political support from clinical and

organizational leaders. Programs at the practice site are highly visible

and can easily engage both groups as faculty. In the era of pay-for-

performance and other ﬁnancial practice incentives, political capital

can also be generated by linking CE programs promoting collabora-

tion and teamwork with improvements in quality and safety. These

improvements are more likely to occur when all the clinicians

responsible for a patient group or service come together to share

knowledge and experiences and then plan for and test practice

changes.

Koppel et al.

also found the learner’s developmental stage to be

an important factor in the success of interprofessional education.

Interprofessional education interventions early in professional training

had a positive impact on attitudes and knowledge, but these effects

did not last over time or translate into behavior change in practice.

However, when interprofessional education was used in CE with

practicing clinicians, it was more likely to produce individual and

organizational behavior change. Perhaps experienced clinicians,

because they have a broader world view, can more quickly recog-

nize the relevance of new knowledge and skills and integrate them

186

Learning to Work Together to Improve the Quality of Healthcare

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 186

187

Ladden

more easily into their practice repertoire.

Professional socialization plays an important role in how health

professionals approach collaboration and teamwork. Socialization

begins prior to and continues during professional training.

Attitudes

are often difﬁcult to change until health professionals have the

opportunity to get to know each other in real-life practice situations.

Until that happens, stereotypes prevail and there can be only a

superﬁcial understanding of other health professionals’ skills and

contributions to practice.

Health professional learners and practicing clinicians learn best when

new knowledge and skills are presented within the context of their

work and they can immediately recognize the applicability to patient

care. Presented outside the appropriate clinical context, collaboration

and teamwork can easily be seen as theoretical and “nice to know”

rather than practical and “need to know.” Quality and safety are

powerful incentives for health professionals. Educational programs

that link collaboration and teamwork with high-quality, safe, and

effective patient care may have the best chance of success.

REFERENCES

1. Institute of Medicine. To Err Is Human: Building a Safer Healthcare System.

Washington, DC: National Academy Press; 2000.

2. Institute of Medicine. Crossing the Quality Chasm: A New Healthcare System for the

21st Century. Washington, DC: National Academy Press; 2001.

3. The Joint Commission. Sentinel Events Statistics, 1995-2005. Accessible at

http://www.jointcommission.org/SentinelEvents/Statistics (Access date: August 25,

2007).

4. Leonard M, Graham S, Bonacum D. The human factor: the critical importance of

effective teamwork and communication in providing safe care. Qual Saf Healthcare.

2004;13:i85–i90.

5. Jain M, Miller L, Belt D, King D, Berwick DM. Decline in ICU adverse events, noso-

comial infections and cost through a quality improvement initiative focusing on

teamwork and culture change. Qual Saf Healthcare. 2006;15:235–239.

6. Dunn EJ, Mills PD, Neily J, Crittenden MD, Carmack AL, Bagian JP. Medical team

training: applying crew resource management in the Veterans Health Administration.

Jt Comm J Qual Patient Saf. 2007;33:317–325.

7. Salas, E, Dickinson, TL, Converse, SA. Toward an understanding of team perform-

ance and training. In: RW Swezey and E Salas, editors. Teams: Their Training and

Performance. Norwood, NJ: Ablex; 1992.

8. Firth-Cozens, J. Multidisciplinary teamwork: the good, bad, and everything in-between.

Qual Healthcare. 2001;10:65–66.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 187

9. Institute of Medicine. Health Professions Education: A Bridge to Quality.

Washington, DC: National Academy Press; 2003.

10. Accreditation Council for Graduate Medical Education: Outcome Project. Accessible

at http://www.acgme.org/outcome/project (Access Date: July 20, 2007).

1. American Association of Colleges of Nursing 2002. Nurse Practitioner Primary Care

Competencies in Specialty Areas: Adult, Family, Gerontological, Pediatric, and Women’s

Health. Accessible at http://www.aacn.nche.edu/Education/pdf/competencies

(Access date: July 20, 2007).

12. O’Neil EH and the PEW Health Professions Commission. Recreating Health

Professional Practice for a New Century. San Francisco: PEW Health Professions

Commission; 1998.

13. Walker PH, Baldwin D, Fitzpatrick JJ, et al., and the National League for Nursing

Appointed Interdisciplinary Health Education Panel. Building community: developing

skills for interprofessional health professions education and relationship-centered

care. Nurs Outlook. 1998;46:88–89.

14. Ladden MJ, Blair K, Moore S, Roberts K. Educating health professionals for collaborative

interprofessional team practice: lessons from the ﬁeld. In: New Paradigms in

Advanced Nursing Practice: Teaching & Technologic Strategies in Nurse Practitioner

Education [monograph]. Washington, DC: National Organization of Nurse

Practitioner Faculties; 2003.

15. Zwarenstein M, Reeves S, Barr H, Hammick M, Koppel I, Atkins J. Interprofessional

education: effects on professional practice and healthcare outcomes. Cochrane

Database Syst Rev. 2001;1:CD002213.

16. McPherson K, Headrick L, Moss F. Working and learning together: Good quality care

depends on it, but how can we achieve it? Qual Healthcare. 2001;10:ii46-ii53.

17. Curran VR, Sharpe D, Forristall J. Attitudes of health sciences faculty members

toward interprofessional teamwork and education. Med Educ. 2007; 41:892–896.

18. Gardner SF, Chamberlin GD, Heestand DE, Stowe CD. Interdisciplinary didactic

instruction at academic health centers in the United States: Attitudes and barriers.

Adv Health Sci Educ. 2002;7:179–190.

19. Thomas EJ, Sexton BJ, Helmreich RL. Discrepant attitudes about teamwork among

critical care nurses and physicians. Crit Care Med. 2003;31:956–959.

20. Koppel I, Barr H, Reeves S. Establishing a systematic approach to evaluating the

effectiveness of interprofessional education. Issues in Interdisciplinary Care.

2001;3:41–49.

21. Langley GJ, Nolan KM, Nolan TW, Provost L. The Improvement Guide. San Francisco:

Jossey-Bass; 1996.

22. Oandasan I, Reeves S. Key elements of interprofessional education. Part 2: factors,

processes and outcomes. J Interprof Care. 2005;1(suppl):39–48.

23. Ladden MJ, Bednash G, Stevens D, Moore G. Educating interprofessional learners for

quality, safety, and systems improvement. J Interprof Care. 2006;20:497–505.

188

Learning to Work Together to Improve the Quality of Healthcare

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 188

189

Remarks by Carol Havens, M.D.

Kaiser Permanente Health Center

If we think of interdisciplinary education as simply putting people

from different disciplines in the same room so they all hear the same

thing, we’re wasting an opportunity to get the best that we can out

of the opportunity of educating team members to work together.

I was fortunate in my training as a family practice resident at UC

Davis, where they had an NP training program. As residents, we

were paired in teams with the NP and PA students. In the clinic, we

had a team with a resident paired with a PA student or an NP stu-

dent paired with a clinic nurse and a medical assistant. In this way,

we had the opportunity, as residents, to discover the highest and

best use of all the people on that team with the patients we were

seeing. Challenging complex patients fascinated the PA student I

was paired with and I liked well baby checks, so she had all the

really difﬁcult patients and I took all the well baby checks.

As long as we are going to blow up this continuing education sys-

tem and start over again, I’d like to put in a plug for another step in

the process. Don Moore talked about establishing activities based on

the need — the gap between what is and what could be — and

designing your activities with that end in mind, so that you know

what you want to get out of the activity as you are planning it. I’d

like to add as a next step what is the process to get there and who

needs to be involved in that process, and then what is the best way

to educate for that. Then we can start looking at this team, this

group of people taking care of patients, and who should be provid-

ing what part of care.

That’s not unlike the collaboration between primary care providers

and specialists. I’m a primary care provider and I’m also a specialist

so I get to do both ends. There’s always that challenge between

when is it no longer a primary care problem and at what point should

I refer, and then who is going to do the follow up EKG after we’ve

done the CABG? If it’s not clear who’s going to do that, either things

don’t get done, or things get done twice and lead to unnecessary

duplication. It’s really important that we look at the process of care

and I think it’s great that we’re talking about systems of care, because

that is where it’s going to happen. We should be looking at the

whole process, at who should be responsible for what part and at

what point that hand off occurs.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 189

We talked about culture change, and this is another culture change.

We talked about the Lone Ranger physician in the United States, and

it’s interesting when I talk to my colleagues in a variety of practice

settings about team based care and the opportunities to work together,

they’re all for it. They think it’s a great idea. But, when you get

down to the nitty gritty, such as who is going to call back patients

with normal labs, the answer is always the physician, because that’s

my job and the buck stops with me. No matter what the job, the

physician should be doing it. I never quite understood the concept of

team care when the physician is supposed to be doing everything.

That’s not true for everybody, but a lot of physicians still feel that

way. Just getting them even to think about the team process, and

what the opportunities are and who could be doing what, is going

to be a challenge.

I am going to describe what we have done at Kaiser around the

issue of interdisciplinary education. The concept of team based care,

I think, is one that’s easier to understand in the practice than it is

in the theory. Our greatest successes have been in two areas. One

centers around patient safety, particularly around things like critical

events practice where we actually train teams and physicians to deal

with critical events. This is the process: First we explain your role,

then show this is where your role changes to this role, and how

you hand it off, and, ﬁnally, we actually have them practice. We do

simulations; we record who did what and we ask questions. How

did it happen? How did the hand offs work? Did anything fall

through the cracks? And, by the way, how did the patient do?

The other has been in chronic conditions management, where we’ve

had some success, and that’s very much team based care. It follows

the same process of looking at where we are with chronic conditions

and where we want to be, also what could be and what we need

to do to get there. Clearly, physicians are not going to be able to

do this all by themselves; they have a role. Nurses can’t do it all by

themselves; they have a role. We have the opportunity to look at

what’s the best way to provide the care for these patients with chronic

conditions. We ask who should be responsible for what and how

do we set it up so that everything happens as smoothly and as easily

as possible so that we get the best use of everybody involved, and

we make sure that the patients get the best care possible.

190

Remarks

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 190

191

All that was done with the end in mind, with a clear picture of how

we wanted those teams to function and what the roles were of

everyone on the team. Everybody agreed to what those roles were.

And our hope is that we can start building on those successes and

start demonstrating the opportunities and the advantages of both

team based care and training as teams so that people will start looking

for other opportunities and other places where that approach might

also be a success.

DISCUSSION HIGHLIGHTS

We need to think in a broader perspective about continuing education.

We fail if we fail to think of the continuing professional development

of all the health professionals with whom we are working.

Nurses and physicians follow a different path; physicians focus on a

specialty right after their residency while nurses tend to be prepared

as generalists and then either may stay generalists or may focus on a

specialty. One thing continuing education providers in nursing do is

role development because a nurse who is a good clinician and then

becomes a manager doesn’t necessarily have management skills.

The same is true with administration.

The whole staff development function is more characteristic of

nursing than medicine, partly because most nurses are “staff,” but also

because staff development is embedded in the culture of nursing.

These programs are not well organized or systematic. Some are

accredited, some are not, but a lot of money goes into this work

and it isn’t looked at in any systematic way.

Maybe where we should do this together is where we come together

in places like hospitals, ambulatory care clinics, and home health.

There we come together around staff development and learning is

practice based.

If we really believe that teamwork and cross-disciplinary and spe-

cialty and professional training are critical and important, we ought

to recommend increasing the credit value of learning in a team rela-

Havens

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 191

192

Discussion Highlights

tive to learning in solo or isolation. That would be an incentive that

would make people pay attention.

The American Association of Colleges of Nurses (AACN) and the

American Association of Medical Colleges (AAMC) meet at the same

time of the year, every year, yet they never meet together. AAMC

seems more involved with CME than AACN with CNE. Bringing these

organizations together to think about the continuum of education

could help.

We need to create measures to count what is valued. We are not

counting the right things now. One measurement is how well a

team works together.

The idea of a whole system that links all the health professions

together in a way that directly affects outcome makes sense, but

how do rural primary care practices get linked into a system? Those

have nurse practitioners and pharmacists who are providing care

and they, too, need to be linked into a system somehow.

Health professionals work together so why shouldn’t they learn

together? We need to get state and national accrediting boards to

agree. We need to ﬁnd ways to give credit for that kind of learning.

Are things changing with respect to teamwork? The composition of

specialty teams seems to be changing. More and more nurses are

PhDs; how will this change teams? Does good teamwork enhance

payment? Perhaps it should.

In the UK, there are inter-, multi- and trans-disciplinary activities.

This is creating a huge research agenda.

The doctor does not have to be captain of the team. In our practice,

the team leader changes. The ofﬁce nurse is team leader for OSHA

issues. The diabetic nurse leads us for diabetes. Also, we ﬁnd we

need a good deal of cross training to make teams work.

The ACGME requires evaluations of residents by nurses. Perhaps

that approach should be built into CE.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 192

193

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 193

194

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 194

195

IV.

Reaction from

the Trenches

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 195

196

Reaction from the Trenches

Remarks by Grant S. Fletcher, M.D., M.P.H.

University of Washington School of Medicine

My topic is CME from the trenches. My ﬁrst thought on hearing

this topic was that I don’t know anything about CME. Having only

recently completed my training, I have never submitted any credit

for CME, so I did a little investigative work and asked a colleague

what he does for CME. He said that, when the time arises, it consists

of going to a website at an academic medical center the night before,

cramming all night, and completing his credits. And that was that.

It sounded scary to me. Through this conference I’ve gotten a better

appreciation of what we’re talking about, and it depends on how

you deﬁne CME.

In the context of CME as continued learning, as a recent graduate of

the medical education system, I think about how we keep that fund

of knowledge and expand on it afterwards. For me, this perspective

comes from several sources. Before going to medical school I had

public health training and looked at decision analysis and cost-effec-

tiveness analysis. I felt I had some tools, but not the questions, going

into medicine. I thought a lot about how we acquire knowledge and

then use it. In medical school, I had my antenna up to understand

how it’s done there. I went to nine different institutions during my

medical training. They ranged from community medical centers to

academic health centers, to tertiary private centers and the VA system,

so I saw how I was taught from a lot of different perspectives. After

my medical education, I spent a couple of years as part of a group

that formed a hospitalist program in a community hospital, so I saw

how it was done in the community as well, and then, in more recent

times, I have come back to an academic medical center.

I found a lot of interesting contrasts in these various settings. In my

medical training, one of the interesting things was what we were

taught and what we were not taught so well. I acquired some skills

and knowledge, I saw a great diversity of patients and I think pro-

fessionalism was instilled during that time. However, I think the

actual learning to learn beyond the residency and medical school

suffered greatly. I think, for the most part, we’re not taught much

about the learning process. During medical school, you are inundated

with facts and you have to pare it down, get very quick information,

and move on. What that means, often, is the path of least resistance,

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 196

and that means looking to fellow residents, students, attendings, and

specialists, asking them what they think, and then looking up very

composite references. You’re not doing an idealized sifting through

the evidence and reasoning through multiple sources of evidence. It

is quick and dirty, but that’s the learning habit that I think is instilled

at the time.

When I went to the community hospital after my medical education,

what struck me ﬁrst of all are the sources of information that people

use. They have their limitations and their advantages, but I don’t

think it is anywhere near the idealized view of strictly evidence

based and thinking through it on your own. And second, I was

struck by the lack of access I had to information. I took access for

granted during my medical education, but suddenly I was having

very limited access. I had an UpToDate subscription, and I kept that

going, but beyond that, I was used to going to various journals to

read more. Access to journals was available in the academic medical

center, but what was available in the community hospital was paltry.

In thinking about ongoing education, access to information always

is important, not just in the formal settings but in self development

as well.

Then, when I went from a community setting back to the academic

setting, it struck me how much easier it is to acquire information

within the academic center. You feel like you’re on an island in the

community center, especially if you’re in a small group. I had this

visceral sense that my knowledge and skills would be decaying and

it would be a struggle to keep those up, whereas with an academic

center, you can take some shortcuts because of the questions that

are raised. If there’s a new set of guidelines and people are talking

about them, you automatically keep abreast of some of the latest

information. You can always go to the guidelines later. Just knowing

that there are developments out there is helpful to begin with. A lot

of our discussion has been around academic medical centers or

large practices, whereas I think it’s much more difﬁcult to access

information in the very large number of small, isolated practices that

we have in this country.

Another area that I like but had very little of in community practice,

was actual feedback on what I was doing. I can recall receiving only

one feedback report in the two and a half years that I was in com-

197

G. Fletcher

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 197

198

munity practice. The report was looking at whether I prescribed the

appropriate antibiotic. They sampled four patients and I had not

given the appropriate antibiotics to one. They didn’t know which

patient this was and there was no connection to the records, so I

had to go through multiple hoops to get the information to learn the

speciﬁcs of the case. That was only one report in the entire two and

a half years. If we really want to know how we’re doing, having

some sort of regular formalized feedback would be ideal.

I believe being instilled with professionalism during medical training

is absolutely key. It is my impression that many situations that arise

during practice are far beyond quality measurement. In these situa-

tions, medical education instills an internal professionalism motor

to go to the trouble to ﬁnd the correct clinical answers. It is very

important to keep that motor running when physicians are out in

practice. We often talk about guidelines but these are very general

approaches for medical conditions that you see again and again.

The reality is, in practice, you see many clinical problems that do

not lend themselves to guidelines and easily measured outcomes.

I’m not sure if there’s an easy way to measure how well a clinician

has developed these habits and has that sense of professionalism.

For example, I was consulting on a neuro-surgery patient who had

an inferior vena cava ﬁlter placed for a deep venous thrombosis.

The patient had been bounced around to many teams and had a

very complicated hospital course. I was being consulted for some-

thing else and noticed that this IVC ﬁlter had been in for eight

weeks. It was a retrievable ﬁlter that was supposed to come out at

six weeks. I called around, but nobody seemed to know whether six

weeks was set in stone, what would happen after the six weeks, or

what were the consequences of keeping the IVC ﬁlter in place. That

was well beyond a 30 second info button question. These kinds of

clinical questions arise all the time, so I’m not sure how you would

measure the quality of care in that patient. I could have taken the

easy route, say somebody said just leave it in, and be done with it.

There may be no information about what is the correct thing to do,

but sometimes even discovering that there’s no information is difﬁ-

cult and time consuming.

I had very little exposure to simulations during my training, but I

Reaction from the Trenches

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 198

think it’s an excellent idea, if done in a realistic fashion. Another

difference between the academic center and the community is that

we’re often sheltered from ﬁnancial pressures in an academic center.

As a practitioner, that’s very much in the forefront. Trying to take

into account time and cost imperatives, and rewarding clinicians for

doing the right thing is very important.

Remarks by James A. Clever, M.D.

Marin County, CA

First, about the title, “From the Trenches.” That’s a military term. The

only other kind of trench I can think of is when the public works

department is digging up the sewers and the water lines, but I think

it is a military term and that goes along with the rubber hitting the

sky when you’re sending up a recognizance balloon to see what’s

going on. Furthermore, trench warfare is way passé; later on, I will

say more about the military terms that pervade our profession and

some things that need correction.

I’d like to say a bit about motivation as far as continuing education

is concerned because that it is really important. A little story about

motivation: Jerry Reinsdorf, who owns the Chicago Bulls, was having

a conversation with Phil Jackson, who was the coach of the Bulls

during the time that Michael Jordan was playing for them, and they

were the outstanding team in the NBA for several years. They were

talking about motivation of players, and Reinsdorf said that what

motivates people is fear and greed, and Phil Jackson said, “Jerry,

you’ve got it all wrong. It’s pride and love.” I think that all four of

those things actually motivate people. I dislike people being motivated

by greed and fear, but fear is important for professionals to keep

up, fear of being sued, fear of not being re-licensed or re-certiﬁed.

If practitioners take on a new project — for instance, a primary care

internist might learn how to shave moles or something like that, and

charge enough for that to be equal to a couple of hours of regular

work — that probably is of some importance. But I think pride and

love are the real things, love of the profession, love of your patient.

You can love your patients but you have to take good care of them

as well. And having some pride that you are practicing well and get

feedback that is good is also an important motivator for continuing

education.

199

Clever

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 199

We haven’t talked about what I see as an increasing lack of collegiality

among professionals. We touched on collegiality in the teamwork

situation between physicians, nurses, pharmacists, students, and

house staff, but the profession of medicine has lost a great deal of

collegiality. Twenty years ago, we had an annual dinner dance and

a lot of staff would come, black tie and nicest gowns, and we had

a great time. We’ve lost that. Now maybe 20 people show up at

quarterly staff meetings; that’s out of an active medical staff that

must be about 600 or 700. For medical grand rounds, it seems like

the majority of the people in the audience are retired physicians,

like myself, and all too few house staff come, in part because morning

report is probably better education and a better use of their time,

but we miss that collegiality. Furthermore, I don’t think there ever

was very good collegiality between physicians, nurses, and other

hospital personnel and that’s something we need to do better.

Now the military aspect, and this comes back to what I think is

detrimental. Doctors write the orders and other people carry out

the orders, just as it’s done in the military, and I think that is wrong.

I don’t know what the medical legal consequences would be if we

changed orders to suggestions, or to advice, but the most effective

way of taking care of the orders situation would be that every time

doctors input the orders into the computer, and maybe even before

those orders are put in, they discuss it with the nurses, in a team

way, to decide what is the best way to do things. Another part of

the military attitude is no fraternization with the enlisted. We shouldn’t

treat the nurses and everyone else as enlisted people; they are our

colleagues.

Remarks by Susan W. Wesmiller, M.S., R.N.

University of Pittsburgh Medical Center

Some interesting things are happening in nursing. When you talk

about who pays for continuing education, in nursing it is mostly the

nurse or the employer. We actually budget for 16 hours a year of

continuing education for every single staff nurse. That doesn’t sound

like a lot, but it’s certainly better than zero. It is something that we’ve

continued, even though we know that we need to pull each staff

nurse off her assignment at least two full days a year to make sure

that, at least, competencies, are maintained. In addition, we pay the

200

Reaction from the Trenches

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 200

entire way for anyone who is speaking at a conference, whether it is

a regional conference or a national conference, so if a nurse shows

professionalism and writes an abstract and has that abstract accepted,

then we support that nurse by paying those expenses. Because we

realized that everybody can’t write an abstract that very ﬁrst year,

we started a system where staff nurses can write an application, an

essay about why they need continuing education and why they should

go to a conference, and get some help. You talked about conferences

being passé, but a lot of socialization occurs for young staff nurses,

for young professionals to be able to network with their peers

across the country. I’m glad that we’re supporting that.

Is the nursing shortage having an effect on continuing education

and nursing? We’d all like to say no, but the answer is yes. It becomes

increasingly difﬁcult for us to pull nurses off a schedule to send

them anywhere when we are just trying to cover the schedule. It’s

sometimes hard to get them to ACLS or CPR, let alone send them to

a conference. That is something we continue to struggle with. We

don’t have the nursing shortage in Western Pennsylvania that the

coasts have, but you deﬁnitely can see changes. We have more nurses

retiring from bedside nursing positions than I have seen before.

We might have six or seven retirement parties a month for nurses

leaving bedside nursing at 65 or 66 years of age.

We heard about team building and about the importance of simula-

tion. We’ve combined those two and have all our code training in

the simulation lab. We have a wonderful simulation lab and we send

nurses, respiratory therapists, pharmacists, and physicians in teams to

do simulation. They are assigned their roles for the team and given

three different scenarios. Then they are videotaped. The debrieﬁng

and the discussion that follows the videotaping are where learning

takes place. But again, it’s hard to get our staff nurses down there,

though we’re working on it.

We also talked about the inﬂuence of generations and I don’t think

we should forget that. For the ﬁrst time in the history of the United

States, we have four generations in the workplace, side by side.

If any of you think that millennials are going to put up with what

worked for baby boomers, think again. They don’t want lectures in

the traditional ways. They want a lecture on their iPod so that they

can listen to it while they’re on the treadmill. Or they want to do

201

Wesmiller

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 201

experiential learning. They are techno savvy. We don’t have to

worry about their technical competence because they come with it.

We just need to make sure that they know how to think critically.

One of my reactions to the discussion about technical competence

versus understanding is that I can see it both ways. Our new nurses

are so techno savvy that they don’t need us to show them how to

use a computer, but our older staff nurses need help, and we need

to help them get to that technical competence.

We have a wonderful nursing grand rounds program that we give

monthly. It’s offered live, but we also Web cast it so that anybody

can catch it 24/7. That is available as continuing education for any-

one, whenever they’re working. We had a 100 percent increase in

the number of people who watched grand rounds when we elimi-

nated paper paychecks and people had to go online to see their

paycheck. If you want to know how much you made, you have to

go online and ﬁnd your paycheck. All of a sudden, everybody was

on the computer because everybody wanted to know their paycheck,

and while they were there, they saw You Learn and Grand Rounds.

It’s really interesting to see how we drove that change. Unintended

consequences.

We talked about SBAR (Situation/Background/Assessment/Recom-

mendation) training. We are adding to SBAR a line called “I Need

Clarity.” That is going to be our code — nurse to nurse, nurse to

physician, nursing assistant to physician, nursing assistant to nurse

— to say, “Please stop. I don’t understand what you are asking me

to do and I need some help.” We are starting an across the board

education program for everyone, to teach us how to work together,

but we needed that code. We needed something to say to the physi-

cian, “please just stop what you’re doing right now and help me

because I don’t understand this. I need some clarity.” I hope it works.

We’ve done a lot of SBAR training so I think our staff nurses under-

stand those steps, but they needed something more and the physicians

needed to know that when they hear “I need some clarity” that they

should stop what they’re doing.

My ﬁnal reaction goes back to what we talked about at the begin-

ning of this conference, and that was the idea of a clean slate, of

starting where we would like to end. I thought wouldn’t it be won-

derful if we could waive all the regulatory, mandatory education

202

Reaction from the Trenches

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 202

that takes up so many resources, or if we could demonstrate that

we’re a top tier hospital in outcomes and they could say, “If you are

top tier and you are meeting all of your outcomes, you don’t have to

do regulatory education this year.” Imagine how many more

resources we’d have for providing good patient care.

Remarks by Regina Benjamin, M.D., M.B.A.

Bayou Clinic, Inc., Bayou La Batre, AL

Most of you around the table are from large institutions or represent

large groups. I would submit to you that most practices, and most

physicians, maybe one in two doctors, are more like me than not

like me. I’m in solo practice. I’ve been in solo practice for 18 years,

but I now have a PA. My town has 2,500 people, mostly uninsured,

and the demographics are 60 percent white, 30 percent Asian —

Vietnamese, Cambodian, Laotian — and 10 percent black and other.

I do house calls, and I know my patients and they know me. I go to

their funerals. In fact, recently we were listed on the program, in the

thank you portion. So we get to know each other and it’s a little

different from what I’ve been hearing.

I’ve been asked to react to the conversation over the past two days,

and I think that it’s very fearful. I’m sure I’m speaking for many

other doctors out there. The question I face is how can I improve

my skills and stay current and give the best care that I can to my

patients. And then I ask how can I possibly do all those perform-

ance measures, every measure that you can think of and still have

time to see patients. That’s very scary.

I guess the risk I see is that physicians will start to do things to look

good on paper and not necessarily to take care of patients. I can

make my HEDIS score look great and never do anything to improve

my practice per se. That’s a risk that we have getting ready to re-

certify for boards. Physicians are stressed out, nervous, cramming to

take a test when they’re going to test me again on the Krebs cycle

and how many times I use the Krebs cycle. I need to know what it

was about, but I don’t need to re-certify on it.

People learn in different ways. We still have our quarterly medical

society meetings, and we still have the CME, but it’s more social

203

Benjamin

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 203

now so the topics are not as didactic as they used to be. I learn a lot

in our doctors’ lounge. I don’t know if many people around the

country are still doing that, but our hospital just renovated the

lounge so that we have our breakfast and our lunch there. There’s a

lot of interaction. We talk to other doctors. We consult with the spe-

cialists, and they ask us questions. A lot happens in that breakfast

lounge and during lunch. I’m 30 miles from the hospital, but when I

go there, I try to go around those times when I know the docs will

be there and I can talk to them.

Lectures are important because they tell us what’s new out there.

I can ask myself questions and I can go ask consultants, but I don’t

know what the new things are. I thought about where I learned the

most things that made me actually change things in my practice.

I used to serve on an underwriting committee for a malpractice

insurance company. Every month we’d meet and I’d see how things

would happen there, and I’d come back to my ofﬁce and make a

change. Those were things like how to follow up on labs or do a

sticker ﬁle. That brings me to the idea of service learning, and we

don’t have a way to measure that. Service learning is what we are

starting to teach kids in high school and in college, but we should

be doing it with physicians as well. And if we can measure it, how

can we evaluate it or how can we get credit for it?

I want to tell you a bit about what I would like to have. I need to

tell you about Katrina. Two-and-a-half years ago, Katrina hit our

town; we got 25 feet of water. Out of 2,500 people, 2,000 had no

homes and I lost my clinic. We lost our medical records. We were

able to rebuild and renovate our building and, thanks to the Institute

of Medicine, we were able to get back. We were going to open on the

day after New Years, but on New Year’s Eve, the building burned

down, so we lost all those records again. Needless to say, my staff

said we don’t want to do any more paper records. We need an

electronic record. We got an electronic health record system and

that’s been great. I think the prices will come down, and, as the

costs come down, more people will get it.

I have a program on the computer so I can look up and see right

then the interactions of drugs. That really helps. I’d like to have

UpToDate and things like that integrated in the patient record, so

that when I have a question and I look something up and get the

204

Reaction from the Trenches

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 204

answer, I can document that answer in the record. That documenta-

tion will be there two years from now if I go back to that record, so

I don’t have to go back and look it up again. Those are the kinds of

things I think would help. We don’t have a lot of time in the ofﬁce,

so anything that would make it more efﬁcient would be better.

The other thing I’d like would be simulation or an interactive game,

a virtual game, where you could do whatever you want. I’d like to

see, for example, what would happen if I give this patient with con-

gestive heart failure a meal with salt in it, or if I changed the diet or

if I didn’t change the diet. Or, if I gave a diabetic a pill but then the

diabetic exercised, what would happen? That’s the kind of game I

think would help me a lot, just being able to play it on my own time.

We talked about CME funding and I think the comments were very

much on point, but I don’t share the level of vilifying the industry

that I heard from some. I don’t feel like I’m a bad person because

I take a drug pen or notepad. The clipboards might have a drug

company’s name. Advertising is advertising, and if you know it is

advertising, you know that’s what it is. I’ve asked patients if it bothers

them to see these things and they’ll tell me they don’t care, that it’s

just advertising. Those notepads and pens save me about $100 a

month in supplies, and for us that’s almost our entire electrical bill.

I can put that money towards treating people who don’t have insur-

ance, versus buying some pens and paper. The other thing is the

drug samples. We didn’t talk about the samples but they serve a

major role. These pharmaceutical companies aren’t taking me out

to $200 dinners because I’m not treating the patients that they want,

but they provide samples and we use them a lot. There’s a prescrip-

tion-in-need program, but it takes 30 days to get those back. You

have to ﬁll out the paper work and then wait. Why do that when

you can have samples? A student run free clinic on the East Coast

had to close because the medical school said no gifts from industry,

so they didn’t get samples anymore.

The idea about the single pool of money sounds good. I do have

one concern, and that’s that it would become like NIH. At NIH, the

large institutions with great grant writers who have the wherewithal

and the means can get the funding, but the small ones can’t. The

individual doctor may not have a role so I’d have to see the details.

205

Benjamin

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 205

206

Reaction from the Trenches

As for the role of the academic health center in CME, I just caution

you that in many parts of the country a town and gown problem

persists. The community and the academic health center don’t

always see eye to eye and it may be a real challenge to get them to

do so. The few practice-based research networks that are around

are working really well, but they’re what we call the early adapters.

It will be a while before we can change some of those other cultures.

DISCUSSION HIGHLIGHTS

Doctors need to get rid of the idea that they are launched into practice

with everything they need to know. They need to see continuing

education as an opportunity to improve the quality of care. How

do we bring evidence that matters to the bedside? How do we stop

doing what doesn’t work and do what does?

Most people think of continuing education as a Holiday Inn

experience, where a bunch of doctors look at a screen, nodding

occasionally, but continuing medical education is truly every bit

of learning from what the patient brings to the ofﬁce to formal

presentations.

Many physicians look at CME courses as something they have to

do, not something that will provide beneﬁt.

I hear lots of community physicians say they don’t have enough

access to information resources. I think this is a real issue that we

have to confront. Some of these resources can be provided by

medical societies. They can both assess and provide information

resources for their members.

A lot of questions in practice are almost impossible to anticipate,

so I don’t think any one information resource should have a

monopoly. We should make the access as broadly based as possible

for community practitioners.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 206

Conferences serve useful roles beyond CME. I think they help com-

munity physicians in professional development, recharging the

batteries, and networking. They deﬁnitely shouldn’t go away.

How many clinicians in practice have mentors? A major way one

learns and is guided is through collegial mentorship. Maybe mentor-

ship ought to be formalized a little more for practicing clinicians

because it is one way we learn an awful lot.

A lot of good things have been happening in journals over the past

decade to help practitioners— publishing systematic reviews and

articles about how to appraise evidence and evaluate diagnostic

testing. But the down side is that more and more is being published,

which is intimidating. It makes it hard to keep up.

If CE requirements just are add-ons to what clinicians are already

doing, it will be very scary. The ideal situation would be that CE and

certiﬁcation and licensure requirements are tied almost completely

to the electronic medical record, so that the evidence of acquisition

of knowledge and keeping up and quality performance are auto-

matic and just part of the clinician’s work.

In my view, the perfect CE experience for practitioners is broken

down into thirds. One third comes from examining one’s practice

and sitting down with the practice team to say, how are we doing

and what do we need to do differently. One third is point-of-care

learning during the course of patient care. Then, I would reserve a

third for course work. I don’t mean lectures necessarily, but getting

together with peers and colleagues different than my practice team

for the infusion of new ideas and skills. This is where I would get

my collegial ﬁx and connection to professionalism.

The patient’s role in all this is important. We need to ﬁgure out some

way to measure communication skills and some way to involve

patients in that.

207

Discussion Highlights

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 207

208

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 208

209

Moving Towards

the Future

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 209

210

i In addition to the Federation of State Medical Boards the participants included:

the Association of American Medical Colleges (AAMC), National Board of Medical

Examiners (NBME), Accreditation Council for Continuing Medical Education (ACCME),

Accreditation Council for Graduate Medical education (ACGME), American Osteopathic

Association (AOA), American Medical Association (AMA), American Board of Internal

Medicine (ABIM), Educational Commission for oreign Medical Graduates (ECFMG),

American Hospital Association(AHA), the American Academy of Family Physicians

(AAFP), Harvard School of Public Health, the Council of Medical Specialty Societies

(CMSS), Blue Cross/Blue Shield Association, Physician Insurers Association of America,

Harvard Medical International, the Milbank Memorial Fund, Robert Wood Johnson

Foundation, ﬁve state licensing boards, some certifying boards from the Osteopathic

community, some State Medical Societies, and Columbia Law School. Subsequent

meetings have attracted the participation of the American Association of Retired

People (AARP) and Consumer Union as well.

Continuing Medical Education:

Some Important Odds and Ends

David C. Leach, M.D.

Accreditation Council for Graduate Medical Education

This paper summarizes three important initiatives that have emerged

from the medical profession and will affect continuing medical edu-

cation (CME) in the near future: the National Alliance for Physician

Competence, the Trusted Agent, and the Conjoint Committee on

CME. These three initiatives, each self-organized and still emerging,

have engaged committed people and organizations in conversations

in which they seek to clarify the purpose and principles of CME,

the assessment of physician competence, and the use of technology

in facilitating these goals. Each aims to strengthen the relationship

between the profession and society and each member of the profes-

sion with the whole. This paper focuses on medicine; however, the

need for relevant lifelong learning applies to all health professions.

Perhaps these models from medicine will inform the larger effort.

The National Alliance for Physician Competence

On March 24 and 25, 2005, 45 participants representing 34 organi-

zations met in Fort Worth, Texas to improve medical care in the

United States by supporting consistency in the deﬁnition and assess-

ment of physician competence.

The ﬁrst meeting was convened by

the Federation of State Medical Boards (FSMB), but very quickly the

National Alliance became a coalition of organizations with the col-

lective goal of assuring the public of physician competence. At that

ﬁrst meeting the group reviewed historical trends for the past 100

years, projected likely trends that might characterize the future, and

created several scenarios. For each of the scenarios the group

explored answers to the question: How would/should the healthcare

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 210

community determine and measure physician competence over

the career of a physician for the purpose of assuring the public?

The dialogue was enabled and invigorated through exploration of

alternative futures and their impact on trust, data availability, changing

needs, and other important variables. Together, the group members

created ﬁve scenarios.

The exercise helped participants envision

the healthcare environment in which physician competence would

be assessed.

A second summit, held in December 2005, produced a draft deﬁni-

tion of physician competence, leading to the document that came

to be known as Good Medical Practice–USA

— and a concept of

the organizational model for the National Alliance for Physician

Competence. During a third summit in June 2006, participants

continued to develop the Good Medical Practice–USA document,

created a basic plan for the Alliance, and proposed concepts for a

portfolio tool that physicians could use to provide information about

and to reﬂect on their competence and performance. A fourth

summit, held in January 2007, reﬁned the document and developed

possible models for assessment of competence. A discovery summit

in August 2007 introduced new models of complex adaptive systems,

the use of positive deviance, and the use of technology to foster

distributed decision making and learning communities. The group

also reviewed various not-for-proﬁt organizational models and out-

lined basic strategies for managing change. Experts joined the group

for these topics.

The mission of the National Alliance for Physician Competence is

to assure the public and the healthcare community that individual

physicians are competent to provide safe medical care of the highest

quality. Through partnership and collaboration, the Alliance will 1)

advocate continuity in the deﬁnition, measurement, and determination

of physician competence across the continuum of education, training,

and practice; 2) support seamless collaboration among organizations

that contribute to physicians’ pursuit of lifelong learning and improve-

ment; and 3) seek ways to enable reform of the system of physician

self-regulation so that it is efﬁcient and effective.

The Good Medical Practice–USA document

organizes physician

competence around the six general competencies: patient care, med-

ical knowledge and skills, practice-based learning and improvement,

211

Leach

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 211

interpersonal and communication skills, professional behavior, and

systems-based practice. It clariﬁes patients’ expectations for their

doctors and doctors’ expectations of themselves.

The Alliance is self-funding, with each organization responding to

requests for donations as needed. The Alliance’s main product has

been the ability to maintain a dialogue about physician competence.

The conversations have been substantive, civil, and professional.

Participants are held together by common purpose and principles.

The Alliance has also developed a concept paper about a Physician

Learning and Continuous Improvement System (PLACIS). The purpose

of PLACIS is to support lifelong learning, self-assessment, reﬂection,

and continuous improvement as speciﬁed by the principles of the

“Good Medical Practice” document and also to support external

reporting requirements. PLACIS is used to describe a system that

provides a broad range of educational, developmental, and admin-

istrative support. If supplied with appropriate data, PLACIS could

greatly simplify how physicians respond to the increasing demand

for information by various stakeholder groups. Such data could

include scores on standardized examinations, certiﬁcation status,

hospital privileges, measures of practice performance, results of

self-assessments, and other related data.

The Trusted Agent

While not directly related to CME, the “Trusted Agent” concept, a

brainchild of Bob Galbraith, M.D., co-head of the National Board

of Medical Examiners (NBME) Center for Innovation, may facilitate a

more coherent approach to lifelong learning. The concept describes

a system in which secure data from multiple sources can be tem-

porarily compiled and displayed for such purposes as licensure,

credentialing, and maintenance of certiﬁcation. The data could,

upon request and with the permission of the physician, be obtained

from multiple sources (“Trusted Agents”), displayed and viewed,

and then, like a sand mandala, all traces are removed. The original

data would continue to reside on the Trusted Agents’ servers.

This concept is now being tested in three states and among three

organizations: the NBME, the FSMB, and the individual state boards.

In the test, a physician applying for licensure in one of the three

states gives permission, and with such permission, data from the

212

Continuing Medical Education: Some Important Odds and Ends

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 212

FSMB’s Federation Credentials Veriﬁcation Service (FCVS) and from

the NBME are transmitted in a temporary display to the relevant state

for veriﬁcation of credentials. Veriﬁcation in the test states (including

assembling relevant documents, such as medical school transcripts,

residency experiences, and board scores) used to take 6 months;

now the process takes 6 minutes from permission to completion.

The concept and the successful pilots have demonstrated the power

of temporary data displays to reduce the burden of and enhance

accountability to regulatory bodies. The ability to accommodate data

about CME remains untested but is a compellingly attractive feature

of the system. Most records of CME now reside on paper in the

individual physician’s ﬁles. No national database exists that can link

continuing education with a given physician’s educational and practice

needs. Successful deployment of this system will encourage develop-

ment of such a database, which in turn, will allow individual physicians

to maintain a portfolio of experiences relevant to their needs.

The Accreditation Council for Graduate Medical Education (ACGME)

has developed a “learning portfolio” for residents. The learning port-

folio, which is currently in alpha testing, is an electronic, Web-based

tool that supports resident learning, evaluation, and professional

development. The portfolio is centered around the learner, allowing

residents to chronicle their learning experiences and to seek feed-

back on these experiences. It contains preloaded evaluation tools

that are competency based, and tools can be created easily from a

pool of evaluation items. Residents entering their programs frequently

have had experience with learning portfolios in college or even in

grade school. To date, they have expressed great enthusiasm for the

portfolio as a way to catalog and reﬂect on their experiences. The

residents are using the system as an interactive tool for ongoing

physician development. It is not yet clear what will happen when

these residents graduate. The data exist and are stored on ACGME

servers. The larger community of organizations would probably

govern the principles and function of physician portfolios after this

test group graduates from residency. In that case ACGME could

become a Trusted Agent and at the request of the physician could

contribute data about the physician’s experiences in residency to

larger data displays or could, alternatively, contribute to the devel-

opment of a learning portfolio for lifelong physician development.

213

Leach

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 213

214

Continuing Medical Education: Some Important Odds and Ends

ii ACCME, ACGME, the Alliance for Continuing Medical Education (ACME), the

American Academy of Family Physicians (AAFP), the American Board of Medical

Specialties (ABMS), the American Hospital Association (AHA), the American Medical

Association (AMA), the American Osteopathic Association (AOA), the Association for

Hospital Medical Education (AHME), the Association of American Medical Colleges

(AAMC), the Council of Medical Specialty Societies (CMSS), the Federation of State

Medical Boards (FSMB), the Joint Commission, the Liaison Committee on Medical

Education (LCME), the National Board of Medical Examiners (NBME), and the

Society for Academic Continuing Medical Education (SACME). The Journal of

Continuing Education in the Health Professions also participates.

The Conjoint Committee on Continuing Medical Education

In October 2002, representatives of 16 major stakeholder organizations

in CME

began to meet and have since met regularly and voluntarily

to explore and propose changes in the existing system of CME.

This

group, the Conjoint Committee on Continuing Medical Education

(CC-CME), drafted a document entitled “Reforming and Repositioning

Continuing Medical Education”.

The document describes perceived

inadequacies of the current system of CME, in particular that it has

not consistently accommodated diverse learning styles; applied

advances in educational research; supported practice performance

assessment; and facilitated rapid integration of new knowledge and

skill; and that it has relied excessively on commercial support.

The CC-CME has stated that effective CME for physicians should

1) enhance quality care, 2) support professional activities, 3) assess

professional educational needs, 4) evoke professionalism,

5) motivate learners, and 6) produce measurable outcomes. The

Committee agreed that the six competencies (and a seventh from

the American Osteopathic Association) should serve as the frame-

work for CME.

The CC-CME recommended the following changes: reinforcement

of the continuum of medical education by all relevant organizations;

improved physician self-assessment of the competencies and life-

long learning; development of specialty-speciﬁc core curricula for

each specialty and subspecialty; evidence-based valid content for all

patient care recommendations in educational programs; continuous

improvement in the evaluation of CME’s effectiveness; metrics to

measure and recognize physician learning and behavioral change;

and convening a “blue ribbon” panel to propose proper ways of

funding CME. The recommendations were assigned to the relevant

organizations, and the work has begun.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 214

215

Leach

A white paper published in 2007 recommended that each specialty

society and corresponding certifying board reach consensus on

the competencies of the physicians in that specialty using the six

competencies as a framework.

A continuum from novice to master

was also proposed.

Summary

These three examples provide evidence that many in organized

medicine are thinking about and engaging in signiﬁcant efforts to

reform CME. A common vocabulary about physician competence

and its assessment; inclusion of the perspective of patients in the

expectations of competence; the opportunities afforded by technol-

ogy; and further consensus about core expectations by specialty

have all been identiﬁed as important needs.

All efforts have developed organizational models to accommodate

the fragmented house of medicine. ACGME has 44 nominating

organizations that select review committee members and participate

actively in graduate medical education. The National Alliance for

Physician Competence attracts between 35 and 45 organizations at its

summits. The Conjoint Committee on Continuing Medical Education

convenes representatives from 16 organizations at each meeting. The

Trusted Agent fully expressed may involve over 70 state licensing

jurisdictions and several more suppliers of data.

Dee Hock

would call these organizational models chaordic. They

are organized around common purpose and principles rather than

by command and control hierarchical structures. Hock would argue

that this approach may be a more effective way of coping with com-

plex problems. Chaordic models function at the interface of chaos

and order and are designed to enable both fidelity to purpose and

intelligent adaptation to particular microenvironments. The reform

of CME and lifelong learning for physicians is certainly a complex

problem. As specialties and subspecialties adapt to particular needs, a

common language, common assessment expectations, and intelligent

use of technology can foster coherence as well as accountability.

Improving patient care by improving the professional development

of health professionals remains the overriding goal.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 215

216

REFERENCES

1. Physician Accountability for Physician Competence Summit. Accessible at

http://www.innovationlabs.com/summit/summit1/ (Access date: March 2, 2008)

2. Good Medical Practice–USA. Accessible at http://www.gmpusa.org (Access date:

March 2, 2008).

3. Physician Accountability for Physician Competence Summit: Session Overview.

Accessible at http://www.innovationlabs.com/summit/discovery1/r6_mission.html.

(Access date: September 5, 2007).

4. Jackson MJ, Gallis HA, Gilman SC, et al. The need for specialty curricula based on

core competencies: a white paper of the Conjoint Committee on Continuing Medical

Education. J Cont Educ Health Prof. 2007;27:124–128.

5. Reforming and Repositioning Continuing Medical Education. Accessible at

www.jcehp.com/vol25/2503_CMEReport.pdf (Access date: September 23, 2005).

6. Hock D. One from Many. San Francisco: Berrett-Koehler; 2005.

DISCUSSION HIGHLIGHTS

A Set of Principles

When systems need to be changed, it is good to think of the

principles upon which you want to base the new system. Canada

followed that approach when the Canada Health Act was developed.

Five primary principles were chosen. Several principles have come

up in these discussions, but maybe they should be made explicit

and listed.

I like the idea of developing foundational principles. One should be

simplicity. Others are inter-professional CE and evidence-based CE.

Another is practicality.

Some of the principles conference participants have pointed out

include minimizing bias, ﬂexibility, burdens aligned with values,

integrating CE into practice, stressing innovation and evaluation,

addressing needs of practitioners from a broad spectrum — from

specialists in academic health centers to solo rural practitioners —

and aligning CE with quality improvement and health systems

improvements.

An Institute for Continuing Education

There is need for a national public/private entity to explore, consid-

er, evaluate and experiment, to keep the continuing education

agenda on the front burner so it is not scattered among the current

Continuing Medical Education: Some Important Odds and Ends

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 216

groups around the country. The public should support an organization

that would allow us to explore policy and outcomes.

The purpose of a national authority or institute would be to improve

patient care and health outcomes through enhancing professional

knowledge, skills, practices, teamwork and decision-making. It

ought to be independent. It ought to be governed predominantly by

the relevant professions, not as representatives of groups but rather

as individuals coming together to form this body for this purpose. It

should be grounded by the principles we’ve been articulating about

what we want continuing education and life long learning to be in

the professions, including the notion that it would stress innovation

and evaluation as well as the ideas of flexibility and outreach to

accomplish all of the needs from the academic health centers to solo

practitioners in the rural community. It should have, perhaps, a way

of putting some seal of performance or quality on those entities that

met certain standards. It has to obtain its support from government

agencies, foundations, or interested professional groups. If it were

independent and managed by professionals, and had all these attrib-

utes, you could legitimately accept corporate sponsorship in addition

to other sources if it were redirected and merged in a non-speciﬁc

way with no one company supporting any one project.

I see the new entity as an exciting way to push forward the science

of continuing education, just as the NIH pushed forward the science

of basic science and clinical medicine. In the long run, continuing

education isn’t going to go far unless we have a better science base.

Over and over again in these discussions, we have pointed out

the lack of information bases and the lack of rigorous studies of

innovations.

I’m looking for an enterprise that will have a capacity to innovate, to

experiment, to evaluate, to disseminate, to bring together the kind

of forward thinking we’ve had here in a regular way that could be

implemented. A national entity that would be a research entity could

explore broader policies and expand the science of education and

quality improvement.

A national entity responsible for continuing education should propose

and promulgate and investigate standards for continuing education.

217

Discussion Highlights

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 217

An Umbrella Organization for Accrediting

Inter-professional CE

We need an accrediting mechanism that incorporates all of the

relevant health professionals so there is an opportunity to reconcile

the differences and commonalities involved in the continuing educa-

tion of healthcare professionals. The ACCME and other accrediting

organizations ought to come together or a task force should be formed

to do this. We need a timeline to create an accrediting mechanism

that represents all the health professionals.

The ANCC and ACPE both have adopted the ACCME’s conﬂict of

interest requirements this year, so there is movement to align the

accrediting requirements of the three organizations. We should

encourage more rapid progress and movement to an approach that

makes it easier to provide accredited inter-professional CE.

We ought to get the ACCME and other accrediting organizations to

work together, or get a task force to facilitate the discussion, with a

timeline, to create an accrediting mechanism that represents all the

health professionals and that operates with these principles.

The current accrediting process, with ACCME and the ANCC and the

other organizations, can meet the principles and concepts we’ve

outlined, if given a deadline of two years and impetus to make that

change.

We ought to be very speciﬁc about a timeline. I’d like to go for a two-

year timeline that people need to meet and come up with something,

and the people who need to meet are the people who see themselves

as currently having responsibility for this. One characteristic of being

a professional is that we are voluntarily self-regulating and we

have established a set of standards by which we accredit and certify

ourselves, and in many cases license ourselves in conjunction with

a public jurisdiction. We need to hold that mechanism accountable,

so there should be a timeline.

218

Discussion Highlights

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 218

219

Conference Conclusions

and Recommendations

After two and a half days of discussion, participants agreed to the

following conclusions and recommendations:

CONCLUSIONS

Continuing Education and the Public

The quality of patient care is profoundly affected by the performance

of individual health professionals.

The fundamental purposes of continuing health professional

education (CE) are:

— To improve the quality of patient care by promoting improved

clinical knowledge, skills, and attitudes, and by enhancing

practitioner performance.

— To assure the continued competency of clinicians and the

effectiveness and safety of patient care.

— To provide accountability to the public.

CE fulﬁlls a critically important, indeed essential, public purpose.

Given the accelerating pace of change in clinical information and

technology, CE has never been more important.

Responsibilities of Individual Professionals, Professional

Teams, and Health Systems

Maintaining professional competence is a core responsibility of

each health professional, regardless of discipline, specialty, or type

of practice.

The individual clinician has been the principal unit of accountability

for performance in the healthcare delivery system. Given that the

performance of health systems also profoundly affects patient care,

CE fails to take into account systems of care.

Effective patient care increasingly depends on well-functioning

teams of healthcare professionals. Therefore, CE must address the

special learning needs of collaborating teams.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 219

Quality improvement efforts and CE activities overlap and ideally

are mutually reinforcing.

CE Methods

Traditional lecture-based CE has proven to be largely ineffective in

changing health professional performance and in improving patient

care. Lecture formats are employed excessively relative to their

demonstrated value.

Professional conferences play an important role in CE by promoting

socialization and collegiality among health professionals. Health

professionals have the responsibility to help one another practice

the best possible care. Meeting together provides opportunities for

cross-disciplinary and cross-generational learning and teaching.

Practice-based learning and improvement is a promising CE

approach for improving the quality of patient care. Maintenance

of certiﬁcation programs (in which clinicians review the care they

actually deliver in their own practices, compare the results with

standards of excellence, and create a plan for improvement) and

maintenance of licensure programs are moving CE in this direction.

Currently, most CE faculty are insufﬁciently prepared to teach

practice-based learning.

Information technology is essential for practice-based learning by:

— Providing access to information and answers to questions at the

time and place of clinical decision-making (point-of-care learning).

— Providing a database of clinician performance at the individual

and/or group practice level, which can be compared to best

practices and used to make plans for improvement.

— Providing automated reminder systems.

Interactive scenarios and simulations are promising approaches to

CE, particularly for skills development, whether the skill is a highly

technical procedure, history taking, or a physical examination

technique.

Insufﬁcient research is currently directed at improving and evaluating

CE. There is no national entity dedicated to advancing the science of

220

Conclusions and Recommendations

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 220

CE as there is for biomedical and clinical research.

Financing CE

The majority of ﬁnancial support for accredited CME, and increas-

ingly for CNE, derives directly or indirectly from commercial entities.

Pharmaceutical and medical device companies and healthcare pro-

fessionals have inherently conﬂicting interests in CE. Commercial

entities have a legitimate obligation to enhance shareholder value

by promoting sales of their products, whereas healthcare profession-

als have a moral obligation to improve patient/public health without

concern for the sale of products.

Commercial support for CE:

— Risks distorting the educational content and invites bias.

— Raises concerns about the vows of health professionals to place

patient interest uppermost.

— Endangers professional commitment to evidence-based decision

making.

— Validates and reinforces an entitlement mindset among health

professionals that CE should be paid for by others.

— Impedes the adoption of more effective modes of learning.

No amount of strengthening of the “ﬁrewall” between commercial

entities and the content and processes of CE can eliminate the

potential for bias.

Academic health centers and other healthcare delivery systems are

not sufﬁciently attentive, either to their roles in planning, providing,

and assessing CE or to their responsibilities in managing their own

conﬂicts of interest and those of individual faculty and administrators

when paid by commercial interests for CE teaching.

Accrediting CE

Current accreditation mechanisms for CE are unnecessarily complex

yet insufﬁciently rigorous. Compared to earlier, formal stages of

221

Conclusions and Recommendations

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 221

health professions education, the CE enterprise is fragmented, poorly

regulated, and uncoordinated; as a result, CE is highly variable in

quality and poorly aligned with efforts to improve quality and

enhance health outcomes.

With the increasing need for inter-professional collaboration,

accrediting bodies of the various health professionals need closer

working relationships.

RECOMMENDATIONS

CE Methods

The CE enterprise should shift as rapidly as possible from excessive

reliance on presentation/lecture-based formats to an emphasis on

practice-based learning.

New metrics are needed:

— To assess the quality of CE. These metrics should be based

on assessment of process improvement and enhanced patient

outcomes.

— To identify high-performing healthcare organizations. The

possibility of awarding CE credit to individual health professionals

who practice in such organizations should be explored.

— To automate credit procedures for point-of-care learning.

Federal and state policymakers should provide ﬁnancial support

for the further development of information technology tools that

facilitate practice-based learning and should strongly encourage all

clinicians to use these tools.

The responsibility for lifelong learning should be emphasized

throughout the early, formal stages of education in all health

professions. Students should be taught the attitudes and skills to

accomplish CE throughout their professional lifetimes.

A national inter-professional CE Institute should be created to

advance the science of CE. The Institute should:

222

Conclusions and Recommendations

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 222

— Promote the discovery and dissemination of more effective

methods of educating health professionals over their profession-

al lifetimes and foster the most effective and efﬁcient ways to

improve knowledge, skills, attitudes, practice, and teamwork.

— Be independent and composed of individuals from the various

health professions.

— Develop and run a research enterprise that encourages increased

and improved scientiﬁc study of CE.

— Promote and fund evaluation of policies and standards for CE.

— Identify gaps in the content and processes of CE activities.

— Develop mechanisms needed to assess and fund research

applications from health professional groups and individuals.

— Stimulate development and evaluation of new approaches to

both intra- and inter-professional CE, and determine how best

to disseminate those found to be effective and efﬁcient.

— Direct attention to the wide diversity and scope of practices

with special CE needs, ranging from highly technical specialties

on the one hand to solo and small group practices in remote

locations, on the other.

— Acquire ﬁnancial resources to support its work and provide

funding for research. Possible funding sources include the

Federal government, foundations, professional groups, and

corporations.

A concerted effort is needed to make the concept of a Continuing

Education Institute a reality. To achieve this, The Institute of Medicine

should convene a group to bring together interested parties to propose

detailed steps for developing a Continuing Education Institute.

CE Financing

Accredited organizations that provide CE should not accept any

commercial support from pharmaceutical or medical device compa-

nies, whether such support is provided directly or indirectly through

subsidiary agencies. Because many professional organizations and

223

Conclusions and Recommendations

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 223

224

institutions have become heavily dependent on commercial support

for current operations, an abrupt cessation of all such support would

impose unacceptable hardship. A ﬁve-year “phase out” period

should be allowed to meet this recommendation.

The ﬁnancial resources to support CE should derive entirely from

individual health professionals, their employers (including academic

health centers, healthcare organizations, and group practices),

and/or non-commercial sources.

Faculty of academic health centers should not serve on speakers’

bureaus or as paid spokespersons for pharmaceutical or device

manufacturers. They should be prohibited from publishing articles,

reviews, and editorials that have been ghostwritten by industry

employees.

CE Accreditation and Providers

Organizations authorized to provide CE should be limited to profes-

sional schools with programs accredited by national bodies, not-for-

proﬁt professional societies, healthcare organizations accredited by

the Joint Commission, multi-disciplinary practice groups, point-of-care

resources, and print and electronic professional journals.

Existing accrediting organizations for continuing education for

medicine (the Accreditation Council for Continuing Medical

Education) and nursing (the American Nurses Credentialing Center)

should meet and within two years develop a vision and plan for a

single accreditation organization for both nursing and medicine.

The new organization should incorporate the guiding principles for

CE and the recommendations laid out in this report where relevant.

The American Academy of Nursing and the Association of American

Medical Colleges should convene the two accrediting bodies for

this purpose.

Academic health centers should examine their missions to determine

how to strengthen their commitment to CE. They should help their

faculty gain expertise in teaching practice-based learning and incor-

porate information technology, simulations, and interactive

scenarios into their CE activities.

Conclusions and Recommendations

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 224

225

Additional References

Suggested by Conference Participants

General References

Agency for Healthcare Research and Quality. Effectiveness of Continuing Medical

Education (Evidence Report/Technology Assessment; 149). Rockville, MD: US

Department of Health and Human Services; 2007.

Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB.

Effectiveness of teaching quality improvement to clinicians. JAMA. 2007;298:1023–1037.

Center for the Advancement of Interprofessional Education (CAIPE). Interprofesional

education: a deﬁnition. CAIPE Bulletin. 1997;13:19.

Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and

research agenda. Acad Med. 2005;80(Suppl):546–553.

Gardner M. Diagnosis using search engines. BMJ. 2006;333:1131.

Golden GA, Parochka JN, Overstreet KM. Medical education and communication compa-

nies: an updated in-depth proﬁle. J Cont Educ Health Prof. 2002;55–62.

Headrick, LA, Wilcox, PM, Batalden, PB. Interprofessional working and continuing med-

ical education. BMJ. 1998; 316:771–774.

Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback

versus altermative strategies: effects on professional practice and healthcare outcomes.

Cochrane Database Syst Rev. 2006;2:CD000259.

Johnson J, Dutton S, Briffa A, Black C. Broadband learning for doctors. BMJ.

2006;332:1403–1404.

Manning PR, DeBakey L, editors. Medicine: Preserving the Passion in the 21st Century.

New York: Springer-Verlag; 2004.

McDonald, FS, Zeger SL, Kolars JC. Factors associated with medial knowledge acquisi-

tion during internal medicine residency. J Gen Intern Med. 2007;22:962–968.

O’Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education

meetings and workshops: effects on professional practice and healthcare outcomes.

Cochrane Database Syst Rev. 2001;1:CD003030.

O’Brien MA, Rogers S, Jamtvedt G, et al. Educational outreach visits: effects on professional

practice and healthcare outcomes. Cochrane Database Syst Rev. 2007;4:CD000409.

Sachdeva AK. Acquiring skills in new procedures and technology: the challenge and

the opportunity. Arch Surg. 2005;140:387–389.

Sachdeva AK. The new paradigm of continuing education in surgery. Arch Surg.

2005;140:264–269.

Sachdeva AK. Surgical education to improve the quality of patient care: The role of

practice-based learning and improvement. J Gastrointest Surg. 2007;11:1379–1383.

Sachdeva AK, Russell TR. Safe introduction of new procedures and emerging technologies

in surgery: Education, credentialing, and privileging. Surg Clin North Am. 2007;87:853–866.

Tang H, Ng JH. Googling for a diagnosis: use of Google as a diagnostic aid: internet

based study. BMJ. 2006; 333:1143–1145.

Wagner EH, Glasgow RE, Davis C, et al. Quality improvement in chronic illness care: a

collaborative approach. Jt Comm J Qual Improv. 2001;27:63–80.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 225

226

Financing Continuing Education

Abramson J, Starﬁeld B. The effect of conﬂict of interest on biomedical research and

clinical practice guidelines: Can we trust the evidence in evidence-based medicine?

J Am Board Fam Pract. 2005;18:414–418.

Agrawal S, Saluja I, Kaczorowski J. A prospective before-and-after trial of an educational

intervention about pharmaceutical marketing. Acad Med. 2004;79:1046–1050.

Allman RL. The relationship between physicians and the pharmaceutical industry:

Ethical problems with the every-day conﬂict of interest. Hospital Ethics Committees

Forum. 2003;15:155–170.

Andersson SJ, Troein M, Lindberg G. General practitioners’ conceptions about treatment

of depression and factors that may inﬂuence their practice in this area: a postal survey.

BMC Fam Pract. 2005;16:21.

Antonuccio DO, Danton WG, McClanahan TM. Psychology in the prescription era:

building a ﬁrewall between marketing and science. Am Psychol. 2003;58:1028–1043.

Ashar BH, Miller RG, Getz KJ, Powe NR. Prevalence and determinants of physician

participation in conducting pharmaceutical-sponsored clinical trials and lectures. J Gen

Intern Med. 2004;19:1140–1145.

Association of American Medical Colleges. The Scientiﬁc Basis of Inﬂuence and

Reciprocity: A Symposium. Washington, D.C: Association of American Medical Colleges;

2007. Accessible at www.aamc.org/publications as a pdf download (Access date: April 1,

2008).

Bellande BJ, Fonarow GC, Nishimura RA, Shah PM. Task force 3: Disclosure of relation-

ships with commercial interests: Policy for educational activities and publications.

Circulation. 2004;110:2524–2528.

Bergeron L. Psychological aspects of gifts from drug companies. JAMA.

2003;290:2406–2407.

Berwick DM. Disseminating innovations in healthcare. JAMA. 2003;289:1969–1975.

Bickell NA. Drug companies and continuing medical education. J Gen Intern Med.

1995;10:392–394.

Boyd EA, Cho MK, Bero LA. Financial conﬂict-of-interest policies in clinical research:

issues for clinical investigators. Acad Med. 2003; 78:769–774.

Brett AS, Burr W, Moloo J. Are gifts from pharmaceutical companies ethically problemat-

ic? A survey of physicians. Arch Intern Med. 2003; 163:2213–2218.

Cain JM. Conﬂict of interest in healthcare: a subtle but signiﬁcant issue in women’s

health. Curr Womens Health Rep. 2003; 3:87–88.

Campbell EG. Doctors and drug companies: scrutinizing inﬂuential relationships.

N Engl J Med. 2007;357:1796–1797.

Carroll AE, Vreeman RC, Buddenbaum J, Inui TS. To what extent do educational inter-

ventions impact medical trainees’ attitudes and behaviors regarding industry-trainee and

industry-physician relationships? Pediatrics. 2007;120:e1528–e1535.

Chabner BA. CME: The “third phase” of a doctor’s education. Oncologist. 2004;9:241.

Chen P. Paul, Weiss, Rifkind, Wharton & Garrison LLP, New York, NY, USA. Education or

promotion?: Industry-sponsored continuing medical education (CME) as a center for the

core/commercial speech debate. Food Drug Law J. 2003;58:473–509.

Chepesiuk R. Supported by an unrestricted educational grant. Can Med Assoc J.

2003;169:421–422.

Additional References

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 226

227

Cleary JD. Impact of pharmaceutical sales representatives on physician antibiotic

prescribing. Pharm Technol. 1992;8:27–29.

Collins J. Professionalism and physician interactions with industry. J Am Coll Radiol.

2006;3:325–332.

osta-Val R. The imminent paradigm shift toward the relationship between pharmaceutical

industry and medicine. Jornal Vascular Brasileiro. 2007;6:104–106.

Coyle SL, Ethics,and Human Rights Committee American College of Physicians-American

Society of Internal Medicine. Physician-industry relations. Part 1. Individual physicians.

Ann Intern Med. 2002; 136:396–402.

Coyle SL. Ethics and Human Rights Committee, American College of Physicians-American

Society of Internal Medicine. Physician-industry relations. Part 2. Organizational issues.

Ann Intern Med. 2002;136:403–406.

Dana J, Loewenstein G. A social science perspective on gifts to physicians from industry.

JAMA. 2003;290:252–255.

Davis DA. CME and the pharmaceutical industry: two worlds, three views, four steps.

CMAJ. 2004;171:149–150.

DelSignore JL. Current guidelines regarding industry-sponsored continuing medical

education. Clin Orthop. 2003;412:21–27

Dietz HP. Don’t regulate—abolish. Med J Aust. 2006;185:673–675.

Donohue JM, Berndt ER, Rosenthal M, Epstein AM, Frank RG. Effects of pharmaceutical

promotion on adherence to the treatment guidelines for depression. Med Care.

2004;42:1176–1185.

Dyer O. Drug company and pharmacy send unsolicited samples of Prozac to Florida

residents. BMJ. 2003 327:950.

Fisher MA. Physicians and the pharmaceutical industry: a dysfunctional relationship.

Perspect Biol Med. 2003;46:254–272.

Fugh-Berman A, Ahari S. Following the script: How drug reps make friends and inﬂuence

doctors. PLOS Med. 2007;4:0621–0623.

Goldblum OM, Franzblau MJ. Academic medical centers and conﬂicts of interest. JAMA.

2006; 295:2845–2846.

Green JA. Pharmaceutical marketing research and the prescribing physician. Ann Intern

Med. 2007;146:742–748.

Gross CP, Gupta AR, Krumholz HM. Disclosure of ﬁnancial competing interests in ran-

domised controlled trials: cross sectional review. BMJ. 2003;326:526–527.

Halperin EC, Hutchison P, Barrier RC Jr. A population-based study of the prevalence and

inﬂuence of gifts to radiation oncologists from pharmaceutical companies and medical

equipment manufacturers. Int J Radiat Oncol Biol Phys. 2004;59:1477–1483.

Hamaty D, Villarreal A, Murphy T. Continuing education or bribe?. Pain Med. 2003;4:295.

Harrison RV. The uncertain future of continuing medical education: commercialism and

shifts in funding. J Contin Educ Health Prof. 2003;23:198–209.

Hemminki E, Karttunen T, Hovi SL, Karro H. The drug industry and medical practice—

the case of menopausal hormone therapy in Estonia. Soc Sci Med. 2004;58:89–97

Henney JE. Safeguarding patient welfare: Who’s in charge? Ann Intern Med. 2006;

145:305–307.

Additional References

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 227

Hensley S. When doctors go to class, industry often foots the bill: lectures tend to

feature pills made by course sponsors; companies deny inﬂuence; a purple heartburn

brochure. Wall Street Journal (East Ed) 2002 A12; Dec 4:A1.

Hilliard T, Chambers T. The relationship between paediatricians and commerce.

Paediatr Respir Rev. 2006;7:54–59.

Holmes JS, Shevrin M, Goldman B, Share D. Translating research into practice: are physi-

cians following evidence-based guidelines in the treatment of hypertension? Med Care

Res Rev. 2004; 61:453–473.

Kerridge I, Maguire J, Newby D, et al. Cooperative partnerships or conﬂict-of-interest?

A national survey of interaction between the pharmaceutical industry and medical

organizations. Intern Med J. 2005; 35:206–210.

Komesaroff PA, Kerridge IH. Ethical issues concerning the relationships between

medical practitioners and the pharmaceutical industry. Med J Aust. 2002;176:118–121

Lexchin J. Interactions between physicians and the pharmaceutical industry: What does

the literature say? CMAJ. 1993;149:1401–1407.

Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and

research outcome and quality: systematic review. BMJ. 2003;326:1167–1170.

Lexchin J, Cassels A. Does the C in CME stand for “Continuing” or “Commercial”?

[comment]. CMAJ. 2005;172:160–162.

Lexchin J, Mintzes B. Transparency in drug regulation: mirage or oasis? CMAJ.

2004;171:1363–1365.

Majumdar SR, McAlister FA, Soumerai SB. Synergy between publication and promotion:

Comparing adoption of new evidence in Canada and the United States. Am J Med.

2005;60:1323–1334.

Marlow B. The future sponsorship of CME in Canada: industry, government, physicians,

or a blend? JAMC. 2004;171:150–151.

McCoy C, Oshman L. Psychological aspects of gifts from drug companies. JAMA.

2003;290:2406–2407.

Mintzes B, Barer ML, Kravitz RL, et al. How does direct-to-consumer advertising (DTCA)

affect prescribing? A survey in primary care environments with and without legal DTCA.

Can Med Assoc J. 2003;169:405–412.

Mizik N, Jacobson R. Are physicians easy marks? Quantifying the effects of detailing and

sampling on new prescriptions. Management Science. 2004;50:1704–1715.

Morgan JM, Marco J, Stock L, Zannad F. Educational governance for the regulation of

industry sponsored continuing medical education in interventional and device based

therapies. Heart. 2005; 91:710–712

Morgan MA, Dana J, Loewenstein G, Zinberg S, Schulkin J. Interactions of doctors with

the pharmaceutical industry. J Med Ethics. 2006;32:559–563.

Moynihan R. Drug company sponsorship of education could be replaced at a fraction

of its cost. BMJ. 2003;326:1163.

Moynihan R. Who pays for the pizza? Redeﬁning the relationships between doctors and

drug companies. 1: Entanglement. BMJ. 2003;326:1189–1192.

Moynihan R. Who pays for the pizza? Redeﬁning the relationships between doctors and

drug companies. 2: Disentanglement. BMJ. 2003;326:1193–1196.

Mueller PS, Hook CC, Litin SC. Physician preferences and attitudes regarding industry

support of CME programs. Am J Med. 2007;120:281–285.

228

Additional References

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 228

Orlowski JP, Wateska L. The effects of pharmaceutical ﬁrm enticements on physician

prescribing patterns. There’s no such thing as a free lunch. Chest. 1992;102:270–273.

Prosser H, Almond S, Walley T. Inﬂuences on GPs’ decision to prescribe new drugs-the

importance of who says what. Fam Pract. 2003;20:61–68.

elman AS. Separating continuing medical education from pharmaceutical marketing.

JAMA. 2001;285:2009–2012.

Relman AS, ACCME. Defending professional independence: ACCME’s proposed new

guidelines for commercial support of CME. JAMA. 2003;289:2418–2420.

Rentmeester CA. Pharma PR or medical education? [comment]. Hastings Cent Rep.

2005;35:6.

Rutledge P, Crookes D, McKinstry B, Maxwell SR. Do doctors rely on pharmaceutical

industry funding to attend conferences and do they perceive that this creates a bias in

their drug selection? Results from a questionnaire survey. Pharmacoepidemiol Drug Saf.

2003;12:663–667.

Schneider JA, Arora V, Kasza K, Harrison RV, Humphrey H. Residents’ perceptions over

time of pharmaceutical industry interactions and gifts and the effect of an education

intervention. Acad Med. 2006;81:595–602.

The scientiﬁc basis of inﬂuence and reciprocity: a symposium sponsored by the

Association of American Medical Colleges. Washington, DC. June 12, 2007.

Steinman MA, Baron RB, Marlow B. Is continuing medical education a drug promotion

tool? Debates. Can Fam Physician. 2007;53:1650–1653.

Steinman MA, Baron RB. Rebuttal: Is CME a drug-promotion tool?: YES. Can Fam

Physician. 2007;53:1877–1879.

Susman JL. Commercial sponsorship of CME: There are alternatives. J Fam Pract.

2004;53:676–690.

Takhar J, Dixon D, Donahue J, et al. Developing an instrument to measure bias in CME.

J Contin Educ Health Prof. 2007;27:118–123.

Tenery RM Jr. Interactions between physicians and the healthcare technology industry.

JAMA. 2000;283:391–393.

Wall LL, Brown D. The high cost of free lunch. [review] [37 refs]. Obstet Gynecol;

2007;110:169–173.

Waring GO III. On what information do ophthalmologists base their clinical practice?

J Refract Surg. 2004;20:414–416.

Watkins C, Moore L, Harvey I, Carthy P, Robinson E, Brawn R. Characteristics of general

practitioners who frequently see drug industry representatives: national cross sectional

study. BMJ. 2003;326:1178–1179.

Wilson A. Giving and receiving of gifts between pharmaceutical and medical specialists

in Australia. Intern Med J. 2006;36:795–796.

Wilson FS. Continuing medical education: ethical collaboration between sponsor and

industry. Clin Orthop. 2003;412:33–37.

229

Additional References

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 229

230

Biographical Sketches and Statements of Potential

Conﬂicts of Interest of Conference Participants

Barbara Atkinson, M.D., one of the country’s 13 women medical school deans,

serves simultaneously as Dean of the Medical School and as Executive Vice

Chancellor at the University of Kansas Medical Center. In those roles, she oversees

the Schools of Medicine and Nursing and Allied Health, with more than 2,700 stu-

dents and 780 faculty members, and leads the medical school’s curriculum change

which is designed to ensure that students are learning basic and clinical sciences

from faculty members on the leading edge of advancing medical knowledge. She

reports no potential conﬂicts of interest related to continuing education in the

health professions.

Denise Basow, M.D., who trained in internal medicine at Johns Hopkins and

practiced in Boston, joined UpToDate as Deputy Editor in 1996. Since then she has

progressed to the position of Executive Editor, where she supervisors the Deputy

Editors, plans new specialties, and provides overall vision for UpToDate. She has

been involved with the CME accreditation process at UpToDate and has experience

with online CME. She reports a potential conﬂict of interest related to continuing

education in the health professions because UpToDate is a CME provider and she

owns UpToDate stock.

Regina M. Benjamin, M.D., M.B.A., is founder and CEO of the Bayou La Batre

Rural Health Clinic in Bayou La Batre, Alabama. As a former associate dean at the

University of South Alabama College of Medicine, she administered the Alabama-

AHEC program and USA Telemedicine Program. She is a past president of the

American Medical Association Education and Research Foundation and serves as

vice-chair of the AMA Council on Ethical and Judicial Affairs. Other positions

include the Council of Graduate Medical Education and the NIH Committee on

Minority Health and Health Disparities. Dr. Benjamin reports no potential conﬂicts

of interest related to continuing education in the health professions.

David Blumenthal, M.D., M.P.P., is Director of the Institute for Health Policy at

the Massachusetts General Hospital/Partners HealthCare System, Samuel O. Their

Professor of Medicine and Professor of Healthcare Policy at Harvard Medical

School, and Director of the Harvard University Interfaculty Program for Health

Systems Improvement. His research interests include the dissemination of health

information technology, quality management in healthcare, and the extent and

consequences of academic-industrial relationships in the health sciences. He

reports no potential conﬂict of interest related to continuing education in the health

professions.

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 230

James A. Clever, M.D., retired in 2005 after 35 years in private practice in general

internal medicine in San Francisco. He was elected to a four-year term on the Marin

Healthcare District Board of Directors in 2006. His CME continues to include one

national and several regional meetings each year, attending 35 to 40 grand rounds,

and reading some 200 journal publications. He reports no potential conﬂicts of

interest related to continuing education in the health professions.

Jordan J. Cohen, M.D., currently Professor of Medicine and Public Health at

George Washington University, is President Emeritus of the Association of American

Medical Colleges. During his tenure at AAMC, Dr. Cohen led Association efforts

to strengthen its programs for medical students, residents, and constituents and its

data gathering, communications, and advocacy capabilities. He also introduced

initiatives to improve medical education, research, and patient care. Dr. Cohen

reports no potential conﬂicts of interest related to continuing education in the

health professions.

Ellen M. Cosgrove, M.D., is Senior Associate Dean for Education and is responsible

for undergraduate, graduate, and continuing medical education at the University of

New Mexico School of Medicine, where she is also a professor of internal medicine.

Both the Accreditation Council for Continuing Medical Education and the Alliance

for CME, where she has held leadership positions, have honored Dr. Cosgrove

for her long-time commitment and service to continuing medical education. She

reports a potential conﬂict of interest related to continuing education in the health

professions because UNM is an accredited CME provider.

Linda Cronenwett, Ph.D., R.N., is Dean and Professor of the School of Nursing at

the University of North Carolina at Chapel Hill as well as the Associate Chief

Nursing Ofﬁcer for Academic Affairs at the University of North Carolina Hospitals.

Her other commitments include: principal investigator for a national initiative on

Quality and Safety Education for Nurses funded by the Robert Wood Johnson

Foundation; the board of directors of the Institute for Healthcare Improvement, and

national advisory committee for Robert Wood Johnson’s Transforming Care at the

Bedside initiative. Dr. Cronenwett reports a potential conﬂict of interest related to

continuing education in the health professions because UNC is an accredited

CE provider.

David A. Davis, Jr., M.D., is Vice President for Continuing Healthcare Education

and Improvement at the American Association of Medical Colleges and founding

director of the University of Toronto’s Knowledge Translation Program at St.

Michael’s Hospital in Toronto. As the author of numerous articles and editor of two

major books on CME, Dr. Davis is widely recognized for his contributions to the

231

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 231

ﬁeld. These include the application of problem-based learning principles, the use of

innovative needs assessment and evaluation techniques, the use of standardized

patients, and a systematic, outcomes-based focus on CME. He reports potential

conﬂicts of interest related to continuing education in the health professions

because of funding by Merck Canada and occasional speaking and consultancy to

medical schools, commercial interests, and specialty societies, with all funds trans-

ferred to the CME research fund at the University of Toronto.

Catherine D. DeAngelis, M.D., M.P.H., is Editor-in-Chief of Scientiﬁc Publications

and Multimedia Applications, and oversees the Journal of the American Medical

Association (JAMA) as well as nine Archives publications. She is also Professor of

Pediatrics at Johns Hopkins University School of Medicine. She is author and editor

of 11 books on pediatrics and medical education and has published more than 200

original articles, chapters, editorials, and abstracts in those ﬁelds. Recent works

have focused on women in medicine, conﬂicts of interest in medicine, and on med-

ical education. Dr. DeAngelis reports no conﬂicts of interest related to continuing

education in the health professions.

Lyn (Lynore) DeSilets, Ed.D., R.N.-B.C., is Assistant Dean of the Villanova

University College of Nursing, where she has served as Director of Continuing

Education in Nursing and Healthcare for 28 years. The program is a nationally

accredited provider of CNE by the American Nurses Credentialing Center

Commission on Accreditation. She also serves as an appraiser for ANCC’s

Commission on Accreditation, is a member of the Continuing Education Review

Panel for the Pennsylvania State Nurses Association, and is an associate editor

for the Journal of Continuing Education in Nursing. Dr. DeSilets reports potential

conﬂicts of interest related to consultancies for the American Public Health

Association, the ANCC, and the Pennsylvania State Nurses Association.

F. Daniel Duffy, M.D., is Senior Advisor to the President of the ABIM and Adjunct

Professor of Medicine and director of the community health track at the University

of Oklahoma College of Medicine-Tulsa, where he heads a program to integrate

a safety-net primary care system in the region. At the American Board of Internal

Medicine, he introduced a new maintenance of certiﬁcation process based on

evaluation of practice performance. He is a member of the Ambulatory Measures

Steering Committee of the National Quality Forum, the Performance Measurement

Committee of the National Committee for Quality Assurance, and the Board of

Directors of the American Board of Medical Specialties. Dr. Duffy reports potential

conﬂicts of interest related to continuing education in the health professions relat-

ing to his work with ABIM.

232

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 232

233

Harvey V. Fineberg, M.D., Ph.D., is currently President of the Institute of Medicine,

a position he assumed in 2001 after serving as Provost of Harvard University from

1997 to 2001 and, for the previous 13 years, as Dean of the Harvard School of

Public Health. His academic career focused mainly on the ﬁelds of health policy

and medical decision making. At IOM, he has chaired and served on a variety of

panels dealing with health policy issues, ranging from AIDS to vaccine safety. Dr.

Fineberg reports no potential conﬂicts of interest related to continuing education

in the health professions.

Grant S. Fletcher, M.D., M.P.H., is an attending physician in the Consultation and

Hospitalist Medicine Program at the University of Washington’s Harborview Medical

Center in Seattle. He completed his masters in public health at the University of

North Carolina-Chapel Hill and helped to start the hospitalist program at a Salt

Lake City hospital. His research interests include developing and assessing clinical

decision support tools with the electronic medical record, and clinical research on

hospital care. Dr, Fletcher reports no potential conﬂicts of interest related to contin-

uing education in the health professions.

Suzanne W. Fletcher, M.D., M.Sc., is Professor Emerita of Ambulatory Care and

Prevention at Harvard Medical School, Adjunct Professor of Epidemiology at the

University of North Carolina School of Public Health, and Adjunct Professor at the

UNC School of Medicine. At Harvard, Dr. Fletcher served as director of the CME

course for primary care clinicians given at PRI-MED. The course was held each

year in ﬁve cities, with total attendance of more than 200,000. She also serves as

co-editor of primary care for the electronic text, UpToDate, on the International

Advisory Board for The Lancet, and as associate editor of the Journal of the

National Cancer Institute. She was founding co-editor of the Journal of General

Internal Medicine, has served as co-editor of the Annals of Internal Medicine

and, in 1995, organized the conference that led to the formation of the World

Association of Medical Editors. She chaired the committee that developed the

first MOC module at the American Board of Internal Medicine. Dr. Fletcher

reports potential conﬂicts of interest related to continuing education in the health

professions because Harvard Medical School and Harvard Pilgrim Healthcare are

accredited CME providers, and because of honoraria from the American Board of

Internal Medicine, Susan G. Komen Foundation, Research Triangle International,

Josiah Macy, Jr. Foundation, The Lancet, and several medical schools for visiting

professorships. She receives royalties from Lippincott Williams & Wilkens, Wolters

Kluwer, and UpToDate.

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 233

234

Melvin I. Freeman, M.D., is Emeritus Director of Continuing Medical Education

at the Virginia Mason Medical Center in Seattle, where he currently is a research

scientist in ophthalmology. Previously in private group practice, he was head of

ophthalmology at Virginia Mason Clinic and Medical Center and is Emeritus Clinical

Professor of Ophthalmology at the University of Washington School of Medicine.

He is Vice Chair of the Washington State Medical Association’s Continuing Medical

Education Committee, a surveyor for the Accreditation Council for Continuing

Medical Education, and for the American College of Surgeons Program for

Accreditation of Educational Institutes and past president of the Alliance for

Continuing Medical Education. Dr. Freeman reports potential conﬂicts of interest

relating to continuing education in the health professions due to honoraria from

ACCME and the American College of Surgeons, and royalties from Elsevier/Mosby

Publishers and Slack, Inc.

Michael Green, M.D., M.Sc., is Associate Professor of Medicine and Associate

Director of the Primary Care Residency Program at Yale University School of

Medicine. His interests include curriculum development, evaluation of clinical

competence, evidence-based practice, and lifelong self-directed learning in

medicine. He has developed an Internet-based “Point of Care Clinical Question”

module and is collaborating on research with ABIM to deﬁne the characteristics

of physicians’ information needs and information seeking behavior, and the

association of self-directed learning with self-directed learning “readiness” evaluation

instruments. Dr. Green reports potential conﬂicts of interest related to continuing

education in the health professions because of his employment, and his funding as

a Kimball Scholar at the American Board of Internal Medicine.

Carol Havens, M.D., is Director of Clinical Education for Northern California

Kaiser-Permanente and a practicing physician in addiction medicine. At Kaiser-

Permanente, she has been involved in CME for 22 years and has served as regional

director of clinical education for the past 14 years. She is chair of the California

Medical Association CME Committee, a member of the Accreditation Review

Committee for the ACCME, and a member of the CME committees for the

Permanente Federation and Audio-Digest. She has been involved in a variety

of educational interventions, including live programs, video conferences, and

academic detailing training and evaluation. Dr. Havens reports potential conﬂicts

of interest related to continuing education in the health professions due to

employment with the Permanente Medical Group, honoraria from the California

Academy of Family Physicians and CMA’s Institution for Medical Quality, and

royalties from Wolters-Kluwer.

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 234

235

Paul C. Hébert, M.D., M.H.Sc., is Editor-in Chief of the Canadian Medical

Association Journal, a critical care physician at The Ottawa Hospital, and clinical

researcher at the Ottawa Health Research Institute. He is also professor in the

department of medicine with cross appointments in the departments of anesthesiol-

ogy, surgery, epidemiology, and community medicine. Dr. Hébert’s research has

focused on transfusion practice, including alternatives to transfusion, blood conser-

vation, and resuscitation ﬂuids. His groundbreaking trial on critical care transfusion

requirements, published in the New England Journal of Medicine, changed the way

clinicians throughout the world approach transfusion. Dr. Hébert reports no poten-

tial conﬂict of interest related to continuing education in the health professions.

Maryjoan D. Ladden, Ph.D., R.N., Assistant Professor in Ambulatory Care and

Prevention at the Harvard Medical School, is a pioneer in developing inter-profes-

sional education and practice models to improve the quality and safety of healthcare,

and to enhance collaboration between nurses and physicians. She is a course

director of Current Clinical Issues in Primary Care, PRI-MED, which is the largest

inter-professional primary care continuing education program in the world. Dr.

Ladden reports potential conﬂicts of interest related to continuing education in the

health professions due to her work with MJ Ladden & Associates, a consulting ﬁrm,

honoraria as co-director for Current Clinical Issues in Primary Care and the National

Medical Advisory Board Take Care Health System, and royalties from UpToDate.

David C. Leach, M.D., Immediate Past Executive Director of the Accreditation

Council for Graduate Medical Education, has focused on how physicians acquire

competence and become practitioners of the art, science, and craft of medicine.

Grants from the Robert Wood Johnson Foundation and the Pew Charitable Trusts

supported his work to develop innovative curricula for both medical students and

residents. His current interests include the teaching of improvement skills, aligning

accreditation with emerging healthcare practices, and the use of educational out-

come measures as an accreditation tool. Dr. Leach reports no potential conﬂicts of

interest related to continuing education in the health professions.

Donald A.B. Lindberg, M.D., has been director of the National Library of Medicine,

the world’s largest biomedical library, since 1984. Prior to his NLM appointment,

he was Professor of Information Science and Professor of Pathology at the University

of Missouri-Columbia. His work has enhanced the role of information and computer

technology in medical diagnosis, artiﬁcial intelligence, and education programs.

He was the ﬁrst President of the American Medical Informatics Association and serves

on numerous boards, including the Computer Science and Engineering Board of

the National Academy of Sciences, the National Board of Medical Examiners, and

the Council of the Institute of Medicine. He has written three books on the use of

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 235

computers and medical information systems and more than 200 articles and reports.

Dr. Lindberg reports no potential conﬂicts of interest related to continuing educa-

tion in the health professions.

Phil R. Manning, M.D., Paul Ingalls Hoagland Hastings Professor of Continuing

Medical Education and Professor of Medicine Emeritus at the Keck School of

Medicine, University of Southern California, is founding president of the Society

of Medical College Directors for Continuing Medical Education. He has been a

governor, regent, and vice president of the American College of Physicians and

serves on several educational committees of the American College of Cardiology.

He is also co-editor of Medicine: Preserving the Passion in the 21st Century and

author of 46 peer-reviewed publications. Dr. Manning reports potential conﬂicts

of interest related to continuing education in the health professions because of

honoraria from the National Library of Medicine.

Paul Mazmanian, Ph.D., serves as Associate Dean for Continuing Professional

Development and Evaluation Studies at the Virginia Commonwealth University

School of Medicine, where he is also a professor in the Department of Family

Medicine with a joint faculty appointment in the Department of Epidemiology and

Community Health. His research interests include learning and change in physician

performance. For the past eight years, he has been Editor of the Journal of

Continuing Education in the Health Professions and also has served on study

sections or as a consultant to the Agency for Healthcare Research and Quality,

the National Cancer Institute, The National Heart, Lung, and Blood Institute, and

the Institute of Medicine. Dr. Mazmanian reports no potential conﬂicts of interest

related to continuing education in the health professions.

Pamela H. Mitchell, Ph.D., M.S., holds numerous positions at the University of

Washington, where she is Associate Dean for Research, Professor of Biobehavioral

Nursing and Health Systems, and the Elizabeth S. Soule Distinguished Professor

of Health Promotion at the School of Nursing, and is Adjunct Professor of Health

Services at the School of Public Health and Community Medicine. She also is

founding director of the Center for Health Sciences Inter-professional Education

and Research, past chair of the American Academy of Nursing’s expert panel on

quality care, and president-elect of the American Academy of Nursing. Dr. Mitchell

reports potential conﬂicts of interest related to continuing education in the health

professions because of her employment (a self-supporting unit of the School of

Nursing that provides CNE) and honoraria for numerous speaking engagements

for non-commercial conferences and symposia that provide CE credit, including

the Washington State University Research Symposium, Washington State Nurses’

Association, and the Association for Prevention Teaching and Research.

236

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 236

Donald E. Moore, Jr., Ph.D., directs the Division of Continuing Medical Education

at the Vanderbilt University School of Medicine, where he is also Professor of

Medical Education and Administration. For more than 25 years, Dr. Moore has held

positions in medical schools, hospitals, and the government, where he helped to

develop and coordinate continuing medical education for physicians and hospital

staff, worked on faculty development, managed an educational technology center,

and established a diabetes care education and research program. His research

interests include practice-based CME, evidence-based medicine, the role of CME

in physician change, and the impact of CME on healthcare outcomes. Dr. Moore

reports potential conﬂicts of interest related to continuing education in the health

professions due to his employment, research funding from Wyeth Pharmaceuticals,

and consultancies with Wyeth, Rockpointe Communications, The Chatham Institute,

Fortis-Spectrum Communications, the Pennsylvania Consortium for Academic

CME, and Yale University School of Medicine.

June E. Osborn, M.D., served as the President of the Josiah Macy, Jr. Foundation

from 1996 until she retired at the end of 2007. Trained in pediatrics as well as

virology and infectious diseases, she previously served as Associate Dean of the

Graduate School for Biological Sciences at the University of Wisconsin-Madison

and as Dean of the School of Public Health at the University of Michigan. She has

served in many advisory roles for federal health agencies, private foundations, and

the World Health Organization, particularly on issues relating to virology, vaccines,

infectious diseases, and public health. She chaired the National Commission on

AIDS from 1989-1993 and was a member of WHO’s Global Commission on AIDS.

Dr. Osborn reports no potential conﬂicts of interest related to continuing education

in the health professions.

Ajit K. Sachdeva, M.D., established and directs the division of education at the

American College of Surgeons, where he oversees the development and implemen-

tation of innovative educational programs for surgeons, surgical residents, and

medical students. His initiatives include core competencies in surgery, enhanced

safety for surgical patients, expanding the repertoire of experiential learning pro-

grams, distant education, the use of simulations and simulators in surgery, and

evaluation of educational outcomes. He is widely recognized for his contributions

to the ﬁeld of continuing medical education. Dr. Sachdeva reports no potential

conﬂicts of interest related to continuing education in the health professions.

Marla E. Salmon, Sc.D., R.N., is Dean of the Nell Hodgson Woodruff School of

Nursing at Emory University, where she is also professor of both nursing and public

health. Her interests are in national and international health policy, administration,

public health, and workforce development. Dr. Salmon has held senior leadership

237

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 237

roles in professional and national government service. Her many positions

include membership on the Nursing Advisory Council of the Joint Commission on

Accreditation of Healthcare Organizations, the Board of Directors for the National

Council on Healthcare Leadership, and the Board of Trustees of the Robert Wood

Johnson Foundation. Dr. Salmon reports no potential conﬂicts of interest related to

continuing education in the health professions.

David C. Slawson, M.D., is the B. Lewis Barnett, Jr. Professor of Family Medicine

and Vice Chair for Academic Affairs for the Department of Family Medicine at

the University of Virginia Health System. He also is founder and director of the

Center for Information Science and holds a joint appointment as Professor in

the Department of Public Health Sciences. He directs the Faculty Development

Program for the Family Medicine Department and is a member of the editorial

board of the American Family Physician. Dr. Slawson reports potential conﬂicts

of interest related to continuing education in the health professions due to his

employment, consultancies with J. Wiley & Sons, honoraria from Tufts University,

and stock ownership/options with InfoPOEM, Inc.

Robert Steinbrook, M.D., is a national correspondent for the New England

Journal of Medicine and an Adjunct Professor of Medicine at Dartmouth Medical

School. He served as deputy editor of the New England Journal of Medicine from

1993 to 2001 and before that was a medical writer for the Los Angeles Times.

Dr. Steinbrook is a graduate of Amherst College and the University of Pennsylvania

School of Medicine and trained in internal medicine at the University of California,

San Francisco. Dr. Steinbrook reports no potential conﬂicts of interest related to

continuing education in the health professions.

William M. Tierney, M.D., is Chancellor’s Professor in the Indiana University

School of Medicine’s Department of Medicine. He also is a senior investigator in

the Regenstrief Institute, where he serves as Executive Director of the Regenstrief

Center for Healthcare Improvements and Research. His research interests include

the effects of innovative, computer-based interventions to improve healthcare

quality and lower costs and a major HIV/AIDS research program in collaboration

with Moi University in Kenya. He also directs ResNet, a PBRN within IU. Dr. Tierney

reports potential conﬂicts of interest related to continuing education in the health

professions because he serves as unpaid co-editor in chief of the Journal of General

Internal Medicine, which has a continuing medical education mission.

Thomas R. Viggiano, M.D., M.Ed., is Associate Dean for Faculty Affairs and

Barbara Woodward Lips Professor at the Mayo Medical School. He specializes in

curriculum integration and evaluation, and designed an innovative system for cur-

238

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 238

riculum evaluation based on quality improvement principles. Among other posi-

tions, he chairs the AAMC Group on Faculty Affairs, where he has helped develop

collaborative research projects involving AAMC’s Council of Teaching Hospitals.

Dr. Viggiano reports no potential conﬂicts of interest related to continuing education

in the health professions.

Susan W. Wesmiller, M.S., R.N., is Director of Nursing Education and Research

at the University of Pittsburgh Medical Center, where she also is a doctoral student

and adjunct faculty member at the School of Nursing. Recent presentations include

the use of technology in nursing education and the research process for nurse

educators. Her recent research involves communication strategies to improve co-

worker relationships and using a learning management system in a computerized

education program in a transplant intensive care unit. She reports no potential

conﬂicts of interest related to continuing education in the health professions.

Michael S. Wilkes, M.D., Ph.D., is Professor of Medicine and Director of Global

Health Programs at the University of California, Davis, School of Medicine. With

NIH funding, he has developed Web-based CME tools on topics such as ethics,

communications, clinical genetics, epidemiology, and end-of-life care. Other

research interests involve medical education and physician behavior change,

particularly around physician prescribing behavior. Dr. Wilkes also is an award

winning journalist and a long-standing medical commentator for National Public

Radio. Dr. Wilkes reports potential conﬂicts of interest related to continuing

education in the health professions because the University of California is a CME

provider and he holds an NIH grant to develop CME.

Patricia S. Yoder-Wise, R.N., Ed.D., is Editor-in-Chief of the Journal of

Continuing Education in Nursing: Continuing Competence for the Future. As a

professor of nursing at Texas Tech University Health Sciences Center-Lubbock, she

teaches in the graduate nursing administration program. She also teaches policy

and politics at Texas Woman’s University-Houston. As President of the American

Nurses Credentialing Center from 2005-2007, she was involved in numerous

discussions about certiﬁcation, Magnet and continuing education accreditation,

and will serve as guest editor for both Nursing Administration Quarterly and Nurse

Leader. Dr. Yoder-Wise reports potential conﬂicts of interest related to continuing

education in the health professions due to honoraria from the Journal of Continuing

Education in Nursing, the ANCC, and Glaxo-Wellcome, and for speaking at several

continuing education events, and royalties received from Elsevier, Jones and Bartlett

and Nursing Administration Quarterly.

239

Biographies

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 239

240

Editors: Mary Hager, Sue Russell, and Suzanne W. Fletcher, M.D., M.Sc.

Design & production: David Klaboe, Klaboe Design, NYC

Conference photos: Ernest McCreight, Bermuda

Cover photos in montage from Corbis, Getty Images, Masterfile, and iPhoto

Printed in USA by Cinnamon Graphics, Inc., New York

CEHP 5-9 with final edits (forhiresPDF):050306 NeuroMngrph11/17/05AA 5/9/08 5:37 PM Page 240

Continuing Education in

the Health Professions

Proceedings of a Conference

Chaired by Suzanne W. Fletcher, M.D., M.Sc.

Edited by Mary Hager, Sue Russell,

and Suzanne W. Fletcher

, M.D., M.Sc.

Continui ng Educ ation in the Health Profess ions

Fl etcher

The Josiah Macy, Jr. Foundation is a private philanthropy dedicated to improving

the health of individuals and the public. Since its establishment in 1930, the

Foundation has focused its support principally on projects and conferences

designed to enhance the education of health professionals, especially physicians.

ISBN 0-914362-49-6