The Newcastle ENDOPREM™: a validated patient reported experience measure for gastrointestinal endoscopy

NeilsonLJ, etal. BMJ Open Gastro 2021;8:e000653. doi:10.1136/bmjgast-2021-000653

The Newcastle ENDOPREM™: a

validated patient reported experience

measure for gastrointestinal endoscopy

Laura J Neilson ,

1,2

Linda Sharp,

2,3

Joanne M Patterson,

Christian von Wagner,

Paul Hewitson,

Lesley M McGregor,

Colin J Rees

1,2,3

To cite: NeilsonLJ, SharpL,

PattersonJM, etal. The

Newcastle ENDOPREM™:

a validated patient reported

experience measure for

gastrointestinal endoscopy.

BMJ Open Gastro

2021;8:e000653. doi:10.1136/

bmjgast-2021-000653

► Additional supplemental

material is published online

only. To view, please visit the

journal online (http:// dx. doi.

org/ 10. 1136/ bmjgast- 2021-

000653).

Received 17 March 2021

Accepted 5 August 2021

For numbered afliations see

end of article.

Correspondence to

Dr Laura J Neilson;

l. neilson@ nhs. net

Endoscopy

employer(s)) 2021. Re- use

permitted under CC BY- NC. No

commercial re- use. See rights

and permissions. Published

by BMJ.

ABSTRACT

Objectives Measuring patient experience of

gastrointestinal (GI) procedures is a key component of

evaluation of quality of care. Current measures of patient

experience within GI endoscopy are largely clinician

derived and measured; however, these do not fully

represent the experiences of patients themselves. It is

important to measure the entirety of experience and not

just experience directly during the procedure. We aimed to

develop a patient- reported experience measure (PREM) for

GI procedures.

Design Phase 1: semi- structured interviews were

conducted in patients who had recently undergone GI

endoscopy or CT colonography (CTC) (included as a

comparator). Thematic analysis identied the aspects

of experience important to patients. Phase 2: a question

bank was developed from phase 1 ndings, and iteratively

rened through rounds of cognitive interviews with

patients who had undergone GI procedures, resulting in

a pilot PREM. Phase 3: patients who had attended for GI

endoscopy or CTC were invited to complete the PREM.

Psychometric properties were investigated. Phase 4

involved item reduction and renement.

Results Phase 1: interviews with 35 patients identied

six overarching themes: anxiety, expectations, information

& communication, embarrassment & dignity, choice

& control and comfort. Phase 2: cognitive interviews

rened questionnaire items and response options. Phase

3: the PREM was distributed to 1650 patients with 799

completing (48%). Psychometric properties were found to

be robust. Phase 4: nal questionnaire rened including

54 questions assessing patient experience across ve

temporal procedural stages.

Conclusion This manuscript gives an overview of

the development and validation of the Newcastle

ENDOPREM™, which assesses all aspects of the GI

procedure experience from the patient perspective. It may

be used to measure patient experience in clinical care and,

in research, to compare patients’ experiences of different

endoscopic interventions.

INTRODUCTION

Around one- third of the population will

undergo gastrointestinal (GI) endoscopy

during their lifetime, with approximately 1.5

million endoscopies performed annually in

England and 17 million in the USA.

1–3

Colonography (CTC) is performed in many

countries as an alternative to colonoscopy

or where colonoscopy is incomplete. There

is significant variation in CTC use between

countries with around 150 000 CTCs under-

taken in England annually, and although less

Summary box

What is already known about this subject?

► Patient experience is a key dimension of high-

quality clinical care.

► Positive patient experience is an important determi-

nant of participation in screening programmes and

repeat attendance for surveillance procedures.

► Current measures of patient experience in gastroin-

testinal (GI) endoscopy are largely clinician derived

and measured and may not address the priorities of

patients themselves.

What are the new ndings?

► The Newcastle ENDOPREM™ is the rst fully

patient- derived patient- reported experience mea-

sure (PREM) for GI endoscopy.

► This is a comprehensive measure, which captures

experience across the entire patient journey, specif-

ically elements of patient experience as prioritised

by patients.

► The psychometric properties of the Newcastle

ENDOPREM™ have been demonstrated to be

robust.

How might it impact on clinical practice in the

foreseeable future?

► The Newcastle ENDOPREM™ is an important tool

for use both in research and evaluation of routine

care.

► Use of the Newcastle ENDOPREM™ will enable

units to measure experience provided by units and

endoscopists, to allow comparison of experience

across different endoscopy modalities (eg, as part

of clinical trials of devices) and will identify specif-

ic areas, which can be targeted to improve patient

experience.

► The PREM is now being adapted for use interna-

tionally and for measuring experience of more novel

technologies

on October 6, 2024 by guest. Protected byhttp://bmjopengastro.bmj.com/BMJ Open Gastroenterol: first published as 10.1136/bmjgast-2021-000653 on 25 October 2021. Downloaded from

NeilsonLJ, etal. BMJ Open Gastro 2021;8:e000653. doi:10.1136/bmjgast-2021-000653

Open access

widely used in the USA, it is performed in at least 700

centres.

4 5

Consistent high- quality endoscopy is essential to

ensure that procedures are safe and that lesions are not

missed, however, there is significant and unacceptable

variation in practice and quality.

6–10

Patient experience

is one of the key dimensions of high- quality clinical

care. Increasing emphasis on patient experience as a

marker of quality has emerged in recent years because

of the recognition that positive patient experience is

associated with better patient outcomes.

In England,

a report into a failing hospital found that losing sight of

patients at the centre of healthcare was a key component

of system failures.

As a result, patient experience is now

routinely assessed across many aspects of primary and

secondary care services in the English National Health

Service. Similarly, in the USA, there is increasing use of

experience measures (such as the Hospital Consumer

Assessment of Healthcare Providers and Systems Survey)

to evaluate healthcare providers.

The value of such

information is that it lets healthcare providers identify

which aspects of a service may need attention or require

improvement.

Within the field of endoscopy, there is evidence that

colonoscopists who deliver high- quality procedures also

deliver better patient experience.

Patient experience

is a key determinant of participation in bowel cancer

screening programmes and influences whether patients

attend for repeat disease surveillance procedures.

11 15

Current measures of patient experience within the

field of GI endoscopy focus almost completely on proce-

dural aspects such as pain or discomfort but ignore

elements such as preprocedural communication, anxiety

and preparation for procedures.

Additionally, almost

all measures are clinician derived and measured. For

example, the Gastrointestinal Endoscopy Satisfaction

Questionnaire focuses on satisfaction and did consult

with patients during question development, however,

these questions were developed based on literature

review and were not generated by the patients them-

selves following patient interviews using best practice

for development of measures of patient experience.

has been reported that patients and clinicians prioritise

different aspects of experience with clinicians prioritising

the procedure itself, including aspects such as comfort,

whereas communication and interaction with the endos-

copist are more important to patients.

Elements of the

endoscopy experience other than just those at the time

of the procedure, for example, bowel preparation and

preprocedural anxiety, are also important. Preproce-

dural anxiety has been found to be a key determinant

of pain and discomfort experience during other clin-

ical procedures.

Moreover, measuring only procedural

comfort scores gives information on only a very small part

of the overall patient experience. Patient experience of

CTC is not routinely measured in clinical practice and

its measurement in research settings tends to focus on

comfort and bowel preparation.

20 21

Endoscopy organisations advise measuring patient

experience but provide little advice on how this should be

done. The European Society of Gastrointestinal Endos-

copy (ESGE) and British Society of Gastroenterology

(BSG) have highlighted the importance of measuring

and improving patient experience and recommend

measuring self- reported patient experience using a ‘vali-

dated scale;’ however, they acknowledge the lack of a

standardised approach.

22 23

Patient- reported experience

measures (PREMs) are tools to measure patients’ expe-

rience of healthcare and should address those aspects of

care defined by patients as being important to them; thus,

patients should be involved in developing such measures

and in defining what they measure.

24 25

The systematic

development and validation of PREMs have been under-

taken in a range of specialties and disease conditions,

for example, cancer care, paediatric emergency care

and sickle cell disease but not in the field of GI endos-

copy or CTC.

26–28

The approach to devising a PREM

includes literature review to identify current tools and

aspects contributing to patient experience; a qualitative

stage involving relevant patients to identify areas of care

important to them; questionnaire design informed by the

qualitative work; cognitive testing of the questionnaire

and field testing to assess validity.

25 29

Tools to measure

the total, or overall, experience of GI endoscopy do not

exist. Those tools which have been developed to measure

some elements of experience have not been systemati-

cally developed with patients. This paper describes the

rigorous process of developing the first patient- derived,

validated, PREM for GI endoscopy.

METHODS

The study was undertaken in the North- East of England.

The development of the PREM involved four phases and

was undertaken according to the established COSMIN

(COnsensus- based Standards for the selection of health

status Measurement INstruments) criteria for develop-

ment of health measurement tools.

Phase 1: concept elicitation

This phase provided in- depth and detailed identifica-

tion and description of the experiences of patients who

had undergone oesophagogastroduodenoscopy (OGD),

colonoscopy or CTC and has been reported in detail

elsewhere.

Between February 2016 and April 2017,

patients aged ≥18 years who had undergone one of

these procedures for symptoms or surveillance (but not

within the national bowel cancer screening programme)

in one hospital were invited to participate in one- to- one

semi- structured interviews. Eligible patients on a series

of endoscopy lists were approached in the department

before their procedure. Purposive sampling was used

to ensure a range of procedures, age and sex among

participants. These were guided by a topic guide devel-

oped from literature review and expert opinion; the

guide was used flexibly so that any new issues identified

on October 6, 2024 by guest. Protected byhttp://bmjopengastro.bmj.com/BMJ Open Gastroenterol: first published as 10.1136/bmjgast-2021-000653 on 25 October 2021. Downloaded from

NeilsonLJ, etal. BMJ Open Gastro 2021;8:e000653. doi:10.1136/bmjgast-2021-000653

Open access

by patients were added to the guide to be explored in

subsequent interviews. Interviews were audio- recorded

and transcribed. Recruitment continued until data satu-

ration. Thematic analysis was undertaken to identify

over- arching themes.

This phase identified aspects of

experience that mattered to patients to inform ques-

tionnaire content. It was also used to identify whether it

was possible to develop a PREM that would apply across

different GI procedures.

Phase 2: development and content validity

This phase was undertaken between April and August

2017 and involved iterative development of the pilot

PREM. Informed by the qualitative interviews and a

focused literature review, a question bank was generated.

This involved decisions on topics to be included, their

order, question format and response format. Several

rounds of revision and review were undertaken by the

research team.

The draft questionnaire—designed for patient self-

completion—was then developed and pretested, using

several rounds of face- to- face cognitive interviews with

patients who had undergone GI procedures, from June

to July 2017. Interviewees were invited to ‘talk aloud’ as

they completed the questionnaire, with verbal probing

used to clarify any problems or issues.

Different styles

of question format and layout were tested. Interviews

were audio- recorded and transcribed. Analysis after each

round used a systematic approach to identify problems.

The questionnaire was refined after each round and the

refined draft tested in the subsequent round. Once no

new issues arose in interviews, the pilot PREM was agreed.

This process ensured patient comprehension of the ques-

tions and face and content validity of the questionnaire

(ie, confirmed relevance and comprehensiveness of the

questions).

Phase 3: psychometric properties and validation

This phase evaluated the psychometric properties of the

pilot questionnaire and validated it in a large cohort.

Between October 2017 and September 2018, the pilot

PREM was given to patients following GI endoscopy or

CTC in four English NHS hospital Trusts. The sample

size was based on the number of questions included in

the pilot PREM and exceeded the recommended 15

completed questionnaires per item for studies of psycho-

metric qualities. Patients were asked to take it home and

complete and return it in the prepaid envelope provided.

A patient information sheet (PIS) was included with the

questionnaire pack and it was made clear to participants

that return of a completed questionnaire was deemed to

signify consent.

Statistical analysis focused on response rates and

patterns, missing values, ‘floor’ and ‘ceiling’ effects (ie,

propensity of respondents to endorse the extreme ends

of the response scale) and correlations between ques-

tion responses (to identify agreement between items and

question redundancy) and with the total score (sum of all

the experience questions). Analysis was performed using

IBM® SPSS® V.24 (IBM Corp. Released 2016. IBM SPSS

Statistics for Windows, Version 24.0. Armonk, NY: IBM

Corp).

Following completion of Phase 3, all questions were

reviewed systematically alongside the Phase 3 data to

determine where refinements in wording or layout were

required. Those questions considered to be redundant

were removed, for example, those which correlated

poorly with others. This resulted in the Newcastle

ENDOPREM™

RESULTS

Phase 1: concept elicitation

Thirty- five patients (15 OGD, 10 colonoscopy, 10 CTC)

were interviewed. Characteristics of these patients are

Table 1 Phase 1 participant characteristics

Variable Number of participants (%)

Sex Male 19 (54.3)

Female 16 (45.7)

Age ≤54 12 (34.3)

55–64 6 (17.1)

65–74 12 (34.3)

≥75 5 (16.3)

Procedure OGD (including transnasal endoscopy) 15 (42.8)

Colonoscopy 10 (28.6)

CTC 10 (28.6)

Indication

Alarm symptoms 12 (34.3)

Routine symptoms 17 (48.6)

Surveillance procedures 6 (17.1)

CTC, CT colonography; OGD, oesophagogastroduodenoscopy.

on October 6, 2024 by guest. Protected byhttp://bmjopengastro.bmj.com/BMJ Open Gastroenterol: first published as 10.1136/bmjgast-2021-000653 on 25 October 2021. Downloaded from

NeilsonLJ, etal. BMJ Open Gastro 2021;8:e000653. doi:10.1136/bmjgast-2021-000653

Open access

listed in table 1. When invited to discuss their experience,

a few participants focused on a single aspect of the experi-

ence which was most important to them: for example, the

taste and volume of the bowel preparation or the effect

of good communication from staff. However, most partic-

ipants described their experience in chronological order,

that is, as stages of a process, starting with the referral

process (henceforth, called ‘before attending for the

test’), then visiting the hospital (‘at the hospital, before

the test’), undergoing the procedure itself (‘during the

test’) and what happened afterwards (‘after the test’).

Six themes emerged: anxiety, expectations, information

& communication, embarrassment & dignity, comfort

and choice & control. All six themes were demonstrated

in more than one procedural stage. All themes and

procedural stages emerged in relation to all three GI

procedures.

The themes were organised by procedural stage, as

described in phase 1. Questions were developed for

each relevant theme and stage; where possible, language

echoed that used by patients in phase 1. Most questions

were in the form of statements (eg, ‘during the test, my

dignity was maintained at all times’) with a five- level

Likert- type response scale, ranging from ‘strongly agree’

to ‘strongly disagree’. To minimise response set bias

(where respondents endorse the same response option

for all questions), both positive and negative questions

were included (ie, some statements were phrased such

that ‘strongly agree’ would indicate a ‘positive’ expe-

rience, and others phrased so that ‘strongly disagree’

would indicate a ‘positive’ experience).

Ten rounds of review and revision were conducted

by the study team. Five rounds of cognitive testing were

then undertaken, with three endoscopy patients in each

round (15 participants in total, table 2). Focused litera-

ture review found no additional aspects of patient experi-

ence not covered by the pilot PREM.

The pilot PREM thus comprised 59 questions which

spanned patient experience; plus a series of questions on

respondent characteristics (eg, age, ethnicity) as these

have been found to be associated with patient experience

in many clinical contexts; plus three questions on poten-

tial ‘explanatory’ factors (which might affect patient

experience; eg, endoscopist gender); and a further two

questions on ‘overall’ experience. The sections of the

pilot PREM were section A—completing this survey (10

questions), section B—before coming to hospital for

your test (16 questions), section C—preparing for your

test (six questions), section D—at the hospital, before

the test (five questions), section E—during the test (21

questions), section F—after the test (11 questions) and

section G—overall experience (three questions).

Phase 3: psychometric properties and validation

The PREM underwent multisite validation and was

given to 1650 eligible patients of whom 799 responded

(response rate=48.4%). The response rate was higher

in older patients and those undergoing lower GI proce-

dures were more likely to respond (table 3). As patients

took the PREM away to complete, the time to complete

it was not measured; however, in phase 2, the interview

time was 10–15 min. Respondents’ ages ranged from

18 to 95, with a mean age of 65.3 (SD 12.6). 43.3% of

respondents were male and the majority (98.4%) were of

white British ethnicity. 41.1% of respondents underwent

OGD (including 0.8% who underwent transnasal endos-

copy), 43.3% underwent colonoscopy, 1.3% underwent

both OGD and colonoscopy on the same day (referred to

as ‘OGD & colonoscopy’ henceforth) and 14.4% under-

went CTC.

There was notable variation in the numbers of partici-

pants recruited per procedure type; 10 patients who had

both OGD & colonoscopy performed responded to the

questionnaire (of 34 invited) and 115 CTC participants

Table 2 Phase 2 participant characteristics

Variable Number of participants (%)

Gender Male 6 (30.0)

Female 9 (60.0)

Age ≤54 5 (33.3)

55–64 5 (33.3)

65–74 3 (20.0)

≥75 2 (13.3)

Procedure OGD (including transnasal endoscopy) 7 (46.7)

Colonoscopy 7 (46.7)

CTC 1 (6.7)

Education level

High school 9 (60.0)

College 4 (26.7)

Higher education 2 (13.3)

CTC, CT colonography; OGD, oesophagogastroduodenoscopy.

on October 6, 2024 by guest. Protected byhttp://bmjopengastro.bmj.com/BMJ Open Gastroenterol: first published as 10.1136/bmjgast-2021-000653 on 25 October 2021. Downloaded from

NeilsonLJ, etal. BMJ Open Gastro 2021;8:e000653. doi:10.1136/bmjgast-2021-000653

Open access

(of 209 invited). Far fewer CTCs are done at each of the

sites than OGD and colonoscopies, and this was reflected

in the number of participants recruited in this group.

There was a discrepancy between the procedure the

patient reported that they had attended for and the

procedure confirmed by the clinical team in 53 cases

(6.6%). The section that should have been completed by

those undergoing colonoscopy and CTC only (covering

issues around bowel preparation) was completed by

96.1% of participants to whom it applied and 18.0% of

those to whom it did not.

Completion rates of individual questions were high; for

only three questions, more than 5% of people failed to

provide an answer. The oldest age group (>75 years) was

significantly more likely to miss questions.

In terms of potential question redundancy, two pairs

of questions correlated strongly (r

>0.8) and four ques-

tions poorly correlated with any others (r

<0.3). Eight

questions, including the four which poorly correlated

with any others, had poor corrected item- total correla-

tion (ITC <0.3), meaning that they did not correlate well

with the overall sum of all questions.

Twenty- five questions had a ceiling effect (>40% of

respondents endorsed the ‘best’ response). No questions

had floor effects (>40% choosing the ‘worst’ option).

Phase 4: item reduction and reﬁnement

Five questions were removed as they were considered

redundant because of poor inter- item correlation and

ITC. Two items (which were strongly correlated) were

merged; they asked a similar question with slightly

different time points. One question was added by the

study team to simplify an item and one explanatory ques-

tion was rephrased with a further explanatory question

added for clarity.

The final Newcastle ENDOPREM™ includes 10 demo-

graphic/patient characteristic questions, 54 patient

experience and four explanatory questions (online

supplemental file 1). A shorter version will be available

for specific procedures, for example, without the bowel

preparation section for upper GI endoscopy.

DISCUSSION

Understanding and improving patient experience are

fundamental to delivering high- quality GI endoscopy. We

give an overview of a systematic and rigorous approach

to developing the first validated and fully patient- derived

PREM for GI endoscopy, the Newcastle ENDOPREM™,

conducted according to the COSMIN criteria.

The Newcastle ENDOPREM™ is robust and compre-

hensive and in this form is designed to capture experi-

ence across the entire patient journey, not simply during

the procedure itself. In its current form, it is applicable

to upper GI endoscopy, colonoscopy and CTC. The

various elements of the patient journey—including how

the patient received their appointment, how they expe-

rienced the facilities in which they waited for the proce-

dure, how results were given and what would happen

next were conveyed—were all things that patients raised

in the qualitative work as being important to them.

A range of measures are available to measure individual

components of the endoscopy process. For example, a

number of procedural comfort scores exist and some

of these have been validated.

However, almost all

measures are clinician or expert designed rather than

being developed based on what patients report as being

important. The Global Rating Scale patient experience

domain is used internationally and was derived from liter-

ature review and expert opinion.

The Gastrointestinal

Table 3 Phase 3 response rates according to participant characteristics

Variable Respondents n (%) Non- respondents n (%) P value

Sex Male 346 (49.7) 350 (50.3) 0.342

Female 444 (47.3) 494 (52.7)

Age ≤54 142 (30.4) 325 (69.6) <0.001

55–64 191 (48.7) 201 (51.3)

65–74 271 (60.4) 178 (39.6)

≥75 181 (56.4) 140 (43.6)

Procedure OGD 328 (45.6) 392 (54.4) <0.001

Colonoscopy 346 (54.3) 291 (45.7)

CTC 115 (55.0) 94 (45.0)

OGD and colonoscopy 10 (29.4) 24 (70.6)

Site

A 98 (43.9) 125 (56.1)

0.045

B 193 (53.8) 166 (46.2)

C 220 (50.1) 219 (49.9)

D 288 (45.7) 342 (54.3)

P value for χ² tests.

CTC, CT colonography; OGD, oesophagogastroduodenoscopy.

on October 6, 2024 by guest. Protected byhttp://bmjopengastro.bmj.com/BMJ Open Gastroenterol: first published as 10.1136/bmjgast-2021-000653 on 25 October 2021. Downloaded from

NeilsonLJ, etal. BMJ Open Gastro 2021;8:e000653. doi:10.1136/bmjgast-2021-000653

Open access

Endoscopy Satisfaction Questionnaire was developed

by including the ‘most relevant’ questions from existing

measures.

Although patients were involved to ensure

face and content validity, they were not involved in

initially defining the questions. While it covers some

preprocedural and postprocedural elements, items about

preparation for the test or later results are not covered

in detail. The Gastronet is a clinician- derived question-

naire for assessing patient experiences of endoscopy in

Europe.

It includes three questions about discomfort

and a question about satisfaction with the information

given about the test and results, which patients complete

the day following their procedure. In the USA, the

American Society for Gastrointestinal Endoscopy recom-

mends that a general satisfaction scale is used, modified

for endoscopic procedures.

38 39

This is not specific to GI

procedures and was developed without patient input,

nor does it assess comfort. The Newcastle ENDOPREM™

addressed the assessment of patient experience differ-

ently by first establishing the aspects of experience that

mattered to endoscopy patients, then systematically devel-

oping and testing a tool to measure all of those aspects

of experience, while adhering to best practice in health

measurement development according to the established

COSMIN checklist.

Patients were also involved in the

process of refining and improving the instrument and in

the psychometric validation study. Thus, the instrument

is both grounded in patient experience and seeks to

measure that experience.

The length of this questionnaire is an important issue to

be considered. The PREM is necessarily long as it covers

the entirety of the patient experience and covers upper

and lower GI procedures with and without sedation.

However, despite this length, the completion time was

around 10–15 min in the cognitive interview round. In the

validation survey, the response rate of 48% is comparable

to similar self- completion questionnaires in other areas

of clinical practice.

Completion rates might be affected

by when the questionnaire is administered. We took the

decision to ask patients to complete the questionnaire

after leaving the department to avoid the possibility that

their ratings of experience would be influenced either by

sedation or by still being present in the hospital within

the department that had provided them with care. There

are different possible modes and timing of questionnaire

administration (eg, paper vs digital, provided at end of

clinic appointment vs sent by post after discharge) and

these will be the focus of further work. Were the ques-

tionnaire too long then the completion rate of questions

might be expected to be impacted, however, the comple-

tion of individual questions was excellent with very low

levels of missing data. Moreover, there was no evidence

that the number of questions missed increased in later

parts of the questionnaire. The full questionnaire covers

upper and lower GI endoscopy and CTC. At the outset,

it was unclear whether it would be possible to develop

a PREM that was applicable to patients who had under-

gone different endoscopic procedures. The work here

demonstrates that the issues that matter to patients are

very similar across GI procedures. The main difference—

as would be expected—is the issue of bowel preparation

for colonoscopy, which patients described in rich and

varied ways in the qualitative interviews.

We designed

the instrument so that the questions relating to bowel

preparation were in a contained section, however, we

note that this section was completed by 18% of those to

whom it did not apply. When applied to clinical practice

or research, only the relevant parts of the questionnaire

will be distributed and, thus, the PREM will be signifi-

cantly shorter and clearer.

The PREM will also need to be modified where clinical

practice varies. In the UK, conscious sedation is used for

colonoscopy. In other countries, for example, USA and

Australia, deep sedation is used. Therefore, non- relevant

sections of the PREM, for example, those covering

intraprocedural pain or comfort, will not be completed

where they are not needed. While redundant sections of

the PREM will not be given to particular patients and in

particular settings, it is important that this is based on

only removing those irrelevant sections and not simply

removing sections that clinicians or researchers consider

of lesser importance.

The inclusion of CTC within this PREM was to allow

a comparator from a non- endoscopy GI investigation,

which may be considered an alternative to colonoscopy.

Much of the argument for the role of CTC relates to

experience, so developing a PREM to allow this to be

measured accurately was one of the goals of this research.

A potential weakness of the study is the study popula-

tion. Phases 1 and 2 were undertaken in an expert centre

with a strong track record in endoscopy and endoscopy

research. Phase 3 was undertaken across four sites to

increase diversity of patients and experiences, but these

were all in the North- East of England, an area with very

limited ethnic diversity. Very few participants reported

their ethnic group as anything other than White British

(and only 0.5% of the participants in phases 1 and 2 were

non- White British).

This means we cannot be certain

that the performance and properties of the question-

naire are the same in non- White British ethnic groups.

Further work is planned to test the PREM further in

international settings and settings of wider ethnic diver-

sity. We also currently lack information on the suitability

of the questionnaire for patients who had other types

of endoscopic procedures (eg, flexible sigmoidoscopy);

testing this is also important.

There are two possible uses of the Newcastle ENDO-

PREM™. As a patient experience measure, the detail and

granularity of the questions across the journey can help

healthcare providers identify which specific aspects of

their service may benefit from improvement. The instru-

ment may also be used to compare patient experiences

of different endoscopic interventions in research studies.

Detailed understanding of the different components

of endoscopy and how these affect patient experience

would be valuable in the context of such head- to- head

on October 6, 2024 by guest. Protected byhttp://bmjopengastro.bmj.com/BMJ Open Gastroenterol: first published as 10.1136/bmjgast-2021-000653 on 25 October 2021. Downloaded from

NeilsonLJ, etal. BMJ Open Gastro 2021;8:e000653. doi:10.1136/bmjgast-2021-000653

Open access

comparisons. While the Newcastle ENDOPREM™ was

developed within the context of the NHS in the UK, rela-

tively few of the questions are specific to the processes

and organisation of care in the NHS. We would expect,

therefore, that the instrument would be applicable inter-

nationally with revisions related to service context. We

are currently investigating this in a rigorous international

validation study across Europe, USA and Australia using

the European Organisation for Research and Treat-

ment of Cancer approach to international translation of

patient- reported outcome measures, with the PREM now

having been translated into Polish, Norwegian, Spanish,

Italian, Dutch and French.

The Newcastle ENDOPREM™ is currently being

adapted for other GI procedures, including capsule

endoscopy and CytoSponge. These procedures are

being introduced into routine clinical practice, in some

instances as an alternative to endoscopy, and, therefore,

the Newcastle ENDOPREM™ will be available to compare

experiences of these procedures, both in the clinical and

research setting.

The Newcastle ENDOPREM™ has been developed

through a robust and comprehensive pathway. The

themes are likely to remain constant despite the changing

endoscopic landscape. This PREM was developed prior

to the novel COVID- 19 pandemic and while delivery of

endoscopy may change, and consequently how patients

rate their experience, this tool should remain a valid way

to measure that experience.

CONCLUSION

The Newcastle ENDOPREM™ is the first patient- derived

PREM that can be used to assess experience of patients

who have undergone different GI procedures. It is now

available for use in GI endoscopy research or evaluation

of routine care.

Author afliations

Department of Gastroenterology, South Tyneside and Sunderland NHS Foundation

Trust, South Shields, UK

Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne,

Newcastle University Centre for Cancer, Newcastle University, Newcastle upon

Tyne, UK

School of Health Sciences, University of Liverpool, Liverpool, UK

Health Behaviour Research Centre, University College London, London, UK

Nufeld Department of Population Health, University of Oxford, Oxford, UK

Division of Psychology, University of Stirling, Stirling, UK

Twitter Laura J Neilson @LauraJNeilson

Contributors CJR secured funding. CJR, LJN and LS conceived the study and

reviewed the literature. CJR, LS and JMP oversaw data collection, analysis and

interpretation. LJN undertook all interviews, analyses and drafted the paper. LMM

double coded a proportion of the interview transcripts. CJR, LJN, LS, JMP, CvW,

PH and LMM contributed to study design, iteratively rened and approved the

nal PREM, critically reviewed manuscript drafts and approved the nal article for

submission. CJR and LJN are guarantors for the data.

Funding This study was an investigator led study funded by Aquilant Endoscopy

and adopted onto the National Institute for Health Research Portfolio (UKCRN ID

18749). The funder had no role in study design, data collection, data analysis,

data interpretation or writing of this manuscript. The corresponding author had full

access to all the data in the study and had nal responsibility for the decision to

submit.

Competing interests CJR has received grant funding from ARC medical, Norgine,

Aquilant and Olympus medical. He was an expert witness for ARC medical. CJR

and LS have received research funding from 3D matrix. No other authors have

competing interests.

Patient consent for publication Not applicable.

Ethics approval Ethical approval was obtained through the NRES Committee

London- Stanmore (IRAS ID: 14689, National Institute for Health Research UKCRN ID

18749). Participants provided informed consent before interview.

Provenance and peer review Not commissioned; externally peer reviewed.

Data availability statement No data are available.

Supplemental material This content has been supplied by the author(s). It has

not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been

peer- reviewed. Any opinions or recommendations discussed are solely those

of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and

responsibility arising from any reliance placed on the content. Where the content

includes any translated material, BMJ does not warrant the accuracy and reliability

of the translations (including but not limited to local regulations, clinical guidelines,

terminology, drug names and drug dosages), and is not responsible for any error

and/or omissions arising from translation and adaptation or otherwise.

Open access This is an open access article distributed in accordance with the

Creative Commons Attribution Non Commercial (CC BY- NC 4.0) license, which

permits others to distribute, remix, adapt, build upon this work non- commercially,

and license their derivative works on different terms, provided the original work is

properly cited, appropriate credit is given, any changes made indicated, and the

use is non- commercial. See:http:// creativecommons. org/ licenses/ by- nc/ 4. 0/.

ORCID iD

Laura JNeilson http:// orcid. org/ 0000- 0002- 7185- 0825

REFERENCES

1 Seeff LC, Manninen DL, Dong FB, etal. Is there endoscopic capacity

to provide colorectal cancer screening to the unscreened population

in the United States? Gastroenterology 2004;127:1661–9.

2 Department of Health Knowledge and Intelligence Team. Further

analysis of HES endoscopy data. London: Department of Health

Knowledge and Intelligence Team, 2010.

3 Peery AF, Crockett SD, Murphy CC, etal. Burden and cost of

gastrointestinal, liver, and pancreatic diseases in the United States:

update 2018. Gastroenterology 2019;156:254–72.

4 Obaro AE, Plumb AA, Fanshawe TR, etal. Post- imaging colorectal

cancer or interval cancer rates after CT colonography: a systematic

review and meta- analysis. Lancet Gastroenterol Hepatol

2018;3:326–36.

5 American College of Radiology. American College of Radiology

Website- my CT colonography, 2021. Available: https://www. acr.

org/ Clinical- Resources/ Colon- Cancer- Screening- Resources/ My- CT-

Colonography

6 Gavin DR, Valori RM, Anderson JT, etal. The National colonoscopy

audit: a nationwide assessment of the quality and safety of

colonoscopy in the UK. Gut 2013;62:242–9.

7 Bowles CJA, Leicester R, Romaya C, etal. A prospective study

of colonoscopy practice in the UK today: are we adequately

prepared for national colorectal cancer screening tomorrow? Gut

2004;53:277–83.

8 Oppong KW, Romagnuolo J, Cotton PB. The ERCP quality network

benchmarking project: a preliminary comparison of practice in UK

and USA. Frontline Gastroenterol 2012;3:157–61.

9 Rembacken B, Hassan C, Riemann JF, etal. Quality in screening

colonoscopy: position statement of the European Society of

gastrointestinal endoscopy (ESGE). Endoscopy 2012;44:957–68.

10 Kaminski MF, Regula J, Kraszewska E, etal. Quality indicators

for colonoscopy and the risk of interval cancer. N Engl J Med

2010;362:1795–803.

11 Black N, Jenkinson C. Measuring patients’ experiences and

outcomes. BMJ 2009;339:b2495.

12 Francis R. Report of the Mid- Staffordshire NHS Foundation

trust public inquiry and the government's response.. London

The Stationery Ofce; 2013. https://www. gov. uk/ government/

publications/ report- of- the- mid- staffordshire- nhs- foundation- trust-

public- inquiry

on October 6, 2024 by guest. Protected byhttp://bmjopengastro.bmj.com/BMJ Open Gastroenterol: first published as 10.1136/bmjgast-2021-000653 on 25 October 2021. Downloaded from

NeilsonLJ, etal. BMJ Open Gastro 2021;8:e000653. doi:10.1136/bmjgast-2021-000653

Open access

13 Agency for Healthcare Research and Quality. CAHPS®: assessing

health care quality from the patient’s perspective 2014 https://www.

ahrq. gov/ cahps/ about- cahps/ index. html

14 Ekkelenkamp VE, Dowler K, Valori RM, etal. Patient comfort and

quality in colonoscopy. World J Gastroenterol 2013;19:2355–61.

15 Doyle C, Lennox L, Bell D. A systematic review of evidence on

the links between patient experience and clinical safety and

effectiveness. BMJ Open 2013;3:e001570.

16 Brown S, Bevan R, Rubin G, etal. Patient- Derived measures of Gi

endoscopy: a meta- narrative review of the literature. Gastrointest

Endosc 2015;81:1130–40.

17 Hutchings HA, Cheung W- Y, Alrubaiy L, etal. Development

and validation of the gastrointestinal endoscopy satisfaction

questionnaire (GESQ). Endoscopy 2015;47:1137–43.

18 Denters MJ, Deutekom M, Derkx B, etal. Patient satisfaction

with the colonoscopy procedure: endoscopists overestimate the

importance of adverse physical symptoms. Frontline Gastroenterol

2012;3:130–6.

19 Baser E, Togrul C, Ozgu E, etal. Effect of pre- procedural state- trait

anxiety on pain perception and discomfort in women undergoing

colposcopy for cervical cytological abnormalities. Asian Pac J

Cancer Prev 2013;14:4053–6.

20 Hansmann A, Burling D. Essential requirements of a CT

colonography service. Eur J Radiol 2013;82:1187–91.

21 Zueco Zueco C, Sobrido Sampedro C, Corroto JD, etal. CT

colonography without cathartic preparation: positive predictive

value and patient experience in clinical practice. Eur Radiol

2012;22:1195–204.

22 Kaminski MF, Thomas- Gibson S, Bugajski M, etal. Performance

measures for lower gastrointestinal endoscopy: a European Society

of gastrointestinal endoscopy (ESGE) quality improvement initiative.

Endoscopy 2017;49:378–97.

23 Rees CJ, Trebble T, von Wagner C. British Society of

gastroenterology position statement on patient experience of GI

endsocopy. Gut 2019;66:1181–96.

24 Coulter A, Fitzpatrick R, Cornwell J. The point of care. Measures

of patients’ experience in hospital: purpose, methods and uses.

London: King's Fund, 2009. https://www. kingsfund. org. uk/ sites/

default/ les/ Point- of- Care- Measures- of- patients- experience- in-

hospital- Kings- Fund- July- 2009_ 0. pdf

25 Hodson M, Andrew S, Michael Roberts C, Roberts C. Towards

an understanding of PREMS and PROMs in COPD. Breathe

2013;9:358–64.

26 Davies F, Vohra J, Hopwood B. The development of a patient

reported experience measure (PREM) survey for children 0- 16

years in urgent and emergency care 2011. London Royal College of

Paediatrics and Child Health; 2011. https://www. rcpch. ac. uk/ sites/

default/ les/ Research_ Report_-_ PREM_ Development_ Oct2012. pdf

27 Department of Health. National cancer patient experience survey,

2011- 2012. UK data service, SN: 7134 2012.

28 Chakravorty S, Tallett A, Witwicki C, etal. Patient- Reported

experience measure in sickle cell disease. Arch Dis Child

2018;103:1104–9.

29 Jenkinson C, Coulter A, Bruster S. The Picker patient experience

questionnaire: development and validation using data from in- patient

surveys in ve countries. Int J Qual Health Care 2002;14:353–8.

30 Mokkink LB, Terwee CB, Patrick DL, etal. The COSMIN

checklist for assessing the methodological quality of studies

on measurement properties of health status measurement

instruments: an international Delphi study. Qual Life Res

2010;19:539–49.

31 Neilson LJ, Patterson J, von Wagner C, etal. Patient experience

of gastrointestinal endoscopy: Informing the development of the

Newcastle ENDOPREM™. Frontline Gastroenterol 2020;11:209–17.

32 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res

Psychol 2006;3:77–101.

33 Willis GB. Cognitive interviewing: a tool for improving questionnaire

design. London: Sage Publications, 2004: 335.

34 Conrad F, Blair J. From impressions to data: increasing the

Objectivity of cognitive interviews. In: Proceedings of the Section on

Survey Research Methods, American Statistical Association 1996.

35 Rostom A, Ross ED, Dubé C, etal. Development and validation of a

nurse- assessed patient comfort score for colonoscopy. Gastrointest

Endosc 2013;77:255–61.

36 Joint Advisory Group on Gastrointestinal Endoscopy. Joint Advisory

group on gastrointestinal endoscopy (JAG) global rating scale (GRS).

version for acute services (all nations). London 2016 https://www.

thejag. org. uk/ Downloads/ JAG/ Accreditation% 20-% 20Global%

20Rating% 20Scale% 20( GRS)/ Guidance% 20-% 20acute% 20GRS%

20standards% 20UK. pdf

37 Hoff G, Bretthauer M, Huppertz- Hauss G, etal. The Norwegian

Gastronet project: continuous quality improvement of colonoscopy

in 14 Norwegian centres. Scand J Gastroenterol 2006;41:481–7.

38 Johanson JF, Schmitt CM, Deas TM, etal. Quality and outcomes

assessment in gastrointestinal endoscopy. Gastrointest Endosc

2000;52:827–30.

39 Rizk MK, Sawhney MS, Cohen J, etal. Quality indicators common to

all GI endoscopic procedures. Gastrointest Endosc 2015;81:3–16.

40 NHS England Analytical Team (Medical and Nursing Analytical Unit).

Statistical bulletin: overall patient experience scores 2015 adult

inpatient survey update NHS England; 2016. https://www. england.

nhs. uk/ statistics/ 2016/ 06/ 08/ overall- patient- experience- scores-

2015- adult- inpatient- survey- update/

41 Ofce for National Statistics. UK data service census support, 2011

census: Digitised boundary data (England and Wales), 2011. https://

borders. ukdataservice. ac. uk

42 Kuliś D, Bottomley A, Velikova G. EORTC quality of life group

translation procedure. Fourth Edition. Brussels: EORTC, 2017.

https://www. eortc. org/ app/ uploads/ sites/ 2/ 2018/ 02/ translation_

manual_ 2017. pdf

on October 6, 2024 by guest. Protected byhttp://bmjopengastro.bmj.com/BMJ Open Gastroenterol: first published as 10.1136/bmjgast-2021-000653 on 25 October 2021. Downloaded from