nm0296: thanks very much for coming to our symposium [0.2] on educational [0.5] 
research [1.1] er [0.3] as you probably know we were officially named 
Department of Undergraduate G-P Medical Education [0.6] on the first of [0.2] 
March [0.3] this year when we have been administering [0.4] the undergraduate G-
P [0.5] course for the [0.3] namex [0.2] Medical School of the namex namex [0.
6] Medical Schools [0.6] er [2.0] so [0.3] what we do here at the moment is to 
mirror [0.7] exactly what happens at the undergraduate [0.6] department [0.3] 
at namex University the undergraduate G-P department it's exactly the [0.4] the 
same curriculum [0.7] er [0.2] inevitably [0.4] our remit is to provide [0.5] 
medical education [0.4] within [0.3] primary care [0.4] and [0.2] teaching 
students er [0.2] competencies [0.3] in the consultation [0.9] but that also [0.
4] involves research [0.4] into medical education [0.6] and [0.8] all the 
different people that are speaking today will be talking about a different 
aspect [0.4] of [0.3] undergraduate [0.4] medical education that will be [0.4] 
related to [0.4] primary care [0.7] but we're also joined by the [1.1] er [0.2] 
group of people who 
are doing a module in effective teaching of the Master's degree [0.4] in 
Medical Education who have joined us today [0.3] of which i am also [0.5] a 
student [1.2] so about [0.7] ten of our colleagues from that [0.4] module have 
joined us [0.6] today [1.2] now [0.3] i'm handing evaluation forms round i'd be 
very grateful if you would [0.3] fill those evaluation forms out [0.4] as the 
day goes on and various people are [0.3] are talking they're the same 
evaluation forms that we're using [0.3] for the Master's in [0.4] in Medical 
Education so it will it will [0.4] help us to [0.2] to evaluate that [0.6] er 
[0.2] inevitably because it's an educational research [0.5] er symposium [0.3] 
it's also become an educ-, [0.2] educational research project [0.3] in itself 
[0.4] and i'm going to ask [0.4] namex from the Centre for [0.4] English 
Language [0.5] Teachers' Education to explain to us [0.7] what is happening [0.
6] namex 
of0297: thanks a lot Dr namex [0.4] er as Dr namex said my name's namex and i 
come 
from the Centre for English Language Teacher Education [0.3] on main campus [0.
9] and er this is part of a research project [0.4] into er [0.5] the language 
of [0.3] academic speech [0.4] er what we're doing is putting together a 
database of a whole lot of lectures and seminars from a whole [0.4] er a whole 
range of subject areas different subject areas around the university [0.5] and 
it's er becoming [0.8] part of a [0.3] corpus [0.3] that is going to be made 
available to researchers into language so [0.4] er [0.6] in fact part of the 
database has already been used in several papers on er [0.3] academic 
vocabulary and syntax and the difference between [0.4] written and spoken 
English er these sort of projects that are going on [0.4] er part of [0.3] er 
[0.6] the clips from the res-, er recordings are also being used for another 
project which is called [0.4] Essential Academic Skills in English [0.3] and 
that's a C-D-ROM project that's aimed at [0.6] univers-, er students who are 
coming to university from [0.5] er [0.2] foreign countries for the first time 
[0.4] and what it aims to do is actually ease 
them into [0.3] the university experience give them exposure to the kind of 
language that they're going to be exposed to [0.4] when they come to university 
[0.4] so as to [0.2] make the ride easier for them [0.2] as it were [0.5] so i 
just need to make sure that you're okay with being filmed for this [0.4] is it 
is it all right [0.7] 
nf0298: yeah 
nm0299: sure [laughter] 
of0297: brilliant thanks a lot [laughter] 
nm0296: i should have introduced myself actually right at the beginning but i 
think [0.2] you all [0.2] know who i am [0.5] er but i'm namex i'm a general 
practitioner in namex and a senior lecturer here [0.6] and i was informed by 
one of my colleagues at the beginning [0.3] the longer the title that you have 
in the university is usually [0.4] er correlates with your [0.2] lesser 
importance [0.2] [laughter] in the university so i've got this grand title of 
being Director of G-P Undergraduate Medical Education [0.5] [laughter] er [1.3] 
so that's a l-, er a little bit about myself [0.4] with relation to the day 
itself all the [0.4] er speakers will [0.2] be allocated half an hour [0.3] the 
idea is that they speak for fifteen minutes because hopefully it's going to 
generate a lot of discussion [0.4] amongst the group so hopefully feef-, 
fifteen minutes discussion [0.5] at eleven-thirty we'll break for coffee [0.4] 
and then at twelve-forty-five we will make our way to the central [0.4] campus 
to eat in what is referred to as the Eat restaurant [0.4] in the Arts Centre [0.
5] where you'll have a very nice meal [0.4] and er then we'll come back here [0.
4] again at two o'clock [0.5] er i should also say that i've organized today 
with my colleague who's sat here namex [0.5] who is a G-P in namex and a 
lecturer here [0.4] in the department so he's been putting in a lot of work [0.
4] behind the scenes [0.3] for today and is also going to be [0.2] speaking a 
little bit later [0.8] so we should introduce our [0.4] 
first speaker [0.5] er Dr namex [0.5] who is from namex University [0.4] he is 
an academic G-P registrar but is shortly to become a lecturer [0.4] in 
communication skills [0.4] at namex University he's also a general practitioner 
[0.5] and he's going to talk about student centred evaluation using [0.4] the 
nominal group technique [1.4] thank you [10.1] 
nm0295: right thank you very much [0.2] er it's a it's a pleasure to come here 
to speak to this group [0.5] er as [1.2] as i slowly sort of spread the word of 
what i've been doing er [0.9] as the the title says er i submitted this for my 
er my M-S-C in primary care [0.4] and yesterday i was er i was fortunate to 
receive the news that it has formally been accepted [0.4] er subject to the 
final exam boards so probably as [0.3] formally accepted as it possibly can be 
[0.7] er [1.3] whereabouts are the [0.3] forward [0.3] and back [0.2] for the 
slides on the [1.5] for the Powerpoint [0.5] is there er something 
i'm not seeing [0.4] aha 
nm0300: just pull it 
nm0295: yes there is there's a hidden drawer [laughter] [0.6] [cough] [0.3] 
marvellous right [0.9] okay [0.2] so [0.4] er the the nominal group technique 
i'll come to describe a bit more but just to mention that it was something that 
perhaps i came upon entirely by accident it may well be something that p-, 
people here don't know an awful lot about [0.2] but hopefully i'll change that 
as the course of this as we go along [1.8] some of what i i wanted to talk 
about really is evaluation and the whole process of evaluation that [0.4] er 
for me [1.7] i i felt that evaluation wasn't always a terribly useful thing in 
the way that it's being done [0.5] and th-, this just a just a way of 
summarizing what happens as part of any educational event [0.3] that you know 
you want to collect data analyse interpret disseminate it make a decision [0.3] 
and this informs then to change your practice [0.8] er [1.1] 
what i found particularly interesting about evaluation was the the the aspects 
really of data collection [1.0] er and with data collection and e-, evaluation 
[0.8] there were two things that i always wanted to know about this is [0.4] 
why we were [0.2] were we collecting this information [0.4] and who were we 
collecting this sort of information for [0.5] and generally what i found was 
that it probably isn't actually for you [0.5] that often evaluations are done 
because somebody has decided that something is important [0.5] but it might not 
necessarily be what's actually important to you the the person that's 
experiencing [0.6] the learning [1.3] so i wanted to create something that was 
a student centred view [0.4] er [0.3] of an educational experience [2.1] now [0.
8] evaluation is a is a difficult thing and i-, [0.2] in different circles it's 
thought of in in sort of very high terms or very low terms [0.4] some people [0.
2] and if i but to me this is 
the at the top of the list is that people believe it's a purely administrative 
task [0.3] it's done because it has to be done because somebody above you says 
[0.4] evaluate this [0.3] you tick the boxes you hand it in and the job is done 
[1.4] there can be resistance within a faculty setting towards evaluation again 
believing that it's that you know it's a waste of time [0.6] er [0.7] often 
there's little opinion in what the students actually think about what's going 
on yes they tick the boxes they fill in the things [0.4] but then these things 
just get cast to one side [0.5] er [0.4] if you're asking for self-reported 
data er i think i'm a wonderful lecturer [0.3] isn't going to be particularly 
useful when it comes to evaluation [0.6] and overall just apathy and 
disinterest in the whole process [1.3] so i wanted to create something that was 
t-, th-, w-, that was as as student centred as i could possibly make it [0.5] 
so i began by taking a focus group of er final year medical students and took 
them through a semistructured [0.5] er focus group 
asking them broadly about what they were doing [0.4] er within the the 
community based medicine course at Birmingham University [2.2] then [0.2] what 
i did is i used the information from this focus group [0.7] to devise 
statements for the use with the nominal group technique [0.5] now by a 
statement i mean it's sort of like a stimulus statement [0.4] so just er a-, an 
example from outside of medicine is [0.3] what do you think it's important to 
do before you go to work in the morning [0.4] i mean it's a silly example but 
it just gives you an idea of what a stimulus statement is so that poses a 
question [0.4] to which people may then generate lots of ideas [1.7] so what is 
the nominal group technique well the nominal group technique [0.4] is a a a 
structured way of generating ideas [0.3] and it consists of several different 
phases [1.3] the first part is a silent phase so you pr-, you y-, you you [0.2] 
provide th-, the the participants with a 
statement [0.4] and [0.4] so for instance the one about what you do before you 
go to work [0.3] then they work individually [1.0] er in response to this so 
they work on paper [0.4] and they write down a list of everything they can 
think of in response to that statement and it has to be individual no 
discussion occurs [1.3] the second part is the round robin phase so [0.7] each 
person [0.2] in turn reads out the first thing on their list it gets 
transmitted er trans-, [0.2] transmitted transcribed [0.3] to a chart on the 
wall [0.6] where [0.5] that goes on in in a cycle process each person saying 
the next thing on their list until you have a grand list of absolutely 
everything that people have written down [0.4] again [0.2] there's no time for 
er [0.5] for s-, speaking [0.3] other than saying what's on your list you can't 
debate whether or not that's a stupid thing a good thing or a bad thing [0.3] 
everything goes on the list [1.9] this is the only time at which speak-, 
speaking is allowed and this is the clarification phase so you can get to 
understand what 
other people mean by their statements [0.6] it is also the only time at which 
anything can be deleted from the list [0.4] so for instance [0.2] a-, you know 
i-, identical duplications which do happen a lot during this technique [0.4] 
but people can't say i don't agree with that being on the list take it off b-, 
if it goes on the list [0.2] it stays on the list [1.1] and then something 
that's quite crucial for the nominal group technique [0.2] is the voting phase 
[0.9] so for instance in response to a statement they may be presented with [0.
3] thirty statements [0.2] you then say to them [0.4] what are the seven most 
important things to you [0.4] so you as the individual learner in this 
experience [0.2] what is important to you [0.6] the most important thing then 
getting seven votes the next thing six votes five votes four fo-, four votes 
and so on [0.8] and then you collect in all the votes for all the statements [0.
5] and then you feed that information back [0.3] to the participants [0.3] so 
it's a way of generating a large amount of information [0.3] but 
then asking people to whittle it down to what really is important [0.3] and 
then using the votes to calculate what the most important thing is [0.3] to the 
group as a whole [2.2] now the nominal group technique has many advantages [0.
7] it's a structured way [0.3] so it's not like filling in evaluation forms and 
then reading thousands of comments that have been written and trying to 
tabulate them in some way [0.2] it's a structured way of doing this [1.0] it's 
non-confrontational if you write it down you say it people can't shout you down 
for saying things that you don't necessarily agree with [0.8] it disperses 
dominance you can't have one voice going on and on about one particular thing 
[0.3] if you've said it once you can't say it again it's on the list it comes 
down to the voting [0.3] as to whether or not that's important to everyone [1.
8] it's also very easy [0.2] to produce data as we'll 
come on to see how much data this sort of thing produces [0.8] so what 
statements did i use [0.9] well through the nominal gr-, through the the focus 
group we discovered various bits that were quite interesting for the students 
and things that provoked the most debates so we generated three statements and 
they're quite similar to some other pieces of research that have been done like 
this [0.8] so we asked them first of all what would you change [0.3] about the 
community based medicine course in the final year [1.2] then we asked them what 
did you enjoy [0.6] about the community based medicine course in the final year 
[0.7] and finally [0.2] what does final year community based medicine teaching 
give you [0.5] that teaching in other settings does not [3.1] so [0.2] [0.3] at 
the nominal group we recruited eighteen students to attend 
and they were split on the day into two groups purely for logistical reasons 
and and handling large groups [0.8] then they were combined [0.3] towards the 
end of the process so that they all had their master lists to look at [0.4] so 
in response to the three different statements [0.2] they generated a hundred-
and-twenty-four separate ideas [0.8] and from that [0.2] they voted the top 
seven in response to each statement so we had twenty-one statements [0.3] that 
according to this group [0.4] were the most important things overall [0.5] 
about community based medicine at Birmingham [0.9] so you may ask well what 
does eighteen students mean in comparison to the whole of the year it was about 
approximately two-hundred students [0.6] so what we did is we then converted 
that into a questionnaire [0.5] so the twenty-one statements we then asked the 
whole year do you agree or disagree [0.3] with this [0.2] as a as an important 
statement [1.0] a hundred-and-eighty students er responded which is a response 
rate of about eighty per cent which we felt was quite 
acceptable [0.8] and overall out of the twenty-one statements that were made 
twenty of them obtained majority support [1.2] er what you say is majority 
support is you know s-, is very much up to you [0.5] the the cut-off for twenty 
year to twenty-one is at fifty per cent so over fifty per cent of the year 
agreed [0.4] but as we'll come on to see that most people did have a very high 
level of agreement [1.8] so here we have [0.2] er [1.0] the layout of it so the 
three colours represent the three different statements [0.4] with [0.2] each 
statement number [0.4] so these are the seven things [0.2] in response to each 
stimulus statement these are the seven items along the bottom [0.7] and you can 
see that on the whole if you draw a rough line across eighty per cent you still 
get most of the people [0.5] agreeing over eighty per cent and then down here 
you've got these two [0.5] statements which really didn't overall get very good 
er agreement within the whole year [2.5] so the poorer scoring items what were 
they [0.3] well the first one was about general 
knowledge on the ophthalmology and dermatology exam and i think this was a 
specific thing to this group that they were upset about the content of the exam 
when they had it [0.5] but it seemed when it was presented to the whole year [0.
2] that overall that just wasn't agreed with [1.5] there was a er er just a s-, 
a slight shift towards agreement with this statement just saying that you know 
we wanted more formal tutorials in general practice they wanted more set taught 
er tutorials on audit [0.3] on screening on cardiovascular disease [0.4] but 
again y-, n-, there was a high level of disagreement with that statement [0.8] 
so i'll i'll just move on now to the the supported items and i won't bore you 
in great detail of absolutely every th-, single thing i just picked out the 
ones that i thought were the most exciting [1.5] so [0.2] what would you change 
about community based medicine course well they wanted more 
dermatology teaching there was no doubt about that and i think this harks back 
to even the days when i was a medical student the dermatology [0.5] among 
students is poorly thought of and on the whole can be quite badly taught [1.3] 
there were small numbers who felt that it was being done well and i think that 
represented small pockets of dermatology teaching within the practices that 
were being done very well [0.3] but on the whole [0.2] it wasn't being done 
very well at all [2.3] the course at Birmingham is split into an inner city 
practice and an ou-, sort of an outer city practice [0.5] and the students were 
flitting between the two centres and they actually felt that it was very bitty 
and er disorganized [0.3] and they'd much rather have [0.3] core central blocks 
[0.4] at one place rather than being spread out and moving all the time [2.8] 
and finally on this one they wanted feedback on their examination stations 
they have a voices exam where they they rotate round different stations [0.3] 
and unless they do particularly badly or there is something of particular 
concern the students don't get [0.3] feedback about that station they were very 
keen to have that [0.3] 'cause they do get that in some of the specialties [1.
4] what did you enjoy about final year community based medicine [0.2] well they 
enjoyed the one to one tuition [0.3] which i thought was very very reassuring 
[0.6] they found the communication skills role plays very useful [0.3] er th-, 
they found the the words they used were both useful and informative so that 
that was good feedback for us [0.9] and also that the voices exam was relaxed 
and fair [0.2] which again for students to volunteer that something about an 
exam i think is a very positive step [1.7] then the final part [0.9] er [0.7] 
is what i call the Heineken question [0.6] [laughter] and [0.2] this was [0.3] 
what does final year community based 
medicine give you [0.2] [laughter] that other settings do not so what's 
community based medicine reaching out to [0.3] that other teaching settings [0.
2] simply aren't getting to [0.4] so we'll just focus on this one a little bit 
more 'cause i think this is the most exciting of them [1.2] first of all er 
they felt that this was the f-, you know their real opportunity to have 
clinical autonomy and responsibility more than in other settings [0.8] holding 
your own consultations they they they seemed to feel that in hospital settings 
[0.3] you don't get to hold your own consultations you go in you you take the 
whole medical history you present it you don't have anything to do with it [0.
4] within the primary care community based setting that was very different [0.
9] it was less focused on the rare they were fed up of seeing ridiculous [0.4] 
er outside er things in hospital whereas they're actually seeing normal things 
happening in community based [0.3] setting [0.8] and also they 
were being treated as a colleague rather than as a student so they were saying 
that they could develop this more professional relationship with those around 
them [1.9] they were able to to see and experience er continuity of care 
something they weren't getting in the secondary care setting [0.6] they were 
improving their confidence in their patient skills [0.5] and i think just an 
important thing for some of us here [0.2] is that they were had more 
enthusiastic teachers and support from their mentors [0.3] which again i think 
it's a great thing to hear coming out of out of this setting [1.8] so overall 
as a technique w-, you know wha-, what do i think about this technique well i 
think it's i think it's a a w-, a wonderful technique [0.5] er it's cheap i 
mean the the whole thing other than supplying s-, er food for the students 
'cause it does take [0.2] a few hours [0.3] was was you know the the manpower 
of me [0.5] er it was easy to run it was just simply setting the questions the 
students 
went a-, went away made their own data [0.3] minimum input from me [0.6] and it 
so it was reasonably quick the students in their feedback to me did suggest 
that it did take quite a while which was three hours [0.6] i mean i suppose in 
student terms three hours is a very long time [1.0] er [0.5] it's a student 
centred way they generated this data my only input was to create the three 
things [0.2] that they could then [0.3] spill out and talk about whatever they 
wanted to about [0.7] and also they created a large amount of data and that's 
to say that [0.4] i know the twenty-one statements are the most important they 
had the highest votes [0.4] but they were [0.3] the er you know a hundred or so 
other items [0.2] that were still interesting that were coming out of this [1.
0] and as the study has has suggested [0.5] the s-, the opinions of the 
students there and then were representative of the year as a whole [0.4] 
and that's just to say that's Birmingham and that was that final year [0.3] i 
don't know if it would ever [0.2] er sort of leak out in to be applicable to 
other settings [1.5] as far as community based medicine is concerned at 
Birmingham it's thought of highly by the students who have participated in the 
course [0.7] it allows personal and professional growth [0.4] and something 
that i that i felt was you know very nice was that it does bridge this gap it 
seems [0.3] from being a student to becoming a doctor [0.4] er and it seems 
that er you're allowed and able to do that in the primary care setting more so 
than other places [1.3] so the future of this i think i-, it can go anywhere 
and everywhere for those people who wish to use it [0.4] er we've already used 
it evaluating our firm one scheme in in the year four [0.3] and we're using it 
soon to look at the communication needs for overseas doctors [0.9] as to where 
it's 
already been for me and [0.4] it's already been disseminated partially within 
the university and it's going to be [0.4] er sent off to the curriculum 
development and implementation group [0.3] along with quality assurance as well 
[0.3] it's been presented at the R-C-G-P research symposium in Birmingham [0.6] 
and it's been accepted as a short communication for AMEE in Bern this year [0.
4] and it's also er sort of [0.3] in the pipeline for publication 
nm0301: i've got one [1.5] hi [0.2] [laughter] how are you [1.0] er [0.2] i was 
just won-, [0.3] what went through my mind er as you as you were talking was 
it's a great way of collecting a majortiy view [0.4] of ar-, of areas er [0.3] 
of improvement things that need to be need to be dealt with [0.6] i was [0.2] i 
was wondering what happens to the other vast amount of data that gets lost [0.
4] during the deletion process because it seems that's where you might get [0.
2] slightly more interesting off the wall [0.3] responses to the programme [0.
3] 
nm0295: 
i mean absolutely i mean that's why i mentioned that the fact that you do get 
this great yield of data [0.2] er [0.4] and as yet we haven't done anything 
with the other data that's there [0.5] er i'm not i don't have the numbers with 
me but the votes that were allocated to [0.4] er these particular [0.2] things 
that made it into the top twenty-one [0.2] were head and shoulders way above 
absolutely everything else [0.4] that the that the the highest one i think was 
scoring seventy to eighty votes [0.4] and then suddenly you get down into the 
mid-twenties and it will drop off into just being three or four votes in total 
for all the other items [0.8] er and it's interesting looking at the data that 
came out of there [0.3] there was some [0.3] er r-, really o-, odd stuff that 
came out which i which i felt was very good because it does give everyone a 
voice and i think one of the things said [0.4] er tutors need to stop taking 
themselves so seriously [0.5] [laughter] er which is of course is not going to 
get something to the top of the 
vote but it's something that some students did feel was quite important [0.8] 
but w-, i mean you qualitatively you could look at the sorts of things then 
group it together in categories that other things [0.2] other things people 
find useful [0.4] but we've not done anything with it yet [0.3] 
nm0302: are you going to send an e-mail round the department about tutors 
taking themselves [laughter] seriously [0.3] 
nm0295: probably wouldn't be the best idea [laughter] [1.6] 
nm0303: can you just clarify for me from the [1.0] how does the focus group 
decide at three statements i'm a bit confused with the process 
nm0295: okay i mean i-, it didn't directly decide the three statements it 
informed the three statements [0.4] that er er [0.4] i mean i i i had an idea 
as to what i thought might be important within 
the or or [0.3] or what what [0.5] i i wanted to do something that would create 
an open enough question [0.4] that would allow me to to find out what was out 
there what i thought was out there [0.5] but at the same time not be so narrow 
and limited that it didn't give students the breadth and off-, of opportunity 
[0.5] to you know to explore the the er what was going on [0.5] so i one of the 
one of the reasons for doing it was to ensure that [0.3] my questions wouldn't 
[0.3] be too narrow [0.4] and [0.3] the data that came out of the focus group 
[0.3] er on the whole in fact probably all of it [0.2] was represented in one 
way or another within the nominal group [0.5] i just wanted to make sure that 
there wasn't one specific area that we were going to completely miss by asking 
those questions [0.2] 
nm0303: so the three statements you decided the three statements 
nm0295: i d-, 
nm0303: you asked about enjoyment change something [0.2] right 
nm0295: 
that's right [0.6] i mean part of the reason behind that was w-, when i [0.3] 
when i ran the focus group one of the f-, er one of the last things i asked [0.
3] was w-, what what would you change about er the the the C-B-M course [0.3] 
and and and that just had them er running on and on and on and on and it was 
just such a [0.3] a big area to be explored [0.2] that so many ideas came out 
of it that i thought well [0.3] i think that will be an important one to do [0.
4] because from the focus group i can't tell what the important things are here 
[0.4] but in a voting s-, er setting [0.2] then perhaps they will be able to 
narrow it down [1.1] 
nm0303: thank you [1.6] 
nf0304: i just want to ask [0.3] er the three hour session that you had [0.7] 
with the when you did the nominal group was that the whole year or was that [0.
2] a certain selected group [0.2] 
nm0295: that that was the eighteen students that agreed to participate 
nf0304: i was going to say do you think you've got some responder bias from 
eighteen people who voluteered to come along 
nm0295: 
i mean [0.3] certainly yes [0.2] because th-, obv-, the-, these are people who 
are motivated to come along and do this sort of thing [0.6] and indeed they may 
well have generated data that to them was important and potentially biased but 
again it did achieve fairly good support from the year as a whole [0.4] that's 
not to say that other things would all would would not have [0.3] have obtained 
similar r-, [0.2] er [0.3] er overall agreement [0.8] but i mean i-, w-, it was 
difficult to find any other way to to reach these people [1.6] yep 
nm0305: mine was on the same lines actually i was wondering how you chose your 
eighteen students and whether you think it would have made a difference if you 
had a different eighteen students [0.3] 
nm0295: er [0.7] it would be really interesting to find out if it had if it 
would have actually have been different [0.4] er [0.3] er for the for the 
choosing we used e-mail to recruit them because [0.2] all of the students at 
Birmingham probably i'm sure it's true of all universities have an e-mail 
address allocated 
to them when they join the first year [0.8] and er the response rate overall 
wasn't too bad to the initial e-mail the people who wanted [0.3] to actually 
take part was much smaller [0.8] er [0.3] but i think the response rates to the 
the e-mails [0.2] after badgering on e-mail was seventy-odd per cent [0.6] so i 
mean we thought it was a reasonably good way of recruiting people because it 
was free [0.7] you know there was no postage no printing [2.4] 
nm0306: er [0.7] if you'd sat down and wrote a twenty-one item questionnaire 
with three headings yourself how [0.3] different would it have been [0.7] from 
what the students came up with [2.2] 
nm0295: that's a good question [0.4] er [5.2] i honestly don't know [0.3] 
nm0306: right 
nm0295: er [0.4] i mean i i'm sure that that [0.3] the faculty and students 
alike would create [0.2] similar sorts of statements [0.6] er [0.2] in part but 
i i still think there are things that they brought up that that 
have been of surprise to us [0.5] i suppose we could have put them in [0.4] er 
and be surprised by the results but i was i think [0.3] part of it was a 
surprise that they came up with them in the first place [1.0] 
nm0306: i mean er the reason i ask is that we use the [0.4] the traditional 
technique of writing a questionnaire from our [0.3] obviously [0.2] hopefully 
good knowledge of what our course entails and then [0.4] offering free text 
comments to to harvest the idiosyncratic or different [0.7] er [0.2] views of 
students [0.6] and we're reasonably comfortable with that it gives us reliable 
results i mean [0.4] if if i was to descrip-, with a [0.2] group of Clinical 
Methods students at Leicester i'm absolutely certain one of the items they 
would come up with is [0.4] er [0.7] student support to travel to practices [0.
2] yet [0.2] i would never ask [0.2] a question like travel to practices 'cause 
it's quite with-, without our control we have to use the practices we have [0.
5] so the students will generate items [0.3] of [0.3] and data about [0.2] 
something that i can't change where of course 
nm0295: 
yeah [0.2] 
nm0306: i wouldn't put that in my [0.3] own questionnaire [0.9] 
nm0295: i i suppose our rationale behind it was [0.3] because er [0.5] you 
can't do anything about it we were still keen to know whether or not the 
students felt it was important [0.3] and i-, and indeed they did come up with 
that that they wanted to have [0.4] i think they wanted to have closer 
practices and they wanted to have better er er monetary reimbursements [0.4] 
but they got voted out [0.4] th-, there were things that were more important to 
the student that came above it [1.8] 
nm0307: were there actually any incentives for the students to turn up and do 
your focus group [0.4] 
nm0295: er the the first focus group er [0.3] there were no incentives for that 
[0.2] er but for the the nominal group the only incentive was food [0.4] 
nm0307: right [0.5] [laughter] 'cause it does [laughter] 
make a difference to what group of students will [0.6] turn up 
nm0295: yeah 
nm0307: 'cause i always went when i was a student [laughter] if there was 
something like ten pound in an envelope or something like there was just er out 
of interest [0.3] sort of whi-, which bias group you get the okay 
nm0295: yeah [0.2] i mean it was it was quite strange because i i did 
evaluation with the group at the end and i said you know w-, what did you enjoy 
what didn't you enjoy about this technique [0.4] and even though i'd spent a 
fortune on food hot pizzas [0.3] [laughter] and i'd s-, supplied them all 
evening [0.3] several people commented we would like to have had more food [0.
4] [laughter] er several said [0.3] that they would like to have beer [0.4] 
[laughter] and have i i can imagine the responses i would have had if i'd given 
them beer [laughter] 
nm0307: 
different probably [0.2] 
nm0295: yes [laughter] [1.2] 
nm0308: do you think with the er [0.6] computer now and Internet is it possible 
to do a similar thing [0.4] but [0.4] through voting in the Internet [0.4] 
nm0295: the-, there is 
nm0308: then you might get more people involved [0.2] 
nm0295: er i mean that er a sort of an allied technique to the nominal group 
technique is the Delphi technique [0.4] which is aim-, th-, th-, this th-, this 
is different 'cause this is face to face [0.4] whereas the Delphi technique 
involves circulating a questionnaire [0.4] er ei-, either by mail or 
electronically [0.3] and then collating the results formulating a new 
questionnaire sending it out and refining and and recycling [0.5] it's 
something that's that's potentially in the pipeline for me because [0.2] i i 
mean i i agree with you i think with the the the the growth of electronic use 
[0.3] then it would be a very g-, easy and a 
very cheap way of doing that sort of research [5.1] 
nm0296: namex thanks very much and congratulations on your [0.2] Master's 
degree [0.8] 
nm0295: thank you [5.8] 
nm0296: it's very exciting what the academic [0.2] training schemes in general 
practice [0.4] have [0.3] er or the individuals like namex who've been involved 
have have generated as as research projects [0.3] for those of you who don't 
know there are academic training schemes in general practice based here at 
namex [0.4] at namex and namex and i suspect there's one at [0.2] namex as well 
i'm not sure [0.6] er 
nm0301: er there isn't any more [0.3] 
nm0296: th-, there isn't 
nm0301: no [0.4] 
nm0296: but but within the namex there are [0.3] there are the three [0.4] and 
there's an 
opportunity if you're a G-P registrar to go on and do an extra six or twelve 
months working in [0.3] at a university based at er [0.3] a primary care [0.2] 
central general practice centre [0.4] and being supervised to undertake such 
research and it's really great that namex [0.5] is going on from strength to 
strength and is now [0.3] taking his skills forward now as a [0.4] as a 
lecturer in communication skills at namex [0.9] er [0.3] we've created [0.2] 
long breaks today [0.3] er particularly at coffee time and lunchtime because [0.
2] it's a wonderful opportunity to share ideas [0.4] and experiences and 
hopefully the talks will generate a lot of discussion [0.5] and [0.2] one of 
the really good things about today is that we've brought together colleagues 
not just from namex [0.5] but er also [0.3] namex and and namex and we're you 
know obviously very grateful that you [0.5] that you have come today