|
COMENIUS
Evaluation of Educational Outcomes Tool
for Quality Management
Within the Quality Management plans of each school, the
project aimed at comparing and evaluating the impact of delivery of
curriculum and assessment modes on school leavers.
The project considered students’, parents’, and
teachers’ expectations at the start of their final year in five
European schools:
-
Escola
Secundária de Silves, (Silves, Portugal)
-
ITC
“A. Volta »
(Florence, Italy)
-
Liiketalousopisto
Helsinki-Malmi (Helsinki, Finland)
-
Städtische
Riemerschmid-Wirtschaftsschule (Munich, Germany)
-
St
Tiernan’s Community School (Dublin, Ireland)
From the year 2006 to 2009 final year students
shared their expectations and
benchmarked their experiences
with their European counterparts. The project also examined
the expectations of the wider community on the capacities of schools
to meet the needs of the business world. Having completed their
secondary education, the students evaluated the effectiveness of
their educational experience in preparing them to meet the
challenges of further studies or the working world. The schools
tracked the students’ progress one year after they had left school
and then two years after they had left school. By comparing the best
practices in each of the schools, the project assisted in
determining future improvement actions and enhanced the quality of
educational management and the experiences of students
The website, http://www.itcvolta.it/comenius/, was created to implement the above process.
The website contains an
online database which allowed the schools to insert common
questionnaires for students, former students, parents, teachers and
the business world. The
answers to the questionnaires are visible online and can be compared
and analysed through the use of graphs. Furthermore, the website
allows the schools to store the email addresses of all students so
that they can be contacted
also in future years.
Regarding Students
Expectations we can say that in general terms no school
presented a meaningful number of negative answers. Concentrating on
the three key questions:
2. How well do you think school prepared you to develop
your potential?
19. How well did school prepare you to take decisions
about your future (choice of university, work or vocational
training)?
20. How well did school prepare you to develop job
seeking skills (C.V., interview skills, application forms and
letters)?
We notice that in many of these questions the
percentage of negative answers was very low. Regarding question 2,
there is a very high percentage of excellent
and good answers (more than
50%), with St. Tiernan’s with more than 85% of answers above good.
Regarding question 2, also here the responses are
positive, with three schools (St. Tiernan’s, Silves, RWS) with
more than 50% of answers excellent and or good.
Concerning question 20, St. Tiernan’s, RWS and Malmi
have answers excellent and or good
(RWS with 73.68%). Only Volta presents a quarter of negative answers
for question n. 20.
From the answers obtained, we can infer that the
students’ expectations were not letdown. The answers analyzed are
on the whole positive and the negative answers (poor)
are residual and generally inferior to 5%.
This survey becomes more meaningful and accurate when
investigating the opinions of the students once they
left school (follow up). In order to do so, the former
students were asked to fill in online questionnaires One
Year after leaving school and Two
Years after leaving school. It is worth mentioning that both
Malmi and RWS are Vocational schools where the students are
mainly prepared for the world of work.
The other three schools offer a general secondary school
education to students and this influences the results of the survey.
Of the ten questions answered by the students, two have
been examined in depth.
1.
What
are you doing now?
2.
How
do you rate the usefulness of your school learning in respect to
your present situation/occupation?
Regarding the first question, we can consider three distinct groups
of behaviour: the schools where the majority of students continue
their studies (St. Tiernan’s, Volta, RWS), Malmi where 72.2% of
the students work and Silves where a high percentage of students are
unemployed (38,1%) and only 42.86% continue their studies. Regarding
the second question, there is a notable difference among the
schools, even if over 70% of the answers are positive and, in total,
more than 70% are
excellent and or good.
The less positive fact is that 28.75% of the former students of
Silves considered of scarce importance the knowledge acquired in
respect to their occupation in the One Year after leaving school questionnaire. On the other hand, for
Volta and RWS the increase of positive answers of the following year
shows a change of
attitude by students towards the importance of their school
learning.
It
is interesting to compare each category of answers ( excellent, good, moderate, poor) one year after and two years after
having left school.
Concerning
the students who continued their studies, St. Tiernan’s revealed
the highest rating at the end of the two years, while Volta
presented a slight regression which is
irrelevant considering the high number of students’
responses; Silves
maintained the same percentage, while RWS and Malmi evolved
positively. The three schools, Silves, RWS and St. Tiernan’s saw a
decrease in the number of employed students, while Volta and Malmi
saw an increase in these numbers.
The
situation regarding unemployment is notoriously unfavourable for
Silves as the readings increase significantly. In all the other
schools the figures diminish.
In
respect to the Questionnaire for Businesses there is a decrease in
the figures even if this detail is not very representative, only
Malmi and Silves have readings over 10% in the 1st year.
2.
How well do you think our school prepares students in
technical skills for being successful for your company
Here we do not have a unique pattern of behaviour. We
can only state that the satisfaction of the work placements is more
visible in Volta and RWS due to the higher number of excellent
and good answers. Malmi and
Silves increase significantly the number of
poor answers. For
Silves, this increase could coincide with the increase of the number
of unemployed within the examined group.
1.
How well do you think our school prepares students for
being an appropriate candidate for your company in all aspects?
Generally, the evaluation of the work placements is
quite positive, with 57.14% of Malmi’s students obtaining excellent and Volta
obtaining very positive answers in relation to the high number of
employers who answered the questionnaire. The negative answers are
rather residual.
Escola Secundária de Silves
Secondary
School of Silves is one of the oldest schools of the south of
Portugal. In fact it’s presence goes back to 1930 then called
Commercial and Industrial School “João de Deus”. Its foundation
is undoubtedly linked to the rising of the
cork industry and its collateral effects on the local economy. The
expansion was sudden and the need of new facilities came quick. In
1959 a new school was inaugurated and the set of buildings are the
ones still in use today.
Although
it’s vocation and conceptuality has mainly been technical –
people still call it escola técnica
– due to changes occurred in educational policy after the
revolutionary period of 1974 it gradually
become a grammar school.
In
recent years again new policies caused a serious impact on the
vocation of this school. New adjustments have been made and the
technical strand returned back to the Secundária
de Silves. A new impulse on the, so called,
professional/vocational studies has been implemented and the school
seemed to gain back its natural course.
The
work now finished took place within the triennium 2007/2009 and was
the first done by this school that overlooked the follow up of a
group of students, the expectations of the parents, impressions of
the local business and teachers.
Probably
the best goals were achieved in the field of school administration
due to the strong involvement of every school’s management team
and the amount of knowledge acquired on the matter on every project
meeting.
Nevertheless
the project itself brought a good deal of information of each school
and gave us the chance of comparing data which showed us that there
was no big difference between all the institutions.
In
terms of benefits attained by this school we can always speak about
changes carried on or to be done in a near future due to this
project and consequent relationship. Examples of how we act when
facing management problems where pointed down and adjustments are
certainly going to be carried out. The exchange of ideas is a must
in this type of partnership.
We
will start our analysis from the parents questionnaire. The used
sample showed us that the parents have high expectations on their
sons’ and daughters’ education, 85% of them hopes their children
get in the university. It is curious to observe that the majority of
students also expect to get in the university. In what concerns to
performance the prospects are also very positive (75% good or
excellent).
The
students showed a high confidence in the education proposed or
offered in Secundária de
Silves. On the expectations questionnaire among the 22
questions, 82% of them presented answers above good. Less good
results were obtained on foreign languages, It and on the way
prepares students to make decisions. Thus here there is margin for
improvement and our management certainly will take note of this.
The
questionnaires submitted to the same students one and two years
after leaving this school were an excellent completion for the
expectation survey. The
general conclusion is that the answers where then less enthusiastic
than before. Although positive the response of the students showed,
for example, that only 43% went to the university.
An
important information was taken when comparing the answers of the
first year after with the two years after. These showed clearly that
the reliability on the acquired knowledge at school was improving
from the first to the second year.
A
sad conclusion came up when we realised that a increasing number of
students where getting unemployed. This could be a cause of their
negative answers regarding the consistency of the given competences.
The
results from the business world come up to be very satisfactory with
88.9% of the answers revealing that our students are considered to
be good or excellent.
In
general terms we all agree in Secundária
de Silves that the results of this project were positive and
practically all the targets were fulfilled.
The
sample used was sometimes small for the purpose and range of the
questionnaires however the main trends of these school clients were
evaluated. Also the follow up done was pioneer in this school and
was well thought-out by
the school’s council, meaning that is going to be classified as
good practice and will in fact be adopted as normal evaluation
procedure.
ITC “A. Volta »
The
main result of the project for ITC “A. Volta” was the
high number of responses to the questionnaires which confirmed the
reliability and effectiveness of the project itself.
The website, www.itcvolta.it/comenius,
allowed the administration of the numerous questionnaires to all
school leavers, former students, the business world, parents and
teachers. Furthermore, it was possible to have an immediate feedback
as the answers could be visualised in graph form, in real time.
Another effective function of the website was the possibility of
storing the email addresses of the students and of contacting them
directly from the website by email. This function facilitated the
communication between the school and school leavers even
after a period of over two years.
From
2006 to 2009 all the students in their final year were asked to sign
up on the website and to
fill in the “Expectations” questionnaire. Over 120
students each year answered this questionnaire. One year after
leaving school and two years after leaving school, the same students
were contacted by email and asked to answer the relative online
questionnaires. About 50% of the school leavers responded to the
request to fill in the online survey: “One Year After leaving
school” . About 25% of the same students answered the “Two Years
After Leaving School” survey.
The
answers to the questionnaires evidenced, first of all, the good
final examination results: the outcomes were all positive, varying
from excellent to satisfactory. Secondly the data confirmed the
Italian statistics regarding the high number of drop outs from
University after the first year of studies. The majority of Volta
school leavers (66%) enrolled at University (answer. 1 of the
Questionnaire After
Leaving School) but the percentage of students who continued
their university career dropped to 10% the following year. This
fact, together with the answers to questions 19
(How
well did school prepare you to make decisions about your future
(choice of university, work or vocational training) and
20 (How
well did school prepare you to develop job seeking skills (C.V.,
interview skills, application forms)
of the Expectation questionnaire point out the need for a the comprehensive
Guidance and Counselling service for all students
The
questions related to the delivery of the curriculum on the part of
the school showed that the students were in general satisfied,
particularly in the study of Modern Languages and in the area
of Cooperative Skills. On the other hand, a relatively high number
of students (28%) rated unsatisfactory their learning in scientific
subjects. Also this fact, should be taken into account when
determining future improvement actions.
The
answers to the questionnaires by businesses and companies, where
Volta students carry out their work experience, were extremely
positive. First of all a very high number of businesses answered
the questionnaire, secondly all the answers regarding how
well the school prepared the students for the business world
received answers from excellent to good.
The
Comenius project was also very useful for the school’s Quality
Management plans. The data collected on the website was used both in
the two systems of Quality certification, EFQM and ISO that the
school has been involved in.
St
Tiernan’s Community School
At
St. Tiernan’s Community School, the results of the Comenius
Project on Evaluating Educational Outcomes was welcomed.
The responses from students, parents, employers and teachers were
all very positive; it is gratifying to know that the hard work of
the school is appreciated by all the educational partners.
In
both years, 2007 and 2008, students had high expectations; more than
three quarters of those surveyed expected to perform “excellent”
or “good” in their final examinations. The results of both
Leaning Cert and Leaving Cert Applied confirmed that their
expectations were justified.
85% of students in 2007 and 93% of students in 2008 felt that the
school did an “excellent” or “good” job in preparing
students to develop their potential.
In 2007 82% of students(and 67% in 2008) stated that the school
performed “excellent” or “good” in preparing students to
make decisions about their future, while 73% believed that the
school did an “excellent” or “good” job in preparing
students to develop their job seeking skills.
This
trend was confirmed in the 2007 survey of employers; 77% of the
employers surveyed responded that St. Tiernan’s was either
“excellent” or “good” at preparing students to work in their
companies.
Parents were also very positive about the work of the school; 86%
say that the school did an “excellent” or “good” job at
preparing their sons and daughters to develop their potential
The very positive responses of the Tiernan students (to
questions 2,19 and 20 across both years) may reflect the fact that
Tiernans is the only school in the project which offers students a
comprehensive Guidance and Counselling service.
Some elements of the service which may be influential are:-
Formalised Career Guidance for all senior students on a one-to-one
and class basis delivered by a qualified guidance counselor.
A mentoring system for senior students delivered in conjunction with
Microsoft Ireland.
A Skills@Work Programme delivered in conjunction with Microsoft
Ireland.
The on-going work of the school care team (Deputy Principal,
Guidance Counsellor, Home School Liaison Officer, School Completion
Officer and the Chaplain) which tries toidentify and help students
who may be at risk.
The school’s Pastoral Care programme.
Städtische
Riemerschmid-Wirtschaftsschule, München, Germany
This
project is for the RWS one very important module for QSE because it
provides the school with a feedback of its competitive ability. In
Vocational education of Bavaria the RWS is an optional school and
not a compulsory one.
The RWS
actually is a kind of pre-professional school leading to the
O-level. And in this period of education there is a large offer of
competing schools which are in concurrence with us.
The
long-term evaluation shows the parents and businesses the efficiency
of our school, its a good confirmation for our work.
Otherwise
there wouldn’t be enough demand for our school which would mean -
in worst case - we would have to close our school.
From the
point of view of the teachers we can be sure that we indeed meet the
demands of our students in terms of preparing them well for the
working world. The figures of the results are very important for us
teachers because the feedback of the teachers rather give evidence
of a lack of self-confidence in the work we do.
The work
within our international team was very rewarding as we’ve learnt a
lot from each other not only by analysing and benchmarking our
educational work in many respects but also in getting to know each
part- taking member with its cultural background.
An
important condition to succeed in a long-term international project
is to meet the partners
with great respect besides the expectations in this particular
project.
A common
project with different countries connects people in a way that you
first meet for a project’s purpose. The longer and the closer you
work together, the more you get to know your European colleagues in
person that in the end a certain respect
for one another develops into a deep confidence in each other
and friendship which indeed is very motivating
to continue a project or to start a new one.
Comments
from Business College Helsinki-Malmi
In
Business College Helsinki-Malmi we have been working with our
quality management tools for quite a many years. The benchmarking of
other schools is very important in quality management processes. In
this project our interest was to get to know and compare schools’
practices and to find out what different educational partners think
about these practices. At same time the expectations were high on
learning new practices from our new foreign partners.
Findings
from the questionnaires
Q1 (In
relation to your final year results, how do you expect to perform?)
In 2007
and 2008 over 80% students of Malmi expected to perform either
excellent or good. Comparison to RWS is remarkable with over 40%
difference. These answers seem to reflect the attitude or
self-confidence of students while they are studying. It is also
interesting to notice that parents in all schools had more positive
answers than students.
Q2 (How
well do you think school prepared you to develop your potential?)
The
students of Malmi, RWS, Silves and Volta have answered that these
schools have prepared them mainly either good or moderately.
Tiernans seem to have different methods or guidance as in 2007 over
80% of students thought that school has prepared them either
excellently or good. After graduating students of RWS and Volta see
their education in the positively new light, as the number of
excellent or good answers have increased significantly. In Malmi and
Silves the opinion has turned vice versa to be more negative toward
usefulness of school learning. However, it should be noted that the
number of answers is very small one and two years after graduating.
Q20 (How
well did school prepare you to develop job seeking skills?)
There is
a big difference between schools in how school prepares job seeking
skills. Malmi, RWS and Tiernans seem to put effort on that while in
Silves and Volta the focus of studies is in other directions.
Businesses
in all countries seem to be very satisfied with student’s skills
and they think that almost all students are appropriate candidates
for their companies. From our perspective, this may be due to our
continuous and intensive relationships with enterprises. Especially
the on-the-job training process helps us to keep in touch with the
Business sector.
Teachers
in all schools seem to think that school could support more students
in forming their personal skills. Also graduates agree with that in
Malmi and Volta.
The
project has been very interesting and demanding at the same time.
The differences between our schools and cultures have created some
difficulties to compare the data. However at the same time this
opportunity to be able to compare your own school data to foreign
school is extremely interesting. The analysis of data would have
been practically impossible without the project meetings. The
understanding of practices of different schools deepened in many
formal and especially informal discussions
In our
opinion most of the aims of the project were fulfilled.
We got lot of information about the attitudes and perceptions
of our students, their parents, the business sector and our
teachers. During the various discussions and e-mail exchanges we
have been able to understand, what are reasons behind the answers of
different schools. Concerning the reliability of the questionnaire
study, it is important to point that the results are suggestive. For
example, the amount of answers was in some cases quite small in
proportion to total target group. Although it is not mentioned in
data analysis we want to stress the meaning of qualitative methods,
in this case the constructive discussions between teachers and
teachers, and teachers and students of different countries.
This
project has not only given us new interesting information. It has
also supported us to new developments in our school practices. From
our point of view one of the main findings of the project is the
meaning of excellent and individual supporting student counseling.
We are starting a new curriculum in August 2009 and in new
curriculum we have increased significantly the hours for personal
student counselling. The Irish school St Tiernans has had a major
impact on that change. Also
we have started other international projects with partners from this
project. These are only few examples of the developments this
project has started.
In
addition to many development ideas, this project has created many
deep friendships between teachers and students. As the cultural
barriers are now taken away this co-operation should continue,
because now it is easy to focus directly to concrete challenges
without any formal barriers.
Overall Conclusion
The set of data for Student Expectations 2007 is very
encouraging for all the schools involved in the project. The data
shows that students in Malmi, Silves, Tiernans, Volta and RWS
express a high degree of satisfaction with their schools’ efforts
to prepare them to develop their potential; 57%, 69%,85%,57% and 68%
respectively( using figures for Excellent and Good responses to Q. 2
)
In relation to student perceptions of how their schools helped them
to make decisions about the future (question 19 ) the data is not so
positive generally; Tiernan students are very positive in this
regard (82% rate their school’s performance as good or excellent )
Of the other schools only RWS exceed a 60% score .
In relation to student perceptions of how their schools prepared
them to develop their job seeking skills students at both Malmi and
Tiernans are very positive in this regard, recording 69% and 73%
respectively ( i.e. Excellent and Good )
The set of data for Student Expectations 2008 is complete and in
relation to Q. 2 students in all the schools responded very
positively regarding their schools’ efforts to prepare them to
develop their potential with scores ranging from 53% at Volta to 93%
at Tiernans.
There is less positivity in relation to student perception of school
performance in relation to helping students make decisions about the
future; 3 schools record scores of 60% or better (using the
Excellent and Good % ) while 2 schools recorded scores of 41 %.Once
again the students at Tiernans recorded the highest level of
satisfaction ( 82% ).
82% of Malmi students rate their school’s performance in relation
to helping students develop job seeking skills as Excellent or Good;
the figures for Silves, Tiernans, Volta and RWS are 15%, 69%, 29%
and 70% respectively.
We should note that there is significant variation in the responses
between 2007 and 2008; in general the students responded more
positively across all the schools in 2008.
2008 was a very good year!!
In addition to the students surveyed, parents, teachers and
businesses were asked for their opinions as part of the project.
The responses from the businesses surveyed in 2007 were generally
very positive; 4 of the schools were rated “excellent” or
“good” by at least 77% of those surveyed in relation to their
ability to prepare students for the workplace.
The work of the schools is highly regarded by parents in all 5
cases; the ability of the schools to help students develop their
potential is described as either “excellent” or “good” by at
least 52% of parents questioned and in the case of Malmi,
Tiernan’s and RWS the figure exceeds 80%.
Teachers were asked how they rated their respective schools
regarding supports for students in developing personal skills. In
Silves, 54% of teachers rated their school as either “excellent”
or “good” in this regard. The figures for Malmi, Tiernan’s,
Volta and RWS are 60%, 80%, 71% and 72% respectively.
What can we learn from these responses?
The overall conclusion is that all the schools are doing a good job
and are well regarded by all the educational partners consulted as
part of the project. There are of course variations in the rates of
satisfaction.
|