HOME

Home Page Image



 

 


 



FINAL REPORT

COMENIUS    


Evaluation of Educational Outcomes Tool 
for Quality Management

Within the Quality Management plans of each school, the project aimed at comparing and evaluating the impact of delivery of curriculum and assessment modes on school leavers.  The project considered students’, parents’, and teachers’ expectations at the start of their final year in five European schools:

  • Escola Secundária de Silves, (Silves, Portugal)

  • ITC “A. Volta »  (Florence, Italy)

  • Liiketalousopisto Helsinki-Malmi (Helsinki, Finland)

  • Städtische Riemerschmid-Wirtschaftsschule (Munich, Germany)

  • St Tiernan’s Community School (Dublin, Ireland)

From the year 2006 to 2009 final year students  shared their expectations and  benchmarked their experiences  with their European counterparts. The project also examined the expectations of the wider community on the capacities of schools to meet the needs of the business world. Having completed their secondary education, the students evaluated the effectiveness of their educational experience in preparing them to meet the challenges of further studies or the working world. The schools tracked the students’ progress one year after they had left school and then two years after they had left school. By comparing the best practices in each of the schools, the project assisted in determining future improvement actions and enhanced the quality of educational management and the experiences of students

The website, http://www.itcvolta.it/comenius/, was created to implement the above process.  The website contains an   online database which allowed the schools to insert common questionnaires for students, former students, parents, teachers and the business world.  The answers to the questionnaires are visible online and can be compared and analysed through the use of graphs. Furthermore, the website allows the schools to store the email addresses of all students so that they can be  contacted also in future years.

Regarding Students Expectations we can say that in general terms no school presented a meaningful number of negative answers. Concentrating on the three key questions:

2. How well do you think school prepared you to develop  your potential?

19. How well did school prepare you to take decisions about your future (choice of university, work or vocational training)?

20. How well did school prepare you to develop job seeking skills (C.V., interview skills, application forms and letters)?

We notice that in many of these questions the percentage of negative answers was very low. Regarding question 2, there is a very high percentage of excellent and good answers (more than 50%), with St. Tiernan’s with more than 85% of answers above good.

Regarding question 2, also here the responses are positive, with three schools (St. Tiernan’s, Silves, RWS) with more than 50% of answers excellent and or good.

Concerning question 20, St. Tiernan’s, RWS and Malmi have answers excellent and or good (RWS with 73.68%). Only Volta presents a quarter of negative answers for question n. 20.

From the answers obtained, we can infer that the students’ expectations were not letdown. The answers analyzed are on the whole positive and the negative answers (poor) are residual and generally inferior to 5%.

This survey becomes more meaningful and accurate when investigating the opinions of the students once they  left school (follow up). In order to do so, the former students were asked to fill in online questionnaires One Year after leaving school and Two Years after leaving school. It is worth mentioning that both  Malmi and RWS are Vocational schools where the students are mainly prepared for the world of work.  The other three schools offer a general secondary school education to students and this influences the results of the survey.

Of the ten questions answered by the students, two have been examined in depth.

1.      What are you doing now?

2.      How do you rate the usefulness of your school learning in respect to your present situation/occupation?

Regarding the first question, we can consider three distinct groups of behaviour: the schools where the majority of students continue their studies (St. Tiernan’s, Volta, RWS), Malmi where 72.2% of the students work and Silves where a high percentage of students are unemployed (38,1%) and only 42.86% continue their studies. Regarding the second question, there is a notable difference among the schools, even if over 70% of the answers are positive and, in total, more than 70% are excellent and or good. The less positive fact is that 28.75% of the former students of Silves considered of scarce importance the knowledge acquired in respect to their occupation in the One Year after leaving school questionnaire. On the other hand, for Volta and RWS the increase of positive answers of the following year shows  a change of attitude by students towards the importance of their school learning.

It is interesting to compare each category of answers ( excellent, good, moderate, poor) one year after and two years after having left school.

Concerning the students who continued their studies, St. Tiernan’s revealed the highest rating at the end of the two years, while Volta presented a slight regression which is  irrelevant considering the high number of students’ responses;  Silves maintained the same percentage, while RWS and Malmi evolved positively. The three schools, Silves, RWS and St. Tiernan’s saw a decrease in the number of employed students, while Volta and Malmi saw an increase in these numbers. 

The situation regarding unemployment is notoriously unfavourable for Silves as the readings increase significantly. In all the other schools the figures diminish.

 In respect to the Questionnaire for Businesses there is a decrease in the figures even if this detail is not very representative, only Malmi and Silves have readings over 10% in the 1st year.

2.        How well do you think our school prepares students in technical skills for being successful for your company

Here we do not have a unique pattern of behaviour. We can only state that the satisfaction of the work placements is more visible in Volta and RWS due to the higher number of excellent and good answers. Malmi and Silves increase significantly the number of   poor answers. For Silves, this increase could coincide with the increase of the number of unemployed within the examined group.

1.        How well do you think our school prepares students for being an appropriate candidate for your company in all aspects?

Generally, the evaluation of the work placements is quite positive, with 57.14% of Malmi’s students obtaining excellent and Volta obtaining very positive answers in relation to the high number of employers who answered the questionnaire. The negative answers are rather residual.

Escola Secundária de Silves

Secondary School of Silves is one of the oldest schools of the south of Portugal. In fact it’s presence goes back to 1930 then called Commercial and Industrial School “João de Deus”. Its foundation is undoubtedly linked to the rising of the cork industry and its collateral effects on the local economy. The expansion was sudden and the need of new facilities came quick. In 1959 a new school was inaugurated and the set of buildings are the ones still in use today.

 

Although it’s vocation and conceptuality has mainly been technical – people still call it escola técnica – due to changes occurred in educational policy after the revolutionary period of 1974 it gradually become a grammar school.

In recent years again new policies caused a serious impact on the vocation of this school. New adjustments have been made and the technical strand returned back to the Secundária de Silves. A new impulse on the, so called, professional/vocational studies has been implemented and the school seemed to gain back its natural course.

The work now finished took place within the triennium 2007/2009 and was the first done by this school that overlooked the follow up of a group of students, the expectations of the parents, impressions of the local business and teachers.

Probably the best goals were achieved in the field of school administration due to the strong involvement of every school’s management team and the amount of knowledge acquired on the matter on every project meeting.

Nevertheless the project itself brought a good deal of information of each school and gave us the chance of comparing data which showed us that there was no big difference between all the institutions.

In terms of benefits attained by this school we can always speak about changes carried on or to be done in a near future due to this project and consequent relationship. Examples of how we act when facing management problems where pointed down and adjustments are certainly going to be carried out. The exchange of ideas is a must in this type of partnership.

We will start our analysis from the parents questionnaire. The used sample showed us that the parents have high expectations on their sons’ and daughters’ education, 85% of them hopes their children get in the university. It is curious to observe that the majority of students also expect to get in the university. In what concerns to performance the prospects are also very positive (75% good or excellent).

The students showed a high confidence in the education proposed or offered in Secundária de Silves. On the expectations questionnaire among the 22 questions, 82% of them presented answers above good. Less good results were obtained on foreign languages, It and on the way prepares students to make decisions. Thus here there is margin for improvement and our management certainly will take note of this.

The questionnaires submitted to the same students one and two years after leaving this school were an excellent completion for the expectation survey.  The general conclusion is that the answers where then less enthusiastic than before. Although positive the response of the students showed, for example, that only 43% went to the university.

An important information was taken when comparing the answers of the first year after with the two years after. These showed clearly that the reliability on the acquired knowledge at school was improving from the first to the second year.

A sad conclusion came up when we realised that a increasing number of students where getting unemployed. This could be a cause of their negative answers regarding the consistency of the given competences.

The results from the business world come up to be very satisfactory with 88.9% of the answers revealing that our students are considered to be good or excellent.

In general terms we all agree in Secundária de Silves that the results of this project were positive and practically all the targets were fulfilled.

The sample used was sometimes small for the purpose and range of the questionnaires however the main trends of these school clients were evaluated. Also the follow up done was pioneer in this school and was well thought-out  by the school’s council, meaning that is going to be classified as good practice and will in fact be adopted as normal evaluation procedure.

 
ITC “A. Volta » 

The main result of the project for ITC “A. Volta” was the high number of responses to the questionnaires which confirmed the  reliability and effectiveness of the project itself.  The website, www.itcvolta.it/comenius, allowed the administration of the numerous questionnaires to all school leavers, former students, the business world, parents and teachers. Furthermore, it was possible to have an immediate feedback as the answers could be visualised in graph form, in real time. Another effective function of the website was the possibility of storing the email addresses of the students and of contacting them directly from the website by email. This function facilitated the   communication between the school and school leavers even after a period of over two years.

From 2006 to 2009 all the students in their final year were asked to sign up on the website and to  fill in the “Expectations” questionnaire. Over 120 students each year answered this questionnaire. One year after leaving school and two years after leaving school, the same students were contacted by email and asked to answer the relative online questionnaires. About 50% of the school leavers responded to the request to fill in the online survey: “One Year After leaving school” . About 25% of the same students answered the “Two Years After Leaving School” survey.

The answers to the questionnaires evidenced, first of all, the good final examination results: the outcomes were all positive, varying from excellent to satisfactory. Secondly the data confirmed the Italian statistics regarding the high number of drop outs from University after the first year of studies. The majority of Volta school leavers (66%) enrolled at University (answer. 1 of the Questionnaire After Leaving School) but the percentage of students who continued their university career dropped to 10% the following year. This fact, together with the answers to questions 19 (How well did school prepare you to make decisions about your future (choice of university, work or vocational training) and 20 (How well did school prepare you to develop job seeking skills (C.V., interview skills, application forms) of the Expectation questionnaire point out the need for a the comprehensive Guidance and Counselling service for all students

The questions related to the delivery of the curriculum on the part of the school showed that the students were in general satisfied,  particularly in the study of Modern Languages and in the area of Cooperative Skills. On the other hand, a relatively high number of students (28%) rated unsatisfactory their learning in scientific subjects. Also this fact, should be taken into account when determining future improvement actions.

The answers to the questionnaires by businesses and companies, where Volta students carry out their work experience, were extremely positive. First of all a very high number of businesses answered  the questionnaire, secondly all the answers regarding how well the school prepared the students for the business world received answers from excellent to good.

The Comenius project was also very useful for the school’s Quality Management plans. The data collected on the website was used both in the two systems of Quality certification, EFQM and ISO that the school has been involved in.

 

 St Tiernan’s Community School

At St. Tiernan’s Community School, the results of the Comenius Project on Evaluating Educational Outcomes was welcomed.
The responses from students, parents, employers and teachers were all very positive; it is gratifying to know that the hard work of the school is appreciated by all the educational partners.


I
n both years, 2007 and 2008, students had high expectations; more than three quarters of those surveyed expected to perform “excellent” or “good” in their final examinations. The results of both Leaning Cert and Leaving Cert Applied confirmed that their expectations were justified.

85% of students in 2007 and 93% of students in 2008 felt that the school did an “excellent” or “good” job in preparing students to develop their potential.
In 2007 82% of students(and 67% in 2008) stated that the school performed “excellent” or “good” in preparing students to make decisions about their future, while 73% believed that the school did an “excellent” or “good” job in preparing students to develop their job seeking skills.


This trend was confirmed in the 2007 survey of employers; 77% of the employers surveyed responded that St. Tiernan’s was either “excellent” or “good” at preparing students to work in their companies.

Parents were also very positive about the work of the school; 86% say that the school did an “excellent” or “good” job at preparing their sons and daughters to develop their potential

The very positive responses of the Tiernan students (to questions 2,19 and 20 across both years) may reflect the fact that Tiernans is the only school in the project which offers students a comprehensive Guidance and Counselling service.
Some elements of the service which may be influential are:-
Formalised Career Guidance for all senior students on a one-to-one and class basis delivered by a qualified guidance counselor.
A mentoring system for senior students delivered in conjunction with Microsoft Ireland.
A Skills@Work Programme delivered in conjunction with Microsoft Ireland.
The on-going work of the school care team (Deputy Principal, Guidance Counsellor, Home School Liaison Officer, School Completion Officer and the Chaplain) which tries toidentify and help students who may be at risk.
The school’s Pastoral Care programme.

Städtische Riemerschmid-Wirtschaftsschule, München, Germany

This project is for the RWS one very important module for QSE because it provides the school with a feedback of its competitive ability. In Vocational education of Bavaria the RWS is an optional school and not a compulsory one.

The RWS actually is a kind of pre-professional school leading to the O-level. And in this period of education there is a large offer of competing schools which are in concurrence with us.

The long-term evaluation shows the parents and businesses the efficiency of our school, its a good confirmation for our work.

Otherwise there wouldn’t be enough demand for our school which would mean - in worst case - we would have to close our school.

From the point of view of the teachers we can be sure that we indeed meet the demands of our students in terms of preparing them well for the working world. The figures of the results are very important for us teachers because the feedback of the teachers rather give evidence of a lack of self-confidence in the work we do.

The work within our international team was very rewarding as we’ve learnt a lot from each other not only by analysing and benchmarking our educational work in many respects but also in getting to know each part- taking member with its cultural background.

 An important condition to succeed in a long-term international project is to meet  the partners with great respect besides the expectations in this particular project.

A common project with different countries connects people in a way that you first meet for a project’s purpose. The longer and the closer you work together, the more you get to know your European colleagues in person that in the end a certain respect  for one another develops into a deep confidence in each other and friendship which indeed is very motivating  to continue a project or to start a new one.

 

Comments from Business College Helsinki-Malmi

In Business College Helsinki-Malmi we have been working with our quality management tools for quite a many years. The benchmarking of other schools is very important in quality management processes. In this project our interest was to get to know and compare schools’ practices and to find out what different educational partners think about these practices. At same time the expectations were high on learning new practices from our new foreign partners.

Findings from the questionnaires

Q1 (In relation to your final year results, how do you expect to perform?)

In 2007 and 2008 over 80% students of Malmi expected to perform either excellent or good. Comparison to RWS is remarkable with over 40% difference. These answers seem to reflect the attitude or self-confidence of students while they are studying. It is also interesting to notice that parents in all schools had more positive answers than students.

Q2 (How well do you think school prepared you to develop your potential?)

The students of Malmi, RWS, Silves and Volta have answered that these schools have prepared them mainly either good or moderately. Tiernans seem to have different methods or guidance as in 2007 over 80% of students thought that school has prepared them either excellently or good. After graduating students of RWS and Volta see their education in the positively new light, as the number of excellent or good answers have increased significantly. In Malmi and Silves the opinion has turned vice versa to be more negative toward usefulness of school learning. However, it should be noted that the number of answers is very small one and two years after graduating.

Q20 (How well did school prepare you to develop job seeking skills?)

There is a big difference between schools in how school prepares job seeking skills. Malmi, RWS and Tiernans seem to put effort on that while in Silves and Volta the focus of studies is in other directions.

Businesses in all countries seem to be very satisfied with student’s skills and they think that almost all students are appropriate candidates for their companies. From our perspective, this may be due to our continuous and intensive relationships with enterprises. Especially the on-the-job training process helps us to keep in touch with the Business sector.

Teachers in all schools seem to think that school could support more students in forming their personal skills. Also graduates agree with that in Malmi and Volta.

The project has been very interesting and demanding at the same time. The differences between our schools and cultures have created some difficulties to compare the data. However at the same time this opportunity to be able to compare your own school data to foreign school is extremely interesting. The analysis of data would have been practically impossible without the project meetings. The understanding of practices of different schools deepened in many formal and especially informal discussions

In our opinion most of the aims of the project were fulfilled.  We got lot of information about the attitudes and perceptions of our students, their parents, the business sector and our teachers. During the various discussions and e-mail exchanges we have been able to understand, what are reasons behind the answers of different schools. Concerning the reliability of the questionnaire study, it is important to point that the results are suggestive. For example, the amount of answers was in some cases quite small in proportion to total target group. Although it is not mentioned in data analysis we want to stress the meaning of qualitative methods, in this case the constructive discussions between teachers and teachers, and teachers and students of different countries.

This project has not only given us new interesting information. It has also supported us to new developments in our school practices. From our point of view one of the main findings of the project is the meaning of excellent and individual supporting student counseling. We are starting a new curriculum in August 2009 and in new curriculum we have increased significantly the hours for personal student counselling. The Irish school St Tiernans has had a major impact on that change.  Also we have started other international projects with partners from this project. These are only few examples of the developments this project has started.

In addition to many development ideas, this project has created many deep friendships between teachers and students. As the cultural barriers are now taken away this co-operation should continue, because now it is easy to focus directly to concrete challenges without any formal barriers.

 

Overall Conclusion

The set of data for Student Expectations 2007 is very encouraging for all the schools involved in the project. The data shows that students in Malmi, Silves, Tiernans, Volta and RWS express a high degree of satisfaction with their schools’ efforts to prepare them to develop their potential; 57%, 69%,85%,57% and 68% respectively( using figures for Excellent and Good responses to Q. 2 )
In relation to student perceptions of how their schools helped them to make decisions about the future (question 19 ) the data is not so positive generally; Tiernan students are very positive in this regard (82% rate their school’s performance as good or excellent ) Of the other schools only RWS exceed a 60% score .
In relation to student perceptions of how their schools prepared them to develop their job seeking skills students at both Malmi and Tiernans are very positive in this regard, recording 69% and 73% respectively ( i.e. Excellent and Good )
The set of data for Student Expectations 2008 is complete and in relation to Q. 2 students in all the schools responded very positively regarding their schools’ efforts to prepare them to develop their potential with scores ranging from 53% at Volta to 93% at Tiernans.
There is less positivity in relation to student perception of school performance in relation to helping students make decisions about the future; 3 schools record scores of 60% or better (using the Excellent and Good % ) while 2 schools recorded scores of 41 %.Once again the students at Tiernans recorded the highest level of satisfaction ( 82% ).
82% of Malmi students rate their school’s performance in relation to helping students develop job seeking skills as Excellent or Good; the figures for Silves, Tiernans, Volta and RWS are 15%, 69%, 29% and 70% respectively.
We should note that there is significant variation in the responses between 2007 and 2008; in general the students responded more positively across all the schools in 2008.
2008 was a very good year!!

In addition to the students surveyed, parents, teachers and businesses were asked for their opinions as part of the project.
The responses from the businesses surveyed in 2007 were generally very positive; 4 of the schools were rated “excellent” or “good” by at least 77% of those surveyed in relation to their ability to prepare students for the workplace.
The work of the schools is highly regarded by parents in all 5 cases; the ability of the schools to help students develop their potential is described as either “excellent” or “good” by at least 52% of parents questioned and in the case of Malmi, Tiernan’s and RWS the figure exceeds 80%.
Teachers were asked how they rated their respective schools regarding supports for students in developing personal skills. In Silves, 54% of teachers rated their school as either “excellent” or “good” in this regard. The figures for Malmi, Tiernan’s, Volta and RWS are 60%, 80%, 71% and 72% respectively.

What can we learn from these responses?
The overall conclusion is that all the schools are doing a good job and are well regarded by all the educational partners consulted as part of the project. There are of course variations in the rates of satisfaction.