Feb
8
Work-based learning programme diagnostic
February 8, 2011 | Evaluation, Model, Survey | Leave a Comment
The WELL project will develop, implement and evaluate a model of flexible WBL to support courses at every stage of the course cycle (engagement, induction, delivery, assessment, progression, etc.)
The project evaluation workshop on 2nd March will be preceded by a programme diagnostic process (see above). Information will be gathered via semi-structured interviews, and written into a report prior to the workshop.
Objectives of the workshop:
- To test the WELL model against the programmes (distance-learning MBA and PGCHEP) as they stand. To what extent do the programmes follow the recommendations of the model? And does the model take account of the practical implications of actual programme delivery at the course team, School/departmental, and institutional levels.
- To identify areas of programme enhancement, and where elements of the WELL model could be adopted further to support the development of the programme particularly in relation to adoption of technologies.
Process
Three stage process:
1. Diagnostics:
semi-structured interviews, followed by a report so that we can learn about the programmes.
2. Workshop to
(a) test the WELL model against the programmes focussing on the issues surrounding technologies; and
(b) identify areas of enhancement and an outline plan.
3. Write a set of recommendations and plan for enhancement for the programmes based upon critical areas revealed. The methodology for this is set out on the pro-forma.
1. Background info about the course (rationale/purpose, partnership approach?):
- What is the rationale for the distance-based design of the course?
- If and to what extent are partners involved in support for the course?
- How was this agreed?
2. Business case + projections + financial model:
- How was the business case made/prepared?
- What is the main business case? (e.g. market demand, promotion of brand) How do you mitigate against potential risk?
3. SWOT analysis:
- What Strengths, Weaknesses, Opportunities, and Threats can you identify?
- This can be at the School/department or institutional level
4. Potential for increasing the market:
- Is there a potential for increasing the market?
- Could a sector approach be adopted? Could more partners be involved?
- Could this model be transferable to other employers/partners?
- Or has this been developed by/for a specific company?
5. Pedagogies:
- How is learning delivered?
- What pedagogies are used? (e.g. taught content with contextual learning? curriculum design)
6. Impact/benefits
- How will you measure and evaluate stakeholder impact and cost-benefits?
7. Technology-enhanced learning (TEL):
- How is the course utilising technological tools for: communication? delivery? assessment? feedback? anything other purpose?
- How have have these tools enhanced the course processes?
- Why were they adopted over other available options?
- Did the teachers’ or learners’ prior experience influence these decisions?
- Describe any efficiencies gained through the utilisation of these technologies.
8. Further opportunities for TEL:
- Are there any opportunities to enhance the programme through further uses of technology, or more imaginative use of current tools?
Aug
2
Distilling project learning and realising benefits
August 2, 2010 | Culture, Evaluation | 1 Comment
This project has a wide remit; consider the scope of the model as presented through the concept map in previous post. In realising the benefits of the project, we have proposed a project which evaluates and aides an institution’s current progress in embedding lifelong learning and workforce development (LLLWFD) programmes and in providing them with information to make informed plans for future development.
The ‘toolkit’ should work at institutional, faculty and programme levels and would include:
- A defined process/ methodology for using the maturity toolkit (to include working in CAMEL cohorts)
- A range of maturity criteria, level statements and indicators which will work at institutional, local (e.g. faculty/School) level and programme level.
- Guidelines for using the toolkit (including within CAMEL groups).
- Templates to support evidence collection, analysis and reporting
- Sector resources (e.g. papers, other toolkits, best practice guides, links etc.)
The toolkit would focus on areas such as institutional processes and systems, staff professional development, pedagogic approaches and models, delivery models, use of ICT/e-learning, support mechanisms, QA/QE, progression, accreditation, collaborative provision, partnerships with employers, employer consortia, sector bodies & professional bodies, organisational approaches, costing approaches and models, impact/benefits evaluation, compliance, systems accessibility and usability, student engagement etc.
The idea would be for institutions/faculties to use the toolkit to determine how ‘mature’ they are in respect of LLL-WBL, to determine where they would like to be and then to formulate plans, based on evidence collected. Support mechanisms would then follow by way of guidance through the CAMEL cohorts. This can be achieved through a wide range of activities such as face-to-face or online meetings, un-conferences, workshops, telephone interviews, etc. The outputs will underpin capacity building to support a growing community of interest.
May
11
Survey of learners’ experiences with technology
May 11, 2010 | Evaluation, Survey | Leave a Comment
Flexible learning questionnaire – WELL project
Carried out with learners of Certificate in Reablement Support and MA Integrated Emergency Management (pilot programmes for the WELL project)
This questionnaire (WELL-project-small-student-survey) was designed to help us understand more about the learners’:
- familiarity with technologies for learning (Questions 1, 3, 4, 5),
- level of comfort in using technologies for learning, (Questions 2, 6),
- interest in technologies for learning (Questions 7, 8, 9).
- preferred location to study (Question 10),
- preferred time to study (Question 11),
Familiarity with technologies for learning (Question 1, 3, 4, 5)
Certificate in Reablement Support:
Most of the students in the group do not use a computer at university or at home on a daily basis or a few times each week. Usage of mobile phones varies across the group, with none of them using handheld devices and MP3 players.
The learners’ evaluation of their own abilities to use these technologies varies from good to poor.
Learners report some use of discussion forums, but not on a daily or weekly basis. All other forms of communication (blogs, wikis, chat rooms, facebook, etc.) were reported as never or, at best, hardly used. For some learners this was also the case with emails and SMS text messaging.
MA Integrated Emergency Management:
Most of the group report that they have access to computers almost every day or a few times each week. This is also reflected in their usage of mobile phones. With regards to handheld devices and MP3 players, most learners reported to not use them at all.
The learners’ evaluation of their own abilities to use these technologies is generally good.
Learners report some use of discussion forums, but not on a daily or weekly basis. Other forms of communication (wikis, SMS text messaging, news Web pages) were reported as used on a monthly basis or thereabouts. Emails, however, came out on top for this group with 50% of learners using them a few times each week.
Level of comfort in using technologies for learning (Questions 2, 6) and interest in technologies for learning (Questions 7, 8, 9).
Certificate in Reablement Support:
The learners’ feelings with regards to comfort vary from ‘comfortable’ to ‘not at all comfortable’. Most learners report some fear, anxiety, and frustration when using a computer, and others report a certain amount of confidence.
Most learners believe that technologies for learning can be interesting and fun. Also a desire to learn about new technologies is apparent.
MA Integrated Emergency Management:
The majority of the learners have positive reactions towards using technologies for learning. The majority feel ‘very comfortable’ or ‘somewhat comfortable’ and confident using a computer. This group of learners, by and large, see technologies as interesting and useful for the learning experience. They also show some desire to utilise new tools.
Preferred location to study (Question 10) and preferred time to study (Question 11)
Certificate in Reablement Support:
This group appear to do most of their studying at home and at work, between ‘a few times a week’ and ‘a few times a month’. There appears to be no indication of learning at University, a library, or whilst travelling.
This infrequency is also apparent in answer to the question of when they study. Learners report that most of their study time is in the evenings, but this is only ‘between once a month and a few times a month’.
MA Integrated Emergency Management:
This group study more often than the above group, with most study activity carried out at home a few times each week. Some report never to use libraries and to study whilst travelling less than once a month. The same can be said of studying at the University campus.
Studying is carried out throughout the day, with no particular period of the day standing out as more popular. The frequency of ‘a few times each week’ is also reflected in this category.
Final comment
After we gain some insight into learners’ reactions towards using technologies for learning and their familiarity with computer usage, it would be of interest to look at whether these factors, i.e., computer familiarity, comfort with use of technologies, perceived ability to use technologies, anxiety experienced in using technologies, and interest in the use technologies, would have any effect on their performances throughout the durations of the pilot programmes, and achievement in e-assessment. As some learners are quite clearly new to using technologies, and even show anxieties, this can be an ideal context in which to test their use and evaluate their effectiveness, for learners and teachers (a topic of a previous survey and post).
May
7
Critiquing the model
May 7, 2010 | Evaluation, Events, Model | 2 Comments
In the workshop we had on 1st March, we evaluated of a draft generic model of technology-supported work-based learning, in order to support personalised learning, quality enhancement, and flexible delivery.
This draft model is depicted in the below image.
Why do we need a model?
There are many reasons. One of which is that without a model which has institutionl buy-in and acceptance WBL initiatives are merely discrete endeavours without long-term and widespread appeal. A successful model will allow any course to “hang” on it whilst typically being based on the following:
- Process-driven curricula
- Autonomous learning
- Experiential learning
- Evidence-based assessment
- Blended and transdisciplinary approaches
- Flexible delivery approaches
Jan
21
Project impact and stakeholder engagement
January 21, 2010 | Culture, Evaluation, Events | Leave a Comment
The WELL project evaluation workshops on 3rd Novemer 2009 showed that WBL arena can be quite a tough nut for HE to crack. The workshops thrashed out some of the key issues in relation to
“success” for all stakeholders and what constitutes a “successful” WBL project; and
developing a stakeholder engagement strategy for WBL projects.
Notes made at the events are available to read. They will also be incorporated into a final evaluation report.
Dec
17
Informal meeting at Middlesex
December 17, 2009 | Methodology | Leave a Comment
On Wednesday 16th December I travelled down to Middlesex University to find out more about their WBL provisions, and particularly how an institution can ‘roll out’ a model of WBL once it has been piloted as part of a project.
I was subject to the wonderful hospitality of Dr. George Dafoulas (MUSKET project), Dr. Barbara Workman Director of CEWBL), and Yongjun Zheng (MUSKET project) from whom I also gained a great deal of information and advice for the WELL project.
The agenda was fairly informal and open, but I posed the following questions over a smorgasbord of Turkish kebabs:
1. Does the CEWBL serve as an anchor for cultural change across the whole University? Or is it a discrete initiative?
2. What about the processes of University validation, how have their programmes accounted for this?
3. What other processes might require changing in an institution hoping to embed cultural change?
4. In your initial stages, what were your biggest obstacles (especially in dealing with intransigence), if at all?
5. What about cost effectiveness of WBL provisions? And a provision being ‘malleable’ as well as ‘flexible’ (i.e. it grows with the employers, or downsizes accordingly)?
I was in for quite a listening activity!
I am currently completing a ‘literature review’ of WBL theories, pedagogies, and practices and how to develop a criteria for a WBL model; this is an important output of the WELL project. Some of what I discussed with the Middlesex people will inform this literature review but the following comments I decided to blog as well:
Should WBL courses follow a separate accreditation and validation process to an institution’s already ‘traditional’ procedure for doing the same thing? This is key to the ‘responsiveness’ of the WBL model, as such a process can take months to finalise through the University’s procedure.
At Middlesex the IWBL accreditation committee meet once every two months and WBL courses follow a separate process, one that is far more responsive and streamlined.
If WBL is to be embraced as a discrete, transdiscipliary pedagogy, as a field of study rather than a mode of study, then should this also necessitate such a separate process, in order to be more responsive to the demands of the tripatite relationship in WBL (the institution, the employer, and the learner/employee)? There are, of course, other implicated stakeolders such as LLNs and Sector Skills Councils as well.
What about attitudes towards the currency of WBL courses, and their equivalence and comparability vis-à-vis ‘traditional courses?
Middlesex is aware of this, and one of the objectives of MODNet is to integrate the WBL activities into the already existing School/Faculty processes, converting existing courses to the WBL model. Naturally, there is some compromise in doing so but what was once wholly discrete and successful can ensure better ‘buy in’ than moving from zero to sum, bearing in mind that Middlesex is 17 years in this process and has successful case-studies to show.
WBL is a radical approach to the notion of Higher Education, but isn’t this where HE belongs? As, the development of traditional courses took their own life from an endeavour to create intellectual capital from vocational situations (the legal, medical, and teaching professions etc.).
In any case, such an informal trip to learn and connect with other another institution and a similar project is certainly something that can reveal more qualitative information than glorified case-studies in a conference paper, for example.
Thanks Vicki and Peter for suggesting this, and I am grateful to George, Barbara, and Yongjun for their excellent company (and food!).
Nov
13
Learner Survey
November 13, 2009 | Methodology, Survey | Leave a Comment
As part of the JISC WELL project, we are conducting research into the views of the students on the two pilot programmes, Cert. in Reablement Support and MA Integrated Emergency Management. A questionnaire carried out helps us to understand more about students’:
- familiarity with technologies for learning (Questions 1, 3, 4, 5),
- level of comfort in using technologies for learning, (Questions 2, 6),
- interest in technologies for learning (Questions 7, 8, 9).
- preferred location to study (Question 10),
- preferred time to study (Question 11),
Survey Question:
What can we do to enhance your experiences of ‘flexible’ work-based learning?
Flexible learning here refers to learning which is negotiated between an employer, learner (employee), and the University, and is not confined to the University’s campus and/or academic timetable. This can include distance learning courses and part-time courses.
Rationale:
The WELL project aims to develop a model for flexible work-based delivery that can be used on any academic course. The results of the survey will be used to inform the development of such a model, as we aim to find out from you what your preferences are.
Part 1: Flexible Learning Questionnaire
Part 2: Semi-structured interviews for selected learners.
The results will be posted in due course and incorporated into the next version of the Baseline Survey.
Oct
30
Evaluation
October 30, 2009 | Evaluation | 1 Comment
The WELL project has adopted an ongoing approach to evaluation, using a reflective approach that is designed to secure continuous improvement. The process will have a stong emphasis on evaluating the project against its objectives, its operations and its impact on the institution, sector, employers, students and other stakeholders. The WBL arena can be quite tough for HE to engage with: industry, particularly SMEs, different cultures, mutual suspicions, different pressures/ways of working etc.
Part of this evaluation process involves a series of workshops by our External Evaluator, Prof. Peter Chatterton, on:
- Impact Evaluation (3rd November 2009) This workshop will thrash out the issues of “success” for all stakeholders and what constitutes a “successful” WBL project.
- Stakeholder Communications & Engagement (3rd November 2009)This workshop will look at developing a stakeholder engagement strategy for WBL projects.
- Agreeing a Criteria for the WELL Project’s Model (possibly January 2010)
- Reflection (possibly September 2010)
- Review (possibly January 2011)
Further details are available on the JISC WELL website.
This ongoing reflective approach adopted by the project enables lessons to be learned whilst the project is ongoing rather than afterwards, and helps identify practice worthy of further dissemination, allowing judgements to be made on the success of the project ab initio. This is in contrast to a summative “outputs achieved – job done” approach.
The following file is an “Evaluation Map” based on a “why – what – who – when – how” approach to Evaluation:
Oct
9
APEL Assembly
October 9, 2009 | Events, Model | 2 Comments
JISC funded projects favour a collaborative approach, where cross-fertilisation of ideas and sharing of good practice can lead to innovation. Assemblies are one such example of such collaboration. On 8th October we hosted an assembly at the University of Bradford (co-organised by TELSTAR) on the subject of APEL (accredited prior and experiential learning) and how HEIs are tackling this issue. APEL is an important process which formally recognises any prior and/or experiential learning and work experience of lifelong learners, and how this may be formally recognised and credited in an existing HE academic framework.
The event programme and presentations were as follows:
Sarah Malone & Ann Minton (E-APEL, University of Derby)
This workshop provided an overview of the e-APEL tools designed by the University of Derby to introduce learners to the concept of APEL, familiarise them with key academic terminology and support them in the initial stages of preparation of an APEL claim. The session discussed the rationale behind the need for such tools and considers both the technical and practitioner challenges by UK HEIs today.
Rob Stilwell & Harriet Dismore (PINEAPPLE, University of Plymouth)
Rob and Harriet provided an overview of this current project’s progress in developing an APEL system that has a specific focus on University staff. They also discussed their project objectives for the coming year.
Beverly Leeds & Garry George Watkins (TELSTAR, University of Central Lancashire)
Beverly provided an overview of this project’s technology enabled Learning Framework which aims to address the needs of work-based learners and their employers. The framework will include the facilitation of the accreditation of learners’ prior experience, credit recognition of in-house and learning provider courses and negotiated credit awarded learning units.
George Dafoulas (MUSKET, Middlesex University)
The MUSKET project is also concerned with the development of an APEL mapping tool. This will support the mapping of an imported XCRI_CAP description (done by the project’s earlier deliverable in the form of a transform tool) to HE and/or FE courses. A variety of mapping relationships are envisaged and the requirements workshops planned during the project will elicit and refine potential relationships. Currently, there is considerable expertise in the IWBL in accreditation services and we expect to build and “implement” this expertise to support “route planning” for learners in specific work places.
Vikki Illingworth (University of Bradford) & Ian McGregor Brown (Leeds Trinity University College), WYLLN APEL project
WYLLN (the West Yorkshire Lifelong Learning Network) has funded five projects with a view to developing APEL processes in partner institutions. Each project will trial/test and introduce learner focused APEL processes, documentation and information to improve the progression of vocational learners and work-based adults into higher education.
