Skip to content.
Sections
Personal tools
You are here: Home » Features » Demonstrator Projects: Putting toolkits through their paces

Demonstrator Projects: Putting toolkits through their paces

Sam Easterby-Smith and Christina Smart
Last modified 15 May, 2006
Published 09 May, 2006
During 2005, JISC commissioned a set of eight demonstrator projects designed to take selected e-Framework for Education and Research Toolkits and integrate them with other systems. In addition to the technical integration work projects were also expected to conduct evaluations to gauge the quality and usefulness of the technology. This article discusses some of the first phase demonstrator projects, their achievements and the issues they faced.

A key part of the e-Learning Programme has been the technical development projects, [1] small development projects to develop web service toolkits. The aim of these toolkits is to provide “re-usable blobs of functionality” - in other words that they do a specific task and that they can use Web Service standards and common data-models to communicate with other systems. These toolkits are intended to be part of the emerging “service orientated architecture” described by the e-Framework for Education and Research [2]. The toolkits are all open source and are written in common programming languages (either Java or C#) to promote their re-use. For example, in the Enterprise domain, the SWEET.net toolkit consists of a custom-built student information database on which are exposed interfaces using the IMS Enterprise WS specification to interface with client applications, examples of which are an intranet-portal for students or a desktop tool for use by the institution’s administrators [3].

To build on the work of the original toolkits and to spread web service expertise through the sector a group of eight toolkit demonstrator projects were funded and began work last spring [4]. These demonstrator projects were designed to take selected phase one toolkits forward and put into practice integrating them with other systems. In addition to the technical integration work projects are also expected to conduct user evaluations to assess the quality and usefulness of the technology. Scott Wilson, one of the architects of the e-Framework for Education and Research explained the rationale behind this approach:

“There is a fat and thin plan. A thin phase with lots of small projects that are doing very particular pieces of work and are very focused. Following that is a fat phase which is a set of demonstrator projects, which use a whole bunch of these toolkits within an MLE in your institution, so they are broad but shallow – where the toolkits are narrow but deep. And these two phases alternate across the lifetime of the programme. The demonstrators identify problems and gaps which then feed into the next phase of toolkit development projects."

Below is a table summarizing the first phase of demonstrator projects.

Demonstrator Project
Lead Organisation
Domain

Toolkits used
 
BEWT
Bodington integrated with Enterprise Webservice Toolkit
[5]

UHI Millennium Institute

Enterprise CETIS Enterprise SDK
DICE
Resource Discovery using D+ [7]

Staffordshire University

Resource Discovery D+ [8]
ELF Search Service Demonstrator [9]

Heriot Watt University

Resource Discovery D+ / MDC [10]
ICT4BIZ [11]

Brockenhurst College

Enterprise Sweet.net [3]
JELFAD
JISC e-Learning Framework Assessment Demonstrator [12]

Kingston University

Assessment, Sequencing, Learning Design APIS [13], SLeD [14], ISIS [15]
Making Tracks
Game based mathematics demonstrator using Simple Sequencing [16]

Hull University

Sequencing and Assessment APIS, ISIS, ASSIS [17]
SLeD-DLD
Demonstrating Learning Design [18]

Open University

Learning Design SLeD
SLiDE
SLeD Integration Demonstrator [19]

Liverpool Hope University College

Learning Design SLeD

Below are some summaries of some of the projects based on interviews and email exchanges with project teams. We asked them why they were interested in using the toolkits and what the projects had achieved.

e-Learning Framework (ELF) Search Demonstrator

The Heriot-Watt ELF Search Demonstrator project took three search tools; the D+ and MDC toolkits and the RDN’s Subject Portal Project and integrated them into two widely used VLE systems; Moodle and WebCT. The idea being that the search tools would provide the VLE systems with new resource discovery abilities, and enable lecturers and tutors to generate reading lists for courses more easily [9].

The D+ toolkit proved relatively easy to integrate with WebCT as the original project team at Edinburgh University had already made some progress in this area. Integrating D+ with Moodle was achieved by using Moodle’s activity interfaces and the SRU (Search/Retrieve via URL) search web service in D+. The team did encounter some problems with the tools in terms of configuration - such as specifying target repositories and search result formats. The team also managed to integrate the MDC toolkit with Moodle. Some problems were encountered in integrating the RDN’s Subject Portal Project with Moodle partly because it doesn’t support the WSRP standard [20].

ICT4BIZ at Brockenhurst College

This project built on the work of the Sweet.NET project developed in the first phase of JISC funding. The demonstrator project adapted the sweet.Net toolkit to enable three colleges to share student data. The colleges are partners in a Centre of Vocational Excellence (CoVE) which provides ‘ICT for Business’ courses for students in the New Forest and Hampshire. The aim of the project was to enable CoVE managers to gain better access to data about student provision, achievement and retention, which was kept on the MIS systems of three different colleges, Brockenhurst College, Farnborough College of Technology and Totton College.

The technical side of the project was successful and further contributed to the development of the SWEET toolkit, producing a test-suite, better extensibility and improved logging and configuration.

However the project did encounter issues in data sharing between partners. While data from one of the partner sites (using unity) was successfully imported using Sweet, the college insisted on anonymising the data making it useless in the context for which it was intended. The other partner college did not provide any data at all. The project team concluded that this project may have been "a step too far in cultural terms" for the colleges involved [11], [21].

MakingTracks

The MakingTracks projects took the ISIS Sequencing engine alongside SCORM Runtime and Content Packaging services and combined these to deliver an innovative games-based tool for learning v-trails, an approach developed by the NRICH project.

The NRICH project at Cambridge University has built a substantial archive of maths resources and the MakingTracks project used simple sequencing to construct a rich trial for learners based on non- inituitive maths problems using some of the NRICH resources. The demonstrator produced a user environment based on a game with a planets theme where learners need to solve problems to move on to the next level or planet. In solving the problems users begin to understand the underlying concepts (figure). The user environment was constructed in Flash and the navigation system used Ajax to dynamically display navigation options available to the learner.

While technically successful with an attractive user interface the project encountered some challenges in terms of complexity for the Simple Sequencing Specification’s model of the world - and is calling into question the boundaries between this and other more workflow-oriented specifications such as Learning Design [17], [23].

MakingTracks interface

The Making Tracks user interface

JELFAD JISC eLearning Framework Assessment Demonstrator

The JELFAD Project aimed to present students with a sequenced set of learning content and associated assessment questions through either a standard web portal or institutional learning management system. Like MakingTracks, the JELFAD project integrated various toolkits (APIS, ISIS, ASAP and Reload) to deliver learning objects with embedded assessments. This learning content was then delivered to Kingston University students via UPortal or Blackboard as part of their revision and retake exams in first and second year Computer Programming courses.

The JELFAD team developed a "JELFAD PLAYER" (along the lines of the ASSIS player) to orchestrate the workflow between several other tools and according to a sequence that follows the IMS Simple Sequencing standard. The learning objects included in the content packages include:

  • "Static" - fixed resources including HTML, DOC, PDF etc
  • "Multiple Choice Questions" - in QTI2 format, rendered and marked using the APIS toolkit.
  • "Fixed response Question" - QTI2 questions generated by the Roboprof service, also rendered and marked using the APIS toolkit
  • "Upload (File) Question" - Assessment questions in the form of programming tasks, where learners submit Java source code to be marked by the ASAP tool's AJM (Automated Java Marker) service.

The project team used the Reload packaging tool to create the basic IMS Content Packages containing standard materials that were enhanced with the IMS simple sequencing rules and the new assessment content. The project also made use of SCORM services, in development at Hull University and Icodeon, in order to un-pack the content packages at runtime and interpret the sequencing rules. So that the system could be used and evaluated with students in a high stakes environment, the project implemented its own Gradebook and Authentication services.

The project was successful in delivering an integrated revision and exam system to students who responded positively to the use of the system, even in a high stakes (examination) environment. However, like the ict4biz project, it encountered some "institutional reluctance" to incorporating the innovative components into the University systems, for fear that they would be disruptive. This meant that the students received their examination workshops via the uPortal system and this necessitated the overhead of developing a JELFAD specific authentication and authorisation system.

The team concluded that "While the reluctance is understandable.....it does illuminate the difficulty of introducing innovative open source systems into increasingly sophisticated institutional IT systems." [13], [24]

SLeD-Demonstrating Learning Design

This project demonstrated the use of the SLED (Service based Learning Design) toolkit/Player and Coppercore Learning Design engine. It worked closely with the SLED2 toolkit project also based at the Open University which integrated QTI 2 functionality into the original SLED toolkit. The team were able to put the SLED toolkit through its paces including improving the ease of installation, and reviewing the Player architecture to enable the incorporation of new services.

A web site with a demonstration system has been developed which includes a repository of Learning Design scenarios of use and key units of learning. These resources are designed to make it easier for other institutions to use the SLED Learning Design Player and have been evaluated by the UNFOLD community [18], [25], [26].

SLeD demonstration system

Figure of web SLeD demonstration system

SLiDe - SLeD Integration Demonstrator

Like the SLeD DLD project the SLiDe project made practical use of the Coppercore Learning Design Engine, the SLeD toolkit/ Learning Design Player and a set of Learning Design enabled content in a second year module on Multimedia Technology. The learning design tools were also integrated into the institution’s various student-record and authentication systems so that learners could access the course via their normal log in.

The project team worked with the course tutor to develop IMS Learning Designs, at level A with some adaptations for level B. Students were asked for their feedback on using the system and were generally positive. The tutor found that using the learning designs approach was useful. “She felt particularly strongly that IMS Learning Design allowed pedagogy to be captured for online learning in a way that couldn’t be done with the institutional VLE”. However, the team concluded that more user friendly Learning Design authoring tools would be needed before widespread take up of the approach could be achieved.

The biggest technical challenge for the project team was one of working with continually evolving software. It was not possible to fall back to older versions of SleD or CopperCore without compromising functionality and stability. In addition some issues emerged when the systems were load tested with a group of students accessing the software at the same time. These issues were fed back to the development team at the Open University which allowed the SLED toolkit to be improved [20], [27].

Conclusions

This first round of demonstrator projects appear to have been successful in terms of trying to put the original toolkits into practice in one way or another, realising the potential of re-use through standards and web services. As anticipated the work has exposed any weaknesses in the original toolkits, and provided the impetus for further development of them.

It has also raised problems with the specifications themselves and solutions which may be pertinent to their future development. It is very important in this respect that the project outputs are made available to feed into the standards process through CETIS and the Special Interest Groups.

Sustainability is emerging as a significant issue for projects and the programme as a whole. The toolkit and demonstrator projects are only funded for six months, developers find other jobs and move on. The challenge is to ensure that expertise built up during a toolkit project is not lost but passed on to new toolkit and demonstrator projects. In the current third phase of toolkit and second phase demonstrator projects [28], teams have been allocated funding to pay original developers to provide consultation making sure that projects can be quicker off the starting block, and don’t need to re-invent any wheels.

References

[1] Framework and Tools strand of the e-Learning Programme.

[2] e-Framework for Education and Research.

[3] SWEET.Net project on the JISC web site.

[4] JISC ITT: e-Learning Framework and Tools Strand: Demonstrator Projects.

[5] BEWT (Bodington integrated with Enterprise Webservice Toolkit) project on the JISC site.

[6] CETIS Enterprise SDK

[7] DICE (demo project for the d+ toolkit) project

[8] D+ (Brokerage for Deep and Distributed e-Learning Resources Discovery ) toolkit.

[9] ELF Search Service Demonstrator project

[10] MDC (Middleware for Distributed Cognition) toolkit

[11] ICT4BIZ (Sweet.Net demonstrator)

[12] JELFAD (JISC eLearning Framework Assessment Demonstrator) project

[13] APIS (Assessment Provision through Interoperable Segments) toolkit

[14] SLED (Service Based Learning Design System ) toolkit

[15] ISIS (Integrating Simple Sequencing) toolkit

[16] Making Tracks Demonstrator project

[17] ASSIS (Assessment and Simple Sequencing Integration Services) project

[18] SLeD-Demonstrating Learning Design

[19] SLiDE SLeD Integration Demonstrator

[20] ELF Search Service Demonstrator blog

[21] ICT4Biz project web site

[22] NRICH project

[23] Making Tracks project web site

[24] JELFAD project web site

[25] SLeD Demonstrator project web site

[26] Patrick McAndrew, Rob Nadolski and Alex Little (2005). Developing an approach for Learning Design Players. Journal of Interactive Media in Education (Advances in Learning Design. Special Issue, eds. Colin Tattersall, Rob Koper), 2005/14. ISSN:1365-893X [http://jime.open.ac.uk/2005/14].

[27] SLiDE project page

[28] Sarah Holyfield Third round of toolkit and second round of toolkit demonstrator projects

 

Supported by JISC Supported by CETIS
Powered by Plone