Skip to content.
Personal tools
You are here: Home » Features » Evaluating the Toolkit and Demonstrator projects

Evaluating the Toolkit and Demonstrator projects

Christina Smart
Last modified 12 Jul, 2007
Published 12 Jul, 2007
Wilbert Kraan and Scott Wilson from JISC CETIS review the 22 JISC toolkit and 12 toolkit demonstrator projects completed to date. One of the findings is that a strong developer community is a vital part of sustaining and building on any individual toolkit or demonstrator project. Another is that these smaller projects were successful in building up open-source development capacity and soa expertise in UK universities and colleges.

When the Frameworks and toolkits strand of the e-Learning Programme began it represented a new venture for the JISC [1]. Instead of funding large three year software development projects the JISC development team decided to fund short six month projects to develop not shrink wrapped applications, but toolkits designed to help developers rapidly produce standards based web services on existing systems. To evaluate the toolkits a number of demonstrator projects were funded in which the toolkits were validated in a different environment than the one they were developed in.

Almost three years on, the JISC asked Scott Wilson and Wilbert Kraan from JISC CETIS to evaluate how the technical developments and the approach of the toolkit and demonstrators programme had evolved and to make recommendations for the future.

Projects were analysed in activity area clusters, noting which projects were reused by subsequent ones. They found that there was a high overall level of reuse – with 50% of toolkit projects being used in subsequent projects. Some clusters were more effective than others, though. Assessment and enterprise toolkits had a particularly high level of reuse.

The report identifies a number of factors linked to successful toolkit and demonstrator projects. These include:

JISC CETIS Special Interest Group engagement: “Where high levels of reuse of toolkits has been observed, this has been strongly correlated with a strong social network of developers operating within the JISC-CETIS SIGs”

Support from original toolkit developers: “A major factor where toolkits have been successfully adopted has been access to, and support from, the original developers”.

Feedback from Demonstrator projects:“Demonstrator projects have had a clear impact on toolkit quality”

Commonly used platforms: In addition, toolkit demonstrator projects benefited from integrating with widely used open source platforms (e.g. Moodle) which ensured that the outputs of the demonstrator were available to a wider community.

The report concludes that the programme of toolkit and toolkit demonstrator projects had a significant impact in several ways, including:

  • Facilitating uptake of e-Learning Standards
  • Sharing and reusing development outputs: 25% of toolkits being reused by 3 or more projects including existing OSS systems.
  • Adding value to the UK e-learning community (at a low cost) by the reuse of toolkits in widely adopted OSS systems like Moodle, Boddington and Sakai.
  • Enabling institutions to undertake more advanced work than would otherwise be feasible in short projects.
  • And crucially establishing e-learning SOA development capacity in the UK.

The full report is available at:

Do you agree with the report? Let us know what you think at


Supported by JISC Supported by CETIS
Powered by Plone