Skip to content.
Sections
Personal tools
You are here: Home » Features » Large Scale deployment of web services at London Met

Large Scale deployment of web services at London Met

Christina Smart
Last modified 26 Oct, 2005
Published 25 Oct, 2005
There are as yet relatively few examples of web services being used on a large scale in an educational institution. Like other universities the newly formed London Metropolitan University faces the challenge of managing information located across different systems and departments. In response London Met have embarked on a cross-institutional project to integrate information from many of the university MIS systems, such as student records and finance systems using a service oriented approach. We talked to Paul Walk the Enterprise Information Integration (EII) Service Oriented Architecture project manager.

Christina Smart: Can you give us a bit of background about London Met and your team?

Paul Walk: London Metropolitan Univerity was recently formed out of a merger between the University of North London and London Guildhall University, and is now one of the largest unitary universities in the UK. The EII Project Team is an ad hoc project team made up of representatives from two existing teams within the University: the MIS Team which manages some of the major MIS systems, and the Intranet Services Team which is responsible for the management of intranet applications and systems, and which also does the majority of the University's bespoke web services development.

CS: What is the main problem that the Enterprise Information Integration project is addressing?

PW: Primarily this project is about information management. It has been concluded that, while information is one of the University's key assets, this asset is sometimes poorly managed. Specific problems include poor data-quality, duplication, multiple potential sources for information with, at best, a tacit acknowledgement of which source is `authoratitive', and an under-developed `business intelligence' capability.

CS:How will the project address this?

PW:The EII approach is intended to allow us to build an enterprise-wide `meta-model' describing the information `assets' of the University, and to then derive an actual business information model of the real data, with appropriately populated metadata. In addition to this it is intended to develop a `relational model', based on the business model, which will form the basis for a new generation of information processing and business intelligence services.

CS: Which benefits attracted you to a service oriented approach?

PW: SOA offers many benefits in theory - the two main benefits for this project would be:

Firstly, as the information model takes shape, we aim to be able to take a model-driven-development approach to the development of end-user applications. The idea is that users' interaction with information within the University's domain should reflect the model of that information. Exposing information entities, for example, as discrete services makes sense because the service can be tightly coupled to the information model. Application developers can then build `composite' applications by assembling the services they need to use, knowing that some core concerns such as data-sourcing and information relationships have been taken care of at the service layer. Of course this is a slightly idealised aspiration, but our intial developments suggest we can get close enough to this ideal for it to pay off.

Secondly, we need to ensure that we can accommodate the increasing development of web service SOAP being offered by our MIS system vendors. Three of our major systems now have some level of SOAP based API, and all evidence points to this trend continuing. I envisage our `in house' SOA producing web services for consumption by applications, and consuming web services provided by third-party system vendors (or even other organisations within the sector and beyond).

CS:Which institutional systems will be linked through the EII project?

PW:Initially we have focussed on linking some of the large, MIS systems, which are typically composed of a large relational database (usually Oracle) and one or more vendor-supplied client applications.

So far we have successfully linked the following MIS systems:

  • the Student Records system
  • the Finance system
  • the Knowledge/document management system (to identify documents through interrogation of classification metadata)
  • the Smart-card gate-access security system (controls turnstiles at entrances to the campuses)
  • the Room Bookings system (in order to generate academic timetables for students and teaching staff)

Systems we plan to connect include:

  • the Library system (initially for borrower-related information such as loans, fines, reservations etc.)
  • the Enterprise meta-directory (for auditing when users have logged in the network etc.)
  • the Human Resources system

Systems we may connect in the longer term:

  • the Institution's email system
  • the VLE, once we replace WebCT CE 4.1 with something more accessible.

The EII Service Oriented Architecture

An overview of the EII Service Oriented Architecture

CS: The service oriented approach can be realised in a couple of different ways - a set of simple, synchronous point to point services, some message broker or hub-based solution, the enterprise bus and more. Have you chosen one such solution over another, and, if so, why?

PW: We've gone for `keep it simple' in a big way. The research that I have done so far indicates that successful SOA frameworks can often be developed iteratively - starting with simple services and straightforward delivery mechanisms, and gradually introducing more complex components, such as orchestration, asynchronous messaging, service discovery etc. into the framework as needed. Of course, this can sound quite glib - the trick is develop the early iterations in such a way that they can form the foundations for a more heavy-weight framework in the future.

Much of the terminology in current use in discussions about SOA is used in quite a loose way. For instance the term `Web Services' is often used in this context as a term which implies the use of SOAP or XML-RPC to deliver a particular type of service generally (but not always) over HTTP. Without the capital letters, the term `web services' is less specific about any particular technology, but definitely implies the use of the HTTP protocol. The `Service' in `Service Oriented Architecture' does not imply anything about technology at all, much less which protocol should be used.

Having said that, we are developing our SOA framework in Java and the first development iteration will be the creation of `web-services', delivered by Java Servlets which will accept HTTP requests and output HTTP responses in XML format. This first set of services will mostly provide discrete `chunks' of information in response to a query which has been initiated by an end-user web-application. The next development iteration will be to extend the framework to become both a producer and consumer of SOAP-based `Web Services'. We have designed our core Java components in such a way that we can build `Web Services' (SOAP) on top of them - we have done some development to prove that this is possible in practice. Our emphasis at this stage has been on getting the low-level, core components right. To this end, as well as the user query services, we have developed core services to provide functionality such as user authentication/authorisation, logging, caching, connection services, XML transformation services etc.

As to the future, we are adopting a more cautious `wait and see' approach. As the framework grows, we will undoubtedly need to address more complex issues such as orchestration and transaction management.

CS: Are you using open source tools to develop your SOA?

PW: We are using opensource tools for development with the exception of the EII modelling which is being done with a product called MetaMatrix (http://www.metamatrix.com). With the exception of the EII model repository, the SOA itself is being developed and will be delivered using open-source software.

Below is a list of some of the software we're using, it's no means exhaustive:

Commercial software:

  • Solaris operating systems for model repository servers
  • MetaMatrix do deliver the model and low-level data-integration services

Open-source server software:

  • Fedora Core 4 Linux operating systems for service/application servers
  • Apache Tomcat to deliver Servlet-based services (http://www.apache.org)

Open-source tools:

  • Eclipse for Java development
  • CVS for source code management

Open-source Java libraries:

CS: Will you be integrating your virtual learning environment into the architecture?

PW:There are no actual plans to do this - the business drive is for management information, but I am very interested in investigating the possibilities. Currently we run a huge WebCT CE 4.1 installation. This version of WebCT is pretty much impossible to integrate in any meaningful way into an EII architecture. However, I anticipate this system being replaced soon, most probably with WebCT Vista, which is a more industry-standards based system.

CS: How have you gathered the requirements of users, both staff and students?

PW: The short answer is: `we haven't....yet'. What we have done is demonstrate a range of potential capabilities, including a unified web-based query application which can display a range of information about students from a range of systems, and a set of management reports. These demonstrations have been very well received by both senior management and the University's Board of Governors. Several new, high-level committees have been formed to begin the process of identifying and prioritising development projects withing the EII project. These committees are about to meet for the first time, so it's early days for this process. However, we do anticipate that one such project will be to deliver a unified web-portal for students. To this end we have just started the process of identifying who we need to talk to to gather requirements and build a specification.

The demonstration student portal application developed in EII

The demonstration student portal application developed in EII

CS: One of the problems for institutions of switching to service oriented approach is the lack of staff with web services expertise - has this been a problem for the EII project?

PW: Not as such. I would describe our approach as both ambitious and relatively simple. I believe the scope of the SOA we are developing is ambitious - we plan to rapidly integrate at least some functionality from a pretty broad range of systems. However, at a technical level, much of what is planned is pretty simple. For instance, the vast majority of services developed in the first phase will be read-only, query services. This suggests that we can, at first, avoid issues such as managing transactions. Using a REST type approach to delivering web-based query services allows us to treat service end-points as URL addressable, atomic information entities. As the SOA framework is being developed in Java, much of the functionality can be delivered using simple, well-understood Java components such as JavaBeans, Servlets etc. A rich variety of open-source components are available for this kind of Java development, so we are by no means starting from scratch. We have good experience within the team with these technologies - the idea is that if, in the medium-term, we decide to `ramp-up' to a more heavy-weight Web Services/ SOAP / ESB solution, we can develope the expertise as we go. We already have some experience with SOAP SOAP, XML-RPC XML-RPC and Web Services in general in any case.

CS: Do you have any concerns about the project?

PW: The two main concerns would be:

  1. Can we model the University's information assets sufficiently well, such that the model increases our ability to manage these assets? This is the `million dollar' question - without achieving this all else becomes pretty much irrelevant.
  2. Can we develop a flexible, high performance SOA to deliver new services based on this model? This is less of a concern - our preliminary experiences are positive in this area.

CS: More and more institutions will be considering using web services to build their systems - what would be your advice to IT managers embarking on such a project?

PW: Keep the design as simple and manageable as possible from the outset. Be thorough in searching for evaluating existing third-party components - especially from open-source developers. We have benefitted greatly from our use of open-source Java components. Be aware of emerging standards and technologies and build for the future, but try to get some simple services implemented quickly - `quick wins' like this can help to maintain interest and support and it is always useful to demonstrate real, working services/applications as opposed to demonstration protoypes.

Other related resources

Paul Walk's talk on the Enterprise Information Integration project at the 10th Enterprise SIG meeting help at Chester Zoo on the 9th June 2005.

LeAP project case study. A case study about the Learning Architecture Project (LeAP) run by the Department of Education in Tasmania. The project also used a web services approach to integrate and deliver a number of online applications to enhance teaching and learning.

Enterprise Web Services projects, article on this site by Vashti Zarach, Enterprise SIG co-ordinator, summarising current projects in UK education in the Enterprise area.

Security

Posted by wilbert at 2005-10-26 06:45 PM
Very interesting mixture of the pragmatic and the cutting edge. One thing I found notable is that Paul doesn't mention security as a concern. To be sure, there are various means of making sure that only the right people have access to the right data, but it's seldom simple.
 

Supported by JISC Supported by CETIS
Powered by Plone