1 DILIGENT, a digital library infrastructure on grid enabled technology http://www.diligentproject.org/ DILIGENT is an infrastructure project that aims to create an advanced test-bed that will allow members of dynamic virtual e-Science organizations to access shared knowledge and to collaborate in a secure, coordinated, dynamic and cost-effective way. This test-bed will be built by integrating the Grid and Digital Library ([DL]) technologies. Merging of these different technologies will lay the foundations for a next generation e-Science knowledge infrastructure. The DILIGENT infrastructure, which will build upon the efforts of the [EGEE] project (IST-2003-508833), will be able to serve many different research and industrial applications. The test-bed will be demonstrated and validated by two complementary real-life application scenarios: one from the cultural heritage domain and one from the environmental e-Science domain. A dedicated work-package will design and implement the functionality required to compose existing services into processes, to validate the correctness of processes, to reliably execute processes on top of the Grid infrastructure, and to automatically optimize the definition of processes according to the resources available and the quality of service characteristics associated with processes. In particular, process optimization will be considered for query processes, i.e., processes that combine existing DL and Grid services in order to support search requests of the DL users. Hence, the process-related services that will be designed and implemented are: - Process Design and Verification - Process Execution and Reliability - Process Optimization Service In terms of designing a service for process definition, the criteria that allow for the verification of process descriptions will be identified based on the description of the individual services encompassed within a process. Furthermore, a graphical toolset will be implemented that will guide a [VDL] (Virtual Digital Library) user in designing processes and in verifying the correctness of the graphically designed processes. In parallel to the process design and verification service, a service for reliable process execution will be designed. A core aspect in the design of this service is support for scalability and flexibility. In particular, this service will make use of the benefits of the underlying Grid infrastructure and the possibility to dynamically distribute the overall load within the grid among several semantically equivalent services. Finally, a service for process optimization will be designed. This optimization service takes process descriptions and, while respecting the inherent correctness guarantees, applies changes to the execution order of invocations to individual services within the process. 1.1 Research Today’s grid infrastructures provide support for the efficient deployment and use of single services, but are limited in terms of combining several services into a coherent whole – into processes. DILIGENT will provide support for processes over grid services. In order to combine workflow and grid technology, the following aspects have to be solved: - Quality-of-service ([QoS]) guarantees have to be associated with processes. In this context, it is important to identify how the combination of services contributes to the overall quality of a process - Quality-of-service guarantees have to be attached to individual grid services in order to allow to derive the quality of a process out of the QoS information of the constituent services. In here, a cost model for services, but also runtime information on the resources needed by a service are necessary and have to be added to the specification of grid services - Correctness criteria that indicate when processes are considered correct have to be identified. These criteria have to take into account that processes are composed of building blocks and are long-running activities. Essentially, this affects the appropriate failure handling strategies. The goal of the verification is to derive provable qualitative prognoses on the execution of processes. Usually, when designing process-based applications, several failure situations can be anticipated and appropriate failure handling strategies can be defined. However, this specification does not allow for the consideration of the resources that are available at run-time. Hence, failure handling strategies, although being correct from an application point of view, might not be optimal. In addition to robustness, availability, and reliability issues, the self-adaptation of an integrated ‘process-and-service-Grid’ infrastructure to the resources that are available at a particular point in time is an important problem. Therefore, the key issues of a service for the execution and reliability of processes are: - high availability: Essentially, this requires that the process execution service will be distributed within the grid, by applying peer-to-peer technologies for process management - high scalability: the execution of processes should not be affected by the number of services and/or processes in the system. Rather, the process execution and reliability service has to be able to provide a high degree of scalability - reliability: processes have to be executed according to their specification. The Process Execution & Reliability service must provide dedicated guarantees to the users of this service, i.e., it must be a service their users can count on. Failure situations and concurrency problems have to be solved appropriately. - self-adaptation: the Process Execution and Reliability service has to make use of the underlying grid technology, especially in terms of identifying appropriate grid service instances to be invoked within a process, thereby balancing the load within the overall system. Most existing workflow engines do not take into consideration timing and Quality of Service issues as defined in the context of DILIGENT in order to carry out process optimization. The main role of currently available workflow systems is monitoring of the execution of subordinate jobs/tasks. In addition, several efforts have successfully addressed various relevant issues including language constructs for task synthesis and decomposition, scheduling of individual jobs and pertinent policies, correctness of execution, and platform heterogeneity made possible with the help of common APIs. In order to meet the requirements of DILIGENT, we intend to pursue several technical issues that remain open in workflow process optimization: - Effective representation of various complex performance aspects through metric indicators; management of time in workflow structures and timely handling of deadlines for the various functional components involved; evaluation of their contribution to overall service quality - Incorporation of the notion of differentiating “quality” of contributing data for the realization of services while taking into account algebraic properties of participating components that will permit the exploitation of semantically equivalent, potentially more efficient process compositions - Efficient management of data sketches and statistics for DL service accesses, as well as timely handling of resources available. 2005-05-31 14:33:08.252 2005-05-31 10:40:03.64