Resource Assessment

Panlab defines an open framework for coordinating arbitrary resources in distributed testing environments. Generally, the number of resources is not limited in Panlab and the process of finding appropriate resources for the specific testing needs of a customer is a challenging task. Panlab defines in its core logic a mechanism to make a suitable pre-selection of resources based on customer input in order to reduce the number of offerings presented to a user and thus to improve the system’s usability. In this section we define reasonable metrics for categorization and assessment of resources in Panlab and the mandatory processes to gather, evaluate, and utilize all required information.

All information collected by the resource assessment framework serves also as input for availability management and capacity management in Panlab as defined by the ITIL framework. Availability management focuses on (among others) availability, reliability, maintainability and manageability, serviceability and security of the offered service. Capacity management deals with performance management, application sizing, and capacity planning and modeling. Resource assessment should enable Panlab to detect bottlenecks, stale service offerings, and improve usability of the offered resources from customers’ perspective. Analysing waiting time distributions for individual resources based on customer functionality requirements should enable Panlab to detect potential bottlenecks and to reduce waiting times for customers, while increasing utilization of individual resources.

Usefulness or usability of a specific resource for a customer cannot be easily anticipated by the Panlab federation. Thus, Panlab defines a process to gather user based assessments of individual resources. This enables Panlab to gather valuable feedback about the usefulness of a resource for the customer’s needs, which may improve future pre-selection processes in Panlab’s core logic. Contrary to objective criteria for resource evaluation, user assessments are subjective impressions gained by human testers acting on behalf of the customer. Therefore, to avoid any legal issues, presenting human assessments to entities outside the Panlab federation should be avoided. Such legal obligations are for further study. However, internal analysis of such data should be integrated into the core logic entities.

Resource usability, in Panlab context, means that the users can use the federation resources efficiently and easily to accomplish their own needs.