The SIMPATICO technical team are adopting the same line of the(STEP) methodology in order to evaluate the SIMPATICO Technological Platform and its related components. The STEP outlines a rigorous process for technology evaluations of one or more COTS (Commercial Off-The-Shelf) products.
The aim of this STEP methodology is:
- formalizing a process that can be adopted to evaluate a wide range of technological product
- producing a set of documents where relying on the consistency of evaluation results
- providing the evaluation team with guidelines in developing goals
- modelling a process easy to understand and fully adopted.
Four main phases compose the STEP process, namely:
- Phase 1 – Scoping and testing strategy: declare the objectives and scope, and performs a market survey to identify potential products in the technology area.
- Phase 2 – Test preparation: a full set of evaluation criteria will be performed. The products/components will be tested against every criteria and any scenario tests that will be performed. The evaluation team then installs the products/components in the test environment
- Phase 3 – Testing, results and final report: the evaluation team tests and scores the products/components against all of the test criteria. The team must ensure that testing for each product is performed under identical conditions.The team produces a final report that incorporates the evaluation results and any supporting information.
- Phase 4 – Integration and Deployment: deploy and integrate the solution into the operational environment. Actions in this phase may include developing configuration guidance and supporting documentation.
The adaptation of SIMPATICO methodology to STEP focuses on performing both manual and automatic activities, namely some tasks will collect data from interaction of the user with the platform, some other ones will demand directly to the users and they will be asked to respond to a survey.
In this sense, the platform evaluation has been conducted from both quantitative and qualitative point of view taking into account different evaluation criteria.
A set of KPIs have been defined, in particular:
- the Integration Use-Cases have been depicted in order to compare the real platform operation with that one foresees.
- the Quality Assessment has been defined for each component. Seven characteristics of software quality have been identified and the whole platform has been evaluated for each of these characteristics.
- in the end, the Platform will be evaluated through the Feedback coming from the users during the installation, configuration and deployment of the platform within the three pilot cities.
The project have two phase of evaluation and after the last phase we can compare what we were able to improve and what the right way to proceed in order to enhance the real added value provided by SIMPATICO Technological Platform.