A concept for performance measurement and evaluation in network industries

The structure and functioning of network industries have a great effect on the well-being of society, being also prerequisites for economic growth and productivity. Social goals, industry trends, ownership structures, and the economic environment play an important role in the development of such networks. Most network industries enjoy a dominant position on the market. Their performance is steered by national regulatory authorities via price control and quality requirements. For this the regulatory authorities need besides financial indicators feedback on the technical performance of the provided services. The vast development in sensing, data transmission, and collection technologies, as well as in analytical methods, has made it possible and feasible to acquire the needed feedback. Such comprehensive data enable to construct a performance measurement system to regulate, develop, and administer the networks. This paper explores the possibility of developing an overall technical performance index and presents a relevant concept. The suggested overall index would be an additional regulatory tool to evaluate the performance of network industries and their compliance with consumers’ requirements. The aim of the proposed concept is to establish empirically verifiable feedback between a given state of technology, state of institutional governance, and the performance of network industries.


INTRODUCTION
* The economic growth and competitiveness of countries or regions rely on the development level and functioning of network industries.People, businesses, and public services depend on infrastructure networks to function efficiently.These networks comprise of roadways, railways, waterways, pipelines, electricity lines, postal services, and telecommunications to transport goods, obtain and transfer information, gain access, provide services, etc. [1].
Network industries can be defined as entities where the institution or its product consists of many interconnected nodes and where the connections among the nodes define the character of commerce in the industry [2].A node in this context can be an institution, a unit of an institution, or its product.
The products and services provided by network industries represent a sizeable input for every country's economy accounting for a large part of their gross domestic product (GDP) [3].On the other hand, the majority of the services provided by network industries are services of general interest.Governments intervene in markets to promote general economic fairness and maximize social welfare.Government intervention through regulation can directly address inefficient markets and cartels as well as other types of organizations that can wield monopolistic power, raising entry costs and limiting the development of infrastructure.Without regulation, businesses can produce negative externalities without any consequences.This all leads to diminished resources, stifled innovation, and minimized trade and its corresponding benefits [4].Therefore it is necessary to evaluate and measure the performance of the network industries to ensure that the current structural changes do not prevent those social and public policy objectives being attained [3].
Interest has recently considerably grown in performance measurement.The topic of performance measurement has generated much coverage over two decades in many disciplines within the private and public sectors [5].
The main objectives of the research reported here are to  explore the possibility of developing a generic performance index (GPI) to evaluate the performance of network industries across traditional borderlines;  propose a high-level reference model of conceptual information and communication technology (ICT) architecture to allow timely data collection, prediction, and analysis in order to support GPI calculation across network industries.Latest developments in the ICT sector, such as Big Data and analytics in conjunction with novel user interaction design patterns, pave the way to measure all important performance parameters, create radically better understanding of the problem domain, and translate that knowledge into management decisions.

Definition
Performance measurement is the use of statistical evidence to determine progress toward specific defined social or organizational objectives (see Fig. 1).In this paper, performance is seen as a broader concept than in the traditional financial approach, quality of supply, or quality of service, embracing the requirements of consumers in relation to the service provided by network industries.Many comprehensive studies have been performed about enterprises, but not covering full industry verticals or going across verticals [2].The National Performance Review of the U.S. Federal Highway Administration provides a complementary definition of performance measurement [8], which is applicable in the context of network industries: 'A process of assessing progress toward achieving predetermined goals, including information on the efficiency with which resources are transformed into goods and services (outputs), the quality of those outputs (how well they are delivered to clients and the extent to which clients are satisfied) and outcomes (the results of a program activity compared to its intended purpose), and the effectiveness of government operations in terms of their specific contributions to program objectives.' Every performance measurement system (PeMS) requires developing and reviewing at a number of different levels as the situation changes.The PeMS should include an effective mechanism for reviewing and revising targets and standards and should be used to challenge the strategic assumptions [9,10].

Performance indices
In this paper the authors propose hierarchical models of performance measurement where there is a synthesis of low-level measures into more aggregated indicators.The 'real-time' data collection gives new possibilities for performance monitoring and management since the services in network industries are 'consumed' at the same time they are 'produced' [3].
The performance index is a management tool that allows multiple sets of information to be compiled into an overall measure [11].Key performance indices (KPIs) provide information to stakeholders about how well the services are being provided.Performance indices should also reflect the satisfaction of the users not only the concerns of the system owner or operator [8,12].
The procedure of combining data into indices is necessary to present simultaneous information from several related areas and data sources.This process provides a statistical measure that describes the change of performance over time.
Figure 2 proposes a general conceptual model (( 1)-( 3)) for performance indices.It can be exploited in network industries to evaluate performance of different network locations and elements and their performance trends over time.The TPs are measurable or observable environment characteristics, whose values vary over time.A PI defines the measurement of a piece of useful information about the performance of a programme or a work expressed as a percentage, index, rate or other comparable indicator that is monitored at regular intervals and is compared at least to one criterion.The use of PIs goes beyond simply evaluating the degree to which goals and objectives have been achieved [6,16].
Several scaling, weighting, and aggregation methods could be applied based on the nature of the data.The maximum function is proposed to combine and propagate the importance of the value through index hierarchy.Other combination techniques can be used.

Inland transportation networks
In the road industry, PIs are defined for different types of pavements and highway categories.On the first level several single PIs describing the characteristics of the road pavement are assessed [16].The next step is the grouping of single PIs into KPIs and finally into representative CPIs as:  functional performance indices (demands made on road pavements by road users);  structural performance indices (structural demands to be met by the road pavement);  environmental performance indices (demands made on road pavements from an environmental perspective).Finally, based on the CPIs a GPI is defined for describing the overall condition of the road pavements (see Fig. 2 and Fig. 3).The GPI can be used for general optimization procedures [6].A PeMS for railway infrastructure can be developed in a similar way.An example of how to model a respective GPI from railway and transportation CPIs and technical parameters is given in Fig. 4.
Inland transportation is a good example how performance can be measured as one comprehensive entity.

Power distribution networks
Attempts were made in [25][26][27][28] to create overall indices for the evaluation of the quality of supply and reliability of power distribution networks.Emerging Smart Grid developments are putting emphasis on understanding the performance of the power network not only from power quality and grid reliability aspects, but as a whole.However, the index systems in the power distribution are not yet as comprehensive as for road networks.Their development remains a topic for further research.
A lot of effort has been put into voltage quality research.Standardization bodies have implemented several indices [29][30][31].However, the commercial quality, quality of service, safety of operations, and socioenvironmental impacts of power distribution network operations have not been investigated thoroughly.
We made an attempt to combine different sources [25,28,32] of information into a single GPI (Fig. 5).This GPI can be visualized and hierarchically combined for substations (areas), feeder lines, phases, and metering points in the distribution network.

Telecommunication networks
No industry is related to so many other business sectors as telecommunications.The sector is regulated, but the competition is usually fierce among several service providers [33].
Measurement of the performance in telecommunications focuses on two major groups of non-financial indicators: operational efficiency and overall customer satisfaction and experience (see Fig. 6).This can be seen as the main difference from other network industries.The focus in performance measurement is clearly on the customer and efficient service delivery is the main requirement.

Synthesis
The examples of performance indices in network industries (what is measured, how the measurements are grouped and presented) reveal similarities and patterns.The indices are built up similarly and share the same principles.
Despite clear similarities, the focus of measured indices differs among industries.The focus reflects the main issues that are currently topical, omitting other aspects that deserve equal management attention.In the long run the control and steering decisions that are based on the subset of visible information are not effecient and optimal.
In order to systematize the numerous parameters affecting performance and to maintain conciseness, cognition, and clarity, it is important to find new ways to combine parameters systematically across network industries into one GPI and standardize the approach to CPIs (see Fig. 7).We see that here exist possibilities of harmonizing performance measurement principles.Such harmonization would allow of common visualization, comparison, and management of performance across network industries within a general ICT framework.

Data context
The process of unlocking additional value from the existing data and combining them with new data sources (e.g.sensors, Smart Meters, Internet of Things) will have a transformative impact on the management of network industries.More efficient, and therefore less expensive, data communication, storage, and presentation and better data processing mechanisms will allow of the handling of data of greater variety and volume with higher velocity in the near future, which will lead to adequate, accurate, and actionable insights [35,36].
We propose a reference model of system architecture developed to provide a high-quality overview of the functional components in a platform that enables processing the technical parameters and combining them into understandable indices as well as visualizing  them for the end-user.By dividing layers of responsibilities between different functional components of the platform, we can get a clear view of the roles and responsibilities and lay the foundation for a common understanding.
The diagram in Fig. 8 presents an overview of the performance evaluation system and specific functional layers of the platform.The reference model is based on the work done at Tallinn University of Technology [6] and TM Forum [37].

Layers of the reference model
The purpose of the reference model is to provide a highlevel overview of the functional components in the performance evaluation system.All layers have a clear responsibility borderline.Only a subset of functionality may be needed to satisfy the requirements of a particular performance evaluation scenario.
The data communication layer is responsible for transporting the data from data sources into the processing platform.The amount and creation speed of data will play the key role in choosing the communication technology.
The data ingestion layer is responsible for integrating various data sources and importing the technical parameters into the platform.The main task of this layer is to handle the volume, velocity, and variety of the data coming into the platform.Modules of this layer must be capable of scaling out in order to accommodate the data input bandwidth and speed.
The responsibility of the data processing layer is to ensure data quality.The validation of values is also done in the data processing layer.When data are missing, prediction algorithms can be applied to estimate the missing values [17].
The data management layer accommodates processes to perform format transformation towards a data model of uniform domain, correlation of events, and enrichment manipulation.Batch queries over the large historical dataset using the Map-Reduce [38] or a similar algorithm are to provide the functionality to implement scenarios that do not require real-time processing.
The Complex Event Processing (CEP) [39] layer on the other hand controls the processing of streaming data and the calculation of indices, doing it on on-going bases.As to large data, the two above-mentioned layers can be implemented by massively parallel-enabled data processors.
The data aggregation layer carries the responsibility of combining the results of CEP and queries into PIs and combined indices.
The visualization layer is often left out.However, it is the key to make the collected data easily understandable and meaningful to the end-users.
Common functions, Geographical Information System (GIS), and data repository are functions required to implement each of the layers.

CONCLUSIONS
Constant performance evaluation of network industries enables their more effective and efficient lifecycle management.Therefore, a lot of research on per-formance measurement and evaluation and even standardization efforts have been made.
For road networks systems of comprehensive indices are available.Indices of power distribution systems focus today mostly on the voltage quality side and do not provide real end-to-end support for decision-makers.Telecommunication indices focus on operational efficiency and customer experience.A railway GPI calculation system is presented only on an idea level.
Unifying and combining various aspects of different indices into a common performance index of network industries is proposed, but harmonized framework remains a topic for further research.The performance of all examined industries can be analysed and visualized using the proposed performance evaluation reference architecture.
The unified conceptual model of performance measurement as a system covering network industries as a whole should be used to understand and compare the performance inside and across network industries in a regulation realm to secure efficient and optimal investments into the structure and functioning of network industries.
first-level performance indicator, i TP is observed technical parameter, k CPI is combined performance indicator, and GPI is generic performance index.

Fig. 8 .
Fig. 8. Reference model of the system architecture of performance evaluation of network industries.