E-government maturity models: more of the same?

As e-government initiatives progressed, several models for measuring e-government maturity were proposed. Many are stage models based on the Capability Maturity Model (CMM) in which e-government maturity is conceptualized as stages of growth that evolve over time. The paper aims to investigate if e-government stage maturity models measure the use and usefulness of e-government. A meta-synthesis technique was used to compare and contrast 11 meta-models (models derived from other models), at the stage level, for their perspectives, concepts, metaphors, and their similarities and differences. We found that although models use different names and metaphors for analogous concepts, similarities exist among them, and individual stages overlap. Results show two gaps in research regarding the assessment of the actual use and usefulness of e-government. First, meta-models primarily assess the supply-side and operational/technology and citizen/service perspectives. Second, the use and usefulness of e-government are not addressed.


INTRODUCTION
Governments have been using Information and Communication Technologies (ICT) for decades. According to Norris [1], traditional Information Technology (IT) in government is inward-looking. It mainly addresses internal government applications such as payroll, accounting, and emergency lines. They aim to automate operations, gain efficiency, and ultimately offer better services to society. In contrast, e-government is outward-looking as the provision of information and services is primarily meant to external stakeholders. E-government connects government to citizens, businesses, its employees, and other instances of government. Despite the variety of motives for implementing egovernment, the principal motive is electronic access (online versus in person) by external parties to governmental information and services.
For the most part, this interaction has been through government portals on the World Wide Web [1]. Hence, egovernment maturity has been associated to the degree a government has established its presence on the Web [2] [5]. Web presence relates to the publication of static and dynamic data on official web portals, access to databases, and a variety of online services. The sophistication of the services offered directly impacts the level of security and the complexity of the infrastructure required [6]. The increasing levels of complexity and sophistication are not built overnight. And egovernment maturity usually represents a continuum of developmental stages, from publishing information to supporting online transactions, with some governments having progressed further than others [7].
Thus, the use of stage maturity models to gauge egovernment maturity. However, traditional maturity models suppose well-controlled processes geared to continuous improvement. The last maturity level is known in advance, and the model is close ended. E-government stage maturity models are not close-ended. New stages are expected as Information and Communication Technologies (ICT) evolve at a fast pace, and the demands of society for new services arise.
Davis [8] developed and validated scales for measuring perceived usefulness and perceived ease of use to understand and influence unwilling users to accept and use computer systems. He found that these variables have a significant correlation with self-reported current usage and self-predicted future usage. Perceived usefulness refers to "the degree to which a person believes that using a particular system would enhance his or her job performance" [8]. In this context, the usefulness or utility of e-government means if it is helpful, beneficial, or serviceable to its intended users [9]. Use refers to the actual usage or utilization of e-government initiatives. This article intends to call the attention of researchers and practitioners to the gap that exists between factors that influence the adoption of e-government and maturity assessment tools such as stage maturity models. Perceived usefulness is a determinant factor of the intention and actual of use of e-government [10], [11]. In contribution, this article identifies and reviews e-government maturity models to determine whether and how e-government usefulness and use are addressed.
We conduct a tertiary study to review articles related to egovernment stage maturity models published between 1988 and 2019. We focus on meta-models, i.e., models derived from other models to reduce the number of studies surveyed and still acknowledge previous research done on the field. A qualitative meta-synthesis is adopted to compare and contrast 11 e-government maturity meta-models to understand, at the stage level, their perspectives, metaphors and concepts, and their similarities and differences. We seek evidence whether any of the stages address the impact, use, and usefulness of egovernment. Moreover, we survey the methods used to develop the models, year of their publication, and background influence.
This paper is organized as follows. Section II describes egovernment maturity models. While section III presents the research methodology. Section IV applies a qualitative metasynthesis to the selected maturity models and reports results. In section V, we present discussions and the limitations of this work. In section VI, we draw our conclusions and suggest future work.

II. E-GOVERNMENT MATURITY MODELS
This section describes capability maturity models and some of the first e-government maturity models published in the literature.
In Information Systems (IS), maturity is usually measured again capabili ie ," o a Capabili Ma ri can be defined as capability measured against some desired state or goal. According to Mettler [12], in IS research, maturity models are understood as tools that can facilitate internal and/or external benchmarking, present possible improvements, and provide guidelines through the evolutionary process of organizational development and growth.
The Software Engineering Institute (SEI) states that Capability Maturity Models (CMMs) focus on improving processes in an organization. They contain the essential elements of effective processes for one or more disciplines and describe an evolutionary improvement path from ad hoc, immature processes to disciplined, mature processes with impro ed q ali and effec i ene . [13]. CMM provides a set of effective practices that addresses productivity, performance, costs, and stakeholder satisfaction for systematic and evolutionary process improvement. One of the most popular IS capability maturity models is the Capability Maturity Model Integration (CMMI) [14]. It is the successor of the Software Capability Maturity Model (Software CMM) [13]. Software CMM and CMMI were created for the development, maintenance, and acquisition of software products and services. However, other models from different areas [15] adopted their five-stage capability maturity level and the means for determining those levels [16], such as in egovernment. This study focuses on governmental stage maturity models as categorized by Concha et al. [17] and as in some of the models presented below.
The Layne and Lee [4] e-government maturity model is one of the most known and influential academic models, with 3038 citations reported on Google Scholar and 1431 on Scopus as of August 2019. Layne and Lee propose an evolutionary four-stage model based on observations of egovernment initiatives in the US. The Hiller and Belanger's [18] model was one of the first to appear in the academic literature that contemplates the political perspective. They proposed a five-stage maturity model for e-government. Moon [19] adop he Hiller and Belanger model i h minor terminology differences to assess the evolution of egovernment in US municipalities. Gartner's [2] four-stage egovernment maturity model surveys the evolution of egovernment online interaction with customers. It also identifies a strategy and other factors that contribute to success in each phase. Accenture 2003 model is similar to the Gartner's with minor adjustments (change stage names, but not concepts) [6]. The United Nations and American Society for Public Administration 2001 [20], also known as the UN 2001, is a five-stage maturity model. The United Nations (UN) model has evolved over the years; it added eparticipation to its stages in the 2003 to 2008 surveys. Davison et al. [21] proposed a five-stage alignment-based maturity model to explain three "typical transition paths" from government to e-government. These models do not combine different e-government perspectives technology, organization, management, and politics, which are fragmented across metaphors and concepts [6], [22]. The UN 2001, Gartner, and Layne and Lee models do not contemplate the improvements of political development and democracy, which are some of the main egovernment goals. The UN 2001 model focus on the benefits of improving the front-office [23] (web-based services) and does not address the transformation of government operations to improve efficiency. Gartner's model focuses on the customer. However, it does not address the improvement of internal operations [24]. Lee [6] contends that the e-democracy stage that Siau and Long [24] added to their model does not represent an evolution of their model previous stages. They argue that e-democracy is not an evolution of transformation. As Heeks [25] states, all maturity models are a product of their time, context, and mindset. As such, the models discussed above are not yet concerned with the impact, use, or usefulness of e-government initiatives. Their focus is on the supply of e-government, i.e., the implementation of Information and Communication Technologies to improve the efficiency of government services provided online.

III. METHODOLOGY
Me a n he i i a echnique that attempts to integrate re l from man differen b in er rela ed q ali a i e studies. It intends to be interpretive, rather than deductive. In con ra o a me a anal i of q an i a i e die , hich aim to increase certainty in cause and effect conclusions. Differen l , q ali a i e me a n he i eek o nder and and explain phenomena [23]. Qualitative meta-synthesis is an exploratory research method designed to build or extract a common frame of reference from qualitative research results [6]. Stage models are mostly developed qualitatively [15]; thus, a qualitative meta-synthesis technique was selected as the research method for this study. We selected Wash and Do ne [23] seven-step technique for conducting this study's qualitative meta-synthesis. The last three steps of the technique are based on Noblit and Hare [26]. Table I presents a summary of these steps and their descriptions.

Meta-Synthesis Steps Description
Framing a metasynthesis exercise Appropriate research question, purpose or aim frames a meta-synthesis.

Locating relevant papers
Undertake a robust search of on the topic area being studied. Acknowledgement of the potential that search directions may be divergent, rather than linear in meta-synthesis. Deciding what to include Identify and compare methods, validity and scope of each study and decide the scope of the meta-synthesis.

Appraisal of studies
Identify and apply predefined criteria used to appraise the rigor of studies.
Compare and contrast exercise Noblit and Hare [26] understanding of key metaphors, phrases, ideas, concepts, and relations in each study is identified, and usually tabulated. These findings are then juxtaposed to both identify homogeneity of categories/codes/themes and, crucially, to note discordance and dissonance.

Reciprocal translation
The ran la ion of one d finding in o ano her, using metaphors and concepts that could be applied to both. 'Refutational translation' and overlap may contribute to another, emergent, category or understanding which has not been identified in the original accounts.

Synthesis of translation
Clusters of metaphors become refined and a consensus emerges as to core themes, new concepts and exploratory theories. The synthesis needs to reflect the tension between contradictory or alternative explanations if reciprocal translations suggest a lack of congruence.

A. Framing the Meta-Synthesis Exercise
The focus of his study is e-government stage maturity models to assess whether and how they address e-government use and usefulness. We investigate what are, at the stage level, their perspectives, metaphors and concepts, and their similarities and differences. We address the following questions. (1) What are the stages of e-government maturity models? (2) What are the characteristics (perspectives, metaphors, and concepts) of e-government maturity models? (3) What are the similarities and differences of e-government maturity models? (4) Do the stages of e-government maturity models measure the use or usefulness of e-government?

B. Locating Relevant Papers
We conducted a review of the literature based on peerreviewed articles published in academic conferences and journals on Scopus 1 (148 results), Science Direct 2 (164 results) and Web of Science 3 (67 results) using the keywords e-go ernmen AND ma ri model" for the year > 1987. Instead of broadening our search with synonyms such as "e-gov" or "stage maturity level," we used a recursive search of a seeding article's citation tree and related work to find related studies. In the first screening process, all 379 articles were perused of keywords, titles, and abstracts to discard all non-relevant articles and duplicates. We only reviewed articles written in English after 1988. Among the remaining 187 articles, Siau and Long [24] was the first, we identified, that created an e-government maturity model using a metasynthesis technique. We conducted a recursive search of that ar icle ci a ion and rela ed ork on Google Scholar o locate additional relevant literature and other maturity models. Supplementary sources include Google Scholar, ResearchGate, books and book chapters, international organizations and government articles, documents, reports, rankings, and benchmarks.

C. Deciding What to Include
Eleven articles, each proposing or adopting a stage maturity model resulting from a qualitative meta-synthesis or literature review, were selected. Our study draws from previous research. It conducts a tertiary-level qualitative metasynthesis of the e-government evolutive staged models (CMM based) as defined by Concha et al. [17]. These meta-models (models derived from other models) are the product of literature reviews or qualitative meta-synthesis proposed by individuals, consulting firms, international organizations, or governments. We did not include models created between 1999 and 2004, because most of them already accounted for in meta-models developed after 2005. The 11 e-government stage maturity models with their publication year, which includes Siau and Long 2005 [24]; Shahkooh et al. [36] reviewed and synthesized nine maturity models, which were published between 2000 and 2004. They included Deloitte and Touche; UN 2001; Layne and Lee; Accenture; Gartner; World Bank; Wescott; West; and Hiller and Belanger to create a five-stage maturity model. The e-government maturity stages identified are Online presence, interaction, transaction, transformation, and digital democracy. The authors posit that the complexity of technology and the number of e-government applications increase at a higher maturity level. The model is technologycentric and focuses on the supply of e-government and its enabling factors. The effective use of services or their helpfulness is not assessed.
Kim and Grant [29] proposed a five-stage maturity model after conducting a qualitative meta-synthesis of Capability Maturity Model Integration (CMMI) and six maturity models, published between 2000 and 2005, including Layne and Lee; UN 2001; Moon; Siau and Long; Andersen and Henriksen; and Hiller and Belanger. The framework is composed of four input areas (human capital, structural capital, relational capital, and IT investment) and five maturity stages (web presence, interaction, transaction, integration, and continuous improvement -a combination of transformation and egovernance stages of other models). These areas are assessed by using the Intellectual Capital (IC) management model and the Capability Maturity Model Integration (CMMI). The continuous improvement stage indicates that e-government is effectively achieved by continuously improving processes, the use of innovative technologies, and cooperation with other governments. It also implies that users participate in various political activities through web sites. It does not mention the usefulness of initiatives. Lee proposed a frame of reference with five metaphors, which are presenting, assimilating, reforming, morphing, and egovernance. Each metaphor is broken into two underlying themes citizen/service and operation/technology. The citizen/service theme is further divided into five stages/concepts information, interaction, transaction, participation, and involvement. The operation/technology stages are integration, streamlining, transformation, and process management. The model is not concerned if egovernment initiatives are used or are useful.
Fath-Allah et al. [31] applied a qualitative meta-synthesis on 25 evolutionary stage e-government maturity models to identify their similarities and differences and also to find their weaknesses and strengths. Models were published between 2000 to 2012. The study was later used to propose a best practice-based e-government portals maturity model [37]. This study drew on four other literature reviews Kim  Transformation (Electronic government), Engagement (Electronic governance), Contextualization (Policy-driven electronic governance). Each phase is further characterized by three Boolean variables Internal government, Transformation affects external relationships, and Transformation is contextspecific [33]. The model focuses on civil participation and the efficiency of the organization. It does not focus on the use or usefulness of e-government.
Almuftah et al. [32] also conducted a qualitative metasynthesis to review 17 e-government stage models developed between 2001 and 2012. The proposed framework has three stages that capture Presence, Communication, and Integration. Two maturity variables were identified in all models: the level of complexity and the level of interaction. The level of complexity indicates that governments offer services that are more advanced and integrated at higher maturity levels. Moreover, the level of interaction between government and citizens increases at higher maturity levels. Nielsen [22] surveyed 34 stage e-government maturity models to find gaps that could lead to future research on stage, cooperation, and governance models to help successfully develop e-services. He adopted Lee's [6] qualitative metasynthesis framework to compare and contrast models, which were created between 1999 to 2015. The models are Hiller and  [31]; and Almuftah et al. 2016 [32], conducted qualitative meta-synthesis on existing maturity models and then provided their maturity models. While Nielsen 2016 [22] and Iannacci et al. 2019 [34] adopted existing frameworks to synthesize their findings, Persson and Goldkuhl 2005 [27] did a comparative study of four models. Janowski 2015 [33] surveyed 292 articles to validate the concepts underlying his model. Over 40 e-government maturity models were identified and considered in the creation of the 11 metamodels. The original maturity models have between two and twelve stages. They were developed between 1999 and 2015, being 73% of them created in the first ten years. Meta-models Klievink and Janssen, which were cited only once by other authors. Siau and Long meta-model [24] influenced five other meta-models. In contrast, the other meta-models were only cited once or twice.

D. Appraising Studies
All academic papers and articles were peer-reviewed and published in indexed conferences proceedings and journals. Governments and reputable international organizations produced surveys, reports, rankings, and benchmarks. Most methodologies are available online free of charge. For this study, the quality of the papers and their results have already been established.

E. Comparing and Contrasting Maturity Meta-Model Stages
Stages were compared based on the level of the stage, its label, and description. Seven concepts (or stages) were distilled from the stages' labels and descriptions. Concepts are Presenting information, Interaction, Transaction, Integration, Transformation, e-Governance, and Policy-driven e-Governance. During the comparing and contrasting exercises, we did not identify concepts to assess the use or usefulness of e-government. For each model, the stage's name and description were compared and placed under the concept/stage it best suited. Models have between two and six stages. Persson and Goldkuhls has two; Almuftah et al. has three; Fath-Allah et al. and Janowski have four; Sandoval-Almazan and Gil-Garcia model has six. The remaining six models have five stages. Pers on and Goldk hl In egra ion in er ice and Jano ki Digi i a ion fir age encompa the four first concepts identified in other models. Integration in service cover initiatives like publishing information on a web site to integrating government services at different levels, such as those of local governments and state governments. Digitization or Technology in Government includes e-government initiatives such as the provision of a government portal and filing taxes online. Shahkooh et al.'s fourth stage, Fully integrated/transformed e-government , encompasses the Integration and Transformation of most models. Next, we briefly describe each concept and summarize the comparing and contrasting exercises. Stage 1 -It seems that publishing information on a public web site is one of the first e-government initiatives implemented. Although the differences in naming (web presence, online presence, rhetorical intentions). All models include the Presenting information stage. Persson and Goldk hl In egra ion in er ice and Jano ki Digi i a ion stages encompass the concept of presenting the information. Sandoval-Almazan and Gil-Garcia Pre ence (le el 1) and Information (level 2) can be combined into this concept, as the levels are described as government providing information to the public. E-government initiatives can be classified based on the demand and/or on the supply of e-government. This stage assesses only the supply-side and is technology centric. The use or usefulness of web sites and data published is not addressed.
Stage 2 -This stage is about Interaction between government and users. It seems to follow the Presenting information stage. Despite the naming differences (interaction, assimilating, strategic planning) and of stage levels (Sandoval-Almazan and Gil-Garcia's Interaction is level 3), all models include this concept. At this stage, twoway communications are available (asking questions, pools). Iannacci et al.'s Strategic plan (start implementing web-based systems, services, or information sites), Lee's Assimilating; Per on and Goldk hl In egra ion in er ice and Jano ki Digitization stages incorporate the concept of interaction. Models are only concerned with the existence of the interaction; they do not address whether the interaction is useful or how many users interacted with the portal.
Stage 3 -For most models, stage 3 represents the ability to conduct transactions online. Sandoval-Almazan and Gil-Garcia Tran ac ion (le el 4) and Per on and Goldk hl Integration in service require secure and identified ran ac ion a hi le el. Lee' Reforming and Jano ki Digitization stages include the idea of conduct transactions. Some authors, Siau and Long; Sandoval-Almazan and Gil-Garcia; and Shahkooh et al. [24], [28], [30] mentioned the increase in the number of services provided, as well as the of the complexity of services. Iannacci et al.
S em development (potential for some integration between planning tasks or between planning and action) also covers the concept of a transaction. This stage focuses on the supply-side; it does not provide insights into who uses the services or if the interaction was useful. Stage 4 -Integration seems to be the "natural" development path after the transaction stage. Layne and Lee were the first to use the terms vertical and horizontal Integration. Lee [6] clarifie vertical integration refers to the system integration of similar functionality across different levels of government local, state and central, while horizontal integration refers to the system integration among different functions of government. . The e concep ere incorporated in all models, even if some models do not have a epara e In egra ion age. Sia and Long Tran forma ion age co er in egra ion, a ell a Per on and Goldk hl Integration in er ice, Lee' A imila ing, and Jano ki Digitization stages. The focus of this stage is on the supply-side. It aims at increasing the efficiency of operations and technology; it does not look if service is being used or is useful.
Stage 5 This stage deals with the Transformation of how services are rendered and the relationship between State and society. At this level, operational services are offered and improved for higher efficiency and user satisfaction as in Persson and Goldkuh; Janowski; Lee; and Kim and Grant. Some models Sandoval-Almazan and Gil-Garcia; Fath-Allah et al.; and Almuftah et al. do not present a separate Transformation stage, although they all include a separate Integration level. Lee [6] argues that models that focus on "integration" offer technology and operation-oriented perspective while models focused on "transformation" are geared towards the service and organization perspectives. This stage is the first to look into the demand-side, as user satisfaction is a dimension of service use and usefulness. Models that do not include a separate "e-governance" stage seem more focused on the technology/operation perspective, not so much in the citizen/service dimension. This stage focuses on the supplyside. Models are not concerned with the usefulness or outcomes of e-governance. Forums and boards are provided, but their use or impact is not measured.
Stage 7 -Jano ki model mo e be ond he egovernance stage. The author proposes the Contextualization or the Policy-Driven Electronic Governance stage. It uses the technology and governance created by the Digital Government to "implement specific public policy and sustainable development goals in support of specific efforts by countries, cities, communities, and other territorial and social uni o de elop hem. [33]. Services target specific users, such as low-income single-parent families, agricultural areas. This stage is concerned with the creation of public policies, not with their impact and outcomes. No other model addresses this concept.

F. Reciprocating Translation
Using a common frame of reference identified in Section E, we translate the models into one another. Seven concepts Presenting information, Interaction, Transaction, Integration, Transformation, e-Governance, and Policy-Driven E-Governance. In addition to the concepts, models are analyzed in two separate perspectives, the operational/technological and citizen/service. Table II presents the translation of each of the 11 models into the meta-concepts/stages identified in Section E. When stage concepts match, the name of the stage appears on the column. Dark grey cells indicate that the concept does not exist in the model. Light grey columns ho ha he model stage spans and encompasses several concepts.

G. Synthesizing Translation
All models provided for, Presenting information online, the first stage metaphor of e-government development. The initial stage "Information" relates both to the citizen/service and technology/operation perspectives. The interaction and transaction stages are more related to the citizen/service perspective. In contrast, integration and transformation are more related to the operation/technology perspective. This stage focuses on the supply-side and the operation and technology perspectives.
The Interaction stage metaphor exists in all surveyed models. It represents the replication online of processes and services offered in the real world. The concept of integration refers to the integration of interfaces, services, and data (both horizontal and vertical integration). Some models, Shahkooh et al.; Kim and Grant; Sandoval-Almazan and Gil-Garcia; and Iannacci et al. consider integration as high (four or above) maturity level, while others, Persson and Goldkuhl; Lee; Almuftah et al.; and Janowski, place the concept in lower levels. The concept of interaction, such as two-way communication, exists in all models this stage is service and technology-oriented and concentrates on supplying egovernment.
The Transaction stage metaphor is about the restructuring of government business processes to take advantage of information systems and technologies, and the reformation of how governments conduct business with citizens. Siau and Long model doe no addre he opera ion/ echnolog dimension (streamlining) of the Reforming stage metaphor. However, all models contemplate the citizen/service dimension. At this stage, the political and administrative processes and services are reformed. The focus is on providing services online efficiently and improving internal operations. This stage is not concerned with the outcomes or impact or usefulness of the provided e-services.
At the Transformation stage metaphor, the political and administrative processes and services are reshaped for improved effectiveness. This stage refers to the changes in the shape and scope of processes. The government business model goes through a profound transformation, as the focus moves from the operationalization of services to planning and de eloping ne er ice for ci i en benefi . In he citizen/service dimension, citizens can conduct more than simple transactions online. They can participate through surveys and pools actively. Transformation is an explicit concept in Siau and Long; Iannacci et al., and Janowski models. The concept also exists with different names in other models, Persson and Goldkuhl; Lee; and Nielsen. The concept of participation, although not explicit in most models, occurs in all of them. At this stage, there is a concern to pro ide/ ppl benefi hro gh e-government.
The last stage metaphor of e-government development is e-Governance. In Lee model, as the government starts its transformation into e-government, it uses the full capability of innovative information and communication technologies to reconfigure business processes of administrative and political services on the fly to support citizens' involvement in the government's decision-making processes. All models address the concept of involvement; however, the process management concept is only covered to a certain degree. The reconfiguration of processes in real-time was not found in any model. We consider the concept Contextualization or Policy-Driven E-Governance, identified in Section E, as part of the e-Governance metaphor. Thus, stages 6 and 7 merged. This stage is concerned with citizen involvement in politics and policymaking. E-government provides the means for involvement, but the use or usefulness of these initiatives are not assessed. Eleven meta-models and their stages were compared and contrasted using a qualitative meta-synthesis. The name and description of the stages were used in a mapping exercise to identify underlying perspectives, metaphors, and concepts. S age imilari ie and difference ere e plored. The method used to create the model, the year of publication, and influence were also surveyed. Researchers [8] [10] established that perceived usefulness is a determinant factor for the adoption of e-government. However, we did not identify any concept related to usefulness or actual use in any of the e-government maturity models surveyed. New stages could be added to the frame of reference to account for use and usefulness, and the provision of services not yet developed.
Qualitative meta-synthesis was the technique used to develop seven meta-models. Two models -Janowski, and Persson and Goldkuhl -did not derive their models from metasyntheses. Two studies did not propose new models Iannacci et al. surveyed early models and adop ed Da i on framework [21]. While Nielsen conducted the meta-synthesis exercise using Lee's [6] frame of reference.
Meta-models were published between 2005 and 2019, and the majority is based on early models, especially on those developed between 1999 and 2003. Basic metaphors and concepts such as presenting information, transaction, and transformation were defined two decades ago and are still being used to measure the level of maturity of e-government initiatives. Several meta-models [6], [29], [31] are based not only on early models but also on other meta-models. Despite the apparent differences, metaphors, concepts, and perspectives extracted in most meta-models seem to be the same. This finding agrees with other authors [6], [22]. All meta-models are concerned with the operation/technology and services perspectives. They have several stages covering both perspectives and only one stage addressing e-governance and citizen participation. Their focus seems to be on the use of ICT as the means to achieve government reform and transformation. Meta-models measure the supply-side, i.e., the existence and level of sophistication of e-services offered on data portals. However, they do not measure the impact and outcomes of e-government initiatives, or the use or usefulness of e-services. Our findings align with other authors [22], [33].
Although the provision of e-government has evolved, none of the maturity models address actual use, as they are only technology and supply oriented [6], [38], [22]. This result deeply contrasts with public administration reform research, be it focused on the new public management efficiency or on the joined-up government approach (JUG) [39] [43]. It also contrasts with the current view of international organizations in that the value-added of e-government can only be realized through the actual and effective use of provided services [44] [46]. This study was limited by the number of meta-models surveyed and by the bias introduced by recursively searching citations for new models.

VI. CONCLUSIONS AND FUTURE RESEARCH
Qualitative meta-synthesis is a well-established technique among researchers for creating e-government stage maturity models. All surveyed meta-models address the e-governance and e-participation concepts that were lacking in some of the early models. E-government stage maturity models compare the state of e-government maturity in different countries at a certain point in time, thus fail to depict the evolution of egovernment. Despite the significant number of e-government stage maturity models identified in the literature, their underlying themes, metaphors, and concepts are similar. Models treat the technological, managerial, organizational, and political perspectives in a disjoint manner. The operation/technology perspective dominates the scene as the use of ICT is considered the enabler of e-government reform and transformation. These models do not address the actual use and usefulness of e-government. Future research will investigate if other types of e-government maturity models and benchmarks assess the impact, outcomes, use, and usefulness of e-government.