Smart Cities — Knowledge Trustworthiness Evaluation Framework for City Governance
1 Scope
This standard gives a knowledge trustworthiness evaluation framework for city governance application scenarios in smart cities, including knowledge trustworthiness needs analysis, characteristics identification, characteristics measurement, and knowledge trustworthiness estimate and feedback.
This standard applies to the evaluation of the trustworthiness of knowledge oriented towards city governance application scenarios in a smart city, in order to filter the trustworthy knowledge that meets the requirements of the user.
2 Normative references documents
This standard has no normative reference documents.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
knowledge
Cognition, judgement or skill acquired through study, practice or exploration.
NOTE1 Knowledge may be explicit or implicit; organizational or personal.
NOTE2 Knowledge can include factual knowledge, knowledge of principles, knowledge of skills and interpersonal knowledge.
NOTE3 Knowledge is ‘edited’ information that can bring real value to an organization after meaningful environmental analysis. It is the power of knowledge that is implicit in proprietary technologies, successful products and effective strategies. The collection of organizational knowledge (accumulated experience, staff, management skills, operating practices, technology applications, strategic partner and supplier relationships, customer and market intelligence) is its intellectual capital.
NOTE4 Data is a record of objective facts, information is processed data, a meaningful interpretation of data, and knowledge is the in-depth understanding and application of information.
[Source: GB/T 23703.2-2010, 2.1, modified]
entity
Concrete or abstract things in the domain under consideration.
[Source: ISO 8000-2:2022, 3.3.3]
model
An abstract representation of systems or other things and processes.
[Source: GB/T 36332-2018, 3.6]
knowledge model
Structured specification of facts that are true for a given domain.
[Source: ISO 18308:2011, 3.38]
trustworthiness
Ability to meet stakeholders' expectations in a verifiable way.
NOTE1 Depending on the context or sector, and also on the specific product or service, data, technology and process used, different trustworthiness characteristics apply and need verification to ensure stakeholders' expectations are met.
NOTE2 Trustworthiness characteristics include, but are not limited to, source reliability, correctness, timeliness, security and privacy, integrity, relevance, consistency, interpretability, accountability, controllability, stability, and universality.
NOTE3 Verifiability includes verifying the correctness of knowledge through the measurability and demonstrability of objective evidence, e.g., by comparing it with the actual situation, conducting experimental validation, inviting experts to review it, and other measurable or demonstrable means.
NOTE4 The evaluation of the degree of trustworthiness includes the identification of trustworthiness characteristics, analysis, and estimate of the degree of trustworthiness of the results of the analysis based on relevant criteria.
[Source: ISO/IEC TS 5723:2022, 3.1.1, modified]
4 Knowledge trustworthiness evaluation framework
Knowledge trustworthiness evaluation framework for city governance includes four units: knowledge trustworthiness needs analysis, knowledge trustworthiness characteristics identification, knowledge trustworthiness characteristics measurement, and knowledge trustworthiness estimate and feedback, see Figure 1.
The knowledge trustworthiness evaluation framework mainly includes:
a) knowledge trustworthiness needs analysis: Determine the scope of the knowledge application scenarios in the city governance scenario, identify stakeholders, determine the tasks and activities of stakeholders, determine the requirements of the knowledge users, and give the criteria of the acceptable degree of knowledge dependability as the basis for the identification of knowledge dependability characteristics, the measurement of the characteristics, and the estimate of the results in the city governance scenario;
b) knowledge trustworthiness characteristics identification: Identify the knowledge trustworthiness characteristics that are of concern to users in city governance knowledge application scenarios, and describe the characteristics;
c) knowledge trustworthiness characteristics measurement: For the identified knowledge trustworthiness characteristics, choose qualitative, quantitative or a combination of the two measurement methods, and collect relevant data at each stage of the knowledge life cycle for the measurement of each trustworthiness characteristic;
d) knowledge trustworthiness estimate and feedback: Based on the user's needs for knowledge trustworthiness, the criteria for acceptable knowledge trustworthiness, and the measurement results of each trustworthiness characteristic, the degree of knowledge trustworthiness in specific application scenarios is estimated, and the whole process of knowledge trustworthiness evaluation is adjusted and optimized through the feedback mechanism.
See Annex A and Annex B for examples of applications of the knowledge trustworthiness evaluation framework for city governance.
Figure 1-Knowledge trustworthiness evaluation framework for city governance
5 Knowledge trustworthiness needs analysis
Knowledge trustworthiness needs analysis includes, but is not limited to:
a) determine the scope of knowledge application scenarios in the city governance domain, such as the objectives and business processes of scenarios such as public safety, environmental protection, and traffic management;
b) identify the stakeholders in the application scenarios, including the evaluation subject (e.g., knowledge users, third-party evaluation organizations), and the evaluation object (mainly referring to the knowledge in the scenarios);
c) identify stakeholder tasks and activities, including but not limited to:
1) identify the tasks to be accomplished by the subject of the evaluation, the activities to be carried out to accomplish the tasks, and put forward the requirements for the business function requirements for knowledge and the quality of knowledge;
2) identify the life cycle stages of knowledge acquisition, knowledge reasoning, knowledge verification, etc. involved in the evaluation object, the resources available for each stage, and the quality assurance measures;
d) criteria for determining the level of trustworthiness acceptable to users of knowledge.
NOTE The criteria for the acceptable level of trustworthiness for knowledge users vary in different scenarios of city governance, such as authoritative and reliable source of knowledge; can be updated regularly as required; covers more than 90% of the knowledge required for the application scenario; no knowledge irrelevant to the scenario; semantic accuracy of knowledge; access to external knowledge associations that are closely related to the application scenario; and anonymization of personal information in the knowledge.
A single trustworthiness characteristic reflects the trustworthiness attributes of a particular aspect of knowledge in supporting various activities of city governance, and different trustworthiness characteristics describe the trustworthiness attributes of knowledge entities, relationships or knowledge models in different dimensions. The subject of knowledge trustworthiness evaluation defines each trustworthiness characteristic of a knowledge entity and its relationships based on the criteria of an acceptable level of trustworthiness according to the application scenarios and user requirements of knowledge in city governance:
Knowledge trustworthiness characteristics include, but are not limited to:
a) source reliability, the degree of reliability of the source of knowledge, both the direct source of knowledge and the reliability of the data source from which the knowledge is acquired;
NOTE1 Reliability of the direct source of knowledge generally refers to the reliability of the channel of knowledge acquisition, e.g. whether the knowledge comes directly from policies and regulations issued by government departments, research reports of professional organizations, and the experience feedback of frontline staff in city management.
NOTE2 The reliability of data sources for acquiring knowledge generally refers to whether the data are acquired objectively and comprehensively, including aspects such as the scientific nature of the data collection and investigation methods, the rigour of the data processing and analysis process, and the degree of transparency of the data sources, e.g., whether the data collected by the city's sensor network are rigorously calibrated and verified.
b) correctness, the degree to which knowledge correctly represents real-world facts;
c) timeliness, the extent to which knowledge is valid in a specific time frame;
NOTE Timeliness includes not only the freshness of knowledge, whether it is in line with the current actual situation, but also the change in the value of knowledge at different points in time. For example, when considering the timeliness of traffic flow data in the city, as traffic conditions change all the time, only the latest traffic flow data can provide an accurate basis for city traffic management decisions, and it is possible to lead to traffic congestion or inefficiency if outdated traffic flow data are used for planning traffic routes or adjusting signal timing.
d) security and privacy, where knowledge models and acquired knowledge are not tampered with, ensuring that private data is protected;
e) completeness, the degree to which knowledge covers domain information, also known as comprehensiveness or coverage;
f) relevance, the degree to which the knowledge is relevant to the domain tasks;
NOTE In city governance scenarios, the degree of relevance of knowledge to domain tasks determines its application value. For example, knowledge about air and water pollution is highly relevant to the tasks when formulating environmental protection policies for the city, while knowledge about the preservation of historical and cultural heritage and the planning of cultural activities is of great importance in the cultural construction of the city.
g) consistency, the definition, representation, and description of knowledge are consistent and not contradictory;
NOTE Cross-departmental and cross-system city governance scenarios can have different definitions and representations of the same knowledge, as well as different descriptions of knowledge content and actual system implementations, all leading to knowledge inconsistencies.
h) interpretability, whether the knowledge is represented using appropriate notation whether the machine can process the knowledge and whether the semantics are clear.
In addition to the above trustworthiness characteristics, the subject of the knowledge trustworthiness evaluation may choose to use the trustworthiness characteristics given in NOTE 2 of 3.5 or other trustworthiness characteristics not listed, depending on the application scenario and needs of the knowledge. For example, in the ‘One network unified management’intelligent order dispatching scenario, the evaluation subject mainly identifies the reliability of the source of knowledge, correctness, timeliness, completeness and other characteristics, specifically, see A.3, in the intelligent government customer service system knowledge trustworthiness evaluation scenario, the evaluation subject mainly identifies the reliability of the source of knowledge, correctness, timeliness, completeness, consistency and other characteristics, see B.3 for details.
Foreword i Introduction ii 1 Scope 2 Normative references documents 3 Terms and defnitions 4 Knowledge Trustworthiness Evaluation Framework 5 Knowledge trustworthiness requirements analysis 6 Identification of Knowledge Trustworthiness Characteristics 7 Knowledge trustworthiness characteristics measurement 7.1 General 7.2 Select measurement method 7.3 Collect relevant data 8 Estimating and feedback on knowledge trustworthiness Appendixes A （informative） "One-network unified management" intelligent ticket dispatching knowledge trustworthiness evaluation example A.1 overivew A.2 Requirement analysis for knowledge trustworthiness A.3 Identification of knowledge trustworthiness characteristics A.4 Knowledge trustworthiness characteristics measurement A.5 Estimating and feedback on knowledge trustworthiness Appendixes B (informative) Example of knowledge trustworthiness evaluation of the intelligent customer system for government service B.1 Overview B.2 Analysis of knowledge trustworthiness requirements B.3 Identification of knowledge trustworthiness characteristic B.4 Measurement of Knowledge Trustworthiness Characteristic B.5 Estimate on knowledge trustworthiness Bibliography
Smart Cities — Knowledge Trustworthiness Evaluation Framework for City Governance
1 Scope
This standard gives a knowledge trustworthiness evaluation framework for city governance application scenarios in smart cities, including knowledge trustworthiness needs analysis, characteristics identification, characteristics measurement, and knowledge trustworthiness estimate and feedback.
This standard applies to the evaluation of the trustworthiness of knowledge oriented towards city governance application scenarios in a smart city, in order to filter the trustworthy knowledge that meets the requirements of the user.
2 Normative references documents
This standard has no normative reference documents.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
knowledge
Cognition, judgement or skill acquired through study, practice or exploration.
NOTE1 Knowledge may be explicit or implicit; organizational or personal.
NOTE2 Knowledge can include factual knowledge, knowledge of principles, knowledge of skills and interpersonal knowledge.
NOTE3 Knowledge is ‘edited’ information that can bring real value to an organization after meaningful environmental analysis. It is the power of knowledge that is implicit in proprietary technologies, successful products and effective strategies. The collection of organizational knowledge (accumulated experience, staff, management skills, operating practices, technology applications, strategic partner and supplier relationships, customer and market intelligence) is its intellectual capital.
NOTE4 Data is a record of objective facts, information is processed data, a meaningful interpretation of data, and knowledge is the in-depth understanding and application of information.
[Source: GB/T 23703.2-2010, 2.1, modified]
entity
Concrete or abstract things in the domain under consideration.
[Source: ISO 8000-2:2022, 3.3.3]
model
An abstract representation of systems or other things and processes.
[Source: GB/T 36332-2018, 3.6]
knowledge model
Structured specification of facts that are true for a given domain.
[Source: ISO 18308:2011, 3.38]
trustworthiness
Ability to meet stakeholders' expectations in a verifiable way.
NOTE1 Depending on the context or sector, and also on the specific product or service, data, technology and process used, different trustworthiness characteristics apply and need verification to ensure stakeholders' expectations are met.
NOTE2 Trustworthiness characteristics include, but are not limited to, source reliability, correctness, timeliness, security and privacy, integrity, relevance, consistency, interpretability, accountability, controllability, stability, and universality.
NOTE3 Verifiability includes verifying the correctness of knowledge through the measurability and demonstrability of objective evidence, e.g., by comparing it with the actual situation, conducting experimental validation, inviting experts to review it, and other measurable or demonstrable means.
NOTE4 The evaluation of the degree of trustworthiness includes the identification of trustworthiness characteristics, analysis, and estimate of the degree of trustworthiness of the results of the analysis based on relevant criteria.
[Source: ISO/IEC TS 5723:2022, 3.1.1, modified]
4 Knowledge trustworthiness evaluation framework
Knowledge trustworthiness evaluation framework for city governance includes four units: knowledge trustworthiness needs analysis, knowledge trustworthiness characteristics identification, knowledge trustworthiness characteristics measurement, and knowledge trustworthiness estimate and feedback, see Figure 1.
The knowledge trustworthiness evaluation framework mainly includes:
a) knowledge trustworthiness needs analysis: Determine the scope of the knowledge application scenarios in the city governance scenario, identify stakeholders, determine the tasks and activities of stakeholders, determine the requirements of the knowledge users, and give the criteria of the acceptable degree of knowledge dependability as the basis for the identification of knowledge dependability characteristics, the measurement of the characteristics, and the estimate of the results in the city governance scenario;
b) knowledge trustworthiness characteristics identification: Identify the knowledge trustworthiness characteristics that are of concern to users in city governance knowledge application scenarios, and describe the characteristics;
c) knowledge trustworthiness characteristics measurement: For the identified knowledge trustworthiness characteristics, choose qualitative, quantitative or a combination of the two measurement methods, and collect relevant data at each stage of the knowledge life cycle for the measurement of each trustworthiness characteristic;
d) knowledge trustworthiness estimate and feedback: Based on the user's needs for knowledge trustworthiness, the criteria for acceptable knowledge trustworthiness, and the measurement results of each trustworthiness characteristic, the degree of knowledge trustworthiness in specific application scenarios is estimated, and the whole process of knowledge trustworthiness evaluation is adjusted and optimized through the feedback mechanism.
See Annex A and Annex B for examples of applications of the knowledge trustworthiness evaluation framework for city governance.
Figure 1-Knowledge trustworthiness evaluation framework for city governance
5 Knowledge trustworthiness needs analysis
Knowledge trustworthiness needs analysis includes, but is not limited to:
a) determine the scope of knowledge application scenarios in the city governance domain, such as the objectives and business processes of scenarios such as public safety, environmental protection, and traffic management;
b) identify the stakeholders in the application scenarios, including the evaluation subject (e.g., knowledge users, third-party evaluation organizations), and the evaluation object (mainly referring to the knowledge in the scenarios);
c) identify stakeholder tasks and activities, including but not limited to:
1) identify the tasks to be accomplished by the subject of the evaluation, the activities to be carried out to accomplish the tasks, and put forward the requirements for the business function requirements for knowledge and the quality of knowledge;
2) identify the life cycle stages of knowledge acquisition, knowledge reasoning, knowledge verification, etc. involved in the evaluation object, the resources available for each stage, and the quality assurance measures;
d) criteria for determining the level of trustworthiness acceptable to users of knowledge.
NOTE The criteria for the acceptable level of trustworthiness for knowledge users vary in different scenarios of city governance, such as authoritative and reliable source of knowledge; can be updated regularly as required; covers more than 90% of the knowledge required for the application scenario; no knowledge irrelevant to the scenario; semantic accuracy of knowledge; access to external knowledge associations that are closely related to the application scenario; and anonymization of personal information in the knowledge.
6 Knowledge trustworthiness characteristics identification
A single trustworthiness characteristic reflects the trustworthiness attributes of a particular aspect of knowledge in supporting various activities of city governance, and different trustworthiness characteristics describe the trustworthiness attributes of knowledge entities, relationships or knowledge models in different dimensions. The subject of knowledge trustworthiness evaluation defines each trustworthiness characteristic of a knowledge entity and its relationships based on the criteria of an acceptable level of trustworthiness according to the application scenarios and user requirements of knowledge in city governance:
Knowledge trustworthiness characteristics include, but are not limited to:
a) source reliability, the degree of reliability of the source of knowledge, both the direct source of knowledge and the reliability of the data source from which the knowledge is acquired;
NOTE1 Reliability of the direct source of knowledge generally refers to the reliability of the channel of knowledge acquisition, e.g. whether the knowledge comes directly from policies and regulations issued by government departments, research reports of professional organizations, and the experience feedback of frontline staff in city management.
NOTE2 The reliability of data sources for acquiring knowledge generally refers to whether the data are acquired objectively and comprehensively, including aspects such as the scientific nature of the data collection and investigation methods, the rigour of the data processing and analysis process, and the degree of transparency of the data sources, e.g., whether the data collected by the city's sensor network are rigorously calibrated and verified.
b) correctness, the degree to which knowledge correctly represents real-world facts;
c) timeliness, the extent to which knowledge is valid in a specific time frame;
NOTE Timeliness includes not only the freshness of knowledge, whether it is in line with the current actual situation, but also the change in the value of knowledge at different points in time. For example, when considering the timeliness of traffic flow data in the city, as traffic conditions change all the time, only the latest traffic flow data can provide an accurate basis for city traffic management decisions, and it is possible to lead to traffic congestion or inefficiency if outdated traffic flow data are used for planning traffic routes or adjusting signal timing.
d) security and privacy, where knowledge models and acquired knowledge are not tampered with, ensuring that private data is protected;
e) completeness, the degree to which knowledge covers domain information, also known as comprehensiveness or coverage;
f) relevance, the degree to which the knowledge is relevant to the domain tasks;
NOTE In city governance scenarios, the degree of relevance of knowledge to domain tasks determines its application value. For example, knowledge about air and water pollution is highly relevant to the tasks when formulating environmental protection policies for the city, while knowledge about the preservation of historical and cultural heritage and the planning of cultural activities is of great importance in the cultural construction of the city.
g) consistency, the definition, representation, and description of knowledge are consistent and not contradictory;
NOTE Cross-departmental and cross-system city governance scenarios can have different definitions and representations of the same knowledge, as well as different descriptions of knowledge content and actual system implementations, all leading to knowledge inconsistencies.
h) interpretability, whether the knowledge is represented using appropriate notation whether the machine can process the knowledge and whether the semantics are clear.
In addition to the above trustworthiness characteristics, the subject of the knowledge trustworthiness evaluation may choose to use the trustworthiness characteristics given in NOTE 2 of 3.5 or other trustworthiness characteristics not listed, depending on the application scenario and needs of the knowledge. For example, in the ‘One network unified management’intelligent order dispatching scenario, the evaluation subject mainly identifies the reliability of the source of knowledge, correctness, timeliness, completeness and other characteristics, specifically, see A.3, in the intelligent government customer service system knowledge trustworthiness evaluation scenario, the evaluation subject mainly identifies the reliability of the source of knowledge, correctness, timeliness, completeness, consistency and other characteristics, see B.3 for details.
Contents of GB/T 45032-2024
Foreword i
Introduction ii
1 Scope
2 Normative references documents
3 Terms and defnitions
4 Knowledge Trustworthiness Evaluation Framework
5 Knowledge trustworthiness requirements analysis
6 Identification of Knowledge Trustworthiness Characteristics
7 Knowledge trustworthiness characteristics measurement
7.1 General
7.2 Select measurement method
7.3 Collect relevant data
8 Estimating and feedback on knowledge trustworthiness
Appendixes A （informative） "One-network unified management" intelligent ticket dispatching knowledge trustworthiness evaluation example
A.1 overivew
A.2 Requirement analysis for knowledge trustworthiness
A.3 Identification of knowledge trustworthiness characteristics
A.4 Knowledge trustworthiness characteristics measurement
A.5 Estimating and feedback on knowledge trustworthiness
Appendixes B (informative) Example of knowledge trustworthiness evaluation of the intelligent customer system for government service
B.1 Overview
B.2 Analysis of knowledge trustworthiness requirements
B.3 Identification of knowledge trustworthiness characteristic
B.4 Measurement of Knowledge Trustworthiness Characteristic
B.5 Estimate on knowledge trustworthiness
Bibliography
