Value-Based Reasoning in Autonomous Agents
, Michail Mokkas2, - DOI
- 10.2991/ijcis.d.210203.001How to use a DOI?
- Keywords
- Reasoning; Model; Goals; Values; Expert systems; Autonomous agents
- Abstract
The issue of decision-making of autonomous agents constitutes the current work topic for many researchers. In this paper we propose to extend the existing model of value-based teleological reasoning by a new, numerical manner of representation of the level of value promotion. The authors of the paper present and discuss proofs of compatibility of both previous and current models, a formal mechanism of conversion of the parameters of the autonomous device into the levels of promotion of values, the mechanism of integration with machine learning approaches, and a comprehensive argumentation-based reasoning mechanism allowing for making decisions.
- Copyright
- © 2021 The Authors. Published by Atlantis Press B.V.
- Open Access
- This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).
1. INTRODUCTION
A distinctive feature of modern technical devices is their increasing self-sufficiency. One may presume that in the future the user will only formulate the most rudimentary rules of how a device works (or goals), whereas the everyday aspects of system operation will be regulated completely autonomously. The most advanced types of such devices are self-driving vehicles, where the user merely sets the journey's destination and the vehicle develops the itinerary on its own and makes hundreds of traffic-related decisions.
The issue of decision-making by autonomous agents has been a research topic of a number of researchers representing various fields of science [1]. Typically, it is perceived as the selection of the best possible action which should be performed. Classical decision theory, developed mostly by economists, relies on the concept of utility, where utility is a way of representing the agent's preferences. Utility theory can be combined with probability theory to build the principle of maximum expected utility (PMEU) which states that an agent is rational if and only if it chooses the action that yields the higher expected utility, averaged over all the possible outcomes of the action [2,3]. In early 2000s an alternative approach to classical decision theory emerged where the basis for modeling a decision-making situation is the logic-based representation of human reasoning and argumentation and the so-called practical reasoning, that is reasoning regarding the action which should be performed by the agent (discussed from a various points of view: [4–8], and others, where, e.g. [9,10], present it in a general discussion of argumentation). In this work we intend to focus on the latter approach where decisions are made on the grounds of a model of human reasoning and argumentation.
In major decision-making models it is usually assumed that the purpose of system operation is to accomplish some state of affairs pre-declared by the user [11–13] (and many others). Many authors (including [14,15]) believe, however, that increasingly complex devices must be endowed with the ability to not only find the best possible way to attain the state of affairs set by the user, but also the ability to set their own goals themselves; the model proposed in this paper is based on this assumption as we believe that the increasing autonomy of devices calls for the realization of systems able to self-sufficiently set themselves a goal and take measures to accomplish it.
Zurek [14] presents a framework allowing for autonomous (based on values which should be promoted) goal-setting by a device. The model relies on the differentiation between several kinds of goals: abstract (i.e., minimal levels to which values should be promoted) and material (i.e., particular states of affairs which fulfil abstract goals).
The objective of this paper consists in proposing for the model from [14] a new method of the representation of the levels to which particular decision options promote various values as well as modification of the reasoning mechanism allowing for autonomous setting and realization of goals. This study is an expansion of the research results presented during the FedCSIS 2017 conference and further discussed in [16]. The proposed mechanism may serve as a formal foundation for the realization of the autonomous agent and allow for an integration of the logic-based reasoning mechanism discussed in [14] with machine learning approaches. A simple example of a partially autonomous mobile device illustrates how the model operates.
1.1. Contribution
This study is an expansion of the research results briefly discussed during the FedCSIS 2017 conference [16]. The paper is built on the background of our two previous papers [14,16]. Sections 3 and 7 briefly summarize paper [14], sections 4.1 and (partially) 4.2 summarize paper [16].
Our main contribution to the previous results can be summarized as follows:
We present and discuss proofs of compatibility of our model (basics of which were presented in [16]) with the general model presented in [14] as well as with rationality postulates (Sections 4.2 and 4.3);
We introduce an exemplary mechanism of conversion of the parameters of the autonomous device and its environment into the levels of promotion of values (Section 6);
We discuss the problem of the discrete and continuous decision space (Section 8);
We introduce and discuss the mechanism of integration of our model with machine learning approaches (Sections 9.1 and 9.2);
We introduce a general and comprehensive argumentation framework allowing for making value- and goal-based decisions (Section 10);
The model is illustrated by the example (Section 11).
The key point of the model from [14] was the possibility of making decision (establishing the desired state of affairs and performing the action leading to such a goal) even for the decisive situations which were not predicted in advance. This could be realized on the basis of the abstract goal (the minimal required level of promotion of values) allowing for the choice of the best practical goal (action leading to the desired state of affairs). The agent, in order to make such a decision, required the evaluation (in terms of the values from his abstract goal) of a possible decisions and the orderings of those evaluations (obtaining of which was an important disadvantage of this approach). Our new approach extends the previous one by introducing the possibility of computing of the levels of promotion of values and ordering between them.
2. THE MAIN IDEA
The framework presented in [14] (and summarized in the following) provides strong possibilities of modellng reasoning in autonomous systems; however, its practical implementation requires the solution of a number of major problems. In our opinion, a serious disadvantage of the system is the requirement of declaration of a great number of orders indicating the relations between levels to which various situations promote various values and sets of values. This task proves simple only in the case of decision support systems which contain a relatively low number of decision options and values. Unfortunately, for the majority of real decision-making situations, these sets are too large. We find it indispensable to extend the model (1) by a mechanism allowing for an automatic evaluation of the levels to which given values are promoted by various situations (2) and by a mechanism which would determine the cumulative evaluation of the promotion of various values. The key assumption of our work is the idea that the estimation of the levels of promotion of values by the set of possible actions and state of affairs is easier to obtain than a declaration of orderings between them (in Zurek [14] we show that the number of orderings can be significantly greater than the number of all values, actions, and states of affairs).
We plan to attain our goal by introducing a numeric representation of the levels of value promotion. Such a method of representation of value levels (for single values as well as for value sets) will facilitate their comparison, leading to easier and quicker (searching order sets
Some comments: Many real-life decision situations are connected with a huge number of options to choose. Since the computer cannot distinguish between “reasonable” and “non-reasonable” options, in most decision systems we have to analyze all possibilities. Since their number can be high, such analysis can be a serious challenge (especially because the analysis requires detailed knowledge about all the options, relations between them and because the result of the analysis has a large impact on the quality of the decision). This is a problem which exists in most (if not all) formal decision-making mechanisms. If we look at the decision-making mechanisms from a very general point of view we can distinguish two main approaches to collect knowledge necessary to make the decision:
Introduce the knowledge directly to the system (knowledge-based approaches);
Train the system by the examples of previous decisions (machine learning-based approaches).
Both approaches have their own advantages and disadvantages: a knowledge-based systems require preparation and formulation of the entire necessary knowledge, but they allow for full control over the decision-making process (and can justify the decision). The machine learning-based approaches require an extensive set of examples to train the system (the quality of the decision depends on the size and quality of a training set) and, since most of them work as a black-box, they cannot justify their decisions, which may exclude such a system from many decision areas (e.g., in the legal matter, which usually requires justification of the decision).
In order to illustrate the problem, we present the knowledge requirements of some models of knowledge-based decision-making models: in the classical PMEU model [2,3], there is necessity of declaration (or computation) of the level of utility and assignment of the level of probability to every option. The AATS+V model initially [7] required declaration of the promotion or demotion of each value for every possible transition between states of affairs (note that there can be much more transitions than states) but in the later works ([21] and others) it allows for calculation of promotion or demotion. In the later versions, the model allowed for the calculation not only the promotion or demotion, but also of the levels of promotion and demotion. In Amgoud et al. [22] there is a necessity of declaration of the orders between all possible options (which represents the relation of preference). Additionally, the preference relations between arguments pro and con each option is computed on the basis of the strength of the knowledge supporting each of the arguments (which also should be declared in advance). In the model of practical reasoning presented in [11] states of affairs (decision options) require declaration of the list of values supporting and demoting each of the states of affairs as well as preorders on states representing the preferences of the agents which allows for analysis of the value system and decision-making (the choice of a state of affairs). In the popular BDI model there is also the necessity of the analysis and comparison of all possible decision options.
In all the above examples, the amount of knowledge which should be declared is (similar to our approach) very high and the quality of the knowledge has a crucial character for the quality of decisions. The difference between our approach and the approaches presented above is in the kind of knowledge which should be introduced to the system. In our model we introduce the levels of promotion of particular values and weight functions instead of various orderings between states, actions, arguments, etc. There is an open problem: which kind of knowledge is easier to obtain? In our opinion, the answer to this question is not trivial and requires further research, especially regarding the possibility of utilisation of machine learning mechanisms to support obtaining such knowledge. In section 6 we introduce the general proposition on how machine learning mechanisms can help to estimate the level of promotion of particular values.
3. GVR MODEL
We will begin with a summarized discussion of the model of teleological reasoning from [14], further referred to as the GVR model.
Firstly, the naming convention will be presented:
By upper case letters we denote sets;
By lower case letters we denote propositions;
Subscripts denote names of propositions;
Superscripts denote names of sets;
Greek letters denote functions;
Other symbols will be defined later (except trivial logical and set-theory ones).
Definition 1.
[State of affairs] Let
Definition 2.
[Actions] As an action we understand an activity which carries a transition from a certain state of affairs to another state of affairs. Actions will be represented by propositions from set
It is worth noticing that a particular action cannot be performed in every state of affairs. The set of all possible actions in all possible states of affairs we denote as
Function
Example 1.
[Running example] John is in a bar and he is thirsty (state of affairs
Definition 3.
[Transition process] Let
Example 2.
[Running example, cont.] If John, being thirsty (
Definition 4.
[Situation] By a situation
Example 3.
[Running example, cont.] In our case, the set of situations
Definition 5.
[Values] Following Rohan [23], we have to separate the two meanings of the word value: a value may be understood as a concept or as a process.
Value as an abstract concept which allows for the estimation of a particular action or a state of affairs and influences one's behavior.
Value as a process of estimation of the level of extent to which a particular situation (state of affairs and / or action)
It is important to emphasize that values can be promoted to a certain degree by a particular state of affairs or action:
Example 4.
[Running example, cont.] We introduce two values: happiness (
The levels of promotion of those values by both available situations (
By
Example 5.
[Running example, cont.] For value
For value
A partial order
Example 6.
[Running example, cont.] We assume that:
which means that staying thirsty promotes frugality to a greater extent than drinking lager. In real-life reasoning people do not rely only on a comparison of the levels of promotion of one value; usually, they compare the levels of promotion of various values. Theoretically speaking, they are incompatible, but practically, people compare not only the levels of promotion of various values, but also the levels of promotion of various sets of values.
Definition 6.
[Sets of values] By
By
Definition 7.
[The level of promotion of a set of values] By
By
Example 7.
[Running example, cont.] Since both available situations in our example promote the same values (but to different extents), then
Definition 8.
[Value-extent preference] A partial order
Relation
If
Example 8.
[Running example, cont.] By
Note that John prefers beer even though frugality is promoted to the lower extent than it is in staying thirsty. It reflects a case in which someone have to balance and accept decision in which some values are promoted to the maximal at the expense of other values.
How to determine whether the extents to which all values promoted by one situation are preferred to the extents to which all values are promoted by another situation? In the simplest case, we can assume some properties connecting orders
By
By
By
By
For the sake of consistency, I also assume that the order declarations are rational, i.e., there are no declarations leading to situations in which
Then we formulate relations between orders
The first and basic mechanism connecting orders
Example 9.
[Running example, cont.] This can be illustrated by a simple example:
Since
Because drinking lager promotes
The next one allows to derive preferences between the sets of values promoted by given situations: if every value promoted by a situation
Example 10.
[Running example, cont.] In order to illustrate the above we have to add a new value: social relations (
If we assume that there are only 2 values in set
Since both values in
If a set of values promoted by a situation
Example 11.
[Running example, cont.] In order to illustrate the above we assume that drinking lager promotes social relations and happiness (
On the basis of the above assumptions and formula 3 we can say that
The above can simply justified: Since social relations are promoted to the equal level by both situations and drinking beer is preferred to staying thirsty in terms of happiness and frugality, then the agent prefers drinking lager.
In any other case, it is impossible to derive a preference order.
As we have noticed earlier, orders from set
3.1. Goals
As it has already been noticed, we need to draw a distinction between a few different kinds of goals [14]:
Definition 9.
[Abstract goals] Abstract goals are goals represented by the minimal extents to which a particular situation promotes a given set of values (in our model the valuation of any target state of affairs can be considered only with the valuation of the action which leads to it, hence by a situation we will understand a particular action performed in a particular state of affairs):
By
By
By
Definition 10.
[Abstract unreachable goals] Abstract unreachable goals are abstract goals where the agent desires one value to be promoted as much as possible, while other values to be promoted no less than to a certain extent:
By
Function
Definition 11.
[Material goals] Material goals are particular situations which satisfy given abstract goals.
A material goal is a particular action performed in a given state of affairs where:
By
Definition 12.
[Practical goal] A practical goal is a material goal which is achievable, which satisfies the agent's abstract goal, and which the agent is going to reach. A practical goal will be denoted as
Example 12.
[Running example, cont.] Let's assume that John has one abstract goal
We have three decision options
which means that drinking lager (or two) promotes happiness to the satisfactory levels, while staying thirsty not.
which means that staying thirsty or drinking one lager promotes frugality to the satisfactory levels, while drinking 2 beers not (two beers are too expensive).
Since our agent has abstract goal
Let's assume that John has an unreachable abstract goal
3.2. Inference Mechanism
Zurek [14] discusses a number of inference rules as well as a simple argumentation framework which allows for reasoning about goals and making decisions concerning the fulfilment of these goals. Our paper includes a discussion of several inference rules (Section 7) and a slightly modified argumentation framework (Section 10). An exhaustive discussion of the inference rules' structure and properties is included in [14].
4. NUMERIC REPRESENTATION OF THE LEVELS OF VALUE PROMOTION BY PARTICULAR SITUATIONS
The underlying objective of this section is to discuss a new semantics (basics of which were presented in [16]) for the model described in [14], which would allow for a numeric representation of the levels to which given situations promote various values. This task consists of two main parts: first, a proposal and discussion of the numeric method of representation of the level to which given values are promoted; then the development of the mechanism of determining this level, of determining the cumulative evaluation of value sets, the mechanism of comparing them (equivalents of the
4.1. Numeric Representation of Levels to Which Various Situations Promote Particular Values
In the model presented in [14], any level to which a given value (e.g.,
Assuming a numeric representation of the level of promotion of values one cannot miss the fact that not all values are equally important to each user. The simplest solution would be to assign some weight to each value (from the range
Definition 13.
(Function of the weight [16]) Let
The most basic kind of function
If the level of promotion of the battery charge (
4.2. Cumulative Evaluation of the Level of Promotion of a Value Set by a Given Situation
In the model discussed in [14], a given situation may promote various values to various extents. It is represented by set
Since our model assumes that the grounds for evaluation are relative levels to which particular values are promoted by situations, in our model the correspondent of set
For the realization of value-based reasoning to be possible, it is indispensable to define the correspondent of order
Firstly, we assume that the cumulative evaluation of the level to which a given value set will be promoted by a given action will be a number from the range
Definition 14.
(Function Θ [16]) Let
If:
In case of a higher number of values in set
The main idea of the function
The cumulative evaluation of staying at home is
The cumulative evaluation of going to the party is
The cumulative evaluation of the values promoted by a party is higher than the cumulative evaluation of values promoted by staying at home (note that the basis of the evaluation are the relative levels of the values' promotion), which can be interpreted as the situation in which a particular agent prefers going to the party to staying at home. The concept of the function
The properties of function
The value returned by function
Function
As we have already noticed, values should not be treated equally (they are not all equally important), and therefore we proposed a set of weight functions
Definition 15.
(Value-extent-weight preference [16]) A total order
We assume that
Since function
If
If
4.3. Relations Between Orders O O R O
Zurek [14] (as well as this paper's sections 3, def. 8 and formulae 1, 2, 3) introduce the properties of order
Knowing the properties of the functions from the
Since order
The other feature of order
Furthermore, we will analyze whether the relations between order
The first relation will be fulfilled if
The second relation will be fulfilled if
The third relation will be introduced differently than in [14]. It will be fulfilled if
5. GOALS
The four kinds of goals defined in [14] (and discussed in Section 3.1 of this paper) remain the same. The only changes are caused by the fact that the level of promotion of values is expressed in numbers, and therefore the threshold values: (
Although we focused on the 4 main kinds of goals presented in [14], we do not exclude the possibility of adding other types of goals. For example, should a need arise for a more sophisticated kind of goal allowing for the balancing between the levels of promotion of values, we can introduce a new kind of abstract goal in which the threshold values (
6. DECISION SPACE
Similarly to other argumentation-based mechanisms of practical reasoning and decision-making, our model assumes a discrete space of available events. When making a decision, the agent is faced with a known number of available options. It is worth noticing, that In our model we assume that all actions are certain (i.e., the result of a given action is always certain), the problem of uncertain results of actions and the behavior of other agents, was discussed in [26]. Adding such a possibility to our model is possible, but requires further developments and will be discussed in the future works.
In the presented model, the decision-making options available for the state of affairs
Definition 16.
[Decision variables] Let
Example 13.
[Running example cont.] The above definition can be illustrated on an example. Let us assume that the following decision-making situation occurs: we are in a pub (state of affairs
If the number of decision variables and the number of values in the domain of each variable are high, then we can face the problem of combinatorial explosion of the number of options to choose (the number can be decreased because not all combinations of decision options are available, e.g., there is no
Definition 17.
Let
If action
The thus determined level of promotion can be used for further inference.
The above definition can be illustrated by an example:
Example 14.
[Running example, cont.] Let's assume that John have such decision options
Let's assume function
The above function shows that staying without drinking does not promote John's happiness, he prefers to drink more than less, but drinking more than 0.8 liter (or drinking unknown drink) promotes happiness to 0.3 only. John also prefers stout to lager. Drinking 0.5 liter of stout promotes his happiness to 0.6, while drinking 0.5 liter lager promotes happiness to 0.5. This example shows that in a new, unexpected, situation (e.g., there is a new menu in the pub in which beer is served in different glasses: 0.2 and 0.6 l.) John can assign the level of promotion of value happiness to the possible decisions.
Under real-world circumstances, it is not always possible to determine separate functions
Definition 18.
Let
Let us assume that
If action
The result of
For decision variables in which higher levels indicate a lower value's promotion we inverse the result of the normalization function:
Much more problems will appear if maximal or minimal values of a given decision variable are known partially (we can estimate
where
If
Alternatively, functions from the sets
7. INFERENCE RULES
In this work, inference rules will be presented in the form of argumentation schemes, which are forms of argument representing stereotypical patterns of human reasoning.
Argumentation schemes have an antecedent part and a consequence part separated by a double bar which stands for the sign of defeasible inference. Argumentation schemes in computational models are usually interpreted as defeasible inference rules and they constitute a part of a whole argumentation framework, providing a basis on which the process of reasoning is conducted (a detailed description of the framework is presented in Section 10).
The author of [14] introduces a number of such argumentation schemes allowing for the realization of value-based and teleological reasoning. The issue of argumentation schemes was widely discuss in many papers and there is a number of different views on their nature. The author of Prakken [27] presents an in-depth analysis of various approaches to argumentation schemes. From the most general point of view he divided them to the two approaches: the logical one, when argument is treated as a defensible inference rule and the dialogical approach in which arguments are dialogical devices. Since in our approach we do not introduce any dialog moves with shifting the burden of proof, we understand the argumentation schemes as a defeasible inference rule, but unlike it is discussed in the H. Prakken paper, our model is not purely logical because it contains some procedural elements (especially the function
Note that, unlike it is in standard approaches to argumentation schemes, we do not introduce critical questions (the ways of challenging of arguments). It was caused by purely technical character of our model (we do not model the human reasoning process, but we are going to create a reasoning framework for autonomous agents). The ways of challenging of arguments were implemented into reasoning framework.
Below are presented mechanisms from [14] and [16] which have been adapted to our reasoning model with a numeric representation of the level of value promotion (the names will correspond to the mechanisms from [14]). Since the AS1 scheme from [14] is very basic and is practically included in AS2, it will be omitted.
- AS2
Generalized practical reasoning: If in circumstances
In the above example, relation
- AS3
Reasoning with abstract goals: If in the current circumstances
- AS4
Reasoning with unreachable abstract goal: In the current circumstances
The abstract unreachable goal can be understand as a specific kind of abstract goal in which one value should be promoted to the maximal possibly extent. The idea is that the agent chooses the decision (action) which promotes this (key) value to the highest possible extent, while rest of the values are promoted at least to their satisfactory levels.
For the sake of clarity we explain the subsequent verses of the above scheme:
In the first verse, we assume that there exist an abstract unreachable goal, and the available action.
The second verse confirms the actual state of affairs.
Third and fourth verses examine if the actual state of affairs does not satisfy the goal (are the ordinary values promoted below the thresholds or exists available action which promote key value to the higher extent). If not, then
Fifth verse examine if exists available action (
The last three verses examine if there exists other available actions which promote the key value to the higher extent than
- AS5
Goal-driven practical reasoning In the current circumstances
Although we focused on the inference rules from [14], note that new inference rules can be added do the above list.
8. DECISION-MAKING AS AN OPTIMIZATION PROBLEM
One can notice that our model has some common elements with multi-attribute utility theory [28]. Obviously, the topic of MAUT was widely discussed and developed and it would be difficult to discuss here a relations between our approach and all versions and approaches to MAUT, but we will try to present a brief look on the similarities and differences between our model and the most basic approach to multi-attribute utility theory.
Similar to MAUT we evaluate possible decision options in the light of different values (in MAUT utilities) and then we compute the cumulative evaluation of a decision (in MAUT the aggregate utility function) which is the basis of the decision. The process of aggregation of utility in MAUT can be additive or multiplicative. In our work we use the
Unlike it is in MAUT, we do not use weights (as numbers), but we use weight functions which can be more complex and we do not implement the reasoning with uncertainty (but we plan to do it in the future works). The goal of MAUT is to find option with the highest aggregate utility function. In our work the goals of the model can be more complex, similar to MAUT we can maximize the cumulative evaluation of all values (such an approach can be achieved with the use of
Concluding, since there is a number of similarities (and some differences) between our model and MAUT, we believe that our approach can be seen as a bridge between a value-based decision-making and the utility theory.
The model presented in our paper was designed, above all, for situations involving a finite and discrete space of events wherein a finite
A considerably more complex situation is observed where at least one of the decision-making variables relates to an infinite set of values. If it is infinite but discrete, it is often possible to limit the range of variability for the given decision variable without prejudice to the quality of the decision. If the domain of at least one of the variables is continuous, the decision space will also be continuous or semi-continuous. In such a situation, the variable should be, if it is possible, discretized. If it is not possible, the argumentation-based approach fails: the number of possible decision-making options is infinite and no viable arguments for or against the respective options can be formulated. In this context, the most effective decision-making models are based not on argumentation, but on single- and multi-criterion optimization aimed at maximizing the target function. Although the mechanism presented herein was developed with a discrete event space in mind, its underlying premise can, to a certain extent, be also applied in the development of an optimization model. The key premise of our model is the assumption that decisions are rooted in values that should be promoted or a goal comprising a set of values that should be promoted up to a certain minimum level. Should we treat such values as criteria in a decision-making process, the entire sequence could be interpreted as single- or multi-criterion optimization.
In Section 6, we introduced a set of functions
At the beginning of our paper, we distinguished several types of goals that an agent can achieve. Abstract goals and unreachable abstract goals correspond to the agent's needs with respect to the levels up to which particular values ought to be promoted. Based on the same, material goals (i.e., specific actions facilitating particular states of affairs) which fulfil abstract goals are defined, and based on these, the practical goal, i.e., the action to be performed by the agent, is determined. Presented below is the method of approaching the decision-making problem, based on the thus defined goals, as an optimization task.
Let us begin by assuming the following:
Let the model of a decision-making situation
The first case is one where no preimposed goal exists. The decision is based on a choice of the option (option vector) which ensures the best total promotion of all values (taking into account the weight of the respective values).
In the analyzed case, we have a defined goal function
Assuming that function
In the second case, we have a declared an abstract goal
An abstract goal allows the definition of a set of limitations
Because the abstract goal itself does not define a goal function as such, the function from case 1 can be used adopted as the goal function:
In the third case, the agent has a declared unreachable abstract goal
In this case, the goal function will entail maximizing the level of promotion for the key value whereas the minimum levels of promotion for ordinary values of the goal
If
The limit set will be defined on the basis of the ordinary values of the unreachable abstract goal:
For each value
Having obtained, after employing one of the aforementioned approaches, the set
As the development of the optimization mechanism is outside the goal and scope of this paper, and because we do not introduce any specific functions
9. SOME COMMENTS ON DECISION-MAKING
In the introduction to this paper, we observed that the ethical context of decisions made by an autonomous agent becomes more important where the environment in which it is to function influences other people and becomes more unpredictable. This has been confirmed by the famous example of the Tay bot [29] which very quickly “learned” to become a racist. The example relates also to the problem of trust placed in learning mechanisms: how far can we trust a device whose actions in the real world can be potentially dangerous to humans? This is particularly true of mechanisms that are not capable of explaining their actions (such as neural networks): A device can “learn” bad behaviors by observing the behavior of other agents (similarly to humans who can be persuaded to do something they would not normally do through the influence of “bad company” or peer pressure).
The proposed framework allows us to include ethical concerns in the decision-making process. It also allows us to view ethics in a somewhat broader perspective, i.e., not only as based on deontic principles of mandatory action and prohibition, but also focused on the promotion of values and their adequate, relative proportions. Indeed, some authors [15,30] go as far as to directly suggest that under certain circumstances deontic rules should be broken and indicate values as a viable justification for the decision to break a rule.
The in-depth discussion of the problem of ethical behavior of autonomous systems was presented in [bench-capon2020]. The author discussed the problem of autonomous system in the light of three main ethical theories: consequentialism, deontological ethics, and virtue ethics. Although such an analysis is outside the scope of this paper, in our opinion (similar to the mentioned in [bench-capon2020] AATS+V model), our model can be treated as an attempt to model the virtue ethics, especially with the use of values in abstract goals and unreachable abstract goals which can represent virtues (however, we have also implemented some elements of consequentialism, e.g., the analysis of the actions in the light of the future consequences).
Moreover, although the author of [bench-capon2020] points out that his approach is focused on modeling of ethical behavior rather than ethical reasoning, we believe that autonomous setting the practical goal (action and state of affairs to be achieved) from abstract or abstract unreachable goal can be seen as a first step to model the ethical reasoning.
9.1. Two Sources of Knowledge
People acquire knowledge necessary in decision-making processes in various ways. Overall, in the most generalized perspective, these can be divided into two groups: knowledge acquired directly from sources: from books, teachers, parents, etc.; and knowledge acquired through experience: a person acquires knowledge through induction, by interacting with the world and evaluating the effects and consequences of certain behaviors. For instance, the first method can allow us to learn road traffic rules, what is and what is not allowed, whether it is acceptable to drive 100 km/h in an urban area, etc. Meanwhile, the second method allows us to learn how to determine the distance from the car in front of us and decide whether it is safe, will the speed with which we enter a corner allow us to negotiate it safely, etc.
The development of formal models of knowledge representation, inference, and argumentation allows us to represent knowledge acquired using the first of the above methods. On the other hand, the spectacular advances made in machine learning in recent years allowed the development of very complex systems focused on the second method of knowledge acquisition.
Based on the above, it is our belief that autonomous systems acting in the real world should take advantage of both of the above approaches. The model presented herein is based on formal reasoning, i.e., it allows the representation of knowledge acquired in the first manner; however, it has been structured in such a way so as to facilitate the implementation of a hybrid system that would draw upon both approaches to the use of artificial intelligence in decision-making.
9.2. Hybrid Model of Decision-Making
In this section, we will propose a way to combine the described model with machine learning systems. The idea of integration of the two approaches entails the division of functions between the respective types of knowledge:
On the one hand we have knowledge in the form of straightforward and clear principles of reasoning and rules for their application (developing arguments, solving arguments, etc.). This type of knowledge can be input into the system directly as it allows a clear and unambiguous choice of the best option based on the values of the agent's goals (what is more, the model is also capable of explaining why this is the case).
On the other hand we have knowledge which allows us to recognize whether the conditions for applying the inference rules have been satisfied, to what extent the respective values are indeed promoted, etc. This type of knowledge is less about inference and more about pattern recognition, classification, etc.
On the basis of the above division, it can be assumed that functions from the set
Another, alternative way of obtaining
Concluding, the framework presented in our paper allows the integration of machine learning based solutions with traditional solutions involving logical inference modeling.
10. ARGUMENTATION FRAMEWORK
In this section we introduce a reasoning mechanism allowing for decision-making. The mechanism allows for establishing of the agent's goal and performing actions allowing for fulfilment of the goal (practical reasoning). The argumentation-based practical reasoning framework introduced in our work is based on the simple reasoning mechanism presented in [14].
We assume that sets
By
Let
We assume that arguments are constructed on the basis of inference rules from set
Two arguments are in conflict if one attacks the other.
There is only one kind of attack: rebuttal, which is an attack on the conclusion of an argument. The attack has a symmetrical character: if
An attack on the conclusion of an argument
(
(
We assume a partial ordering between arguments
We assume that the basic grounds for determining an order (
In order to eliminate conflicts between arguments constructed on the basis of
Argument
Reasoning about priorities: we assume that priorities between conflicting arguments built on the basis of the same inference rule depend on values whose application the argument promotes.
If both arguments (
If both arguments (
If two arguments (
Argument
If an argument whose conclusion sets a practical goal (i.e.,
If an argument (i.e.,
If one of the arguments concludes that
Generally speaking, the argumentation framework used in the example was inspired by the ASPIC+ argumentation framework [31].
11. EXAMPLE
This section illustrates the process of deciding which action should be performed by the agent. We assume that there exists a smart device
Three values are important for the agent:
The agent declares an abstract unreachable goal
The key value of the goal
The process of deciding which action should be performed is carried out by setting a practical goal by the device
11.1. STAGE 1: Decision Space
This stage involves the definition of the decision space. First of all, we have to define a set of the decision vectors which we will denote as
In our case,
Now we declare the decision space:
We assume that we have four decision variables (parameters):
encryption algorithm,
encryption key length,
quality level,
network speed
Next, we assume the domains of particular decision parameters:
These domains specify all the possible physical values that a parameter can hold.
The next step is to define the vectors for all combinations of possible actions.
These vectors are defined as follows:
11.2. STAGE 2: Calculation of Values (Φ
During the second stage of our decision-making process, we have to calculate the level of promotion of values for all possible actions that can be performed initial state of the device.
The set
If by
Definition 19. [Definition of Φ security
For the special case in which
Then
For example,
The levels of promotion of the value
a
s
Definition 20.
[
For the initial state of affairs
Note that quality is inversely proportional (lower compression = higher quality).
On the basis of the function
Definition 21. [Φ speed
Let us assume that:
For the initial state of affairs
and if:
and if:
then:
On the basis of the function
Since all values have different importance, we assume three functions of the weight:
11.3. STAGE 3: Construction of Arguments
In the previous stages we declared an abstract unreachable goal
Since we assumed that:
we can notice that the initial state of affairs
This is because both situations satisfy the requirements (
To sum up, we have two arguments setting two different practical goal declarations. Since we can only have one practical goal, we will need to resolve the conflict in the next stage.
11.4. STAGE 4: Conflict Resolution
We have two arguments:
with two different conclusions. Arguments
In our case, argument
In the first step, we calculate the relative level of promotion of both values:
In the second step, we calculate the cumulative evaluation of the two conflicting arguments:
In consequence, we obtain that:
which means that
11.5. STAGE 5: Construction of the Final Argument
The last stage involves the construction of the final argument which will become the chosen action that the device
Since the system established a single practical goal, we can create the following argument
The argumentation scheme indicates that action
Finally, the device has to perform the action to use security AES-256, medium level of compression and chose a network that has fast upload speed. (
12. RELATED WORKS
The issue of using the numeric representation of levels to which values are promoted during decision-making has not been much discussed in the literature yet. The author of [24] proposes a simple model of representation of the levels of value promotion, though it is highlighted that “This is not meant to suggest that one can usually balance competing legal values through numerical computations.” Atkinson and Bench-Capon [26] (which is a developed version of an approach from [25] include a proposition of extending the AATS+V model by a simple numeric representation of the agent's preferences (using the product of values and their weights) and the level of utility regarding a decision-making situation. Our concept of the weight's and the level of promotion is a little bit more general: In our work the importance of value is represented by weight function, which do not have to be linear. In order to compute the relative levels of promotion of value (
The process of transition from physical value to evaluation of value itself (which in our work is represented by functions form set
The authors of [2] present a decision-making model for multi-agent systems with agent architecture (named BVG), making it possible to reason on the basis of values. The authors provide the possibility of numeric evaluation of the value level, but they establish constant weights and do not allow for comparing various options promoting various values. However, the system presented in [2] contains an important feature which is absent in our work—the system has a possibility of representation of failure of the action (the action does not leads to the planned state of affairs). This is an interesting property, which should be discussed in the future. Several other works (e.g., [32,33]) introduce methods of numeric representation of the agent's preferences, though lacking any link to values or the possibility of autonomous goal-setting.
An interesting mechanism of numeric representation of the levels of value promotion and relations between the same is proposed in [34]. It is based on a premise largely similar to that described herein (decision-making options promote values to a different extent, and the levels thereof can be compared). However, despite said similarities, the two mechanisms differ in certain key aspects: Sartor introduces the concept of utility which is absent in our model as in our opinion it provides no additional benefits: in our model, the level of value promotion is determined exactly on the basis of its utility. Moreover, the weight function proposed by Sartor is linear which, as observed in Section 4.1, often fails to correspond to real-life decision-making situations. The two models (ours and Sartor's) also differ in terms of the manner in which certain values are determined (e.g., the total level of value promotion which in Sartor's approach is obtained through simple addition). Furthermore, unlike ourselves, Sartor does not introduce any inference mechanism and offers no formal, in-depth analysis of the model he proposes.
An interesting decision-making model for an autonomous vehicle is proposed in [35] where the authors describe a very intricate framework based on the partially observable Markov decision process. The authors' primary goal is to provide a general solution for AV decision-making at any intersection. The model is developed in such a way so as to interact with any and unspecified number of other road users while at the same time accounting for the unpredictability of their behavioor. The model is based on a value function rooted in the expected reward for a given policy. The value function allows the determination of the total regret, whose minimization is the primary goal of the model. The solution proposed in [35] is dedicated to a very specific type of an autonomous device and provides a very advanced decision-making model in the context of behavior at a road intersection, but does not account for a broader decision-making context based on the promotion of a variety of values and does not allow for autonomous goal-setting by a device.
13. DISCUSSION AND CONCLUSIONS
The framework included in [14] allows for the modeling of reasoning in autonomous systems. Regretfully, its practical implementation requires a declaration of a large number of orders describing relations between levels to which various situations promote various values and sets of values. With real decision-making problems, the declaration of such a high number of orders is very challenging and can only be feasible in the case of a situation with relatively small sets of values and actions which are possible to perform. The main objective of our work is to propose modifications of the framework from [14] which would allow a facilitated implementation of the decision-making systems for autonomous agents. Our model can be seen as a kind of theoretical framework constituting a formal background for decision-making mechanisms. The basics of the framework are constructed in a flexible way, which allows for the development of the model by adding new functions to sets
While making a decision, a person intuitively evaluates available decision options, dividing them into better and worse ones (as presented in [14] and other works), but does not attach any numeric values to them. This paper introduces a modified approach, where the level to which particular situations promote various values is represented as a number from the range
The problem of reasoning with values and goals has been discussed by a number of authors; however, in our opinion, there are still many issues requiring further development. One of the most important drawbacks of the existing models consists in the lack of comprehensive discussion of the definition of values, goals, and relations between them. Especially the problem of relations between values and goals has been, in our view, insufficiently analyzed and stands in need of broader discussion.
The main objective of this paper is to present and discuss a new view of the problem of the representation of values and goals in reasoning. We have pointed out some elements in which our idea differs from other concepts of utilization of values in reasoning, particularly those presented in [7] and [21]. We do not intend to prove wrong the ideas included in the aforementioned works or claim they are irrelevant for the representation of various aspects of reasoning with goals and values; it is rather an attempt to demonstrate a new and more intuitive view of the problem. Obviously, the notion of intuitiveness is a subjective one, but we believe that establishing connection between the minimal acceptable levels of promotion of values, understood as goals and motivations of decision makers, allow us to establish a simpler and more abstract (trans-situational) way of representing the goals of agents. On the basis of such goals the decisions can be made.
The model presented in this paper provides a comprehensive tool for representation of values and goals in practical reasoning. Although there are decision-making models which allow for reasoning about values and goals (e.g., [21,36]), we are convinced, that our model of relations between goals and values makes a clear connection between those two concepts and allows us to create an autonomous system which can independently set itself a goal. We believe this is the most important novelty of the paper.
Most models presented in the literature view values as promoted (or demoted) by a change of a state of affairs. Our idea is slightly different: in the model we propose, values are promoted by a particular state of affairs or actions leading to a particular state of affairs, while the level of their promotion by a change of states can be seen as the difference between the promotion of values by the initial state and the action which leads to the target state of affairs. To our minds, such an idea is more intuitive and allows for a more adequate representation of human reasoning in which not only a change of states can be the subject of valuation, but also a group of given states of affairs. Such assumptions make our approach closer to the multi-criteria decision-making problems and we believe that our approach can be seem as a kind of bridge between value-based and multi-criteria decision systems.
On the basis of the above definitions, value-based inference patterns have been introduced with the use of argumentation schemes. The above definition of values allows for the formulation of the concept of abstract, material, and practical goals, which in consequence leads to an improved description of various kinds of teleological reasoning.
Most models presented in the literature view goals as a particular state of affairs which the agent is going to reach. In my conception, an abstract goal is the set of extents to which values should be promoted. On the basis of such a goal, the agent can infer which action brings about the desired goal. This view on the process of teleological reasoning allows for autonomous formulation of a goal by the agent.
Generally speaking, the model presented in the paper introduces a bridge between two separate but connected concepts of values and goals, providing a precise description of the relations between them as well as a possibility of reasoning about goals which, in consequence, allows us to model the motivations of the agent.
There are a few important issues calling for further discussion which I am going to elaborate in my future work. The first one is the discussion of the problem of joint actions with multiple agents. We have assumed that by an action we understand not only a single activity performed by a single agent, but also a joint action made by a group of agents. This is, however, the topic which requires further elaboration (the most comprehensive discussion of the problem is presented in [21]). The second issue is the potential possibility of the incorporation of the model into any formal framework of legal argumentation (ASPIC+ [31] or Carneades [37]). I see this possibility as important in the context of the implementation of this model, but it still requires some elaboration which can be performed in future works. The third issue is the discussion of the problem of decision-making in a legally-regulated environment, including an analysis of various reasoning mechanisms [38,39], conflict resolution [40,41], interpretation [42,43], and other.
CONFLICTS OF INTEREST
The authors declare of no conflicts of interest.
AUTHORS' CONTRIBUTIONS
Tomasz Zurek: 80% (The idea, formal modeling, proofs, analysis, discussion), Michail Mokkas: 20% (Case analysis).
APPENDIX: PROOFS
A.0.1 Independence of the Ordering of Values
Proof.
The proof is trivial. If set
Then:
For any accumulation order
If we accumulate
If we accumulate
If we accumulate
For a higher number of values in
A.0.2. Proof that Θ
Proof.
Assuming that the cumulative evaluation of the relative levels of promotion of the values from set
A.0.3. Proof that O R O O R
Proof.
If
A.0.4. Proof sthat Relations between O O R O O O R
The first relation will be fulfilled if:
Proof.
Since every function in
The second relation will be fulfilled if:
Proof.
Since every function in
The third relation will be introduced differently than in [14]:
Proof.
The above formula may be also presented as:
The set of values promoted by a situation
REFERENCES
Cite this article
TY - JOUR AU - Tomasz Zurek AU - Michail Mokkas PY - 2021 DA - 2021/02/19 TI - Value-Based Reasoning in Autonomous Agents JO - International Journal of Computational Intelligence Systems SP - 896 EP - 921 VL - 14 IS - 1 SN - 1875-6883 UR - https://doi.org/10.2991/ijcis.d.210203.001 DO - 10.2991/ijcis.d.210203.001 ID - Zurek2021 ER -