An agent is a mechanism through which the future is able to affect the past. An agent models the future consequences of its actions, and chooses actions on the basis of those consequences. In that sense, the consequencecausesthe action, in spite of the fact that the action comes earlier in the standard physical sense.
TL:DR;
This model sounds wrong (if still useful) in a way you are probably already aware of.
The agent chooses to take an action based on its calculations and model of the consequences of said action. The agent can be wrong about future, and thus it is not the consequence (in the (real) future) which causes the action, but an error or the past version of the environment (specifically the agent’s observations/records) which causes the action.
An attempt at a concrete example: imagine bots trading on the stock market. Do you expect those bots to always predict the future accurately? (Even without cosmic radiation or mundane technological failures, I expect someone to write a program that (eventually) compiles, but has a ‘wrong line’ or uses the wrong formula, etc. Not ‘inverting the utility function bad’ - unless there’s a lot of traders, but maybe something analogous to’ P(A or B) = P(A) + P(B) + P(A&B)’*.)
TL:DR;
This model sounds wrong (if still useful) in a way you are probably already aware of.
The agent chooses to take an action based on its calculations and model of the consequences of said action. The agent can be wrong about future, and thus it is not the consequence (in the (real) future) which causes the action, but an error or the past version of the environment (specifically the agent’s observations/records) which causes the action.
An attempt at a concrete example: imagine bots trading on the stock market. Do you expect those bots to always predict the future accurately? (Even without cosmic radiation or mundane technological failures, I expect someone to write a program that (eventually) compiles, but has a ‘wrong line’ or uses the wrong formula, etc. Not ‘inverting the utility function bad’ - unless there’s a lot of traders, but maybe something analogous to’ P(A or B) = P(A) + P(B) + P(A&B)’*.)
*It’s supposed to be -, not +, P(A&B).