Environments

Environments

Nature of Environment

• In previous section we have seen various types of agents, now let us see the details of environment where in agent is going to work. A task environment is essentially a problem to which agent is a solution.

• The range of task environments that might arise in AI is obviously vast. We can, however, identify a fairly small number of dimensions along which task environments can be categorized. These dimensions determine, to a large extent, the appropriate agent design and the applicability of each of the principle families of techniques for agent implementation.

Types of Task Environment

1. Fully Observable Vs Partially Observable

• If an agent's sensors give it the access to the complete state of the environment at each point of time, then it is fully observable.

• In some environment, if there is noise or agent is with inaccurate sensors or may be some states of environment are missing then such environment is partially observable.

Example-

Fully Observable

The puzzle game environment is fully observable where agent can see all the aspects, that are surrounding it. That is agent can see all the squares of the puzzle game along with values (if any added) in them.

More examples -

1) Image analysis. 2) Tic - tac toe.

Partially Observable

The pocker game environment is partially observable. Game of pocker is a card game that shares betting rule; and usually (but not always) hand rankings. In this game agent is not able to perceive other player's betting.

Also agent cannot see other player's card. It has to play with reference to its own cards and with current betting knowledge.

More examples-

1) Interactive Science Tutor.

2) Millitary Planning.

2. Deterministic Vs Stochastic

• If from current state of environment and the action, agent can deduce the next state of environment then, it is deterministic environment otherwise it is stochastic environment.

• If the environment is deterministic except for the actions of other agents, we say that the environment is strategic.

Examples -

Deterministic: In image analysis whatever is current percept of the image, agent can take next action or can process remaining part of image based on current knowledge. Finally it can produce all the detail aspects of the image.

Strategic: Agent playing tic-tac toe game is in strategic environment as from the current state agent decides next state action except for the action of other agents.

More examples -

1) Video analysis.

2) Trading agent.

Stochastic: Boat driving agent is in stochastic environment as the next driving does not based on current state. In fact it has to see the goal and from all current and previous percepts agent needs to take action.

More examples-

1) Car driving.

2) Robot firing in crowd.

3. Episodic Vs Sequential

• In episodic environment agent's experience is divided into atomic episodes such that each episode consists of, the agent perceiving process and then performing single action. In this environment the choice of action depends only on the episode itself, previous episode does not affect current actions.

• In sequential environment on the other hand, the current decision could affect all future decision.

• Episodic environments are more simpler than sequential environments because the agent does not need to think ahead.

Example -

Episodic Environment: Agent finding defective part of assembled computer machine. Here agent will inspect current part and take action which does not depend on previous decisions (previously checked parts).

More Examples -

1) Blood testing for patient.

2) Card games.

Sequential Environment: A game of chess is sequential environment where agent takes action based on all previous decisions.

More examples -                                 -

1) Chess with a clock.

2) Refinery controller.

4. Static Vs Dynamic

• If the environment can change while agent is deliberating then we say the environment is dynamic for the agent, otherwise it is static.

• Static environments are easy to tackle as agent need not worry about changes around (as it will not change) while taking actions.

• Dynamic environments keep on changing continuously which makes agent to be more attentive to make decisions for act.

• If the environment itself does not change with time but the agent's performance does, then we say that environment is semidynamic.

Examples -

Static: In crossword puzzle game the environment that is values held in squares can only change by the action of agent.

More examples -

1) 8 queen puzzle

2) Semidynamic.

Dynamic: Agent driving boat is in dynamic environment because the environment can change (A big wave can come, it can be more windy) without any action of agent.

More examples -

1) Car driving

2) Tutor.

5. Discrete Vs Continuous

• In discrete environment the environment has fixed finite discrete states over the time and each state has associated percepts and action.

• Where as continuous environment is not stable at any given point of time and it changes randomly thereby making agent to learn continuously, so as to make decisions.

Example:

Discrete: A game of tic-tac toe depicts discrete environment where every state is stable and it associated percept and it is outcome of some action.

More examples -

1) 8 - queen puzzle

2) Crossword puzzle.

Continuous: A boat driving environment is continuous where the state changes are continuous, and agent needs to perceive continuously.

More examples -

1) Part Picking Robot

2) Flight Controller.

6. Single Agent Vs Multiagent

• In single agent environment we have well defined single agent who takes decision and acts.

• In multiagent environment there can be various agents or various group of which are working together to take decision and act. In multiagent environment agents we can have competitive multiagent environment, in which many agents are working parallel to miximize performance of individual or there can be co-operative multiagent environment, where in all agents have single goal and they work to get high performance of all of them together.

Example:

• Multiagent independent environment

  • Many agent in game of Maze.

• Multiagent cooperative environment

  • Fantasy football. [Here many agents work together to achieve same goal.] Multiagent competitive environment

  • Trading agents. [Here many agents are working but opposite to each other]

• Multiagent antagonistic environment

  • Wargames, [Here multiple agents are working opposite to each other but one side (agent/agent team) is having negative goal.]

• Single agent environment

  • Boat driving[Here single agent perceives and acts]

7. Complexity Comparison of Task Environment

Following is the rising order of complexity of various task environment.

More Types of Task Environment

Based on specific problem domains we can further classify task environments as follow.

1) Monitoring and Surveillance Environment

Example: Agent monitoring incoming people at some gathering where only authorized people are allowed.

2) Time Constrained Environment

Example: Chess with a clock environment where the move should be done in specified amount of time.

3) Decision Making Environment

Example: The executive agent who is monitoring profit of a organization, can help top level management to take decision..

4) Process Based Environment

Example: The image processing agent who can take input and synthesize it to produce required output, and details about the image.

5) Personal or User Environment

Example: A small scale agent which can be used as personal assistance who can help to remember daily task, who can give notifications about work etc.

6) Buying Environment

Example: A online book shopping bot (agent) who buys book online as per user requirements.

7) Automated Task Environment

Example: A cadburry manufacturing firm can use a agent who automates complete procedure of cadburry making.

8) Industrial Task Environment

Example: An agent developed to make architecture of a building or layout of building.

9) Learning Task Environment (Educational)

Example: We can have a agent who is learning some act or some theories presented to it and later it can play it back which will be helpful for others to learn that act or theories.

10) Problem Solving Environment

Example: We can have agent who solve different types of problems from mathematics or statistics or any general purpose problem like travelling salesman problem.

11) Scientific and Engineering Task Environment

Example Agent doing scientific calculations for aeronautics purpose or agent develop to design road maps or over bridge structure.

12) Biological Task Environment

Example Agent working for design of some chemical component helpful for medicine.

13) Space Task Environment

Example Agent that is working in space for observing space environment and recording details about it.

14) Research Task Environment

Example Agent working in a research lab where it is made to grasp (learn) knowledge and represent it and drawing conclusions from it, which will helps researcher for further study.

15) Network Task Environment

Example: An agent developed to automatically carry data over a computer network based on certain conditions like time limit or data size limit in same network (same type of agent can be developed for physically transferring items or mails) over same network.

16) Repository Task Environment

Example: If a data repository is to be maintained then agent can be developed to arrange data based on criterias which will be helpful for searching later on.