Reinforcement Learning

Reinforcement learning is a machine learning approach in which an agent interacts with an environment over a sequence of decisions. At each time step, the agent receives a state, selects an action using a policy, and receives a reward plus a state transition. The objective is to learn behavior that maximizes expected return, often computed with a discount factor. A complete interaction may form an episode that ends at a terminal state. This setup is commonly described as a Markov Decision Process (MDP), while a value function estimates long-term return.

Japanese terms

1. Reinforcement learning — 強化学習きょうかがくしゅう: A machine-learning approach in which an agent learns behavior through interaction and feedback from rewards.
2. Machine learning — 機械きかい学習がくしゅう: A field of AI in which systems learn patterns or behavior from data and experience.
3. Agent — エージェント: The decision-making entity that observes the environment, chooses actions, and receives rewards.
4. Environment — 環境かんきょう: Everything outside the agent that responds to its actions and supplies states and rewards.
5. Time step — 時間じかんステップ: One discrete point in an interaction sequence at which the agent observes, acts, and receives feedback.
6. State — 状態じょうたい: A representation of the information needed to describe the environment at a particular time.
7. Action — 行動こうどう: A choice made by the agent that can affect the environment and future rewards.
8. Policy — 方策ほうさく: A rule or probability distribution that determines which action an agent selects in each state.
9. Reward — 報酬ほうしゅう: A numerical feedback signal that indicates the immediate desirability of an outcome.
10. State transition — 状態遷移じょうたいせんい: The change from one state to another after the agent takes an action.
11. Return — 収益しゅうえき: The total reward accumulated from a time step onward, usually with future rewards discounted.
12. Discount factor — 割引率わりびきりつ: A value between zero and one that controls how strongly future rewards affect present decisions.
13. Episode — エピソード: One complete sequence of interaction from an initial state until a terminal state.
14. Terminal state — 終端状態しゅうたんじょうたい: A state that ends an episode and after which no further action is taken in that episode.
15. Markov Decision Process (MDP) — マルコフ決定過程けっていかてい: A mathematical framework for sequential decisions defined by states, actions, transition probabilities, rewards, and a discount factor.
16. Value function — 価値関数かちかんすう: A function that estimates the expected return from a state, or from taking an action in a state, under a policy.
17. Exploration — 探索たんさく: Trying actions whose outcomes are uncertain in order to discover potentially better behavior.
18. Exploitation — 活用かつよう: Choosing actions currently believed to produce the highest return.
19. Model-based reinforcement learning — モデルベース強化学習きょうかがくしゅう: Reinforcement learning that uses a learned or supplied model of environment dynamics to plan or improve behavior.
20. Model-free reinforcement learning — モデルフリー強化学習きょうかがくしゅう: Reinforcement learning that learns values or policies directly from experience without learning a model for planning.
21. Value-based method — 価値かちベース手法しゅほう: A method that learns value estimates and derives action choices from them.
22. Policy-based method — 方策ほうさくベース手法しゅほう: A method that optimizes a policy directly instead of deriving it only from learned values.
23. Actor–critic — アクタークリティック: A method combining an actor that updates the policy with a critic that evaluates the actor using value estimates.
24. Q-learning — Q学習がくしゅう: A model-free, value-based algorithm that learns the expected return for taking each action in each state.
25. Policy gradient — 方策勾配ほうさくこうばい: A family of policy-based methods that improves a policy by following the gradient of expected return.
26. Deep reinforcement learning — 深層強化学習しんそうきょうかがくしゅう: Reinforcement learning that uses deep neural networks to represent policies, value functions, or environment models.

Related topics

The central practical tradeoff is between exploration, which gathers new information, and exploitation, which uses what the agent already knows. Model-based reinforcement learning plans with a representation of the environment, while model-free reinforcement learning learns behavior directly from experience.

Common method families include value-based methods, such as Q-learning; policy-based methods, including policy gradient methods; and actor–critic methods, which learn a policy and value estimates together. Deep reinforcement learning applies deep neural networks when states, actions, or learned functions are too complex for simple tables or linear representations.