Machine Learning Specialization · Unsupervised Learning, Recommenders, Reinforcement Learning

State-Action Value Function Explorer

A compact Mars-rover-style value-function example for understanding how rewards, discount factor, and missteps change action preference.

State-action valuesDiscount factorReward shapingPolicy intuitionInteractive RL concepts

GitHub

What this project is about

This compact RL project isolates Q(s,a) intuition before deep RL. By changing terminal rewards, step reward, discount factor, and misstep probability, the value of each action changes visually.

Implementation focus

It made the meaning of discounting and reward design clearer: the best action is not fixed by the environment alone, but by how future reward and uncertainty are modeled.

Key results and artifacts

6 states and 2 actions
Terminal rewards: left 100, right 40
Default discount factor: 0.5

What I learned

This conceptual project made the mathematics of Q(s,a) easier to reason about before moving into neural Q-functions.

Milestone deep dive

What this project proves in my learning path.

A compact Mars-rover-style value-function example for understanding how rewards, discount factor, and missteps change action preference. The milestone is included because it shows a complete learning unit: a problem, a data representation, an implementation path, measurable outputs, and a lesson that later connects to larger AI systems.

Input layerstate, action, reward, and next-state transitions

Model layervalue estimation or policy learning loop

Evaluation layertests, metrics, plots, or qualitative output

Artifact layerlearned action preferences and reward curves

Problem framing

What I was trying to solve

This page is not just a lab description. It explains how I transformed the assignment idea into an engineering story: what the model receives, what it optimizes, what output it should produce, and how the result can be inspected.

Implementation

What I coded or assembled

The implementation focus was the mechanism behind the result: functions, model blocks, training logic, preprocessing, evaluation, and the small technical details that make the notebook work rather than only look correct.

Concepts

Core concepts

State-action valuesDiscount factorReward shapingPolicy intuitionInteractive RL concepts

These tags show the technical ideas this milestone added to my toolkit.

Learning result

What changed after it

This conceptual project made the mathematics of Q(s,a) easier to reason about before moving into neural Q-functions.

Important outputs and evidence

6 states and 2 actions
Terminal rewards: left 100, right 40
Default discount factor: 0.5

How I would explain it in an interview

I would describe this milestone by starting from the data representation, then explain the model or algorithm, then show what was measured. The important part is not only the final number; it is the ability to reason about why the method works, what assumptions it makes, and what limitations should be improved next.