IE437 · Data-Driven Decision Making and Control — Lecture 0

Data-Driven Decision Making and Control

Deciding well under uncertainty — three criteria, four categories, four real systems
Jinkyoo Park · KAIST — Industrial & Systems Engineering
The map — how every chapter fits one story, and what each one decided in the field
01

The one question

Every lecture this term is the same question, asked again with one more thing made harder.
The one questionLecture 0

Learning route — identify the decision before naming the method

Bring: No algorithm knowledge is assumed here.

First pass What to do
Follow the idea Decision → information → one choice or a sequence → evidence
Work without the solution Identify the action, objective and available feedback in the temperature example.
Return later The detailed industry architectures and research outlook are reference cases.
For the temperature thread: predict → calculate → reveal and check → change one condition. Complete the core calculation before reading the research extensions.
Lecture 0 · Data-Driven Decision Making and Control3 / 71
The one questionLecture 0

What this course is about — in one sentence

How do we make good decisions under uncertainty?

That is the entire course. We ask it under changing assumptions: a known objective, an uncertain model, decisions through time, and eventually a fixed dataset. Known-model chapters establish tools before we learn from experience. Multi-agent games are a preview of IE579.

Two disciplines answer it differently, and this course lives where they meet — decision-centric AI: use data and learning to make high-dimensional decisions that classical optimisation alone could not reach.

Lecture 0 · Data-Driven Decision Making and Control4 / 71
The one questionLecture 0

Temperature thread — name the decision before choosing a methodshared teaching example · predict before revealing the calculation

A room is at 20°C and the target is 22°C. Let x=T−22=−2x=T-22=-2. A normalized heater command u∈[0,2]u\in[0,2] gives x′=x+ux'=x+u; the one-step cost is c=(x′)2+u2c=(x')^2+u^2. This is a small teaching model, not calibrated building physics.

Predict: Does the largest heating command necessarily give the best decision?

Calculate and check.

Command Next temperature Comfort error + effort
u=0u=0 20°C 4+0=44+0=4
u=1u=1 21°C 1+1=21+1=2
u=2u=2 22°C 0+4=40+4=4
The decision is the heater command; the measured temperature is information. A target temperature alone does not specify how to trade comfort against effort.
Lecture 0 · Data-Driven Decision Making and Control5 / 71
The one questionLecture 0

Try it — the temperature response is unknownwork independently · reveal only after writing an answer

Keep the same goal and heater, but suppose the relation between command and next temperature is unknown. Name what you would measure, what you would learn, and how you would evaluate a decision.

Check your answer. Measure command–temperature transitions and their cost. Learn a response model or learn a value/policy from experience. Evaluate the resulting comfort–effort cost on new allowed trials. If trials are unavailable, evaluation must use the existing log and explicit assumptions.

Across the course, keep asking what is known, what is learned and whether another experiment is allowed.
Lecture 0 · Data-Driven Decision Making and Control6 / 71
The one questionLecture 0

AI, decision making, and what joins them

Lecture 0 · Data-Driven Decision Making and Control7 / 71
The one questionLecture 0

From a real system to a solved problem

Formulation and solution are two different arrows, policed by two different questions. Data helps on the left arrow too — a model fitted to data represents the world more faithfully, not merely faster.
Lecture 0 · Data-Driven Decision Making and Control8 / 71
The one questionLecture 0

Two kinds of uncertainty — name them once

Everything we model is uncertain in one of two ways. The distinction recurs all term.

  • Aleatoric (statistical) uncertainty — the world is genuinely stochastic. Repeating the experiment gives different outcomes. More data alone does not remove this variability under the chosen observation model; improved sensing or a changed process can alter it.
  • Epistemic (model) uncertainty — we simply don't know the model: its parameters, its state, its dynamics. Relevant data can reduce it; misspecified models may remain confidently wrong.
The whole course is a campaign against epistemic uncertainty.

Learning the parameter (Ch 2), the structure (Ch 3), the function (Ch 4), the design map (Ch 5–6), the dynamics (Ch 8–11) — while respecting the aleatoric uncertainty that no data can erase.

Lecture 0 · Data-Driven Decision Making and Control9 / 71
The one questionLecture 0

The two, drawn — only one of them answers to data

Drag the sample count. The blue band collapses like 1/n1/\sqrt{n} — that is what data buys. The amber band is the world's own spread, and it does not move at all however many samples you take.
Lecture 0 · Data-Driven Decision Making and Control10 / 71
02

Three criteria, four categories

Decision making under uncertainty, split by three questions — and the four cells this course actually visits.
Three criteria, four categoriesLecture 0

The three criteriap. 8 of the source deck — the sentence the whole taxonomy hangs on

"Decision making under uncertainties." Three questions divide every method in the zoo, and the answers are independent of one another.

# the criterion one end the other end
1 Is the model known? model-based — handed to us data-driven — learned from data
2 How many stages? static — one decision dynamic — a sequence
3 How many decision makers? single agent — an optimum many agents — an equilibrium

Criterion 1 is where the two uncertainties live: epistemic uncertainty is exactly not knowing the model, and it is the axis this course is named after. Criteria 2 and 3 say what shape the problem has once you do know it.

Lecture 0 · Data-Driven Decision Making and Control12 / 71
Three criteria, four categoriesLecture 0

One familiar decision, three versions — stocking a shop

Suppose selling one item earns 5, ordering it costs 2, and unsold items have no value. Profit is 5min⁡(q,D)−2q5\min(q,D)-2q.

setting what must be decided what is needed
demand is known to be 10 today's order qq compare profits: q=8,10,12q=8,10,12 gives 24, 30, 26
demand is uncertain today's order before seeing demand a demand distribution; compare expected profit
orders repeat daily order based on stock and new information a policy; today's stock affects tomorrow
The product is the same. Changing information and time changes the decision problem. Start with this distinction before learning the names of algorithms.
Lecture 0 · Data-Driven Decision Making and Control13 / 71
Three criteria, four categoriesLecture 0

Three criteria make a cube

The origin is plain optimisation. Step through the numbered route — step ④ is the one to watch, where the data-driven axis is crossed a second time and the count of unknowns doubles.
Lecture 0 · Data-Driven Decision Making and Control14 / 71
Three criteria, four categoriesLecture 0

Criterion 1 — model-based becomes data-driventhe deepest axis, and the course's namesake

Two ways to know the world you decide in:

  • Model-based — the objective, constraints, transition and dynamics are given. You compute the optimum. (Lectures 1, 7, 9)
  • Data-driven — you have only data. You must learn enough of the world to act: a belief (Ch 2–4), a surrogate (Ch 5–6), a value or a policy (Ch 8, 10, 11).
The move that defines the course

Take a model-based method, delete the model, and replace it with data.

optimisation→Bayesian optimisationDP→value-based RLoptimal control→policy-based RL\text{optimisation} \to \text{Bayesian optimisation} \qquad \text{DP} \to \text{value-based RL} \qquad \text{optimal control} \to \text{policy-based RL}

This is a teaching map. In standard RL terminology, model-based RL can itself be data-driven when it learns a model, as in Lecture 11.

Lecture 0 · Data-Driven Decision Making and Control15 / 71
Three criteria, four categoriesLecture 0

Criterion 2 — static becomes dynamic

Single agent Multi agent
Static optimisation static game
Dynamic dynamic optimisation / control dynamic game

Static — one decision. You choose xx once; the modeled decision is chosen once. BO may query repeatedly to find that one design. This is classical optimisation (Lecture 1) and its uncertain, data-driven cousins (Lectures 2–6).

Dynamic — a sequence. Each decision reshapes the state the next one faces. This is the realm of MDPs, optimal control and reinforcement learning (Lectures 7–11).

The single optimisation splits into time — and the value function is born.
Lecture 0 · Data-Driven Decision Making and Control16 / 71
Three criteria, four categoriesLecture 0

Criterion 3 — one decision maker becomes many

Optimisation Game
single objective one optimum x∗x^* —
coupled objectives — equilibrium (Nash, …)

When other agents — also optimising, also learning — share the world, the very notion of "optimal" changes. There is no single best action independent of what the others do. The solution concept shifts from an optimum to an equilibrium.

This course stops before that axis. Crossing it is the subject of the follow-on course, IE579 Game Theory and Multi-Agent Reinforcement Learning — and the cube above shows exactly which face it takes over. Naming the axis here is what makes the boundary of this course legible.

Lecture 0 · Data-Driven Decision Making and Control17 / 71
Three criteria, four categoriesLecture 0

The four categories, and the four systems that prove thempp. 13–16, 19, 57–58 of the source — the badges ①②③④ you will see all term

category what is given lectures the system it decided
① Fundamentals — static, model-based the model 1–3 the template every other cell inherits
② Data-driven static decision making data only 4–6 wind farm layout · traffic signal split
③ Fundamentals — dynamic, model-based the model, over time 7, 9 the template Part IV deletes
④ Data-driven dynamic decision making data only, over time 8, 10–12 semiconductor furnace · offline meta control
The rest of this lecture walks the four categories — and shows, for each, a system where the decision was actually made.
Lecture 0 · Data-Driven Decision Making and Control18 / 71
Three criteria, four categoriesLecture 0

Check — reading the cube

The course maps every decision problem onto three axes. Which three?

Stages, model, and agents. Every lecture in this course is one position on that cube, and the course as a whole is a route through it: the chapters compare different assumptions. The location helps identify candidate methods; the details of the task determine which are appropriate.

Lecture 0 · Data-Driven Decision Making and Control19 / 71
03

Categories ① and ② — the static half

One decision, made once. First with the model in hand, then with nothing but data — and two systems that were decided this way.
Categories ① and ② — the static halfLecture 0

① Fundamentals — optimisation, and beliefLectures 1–3 · static, model-based

Given a model, a decision is a program:

min⁡x  f(x)s.t.g(x)≤0,  h(x)=0\min_x\; f(x) \quad \text{s.t.}\quad g(x)\le 0,\; h(x)=0

Make the model uncertain rather than absent and the same program acquires a belief. Bayes' rule is how the belief moves:

p(θ∣data)=p(data∣θ) p(θ)p(data)p(\theta\mid \text{data}) = \frac{p(\text{data}\mid\theta)\,p(\theta)}{p(\text{data})}

Lecture 2 carries the belief over one parameter; Lecture 3 gives it structure, and then adds decision and utility nodes — at which point the graph is a one-stage decision problem.

Lecture 0 · Data-Driven Decision Making and Control21 / 71
Categories ① and ② — the static halfLecture 0

From a Bayesian network to a decisionp. 18 of the source — a chance node, a decision node, a utility node

Add a decision node and a utility node to a Bayesian network and you have an influence diagram. Lecture 7 adds repeated decisions, a Markov
Add a decision node and a utility node to a Bayesian network and you have an influence diagram. Lecture 7 adds repeated decisions, a Markov state and an explicit transition model to build an MDP.
Lecture 0 · Data-Driven Decision Making and Control22 / 71
Categories ① and ② — the static halfLecture 0

② Data-driven static — three routes when the model is gonepp. 19 and 32 of the source · static, data-driven

OnlineBayesian optimisation — you may still query the system, so choose the next query well (Ch 4)
Offline · surrogatefit a forward model f^\hat f to a fixed dataset, then search it (Ch 5)
Offline · generativelearn the inverse map and sample good designs (Ch 6)

A research extension is offline meta optimisation with online adaptation — learn across many past tasks, then adapt to today's in a handful of trials. The traffic-signal case below is exactly this.

Lecture 0 · Data-Driven Decision Making and Control23 / 71
Categories ① and ② — the static halfLecture 0

Case A · Wind farm layoutApplication Example 1 — where do N turbines go, when each steals wind from the next?

The decision is , the turbine positions. The obstacle is that power has no closed form: turbines sit in each other's wake, so the objective
The decision is x\mathbf{x}, the turbine positions. The obstacle is that power Pi(x;θ,U)P_i(\mathbf{x};\theta, U) has no closed form: turbines sit in each other's wake, so the objective couples every pair.
Lecture 0 · Data-Driven Decision Making and Control24 / 71
Categories ① and ② — the static halfLecture 0

Case B · Traffic signal splitApplication Example 2 — offline meta data-driven optimisation

The same decision — the green split at every intersection — in Hangzhou, Manhattan and two synthetic districts. Every intersection is a new
The same decision — the green split at every intersection — in Hangzhou, Manhattan and two synthetic districts. Every intersection is a new task, and you cannot run a thousand trials on a live junction.
Lecture 0 · Data-Driven Decision Making and Control25 / 71
Categories ① and ② — the static halfLecture 0

Check — what makes a decision problem hard

What extra question does a decision problem ask beyond predicting an outcome?

A demand prediction is an input to an inventory decision. Choosing inventory also requires costs, constraints and a model or evidence about consequences. In logged interaction data, only the chosen action's outcome is observed; a simulator or known model may provide additional information about alternatives.

Lecture 0 · Data-Driven Decision Making and Control26 / 71
04

Categories ③ and ④ — the dynamic half

Now the decision is a sequence, and each one reshapes the state the next one faces.
Categories ③ and ④ — the dynamic halfLecture 0

③ Fundamentals — two scientific parentsLectures 7 and 9 · dynamic, model-based

Lineage A · OR / Dynamic Programming
Lineage B · Control Theory
Model-based
(origin)
MDP & Dynamic ProgrammingLecture 7
Optimal Control / PlanningLecture 9
Data-driven
(extension)
Value-Based RLLecture 8
Policy-Based RLLecture 10
We teach two useful routes into RL: value-based and policy-based methods. They overlap in actor–critic algorithms.
  • Value-based RL is dynamic programming with the model deleted — discrete, value-centric, Bellman's operations research.
  • Policy-based RL is optimal control with the dynamics deleted — continuous, control-law-centric, Pontryagin and Kalman's control theory.
Lecture 0 · Data-Driven Decision Making and Control28 / 71
Categories ③ and ④ — the dynamic halfLecture 0

The same claim, in two gridspp. 57–58 of the source deck — where Lectures 7 to 10 come from

Zoom into the dynamic, single-agent cell and split it twice: by action space and by time. Handed a model, the four boxes are the classical theories.

model-based finite action space infinite action space
discrete time discrete-time MDP · P(st+1∣st,at)P(s_{t+1}\mid s_t,a_t) discrete-time dynamic system · xt+1=f(xt,ut)x_{t+1}=f(x_t,u_t)
continuous time continuous-time MDP · P(st+h∣st,at)P(s_{t+h}\mid s_t,a_t) continuous-time dynamic system · x˙t=f(xt,ut)\dot x_t = f(x_t,u_t)

For this course’s main examples, deleting the given model leads to the methods below. These are not exclusive categories: policies can be discrete and value functions continuous.

model-free finite action space infinite action space
discrete time value-based reinforcement learning policy-based reinforcement learning
Lecture 0 · Data-Driven Decision Making and Control29 / 71
Categories ③ and ④ — the dynamic halfLecture 0

④ The doubling — why the second crossing is not the first again

First crossing · Lecture 1 → 2

The static world. In our first examples, the objective is the main unknown:

min⁡x  f(x)s.t.g(x)≤0\min_x\; \hl{f(x)} \quad \text{s.t.}\quad g(x)\le 0

The answer is a belief over ff — a posterior, a surrogate, a generative inverse.

Second crossing · Lecture 7 → 8

The dynamic world. Delete the model and two objects go missing at once:

Q∗(s,a)=∑s′P(s′∣s,a)[R(s,a,s′)+γmax⁡a′Q∗(s′,a′)]Q^*(s,a)=\sum_{s'} \hl{P(s'\mid s,a)}\big[\hl{R(s,a,s')}+\gamma\max_{a'}Q^*(s',a')\big]

Reward rr and transition PP — the count doubles.

This is why RL is not merely optimisation done with data.
Lecture 0 · Data-Driven Decision Making and Control30 / 71
Categories ③ and ④ — the dynamic halfLecture 0

Three answers to the same doubling

Value-basedCh 8 · learn QQ from samples — rr and PP are fused and neither is modelled
Policy-basedCh 10 · output the action — the dynamics cancel in the gradient
Model-basedCh 11 · estimate PP explicitly, then plan with it

Each answer pays a different price: fusing loses the ability to plan ahead; cancelling costs sample efficiency; estimating invites model bias. No answer dominates — which is why all three are taught.

Lecture 0 · Data-Driven Decision Making and Control31 / 71
Categories ③ and ④ — the dynamic halfLecture 0

Case C · Semiconductor furnace heat controlmodel-based RL, in production — the dynamics were learned, then planned with

An input-convex graph recurrent network learns the heat-transfer dynamics from operating logs; model-predictive control then optimises the w
An input-convex graph recurrent network learns the heat-transfer dynamics from operating logs; model-predictive control then optimises the work-set over a horizon and executes only the first action. Convex in the control is the design choice that makes the inner optimisation solvable at every step.

The balance the professor's own slide names: model expressivity against optimisation solvability. A more faithful dynamics model is worthless if the controller cannot optimise through it in the time between two ticks.

Lecture 0 · Data-Driven Decision Making and Control32 / 71
Categories ③ and ④ — the dynamic halfLecture 0

Check — follow the changing assumptions

Which sequence describes the main learning route of this course?

This is a route through different assumptions, not a claim that every chapter removes exactly one assumption. Lectures 7 and 9 restore a known model to teach sequential decision making. Lectures 8, 10 and 11 use experience; Lecture 12 removes new interaction during training.

Lecture 0 · Data-Driven Decision Making and Control33 / 71
05

The single spine

Read the whole course as a slow stripping-away of what you were handed.
The single spineLecture 0

Each chapter removes one given

Walk the lectures and watch the ledger empty. The counter on the right is the argument of the previous act: it reads 1 for the whole static half, and 2 from Lecture 8 onward.
Lecture 0 · Data-Driven Decision Making and Control35 / 71
The single spineLecture 0

The four systems, and the categories they belong to

Every method in this course was built against a real system. These four run through the lectures, and the numbers are the professor's own.

system category what is decided result
Wind farm ② static, data-driven where to place NN turbines, in a shared wake 1.5% MAPE prediction; total power 7.5 → 20.0
Traffic signals ② static, data-driven (meta) the green split at every intersection 408.5 → 395.5 waiting vehicles in Hangzhou; 920.1 → 859.7 in Manhattan
Semiconductor furnace ④ dynamic, data-driven the heater work-set, every step 2.03 °C average error at a 100-step horizon
Combinatorial design ④ and beyond a chip placement, a route, a schedule search space >1090,000>10^{90{,}000} states — beyond Go's 1036010^{360}

These are source-deck case studies, including simulations and reported applications. Their figures apply to the displayed setting; they are not performance guarantees on a new system — which is why the course spends its first six lectures on how to state the problem before it spends any on how to learn one.

Lecture 0 · Data-Driven Decision Making and Control36 / 71
The single spineLecture 0

The course, as a route through the map

  • Part I — the given world. Ch 1: classical optimisation, the atom of every later method.
  • Part II — the uncertain world. Ch 2 Bayesian statistics (belief as a distribution) → Ch 3 Bayesian networks (structured belief, plus decisions) → Ch 4 Bayesian optimisation (act on an unknown function).
  • Part III — design from data alone. Ch 5–6 data-driven design optimisation: surrogate (forward) and generative (inverse).
  • Part IV — the world that unfolds. Ch 7 MDP & DP → Ch 8 value-based RL ‖ Ch 9 optimal control → Ch 10 policy-based RL → Ch 11 model-based RL.
  • Part V — when you cannot even try. Ch 12 offline RL: the same dynamic, data-driven cell, with the right to interact withdrawn.

Appendix: a probability review — the toolbox underneath all of it.

Lecture 0 · Data-Driven Decision Making and Control37 / 71
06

What comes next

This final section is a research and deployment outlook. Connect each example to a decision, objective and constraint; detailed solver architectures are extensions to the core course map.
What comes nextLecture 0

The bottleneck was never the solverp. 69 of the source — the modelling loop, seen again with fresh eyes

The same three boxes as the opening slide. Solvers have become extraordinary; the arrow that has not improved is the first one. Defining the
The same three boxes as the opening slide. Solvers have become extraordinary; the arrow that has not improved is the first one. Defining the problem mathematically is still the realm of advanced experts, and there are not enough of them.
Lecture 0 · Data-Driven Decision Making and Control39 / 71
What comes nextLecture 0

The paradigm shiftfrom writing a formulation to generating a solver

Formulation-based

A human expert reads the operation, writes min⁡f\min f s.t. g≤0g \le 0, chooses a solver, tunes it.

Weeks per problem. Rewritten whenever the operation changes. The expert is the bottleneck, and the expert does not scale.

Generative

A model is trained to emit a solution given an instance — optimisation as inference.

fθ:  X→Yf_\theta:\; \mathcal{X} \to \mathcal{Y}

Milliseconds per instance, once trained. The instance may be one the expert never saw.

The source deck's own phrase for where this goes: “ChatGPT for Optimization — generative AI for optimisation.”
Lecture 0 · Data-Driven Decision Making and Control40 / 71
What comes nextLecture 0

Neural combinatorial optimisation, defined

Learn a solver, not a solution: , trained over a distribution of instances with the aim of generalizing to ; a changed instance distribution
Learn a solver, not a solution: fθ:X→Yf_\theta: \mathcal{X}\to\mathcal{Y}, trained over a distribution of instances x∼g(⋅)x \sim g(\cdot) with the aim of generalizing to x′∼g′(⋅)x'\sim g'(\cdot); a changed instance distribution must be tested. Because the training signal spans tasks, learning an NCO solver is inherently multi-task learning — which is why the meta-learning of Case B returns here.
Lecture 0 · Data-Driven Decision Making and Control41 / 71
What comes nextLecture 0

What is learned in neural combinatorial optimisation?original introduction PDF pp. 69–75 · separate the solver role from the training method

Role of the learned component What it does Example
Construct a solution choose the next element until a solution is complete add the next customer to a route
Improve a solution propose a change to an existing solution exchange two route segments
Assist mathematical programming guide a component of an exact solver suggest branching or variable selection
Configure a solver select settings for a problem instance choose an algorithm or its parameters

Imitation learning and reinforcement learning are ways to train these components. They are a different classification from construction versus improvement.

Ask which decision the network makes inside the solver. A learned heuristic's speed or solution quality does not by itself prove global optimality.
Lecture 0 · Data-Driven Decision Making and Control42 / 71
What comes nextLecture 0

Two ways to train it

Imitation learning fits solver-provided solutions. Reinforcement learning uses reward or objective feedback. Their sample efficiency, soluti
Imitation learning fits solver-provided solutions. Reinforcement learning uses reward or objective feedback. Their sample efficiency, solution quality, and generalization depend on the training setup; neither has a universal advantage. The choice is Lecture 8 and Lecture 10 arriving in a new domain.
Lecture 0 · Data-Driven Decision Making and Control43 / 71
What comes nextLecture 0

A representative NCO paper — learn the next cityKool, van Hoof & Welling · ICLR 2019 · a four-city teaching example

An attention model encodes the cities. A policy selects the next unvisited city, masks visited cities, and returns to the start after all cities have been visited.

The same four locations, two valid tours. These distances are an illustrative calculation, not a reported benchmark from the paper.
The same four locations, two valid tours. These distances are an illustrative calculation, not a reported benchmark from the paper.

For unit-square cities, the perimeter tour costs 4; the crossing tour costs 2+22≈4.8282+2\sqrt2\approx4.828. Training by REINFORCE compares a sampled tour with a greedy-rollout baseline and adjusts its probability using the cost difference.

The policy learns how to construct a solution; the objective still evaluates the resulting tour. Compare feasibility, tour length and computational budget with a conventional solver. Attention, Learn to Solve Routing Problems!
Lecture 0 · Data-Driven Decision Making and Control44 / 71
What comes nextLecture 0

Your class project

Carry an industrial decision from its context to a tested prototype. Choose one appropriate core method and a baseline; explain the evidence for the resulting decision.

part what you hand in the gate it must pass
1 · Business context the decision as it is made today, and what a better one is worth a reader can state the scope in one paragraph
2 · Problem definition objective, variables, hard and soft constraints, and the data contract report feasibility checks and compare the KPI with the incumbent under the same conditions
3 · Product & delivery the surface an operator would actually use someone else can run it end to end

Every method in Lectures 1–12 is a candidate for part 2. The point of the gate is that the choice of method is not the deliverable — a validated decision is.

Lecture 0 · Data-Driven Decision Making and Control45 / 71
What comes nextLecture 0

Check — two views of sequential decisions

Why does this course compare dynamic programming with optimal control?

Both traditions use values, policies and Bellman reasoning. Lectures 7–10 organize the comparison as known-model methods followed by learning from experience. The columns are a teaching map, not disjoint historical categories.

Lecture 0 · Data-Driven Decision Making and Control46 / 71
07

Closing

One story, told in chapters — and a map to keep in view for the rest of the term.
One question — decide well under uncertainty — asked along three criteria, answered by two lineages.
Keep the decision, information and evaluation rule explicit as the assumptions change.
Let us begin — Lecture 1.
Start with a known objective and constraints. Each chapter will connect a previous skill to a new question, with a small calculation to try.
APPENDIX

Examples, readings and derivations

Reference material for a second pass. The main teaching route ends before this section.
APPENDIXExamples, readings and derivationsLecture 0

Reading guide — learn a solver, then recognise the real decisionone main idea to explain, one comparison, one application

Role Read or revisit Question to answer
Core Kool, van Hoof & Welling, Attention, Learn to Solve Routing Problems! (ICLR 2019) How can a policy construct a route one choice at a time?
Compare A conventional routing heuristic and the NCO role table later in this lecture What is learned: the solution, an improvement step, or a solver component?
Apply The original wind-farm, traffic, furnace and routing cases Which objective, constraints and evaluation budget define success?
First identify the decision. Read the routing paper for its construction loop and experiment design; detailed network architecture is optional. The engineering cases show where this pattern must meet physical and operational constraints.
Lecture 0 · Data-Driven Decision Making and Control51 / 71
EXTENSION

Case studies — engineering models and measured results

Read after completing the main route.
EXTENSIONCase studies — engineering models and measured resultsLecture 0

The farm is a graph, so the model should be one

The representation

Why a graph network

Nodes are turbines and carry free-flow wind speed; edges carry the down-stream wake distance dd and radial distance rr; the global feature is the wind itself.

An appropriate graph readout is permutation-invariant, while node outputs are equivariant. It can accept different graph sizes; accurate transfer from five to twenty turbines still needs validation.

Lecture 0 · Data-Driven Decision Making and Control53 / 71
EXTENSIONCase studies — engineering models and measured resultsLecture 0

Physics-induced graph network — do not learn what you already know

Four trainable modules and three aggregators, stacked. The edge update is where the physics enters: , with a wake weight rather than a free
Four trainable modules and three aggregators, stacked. The edge update is where the physics enters: eij′=fw(eij)×fe(⋅)\mathbf{e}'_{ij} = f_w(\mathbf{e}_{ij}) \times f_e(\cdot), with fwf_w a wake weight rather than a free function.
Lecture 0 · Data-Driven Decision Making and Control54 / 71
EXTENSIONCase studies — engineering models and measured resultsLecture 0

Let the network learn α,κ,R0\alpha, \kappa, R_0

The classical wake model gives the deficit in closed form, but its constants were hand-tuned to observed data. Keep the functional form, and
The classical wake model gives the deficit δu(d,r)\delta u(d,r) in closed form, but its constants were hand-tuned to observed data. Keep the functional form, and learn the constants. This is the whole idea of an inductive bias, in one figure.
Lecture 0 · Data-Driven Decision Making and Control55 / 71
EXTENSIONCase studies — engineering models and measured resultsLecture 0

Does it work — and then, does it decide?

Prediction

1.5% MAPE on farm power.

Decision

Total power 7.5 → 20.0 over the optimisation, by gradient — because the surrogate is differentiable.

This is Lecture 5's thesis arriving early: a surrogate is worth building when it turns a problem you could only simulate into one you can differentiate. The warning that comes with it — that optimization can select optimistic surrogate errors — is Lecture 5's other half.

Lecture 0 · Data-Driven Decision Making and Control56 / 71
EXTENSIONCase studies — engineering models and measured resultsLecture 0

Why meta Bayesian optimisation

Ordinary BO can use an informative prior, but does not automatically transfer experience across intersections. Meta-BO pre-trains on a buffe
Ordinary BO can use an informative prior, but does not automatically transfer experience across intersections. Meta-BO pre-trains fθf_\theta on a buffer of past tasks, then adapts online — so the first trial on a new junction is already informed.
Lecture 0 · Data-Driven Decision Making and Control57 / 71
EXTENSIONCase studies — engineering models and measured resultsLecture 0

The result

Average number of waiting vehicles. Ours is best on every column, and the gain is largest where trials are scarcest — 395.5 vs 408.5 in Hang
Average number of waiting vehicles. Ours is best on every column, and the gain is largest where trials are scarcest — 395.5 vs 408.5 in Hangzhou, 859.7 vs 920.1 in Manhattan.
Lecture 0 · Data-Driven Decision Making and Control58 / 71
EXTENSIONCase studies — engineering models and measured resultsLecture 0

Case C · the result

2.03 °C average error at a 100-step prediction horizon, and stable over 900 steps — enough that the overshoot region, the part an operator a
2.03 °C average error at a 100-step prediction horizon, and stable over 900 steps — enough that the overshoot region, the part an operator actually worries about, is predicted rather than merely tracked.
Lecture 0 · Data-Driven Decision Making and Control59 / 71
EXTENSIONCase studies — engineering models and measured resultsLecture 0

Extension toward practical RLpp. 59 and 66 of the source — the professor's own roadmap

Many benchmark RL experiments allow extensive simulator interaction. The MDP formalism itself does not require a simulator or unlimited queries. Three extensions close the gap to a real plant, and this course takes two of them.

1 · Model-based RLlearn the dynamics, then plan (Lecture 11)
2 · Meta RLone policy across tasks whose dynamics and objectives change
3 · Offline RLlearn from a log, with no interaction at all (Lecture 12)
Why offline RL — in the source deck's own words

"Industrial systems typically do not have simulators, and it can be prohibitive to learn a policy by directly interacting with the real system. Deriving a policy using previously collected data (offline data) is preferable."

Lecture 0 · Data-Driven Decision Making and Control60 / 71
EXTENSIONCase studies — engineering models and measured resultsLecture 0

Case D · Offline meta policy learning, then online adaptation

Collect operation data across many tasks; train a general controller offline — the encoder, the critic and the actor share a buffer per task
Collect operation data across many tasks; train a general controller offline — the encoder, the critic and the actor share a buffer per task; then adapt on the target task with a handful of interactions, not a million. Offline pretraining connects to Lecture 12. The later online adaptation is an extension beyond its fixed-data learning setting.
Lecture 0 · Data-Driven Decision Making and Control61 / 71
EXTENSION

Research outlook — systems and applications

Read after completing the main route.
EXTENSIONResearch outlook — systems and applicationsLecture 0

One engine, three domains — ① Vehicle routingCase 1 of 3 · in production — generation, then test-time improvement search

Pre-training generates several complete tours at once; test-time search improves each one and reports its length (62 · 64 · 61); the runs land in an experience buffer that post-training uses to re-learn the solver. Nothing here is a hand-written heuristic — the search operator is itself learned.
Lecture 0 · Data-Driven Decision Making and Control63 / 71
EXTENSIONResearch outlook — systems and applicationsLecture 0

② Robot fleet — multi-agent path findingCase 2 of 3 · same engine, new domain — generative assignment, then parallel plan improvement

The same pipeline, retargeted

SILAB's own run

Agents blue, tasks red.

Generation now emits a task assignment; test-time search is a path search that reports the conflict count it is driving down (iter 4 · conflicts: 12). The plan is not an answer until the conflicts reach zero — which is a validator, not an objective.

Lecture 0 · Data-Driven Decision Making and Control64 / 71
EXTENSIONResearch outlook — systems and applicationsLecture 0

③ Chip placement and routingCase 3 of 3 · beyond logistics — the same relational search engine, applied to silicon

Generation, then collision-free wire routing

Meta-learned placement

Placement is meta design: the task changes with the higher-level design flow, so what must be learned is a solver that adapts, not a placement that is fixed. (NeurIPS'23)

Lecture 0 · Data-Driven Decision Making and Control65 / 71
EXTENSIONResearch outlook — systems and applicationsLecture 0

The same engine, compared with a solver baselinedelivery routing, in paid production

Package delivery · Gangnam

Distance 25.32 → 21.98 km, delivery time 18 → 14 min against Google OR-Tools.

Food & pharmacy delivery

200 couriers, ~25,000 pharmacies. Total distance 873.5 → 695.1 km; longest single route 67.4 → 37.4 km.

Lecture 0 · Data-Driven Decision Making and Control66 / 71
EXTENSIONResearch outlook — systems and applicationsLecture 0

Running in the field, across domainsone engine, wrapped per domain — these are operating systems, not demonstrations

Delivery routing

Vessel stowage

Airport operations

Parking allocation

Logistics network

Defense logistics

Each is the same combinatorial engine behind a different domain grammar and a different operator screen. What is reused is the solver; what is rebuilt is the contract with the operation.

Lecture 0 · Data-Driven Decision Making and Control67 / 71
EXTENSIONResearch outlook — systems and applicationsLecture 0

The architecture this is heading towardfoundation model → solver → OCP server → agent

One foundation model feeds a small set of problem-class solvers; each solver is wrapped, per domain, into an OCP server; and the applications a field engineer opens are assembled from those. Watch the paths light in turn — one model, one solver, four different operations. Every layer exists to make the one before it reusable.

Each layer makes the one above reusable. The foundation model creates the intelligence, the OCP server (Optimization Context Protocol) makes it reusable, and the platform makes it operable — deploy, connect, monitor, govern, remember, improve.

Lecture 0 · Data-Driven Decision Making and Control68 / 71
EXTENSIONResearch outlook — systems and applicationsLecture 0

The optimisation engine, talking to a language modelthis is the structure that puts optimisation in a field engineer's hands

A request in words (ll) is turned by the Formulator into an instance x=(xD,xC)x = (x_D, x_C) — problem data and solver configuration — which the OCP server solves; the Analyzer returns the answer in the engineer's language. Language models do the reading, writing and explaining; the solver does the deciding. Nothing here asks an LLM to optimise, and nothing asks the solver to understand English.
Lecture 0 · Data-Driven Decision Making and Control69 / 71
EXTENSIONResearch outlook — systems and applicationsLecture 0

The knowledge that makes the translation possible

An ontology over objective, constraints, data and parameters is what turns a sentence into a structured formulation, while exposing missing information and ambiguities that still need resolution. Supply it once per problem class, and every later run inherits it.
Lecture 0 · Data-Driven Decision Making and Control70 / 71
EXTENSIONResearch outlook — systems and applicationsLecture 0

The OI Factorythree gated loops inside one feedback loop — and the shape of your project

Three gated loops — business context, problem definition, product and delivery — each turning until the next role can act, all inside one outer loop that reopens when the operation talks back. Your project is one pass through this.
The rule that makes it a factory, not a project

Each part is finished only when the next role can act on it. Part 2 is done when another person — or a model — could rebuild a working solver from it alone.

Lecture 0 · Data-Driven Decision Making and Control71 / 71

Keys

→ / space
next reveal, then next slide
←
back
↑ ↓
skip a whole slide
R
reset this slide's widgets and quizzes
M
slide index
P
print → save as PDF (1280×720)
F
fullscreen