IE437 · Data-Driven Decision Making and Control — Lecture 6

Data-Driven Design Optimization — Generative-Based

Learn where good designs live — then search, refit, condition or steer
Jinkyoo Park · KAIST
Offline, part 2 of 2 — a generator decides where to look; a performance model decides what to trust
Data-Driven Design Optimization — Generative-BasedLecture 6

Where we are — from judging a design to proposing one

Lecture 4 · Observe Lecture 5 · Trust Lecture 6 · Generate
the question which experiment to run next on what evidence to judge a design no one has measured which designs to propose, and how to steer that toward good ones
the model a GP of ff a performance model f^\hat f, trusted with care a generator of plausible designs, plus a performance model
the decision query the acquisition's arg-max return the surrogate's arg-max, carefully search, refit, condition or steer the generator
Lecture 5 gave the performance model grounds for trust. This lecture adds a model of where to look. A generator structures the search; it does not create evidence about performance — that still comes from the data, and from Lecture 5's tools.
Lecture 6 · Data-Driven Design Optimization — Generative-Based2 / 78
Data-Driven Design Optimization — Generative-BasedLecture 6

Learning route — two sessions, four places to intervene

Bring: the surrogate, the decision bound and how an optimiser exploits it (Lecture 5); Bayes' rule and likelihood (Lecture 2); sampling and weighting (Lecture 3).

Session What it does
1 · Generate, then search or refit What surrogates leave → two models, two jobs → KL, VAE and diffusion as tools → G1: search in learned coordinates → G2: refit the generating distribution, with CbAS worked through
2 · Ask, and steer G3: condition on the performance you want — CVAE, MINs, DDOM → a KL-regularised view of candidate distributions → G4: steer the sampling process → the four side by side, with Lecture 5's tools in each
Keep three questions apart throughout — Lecture 5 closed on them: is it good (performance), is it possible (validity), can the prediction be trusted (reliability). Each method buys one and can spend another.
Lecture 6 · Data-Driven Design Optimization — Generative-Based3 / 78
Session 1 · Act 1

What surrogates still leave

Q1. Lecture 5 gave the surrogate grounds for trust. Three problems are still standing.
ACT 1What surrogates still leaveLecture 6

The problem, unchanged — and what the algorithm may use

The same problem as Lecture 5, with the valid set written in:

findx∗=arg max⁡x∈Xvalidf(x)with only D={(xi,yi)}i=1N,yi=f(xi)+ϵi\text{find}\quad x^* = \argmax_{x\in\hl{\mathcal X_{\text{valid}}}} f(x) \qquad\text{with only } D = \{(x_i,y_i)\}_{i=1}^N,\quad y_i=f(x_i)+\epsilon_i

What the algorithm may use

The fixed dataset, and any knowledge stated up front — a known constraint can be checked directly. Strict offline: ff is never called while training, choosing or tuning.

What it may not

The hidden oracle that scores the final candidates — as in Lecture 5's Design-Bench. The widgets draw the true ff for the teacher's eyes only.

Source alignment: Part 1 pp. 2–3, repeated as Part 2 pp. 3–4 — the source opens Lecture 6 with Lecture 5's slide; the valid set and the information boundary are made explicit here.

Lecture 6 · Data-Driven Design Optimization — Generative-Based5 / 78
ACT 1What surrogates still leaveLecture 6

What Lecture 5 left standing

Q1 What do surrogates leave?Q2 How do we model designs?Q3 Coordinates or distribution?Q4 Condition or sampler?

A huge space

The optimiser still moves one point through every coordinate of xx — a protein of 237 residues has 2023720^{237} sequences. Conservatism made each move safer, not the space smaller.

Validity

Most of xx is not a design at all — not a molecule, not a folded protein, not an image of a digit. Lecture 5's last slide: out of distribution and invalid are different problems, and a surrogate scores both.

One answer

The arg-max returns one design; a lab wants a batch of different good candidates. Reliable scoring alone does not provide convenient coordinates, or a distribution of diverse, plausible candidates.

The source's answer: model what plausible designs look like, and steer that model toward good ones — a distribution over designs, plus a performance signal.

Source alignment: Part 1 p. 4 — the main problem, conservatism limiting expressivity, and three promises of sampling (no optimisation, diverse designs, constraints satisfied implicitly). The first and third are softened here: several methods below still optimise, and a learned prior encourages validity without guaranteeing it.

Lecture 6 · Data-Driven Design Optimization — Generative-Based6 / 78
ACT 1What surrogates still leaveLecture 6

One yy, many xx — the average answer can be invalid

Run the forward map backwards — from the yy you need to the xx that delivers it — and it is not a function: a curve in two pieces. The red mark is the midpoint of that set — what a least-squares inverse returns — stranded in the valley between the branches.
To keep several answers, make the inverse stochastic: x=fθ−1(y,z)x=f^{-1}_\theta(y,z) with z∼p(z)z\sim p(z) choosing among them. That is a generative model.

The same point in one line: for y=x2y=x^2 and y=4y=4, the two answers are ±2\pm2; their average 00 scores 00. The widget's numbers: every point of the curve is worth 2.002.00, their midpoint 0.810.81. Source alignment: Part 2 p. 20 ("make the inverse mapping stochastic … cf. CVAE").

Lecture 6 · Data-Driven Design Optimization — Generative-Based7 / 78
ACT 1What surrogates still leaveLecture 6

Two models, two jobs

The performance model — Lecture 5

How good is this candidate? Trained on pairs (x,y)(x,y). Wrong off the data, and exploited by whatever searches it — so Lecture 5 gave it grounds for trust: conservatism, uncertainty, knowledge.

The generator — this lecture

Which candidates should we look at? Trained on designs xx (sometimes with yy). It offers plausibility, diversity and coordinates — but alone it only samples designs like the data, mediocre ones included.

A generator is not an optimiser: every generative optimiser needs a performance signal — the dataset's labels or a performance model. Where can it enter? Four places, G1–G4 — taken up after the tools:
G1 · coordinatessearch a code, then decode
G2 · distributionrefit the generator toward good designs
G3 · conditionask the generator for a performance
G4 · samplersteer the sampling process
Lecture 6 · Data-Driven Design Optimization — Generative-Based8 / 78
ACT 1What surrogates still leaveLecture 6

Check — reading the function backwards

Lecture 5 learned f^:x↦y\hat{f}: x \mapsto y and searched it. A score-conditioned inverse model — one of this lecture's four routes — learns the other direction. What makes that harder than it sounds?

Many different designs can score the same yy. A model trained to regress xx from yy under a squared loss returns the average of those designs — very often not a valid design at all. That is why the answer has to be a generative model that samples, rather than a regressor that predicts.

Lecture 6 · Data-Driven Design Optimization — Generative-Based9 / 78
Session 1 · Act 2

Generative models, compressed

Q2. Before steering a generator toward good designs, you need one that produces designs at all — and a way to say how close two distributions are.
ACT 2Generative models, compressedLecture 6

Comparing two distributions — the KL divergence

Q1 What do surrogates leave?Q2 How do we model designs?Q3 Coordinates or distribution?Q4 Condition or sampler?
DKL(a ∥ b)=Ex∼a[log⁡a(x)b(x)]  ≥0,=0 only when a=b.D_{\mathrm{KL}}(a\,\|\,b)=\E_{x\sim a}\Big[\log\frac{a(x)}{b(x)}\Big]\;\ge 0,\qquad =0 \text{ only when } a=b.
Press → twice: average with the data's eyes, then with the model's.
This lecture uses both directions: fitting a model averages over the data — cover the data; keeping a search near the generator averages over the search — stay where the generator lives.

Not met before in this course. It is finite only if b>0b>0 wherever a>0a>0 — the widget's last step. Fitting: DKL(pdata∥pθ)D_{\mathrm{KL}}(p_{\text{data}}\|p_\theta) (next slide) and CbAS's fit. Staying near: DKL(q∥pgen)D_{\mathrm{KL}}(q\|p_{\text{gen}}), the tilt of Session 2. The VAE's code term, DKL(qϕ(z∣x)∥p(z))D_{\mathrm{KL}}(q_\phi(z\mid x)\|p(z)), is the second direction applied to codes. KL is not a distance: it is not symmetric and has no triangle inequality.

Lecture 6 · Data-Driven Design Optimization — Generative-Based11 / 78
ACT 2Generative models, compressedLecture 6

The goal — a distribution concentrated on plausible designs

The goal

Train pθ(x)p_\theta(x) to match the data distribution pdata(x)p_{\text{data}}(x), given only samples {xi}i=1N∼pdata\{x_i\}_{i=1}^N\sim p_{\text{data}}.

arg min⁡θ DKL(pdata ∥ pθ)  =  arg max⁡θ Epdata[log⁡pθ(x)]\argmin_{\theta}\ D_{\mathrm{KL}}\big(p_{\text{data}}\,\|\,p_\theta\big) \;=\; \hl{\argmax_{\theta}\ \E_{p_{\text{data}}}[\log p_\theta(x)]}

The first term of the KL does not depend on θ\theta: fitting a generative model is maximum likelihood — Lecture 2's principle, for a model whose likelihood has no closed form.

What is learned is where plausible designs concentrate — not a feasibility certificate. A learned density puts low, not zero, probability away from the data, and like Lecture 5's surrogate it is unconstrained where there is no data (Backup 4). Known constraints still need a check, or a decoder that enforces them.

Source alignment: Part 1 pp. 6–10 (generative models, the KL-to-likelihood step, the latent-variable model). Four families do this — VAEs, GANs, flows, diffusion (Backup 2); design methods mostly build on two.

Lecture 6 · Data-Driven Design Optimization — Generative-Based12 / 78
ACT 2Generative models, compressedLecture 6

VAE — encode, decode, and an objective with two terms

Assert that designs come from a few simple hidden factors: p(x)=∫pθ(x∣z) p(z) dzp(x)=\int p_\theta(x\mid z)\,p(z)\,dz, z∼N(0,I)z\sim\mathcal N(0,I). A decoder pθ(x∣z)p_\theta(x\mid z) produces the design; an encoder qϕ(z∣x)q_\phi(z\mid x) approximates the posterior pθ(z∣x)p_\theta(z\mid x) — Lecture 3's hidden-state inference, done by a network.

L(x)=Eqϕ(z∣x)[log⁡pθ(x∣z)]⏟reconstruct the design  −  DKL(qϕ(z∣x) ∥ p(z))⏟keep codes near the prior\mathcal{L}(x) = \underbrace{\E_{q_\phi(z\mid x)}\big[\log p_\theta(x\mid z)\big]}_{\text{\hl{reconstruct the design}}} \;-\; \underbrace{D_{\mathrm{KL}}\big(q_\phi(z\mid x)\,\|\,p(z)\big)}_{\text{\hl{keep codes near the prior}}}

Generating

Draw z∼N(0,I)z\sim\mathcal{N}(0,I), decode. No data, no optimiser. The reparameterisation z=μϕ+σϕ⊙ϵz=\mu_\phi+\sigma_\phi\odot\epsilon lets gradients through the sample.

Which prior?

p(z)p(z) is a prior over codes, not Lecture 2's prior over parameters: the network's weights are point estimates, and drawing zz does not express uncertainty about them.

The KL term makes the latent space samplable: codes from the prior decode to plausible designs. The latent space is a coordinate system for designs — G1 will search in it.

The standard ELBO has weight 1 on the KL; the β\beta-VAE of the next slide changes that weight and is a different objective. The bound and the reparameterisation are derived in Backup 1; a one-coordinate ELBO calculation is in the appendix. Source alignment: Part 1 pp. 11–13.

Lecture 6 · Data-Driven Design Optimization — Generative-Based13 / 78
ACT 2Generative models, compressedLecture 6

Turning the dial

An exactly solvable VAE with a weight β\beta on the KL term. Each dot is a design's code, each halo its blur, the ring the prior. At β=1\beta=1 the codes fill the prior exactly; below it they spill outside, above it they collapse to a dot.

Spread plus blur sum to exactly 1.0001.000 at β=1\beta=1, 1.5651.565 at β=0.05\beta=0.05 and 0.9300.930 at β=20\beta=20 (Backup 1 has the formula).

Lecture 6 · Data-Driven Design Optimization — Generative-Based14 / 78
ACT 2Generative models, compressedLecture 6

Diffusion — learn the noise, then remove it step by stepHo, Jain & Abbeel · NeurIPS 2020

A VAE stacked TT times, with the encoder fixed to a Gaussian noising process and the latent the size of the data — so training needs no inference network.

Train — corrupt once, predict the noise

xt=αˉt x0+1−αˉt ϵ,Lsimple=E ∥ϵ−ϵθ(xt,t)∥2x_t=\sqrt{\bar\alpha_t}\,x_0+\sqrt{1-\bar\alpha_t}\,\epsilon,\qquad L_{\mathrm{simple}}=\E\,\big\|\epsilon-\epsilon_\theta(x_t,t)\big\|^2

Draw a clean design, corrupt it by a known amount, and predict the noise that was added — a supervised problem at every noise level.

Generate — start from noise, denoise

xt−1=1αt(xt−βt1−αˉt ϵθ(xt,t))+σtzx_{t-1}=\frac{1}{\sqrt{\alpha_t}}\Big(x_t-\frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\,\epsilon_\theta(x_t,t)\Big)+\sigma_t z

Start from xT∼N(0,I)x_T\sim\mathcal N(0,I), not from a training design; each step is a small denoising step.

Denoising steps are steps of the algorithm, not physical time — and each one is a place where the sampler could be steered: Session 2 shows two ways.

αt=1−βt\alpha_t=1-\beta_t, αˉt=∏j≤tαj\bar\alpha_t=\prod_{j\le t}\alpha_j, z∼N(0,I)z\sim\mathcal N(0,I) at stochastic steps and none at the last. The full variational derivation and its weights are Backups 2b and 3; a one-number training step is in Backup 3. Source alignment: Part 1 pp. 15–22, 47–63.

Lecture 6 · Data-Driven Design Optimization — Generative-Based15 / 78
ACT 2Generative models, compressedLecture 6

Check — turning the KL weight

In a β\beta-VAE, you raise β\beta, the weight on the KL term. What do you get?

The KL term pulls the codes toward the prior; the reconstruction term pulls them apart so that designs stay distinguishable. Raising β\beta wins the first and loses the second: prior samples decode to plausible designs, but detail is lost. It is the same trade as Lecture 5's conservatism dial — fidelity to the data against freedom to generate.

Lecture 6 · Data-Driven Design Optimization — Generative-Based16 / 78
Session 1 · Act 3

Coordinates and distribution

Q3. A generator and a performance model are on the table. Where does the performance signal get in?
ACT 3Coordinates and distributionLecture 6

Four places to intervene

Q1 What do surrogates leave?Q2 How do we model designs?Q3 Coordinates or distribution?Q4 Condition or sampler?
where the performance signal enters what changes in depth where Lecture 5's failure can re-enter
G1 · coordinates the search over codes search a code zz; decode x=g(z)x=g(z) latent search (2018) the predictor on codes is a surrogate
G2 · distribution the generator's parameters refit p(x)p(x) toward predicted success CbAS (2019) the refit follows the success model's errors
G3 · condition the generator's input ask p(x∣y)p(x\mid y) for a high yy MINs (2020), DDOM (2023) a requested yy beyond the data
G4 · sampler the sampling process performance information moves the sample (a gradient) or sets the guidance strength classifier guidance (2021), RGD (2024) the sampler follows the performance model where it is wrong
Intervention points, not rival classes: a method can use several. For each, ask what it assumes, what it solves, what it leaves — and wherever a surrogate enters, Lecture 5's three streams still apply.
Lecture 6 · Data-Driven Design Optimization — Generative-Based18 / 78
ACT 3Coordinates and distributionLecture 6

The papers of this lecture — why each is here

Press → for Session 2's papers; click any paper for what it does, why we read it and how it connects. The same list is a table in the appendix.
Lecture 6 · Data-Driven Design Optimization — Generative-Based19 / 78
ACT 3Coordinates and distributionLecture 6

G1 — search in learned coordinatesGómez-Bombarelli et al. · ACS Central Science 2018

Lecture 4 met this paper as BO in a learned space. A VAE turns molecules (SMILES strings) into continuous codes, with a property predictor trained on the codes; search the codes — by gradient steps or Bayesian optimisation — and decode the best.

The search becomes low-dimensional and continuous, and every point it visits decodes to something. Whether that something is valid, and whether the predictor is right about it, are two separate bets.
Encode, predict, optimise in the latent space, decode. {p}(Gómez-Bombarelli et al., 2018)
Encode, predict, optimise in the latent space, decode. (Gómez-Bombarelli et al., 2018)

Lecture 4 used it for its dimension reduction: a 196-dimensional latent space for ZINC molecules, searched with a GP. Here the point is validity: the coordinates were learned from valid molecules. About 74% of random latent points decode to a valid molecule (Lecture 4, from the paper's Table 4).

Lecture 6 · Data-Driven Design Optimization — Generative-Based20 / 78
ACT 3Coordinates and distributionLecture 6

Search the plane, or search the code

One 2-D design problem whose valid designs form a thin curve. Press → twice: climb the surrogate in the plane, then search the code. The decoder here is the curve itself, known exactly — a teaching device; a learned decoder is only approximately valid.

Numbers behind the picture: ascent on the quadratic surrogate f^(x)\hat f(x) (data RMSE 0.0650.065) leaves the curve within two steps and reaches the corner, where f^\hat f promises 4.974.97. The latent surrogate f^(t)\hat f(t) picks t=2.8t=2.8: f^=1.88\hat f=1.88, truth 0.390.39; the true best is 2.402.40 at t=1.8t=1.8, beyond the last measured code (t≈1.05t\approx1.05). Lecture 5's BB, now on a valid design.

Lecture 6 · Data-Driven Design Optimization — Generative-Based21 / 78
ACT 3Coordinates and distributionLecture 6

What latent search buys, and what it costs

Assumption added

The latent space is faithful: nearby codes decode to similar, valid designs, and the predictor on codes is honest.

What it solves

A continuous, low-dimensional search whose candidates are designs — validity moves into the decoder.

What it leaves

Codes the decoder handles badly decode to invalid designs; the latent predictor is a surrogate, exploitable exactly as in Lecture 5.

the fix how paper
validity by construction generate a tree of chemically valid substructures, then assemble — every output is a valid molecule (valid chemistry, not yet synthesisable or good) JT-VAE (Jin, Barzilay & Jaakkola, ICML 2018)
reliability, from Lecture 5 train the predictor on codes with COMs' penalty LCOMs (Qi et al., 2023 workshop)
Validity and reliability are repaired separately: a better decoder for one, Lecture 5's tools for the other.
Lecture 6 · Data-Driven Design Optimization — Generative-Based22 / 78
ACT 3Coordinates and distributionLecture 6

G2 — condition the generator on success

Move the distribution, not a point. With the generator as the prior and a success model as the likelihood — SS: "meets the requested performance" — Bayes' rule gives the designs we want:

p(x∣S)=P(S∣x) p(x)∑x′P(S∣x′) p(x′)p(x\mid S)=\frac{P(S\mid x)\,p(x)}{\sum_{x'}P(S\mid x')\,p(x')}

The prior keeps it plausible

Designs the generator finds implausible keep little mass, however promising the success model says they are.

The success model is Lecture 5's

Offline, P(S∣x)P(S\mid x) is a prediction — e.g. P(y^≥γ∣x)P(\hat y\ge\gamma\mid x) from a surrogate. Its errors are inside the target; a pessimistic one (μ−κσ\mu-\kappa\sigma) gives a pessimistic target.

Lecture 3 conditioned a fixed model on observed evidence. Here the "evidence" is a requested outcome, judged by a model — the target is only as good as the success model.
Lecture 6 · Data-Driven Design Optimization — Generative-Based23 / 78
ACT 3Coordinates and distributionLecture 6

Temperature thread — condition a distribution over heater designsshared teaching example · predict before revealing

Lecture 0's heater as a discrete design: which of two heaters to switch on, instead of how much command to apply. Exactly one may be on.

Predict: After conditioning on success, which design becomes more likely — and is the average of the two a valid design?

The left heater: conditioning multiplies the prior by the success probability. And no — the average turns both heaters half on. Press → to see both.

Numbers behind the picture: prior (0.5,0.5)(0.5,0.5), P(S∣L)=0.8P(S\mid L)=0.8, P(S∣R)=0.2P(S\mid R)=0.2; products (0.4,0.1)(0.4,0.1), so P(L∣S)=0.8P(L\mid S)=0.8, P(R∣S)=0.2P(R\mid S)=0.2. The average (0.5,0.5)(0.5,0.5) activates both heaters and breaks the design rule.

Lecture 6 · Data-Driven Design Optimization — Generative-Based24 / 78
ACT 3Coordinates and distributionLecture 6

Try it — the generator now prefers the right heaterwork independently · reveal only after deciding

The same success model; the generator now proposes the right heater four times as often.

Which design is more likely after conditioning on success?

Check your answer. Neither — the two pulls cancel, half and half. What you sample is set by the prior and the success model together, and every draw is still L or R, never their average. Press → to see it.

Numbers behind the picture: prior (0.2,0.8)(0.2,0.8); products (0.2×0.8,  0.8×0.2)=(0.16,0.16)(0.2\times0.8,\;0.8\times0.2)=(0.16,0.16), so the conditional is (0.5,0.5)(0.5,0.5).

Lecture 6 · Data-Driven Design Optimization — Generative-Based25 / 78
ACT 3Coordinates and distributionLecture 6

CbAS — fit a generator to the conditionalConditioning by Adaptive Sampling · Brookes, Park & Listgarten · ICML 2019

The target, with the trained generator p(x∣θ(0))p(x\mid\theta^{(0)}) as the prior, cannot be sampled directly — so fit a second generator q(x∣ϕ)q(x\mid\phi) to it, averaging over the target (cover it):

p(x∣S,θ(0))=P(S∣x) p(x∣θ(0))P(S∣θ(0)),ϕ∗=arg min⁡ϕ DKL(p(x∣S,θ(0)) ∥ q(x∣ϕ))p\big(x\mid S,\theta^{(0)}\big) = \frac{P(S\mid x)\,p\big(x\mid\theta^{(0)}\big)}{P\big(S\mid\theta^{(0)}\big)},\qquad \phi^* = \argmin_\phi\ D_{\mathrm{KL}}\big(p(x\mid S,\theta^{(0)})\,\|\,q(x\mid\phi)\big)

Derivation. Expand the KL and drop what does not depend on ϕ\phi:

DKL=−Ep(x∣S,θ(0))[log⁡q(x∣ϕ)]−H0=−1P(S∣θ(0)) Ep(x∣θ(0))[P(S∣x)log⁡q(x∣ϕ)]−H0D_{\mathrm{KL}} = -\E_{p(x\mid S,\theta^{(0)})}\big[\log q(x\mid\phi)\big]-H_0 = -\frac{1}{P(S\mid\theta^{(0)})}\,\E_{p(x\mid\theta^{(0)})}\big[P(S\mid x)\log q(x\mid\phi)\big]-H_0
⟹ϕ∗=arg max⁡ϕ Ep(x∣θ(0))[P(S∣x) log⁡q(x∣ϕ)]\Longrightarrow\qquad \phi^*=\hl{\argmax_\phi\ \E_{p(x\mid\theta^{(0)})}\big[P(S\mid x)\,\log q(x\mid\phi)\big]}
Weighted maximum likelihood: draw designs from the generator, weight each by how likely it is predicted to succeed, refit. Any generative model can be trained this way, and qq need not be of the prior's form.

H0=−Ep(x∣S,θ(0))[log⁡p(x∣S,θ(0))]H_0=-\E_{p(x\mid S,\theta^{(0)})}[\log p(x\mid S,\theta^{(0)})] is constant in ϕ\phi. Source alignment: Part 2 pp. 9–11.

Lecture 6 · Data-Driven Design Optimization — Generative-Based26 / 78
ACT 3Coordinates and distributionLecture 6

Why one shot fails — Lecture 3's rare cause, again

Draw from the prior and weight by P(S∣x)P(S\mid x): success is exceedingly rare, so almost every weight is near zero — Lecture 3's likelihood weighting with a rare cause, where almost no draw lands where the evidence points.

Sample from last round's model

Use the previous fit as the proposal, r(t)=q(x∣ϕ(t−1))r^{(t)}=q(x\mid\phi^{(t-1)}), and correct with the importance ratio:

wi=p(xi∣θ(0))q(xi∣ϕ(t−1))  P(S(t)∣xi)w_i=\frac{p(x_i\mid\theta^{(0)})}{q(x_i\mid\phi^{(t-1)})}\;P\big(S^{(t)}\mid x_i\big)

Raise the bar gradually

  • (A) each relaxed condition S(t)S^{(t)} is met often enough under the current proposal;
  • (B) S(t)⊃S(t+1)⊃SS^{(t)}\supset S^{(t+1)}\supset S — the bar only rises, and reaches the target.
A hard conditional query, replaced by a sequence of easy ones. Unlike Lecture 3, the model is not fixed: the proposal and the condition both move.

The source sets S(t)S^{(t)} to y≥γ(t)y\ge\gamma^{(t)} with γ(t)\gamma^{(t)} the QQ-th percentile of the oracle's predictions on the current samples, capped at the target. The ratio corrects the change of proposal; it is not a validity check, and it fails if the proposal misses part of the target. Source alignment: Part 2 pp. 12–15.

Lecture 6 · Data-Driven Design Optimization — Generative-Based27 / 78
ACT 3Coordinates and distributionLecture 6

Why the importance ratio belongs in the weight

Three designs, sampled from last round's model instead of the prior. Press → four times.

Numbers: prior (0.6,0.3,0.1)(0.6,0.3,0.1), success (0.1,0.5,0.8)(0.1,0.5,0.8), target p×P(S)=(0.06,0.15,0.08)p\times P(S)=(0.06,0.15,0.08). Proposal q=(0.2,0.3,0.5)q=(0.2,0.3,0.5); weights (0.6/0.2)(0.1)=0.30(0.6/0.2)(0.1)=0.30, (0.3/0.3)(0.5)=0.50(0.3/0.3)(0.5)=0.50, (0.1/0.5)(0.8)=0.16(0.1/0.5)(0.8)=0.16; q×q\timesweight gives back the target. Without the ratio, q×P(S)=(0.02,0.15,0.40)q\times P(S)=(0.02,0.15,0.40).

Lecture 6 · Data-Driven Design Optimization — Generative-Based28 / 78
ACT 3Coordinates and distributionLecture 6

One shot, then the ladder

Press → to ask for the target in one shot, then to climb. Here qq is a Gaussian refit by weighted maximum likelihood — the pattern Lecture 4's CMA-ES also follows: sample, weight the good candidates, refit the distribution.

Target 2.652.65: one shot keeps 7070 effective draws of 4,0004{,}000; the ladder's worst round keeps 27%27\%; the final design is worth 2.6472.647 against the dataset's best 2.5232.523 (true optimum 2.6572.657). Lecture 5's objective, with 200 designs.

Lecture 6 · Data-Driven Design Optimization — Generative-Based29 / 78
ACT 3Coordinates and distributionLecture 6

What CbAS does on a real design problemprotein fluorescence, 5,000 measured sequences · Brookes et al., 2019

Each round, dashed: the oracle's 80th percentile of the samples; solid: their true mean.

Methods that chase the oracle — RWR, DbAS, and CEM-PI, a distribution search like Lecture 4's — climb on the dashed line while the solid line falls: they have found where the oracle is wrong. CbAS keeps the two together.

Lecture 5's failure, now in a distribution: a generator refit to a surrogate's favourites drifts to where the surrogate is wrong, unless the prior holds it. The ratio p/qp/q is that hold — a partial one.
80th percentile of oracle predictions (dashed) and mean ground truth of the same samples (solid). {p}(Brookes et al., 2019, Fig. 2b)
80th percentile of oracle predictions (dashed) and mean ground truth of the same samples (solid). (Brookes et al., 2019, Fig. 2b)

The paper's caption: high dashed and low solid values mark methods "led astray into pathological regions of the oracle". DbAS is CbAS without the prior ratio. Source alignment: Part 2 p. 16.

Lecture 6 · Data-Driven Design Optimization — Generative-Based30 / 78
ACT 3Coordinates and distributionLecture 6

Two more ways to move the distribution — GFlowNet and BootGen

GFlowNet (Bengio, Jain, Korablyov, Precup & Bengio · NeurIPS 2021)

Build a design step by step — an atom, a fragment, a residue — and train the steps so that a finished design is drawn in proportion to its reward, P(x)∝R(x)P(x)\propto R(x): a flow-consistency loss, not a refit to weighted samples.

It needs reward calls: online if they query the real objective; offline, a proxy — Lecture 5's surrogate. Our lab's Genetic-guided GFlowNets (NeurIPS 2024) adds a genetic search under a small budget of real oracle calls — online.

BootGen (Kim, Berto, Ahn & Park · NeurIPS 2023 · our lab)

Train pθ(x∣y)p_\theta(x\mid y) with weights by rank of yy; sample at a high y†y^\dagger, let a proxy label the samples, add the top KK, retrain; pool several such generators.

G2 — it refits the training distribution toward its own best samples, as CbAS does — and G3 — it is conditioned on yy. The added samples are proxy-labelled synthetic samples — no new performance measurements.

Both shift probability toward good designs, by objectives different from CbAS's — and both rely on a proxy wherever the real objective is not called.

Source alignment: Part 2 pp. 35–41 (BootGen). Rank weights w∝(k∣D∣+rank⁡(y,D))−1w\propto(k|D|+\operatorname{rank}(y,D))^{-1}; the loop figure, Algorithm 1 and the results table are on the appendix slides.

Lecture 6 · Data-Driven Design Optimization — Generative-Based31 / 78
ACT 3Coordinates and distributionLecture 6

What moving the distribution buys, and what it costs

Assumption added

The success model is right where the refit goes, and the prior gives the good designs some mass to begin with.

What it solves

Many good candidates at once, from a model that stays tied to plausible designs.

What it leaves

The refit follows the success model's errors as far as the prior allows; a bar set too high has nothing to climb to.

Lecture 5's tools plug in here: build P(S∣x)P(S\mid x) from a pessimistic prediction (μ−κσ\mu-\kappa\sigma) or a conservative surrogate, and the refit has less to chase.
Lecture 6 · Data-Driven Design Optimization — Generative-Based32 / 78
ACT 3Coordinates and distributionLecture 6

Check — coordinates or distribution?

A latent-space search returns a code whose decoded molecule is chemically invalid; the property predictor rates it the best ever seen. Before any evaluation, what has been established?

Invalidity is observed directly: the decoder left the valid set — a problem of the coordinates, which JT-VAE addresses by construction. The predicted value is not a measurement: for an invalid design the property is not even defined, and for valid ones the predictor's reliability is Lecture 5's question, settled only by evaluating designs. CbAS keeps samples where the prior has mass, which helps, but a learned prior has tails: nothing here guarantees validity.

Lecture 6 · Data-Driven Design Optimization — Generative-Based33 / 78
SESSION 1Coordinates and distributionLecture 6

Session 1 — what generation fixed, and what it left

Mitigated

Plausibility and diversity: candidates come from a model of plausible designs. G1 searched its coordinates; G2 moved its distribution toward predicted success.

Left

  • the performance signal still comes from a surrogate, exploited by the search (G1) or the refit (G2);
  • G2 needed a bar γ\gamma, a success model and many refits.
Session 2 moves the signal closer to the generator: ask for the performance directly (G3), or use it to steer the sampling process (G4) — and a KL-regularised view connects several of them.
Lecture 6 · Data-Driven Design Optimization — Generative-Based34 / 78
Session 2 · Act 4

Condition and sampler

Q4. Instead of refitting the generator, tell it what you want — or steer it at every step.
ACT 4Condition and samplerLecture 6

Session 2 — which distribution is this?

Q1 What do surrogates leave?Q2 How do we model designs?Q3 Coordinates or distribution?Q4 Condition or sampler?
distribution the question it answers where we met it
pθ(x)p_\theta(x) which designs to propose Session 1: the generator
pθ(x∣y)p_\theta(x\mid y) which designs for a requested yy today: G3
f^(x)\hat f(x), P(S∣x)P(S\mid x), μ±κσ\mu\pm\kappa\sigma how good a design is predicted to be Lecture 5: the performance model
p(z)p(z) how to draw a code the VAE's latent prior
p(w∣D)p(w\mid D) which model weights are credible, given the data — a Bayesian posterior Lecture 2
ensemble disagreement how much independently trained models differ — a non-Bayesian uncertainty signal Lecture 5 (Deep Ensembles)
The diversity of pθ(x)p_\theta(x) is not uncertainty about performance, and a requested yy is not a measured yy. Today: the condition (G3), the sampler (G4), and a KL-regularised view of both.
Lecture 6 · Data-Driven Design Optimization — Generative-Based36 / 78
ACT 4Condition and samplerLecture 6

The papers of Session 2

The two right-hand columns are today's. Click a paper for what it does, why we read it and how it connects.
Lecture 6 · Data-Driven Design Optimization — Generative-Based37 / 78
ACT 4Condition and samplerLecture 6

Condition the encoder and the decoder — the inverse map is builtthe conditional VAE — G3 in its simplest form

max⁡θ,ϕ Eqϕ(z∣x,y)[log⁡pθ(x∣z,y)]  −  DKL(qϕ(z∣x,y) ∥ pθ(z∣y))\max_{\theta,\phi}\ \E_{q_\phi(z\mid x,\hl{y})}\big[\log p_\theta(x\mid z,\hl{y})\big] \;-\; D_{\mathrm{KL}}\big(q_\phi(z\mid x,\hl{y})\,\|\,p_\theta(z\mid \hl{y})\big)

Feed the condition yy into both networks. Trained on street-number photographs, each row is the digit asked for and each column a style: the condition fixed, the rest of the freedom sampled.

And the source says the quiet part: yy can be the desired property, and xx the corresponding design — with the warning that a requested yy is not a measured one.
Samples from a CVAE trained on SVHN: row requested digit. {p}(source deck, Part 1 p. 14)
Samples from a CVAE trained on SVHN: row = requested digit. (source deck, Part 1 p. 14)
Lecture 6 · Data-Driven Design Optimization — Generative-Based38 / 78
ACT 4Condition and samplerLecture 6

G3 — ask the inverse map for a scoreModel Inversion Networks · Kumar & Levine · NeurIPS 2020

Training — which yy to learn on?

Accuracy at high yy matters most, but the best few points alone are too noisy. So reweight:

wi=p(yi)pD(yi),p(y)∝NyNy+K e y−y∗w_i=\frac{p(y_i)}{p_D(y_i)},\qquad p(y)\propto \frac{N_y}{N_y+K}\,e^{\,y-y^*}

Up-weight high scores — provided enough data share that score.

Querying — how far past the data to ask?

An optimisation over (y,z)(y,z): the largest yy that an independently trained forward model still confirms:

y~,z~=arg max⁡y,z fθ(fθ−1(y,z))\tilde y,\tilde z = \argmax_{y,z}\ f_\theta\big(f^{-1}_\theta(y,z)\big)
s.t.  ∥y−fθ(fθ−1(y,z))∥2≤ϵ1,p(z)≥ϵ2\text{s.t.}\ \ \big\|y - f_\theta(f^{-1}_\theta(y,z))\big\|^2 \le \epsilon_1,\quad p(z)\ge\epsilon_2
Even the inverse route keeps a forward model — as a consistency check on how far to ask, Lecture 1's trust region in a new costume.

NyN_y is the number of points with score yy; y∗y^* the best score in DD. The weight comes from a bias–variance bound (Backup 5). Source alignment: Part 2 pp. 17–24.

Lecture 6 · Data-Driven Design Optimization — Generative-Based39 / 78
ACT 4Condition and samplerLecture 6

An inverse map returns designs; the forward map's arg-max did not

"The thickest 3 that is still recognisable": MIN and its ablations (left) against the forward map's arg-max (F), and the scores. {p}(Kumar &
"The thickest 3 that is still recognisable": MIN and its ablations (left) against the forward map's arg-max (F), and the scores. (Kumar & Levine, 2020)
On two of these tasks the forward map's answer is not a wrong design but no design at all — an array of pixels that is not a digit. Lecture 5's failure, reported as a table entry: Invalid.

Source alignment: Part 2 pp. 25–28; the full table, including faces and controllers, is the appendix slide "What MINs does on real design problems".

Lecture 6 · Data-Driven Design Optimization — Generative-Based40 / 78
ACT 4Condition and samplerLecture 6

How far past the data can you ask?

The same two hundred designs and oracle, driven by the bar γ\gamma alone. Raise it and the samples move toward the good region, then thin out and decay. The red marker is Lecture 5's answer on the identical problem.
A requested score beyond the data is extrapolation moved from xx to yy. An exact conditional puts no mass where the prior has none; learned generators have tails, so over-asking still produces poor or invalid designs.

Mean true value of the samples: 2.132.13 at γ=2.00\gamma=2.00, 2.5502.550 at 2.652.65 (best, above the dataset's 2.5232.523), then 2.5372.537, 2.4272.427, 2.1632.163 at 2.802.80, 3.003.00, 3.803.80 — while the effective sample count falls from 3,7503{,}750 to 1.51.5. Gradient ascent on the same oracle returns a design it rates 4.184.18 and the world rates −0.14-0.14.

Lecture 6 · Data-Driven Design Optimization — Generative-Based41 / 78
ACT 4Condition and samplerLecture 6

Classifier-free guidance — push a conditional model toward its conditionHo & Salimans · NeurIPS 2021 workshop

Train one diffusion model with the condition and without it — drop yy at random — and combine its two noise predictions at sampling:

ϵ^=(1+w) ϵθ(xt,t,y)−w ϵθ(xt,t)\hat\epsilon=(1+w)\,\epsilon_\theta(x_t,t,y)-w\,\epsilon_\theta(x_t,t)

What it amounts to

Adding (1+w) ∇log⁡pt(y∣xt)(1+w)\,\nabla\log p_t(y\mid x_t) for an implicit classifier, p(y∣x)∝p(x∣y)/p(x)p(y\mid x)\propto p(x\mid y)/p(x). The strength ww trades diversity for agreement with the condition.

Why it is G3, not G4

There is no separate model of performance: the only signal is the labels the model was trained on. Its risk is G3's — asking for a yy the data cannot vouch for.

The derivation is the appendix slide "Backup — guidance, with and without a classifier". Classifier guidance — the G4 form — comes after the tilt.

Lecture 6 · Data-Driven Design Optimization — Generative-Based42 / 78
ACT 4Condition and samplerLecture 6

DDOM — a diffusion model as the inverse mapKrishnamoorthy, Mashkaria & Grover · ICML 2023

ReweightMINs' weights: favour high scores with enough support
Traina conditional diffusion model of xx given yy, dropping yy at random
Generatecondition on the dataset's best yy, with strong classifier-free guidance
A three-mode toy problem: the reverse trajectory splits into all three modes; the samples' mean value climbs to the optimum. {p}(Krishnamoor
A three-mode toy problem: the reverse trajectory splits into all three modes; the samples' mean value climbs to the optimum. (Krishnamoorthy et al., 2023)
The one-to-many inverse, represented properly, and no surrogate anywhere. It asks only for the best label it has seen — whether samples exceed it is for the evaluation to say.

Source alignment: Part 2 pp. 30–34. Benchmark numbers (rank 2.8 against COMs' 3.7 on six Design-Bench tasks) are on the appendix slide.

Lecture 6 · Data-Driven Design Optimization — Generative-Based43 / 78
ACT 4Condition and samplerLecture 6

GTG — ask for an improvement, not a pointYun, Yun, Lee & Park · NeurIPS 2024 · our lab

Buildsynthetic trajectories through the data toward higher scores, with a locality bias
Traina diffusion model that generates whole trajectories, conditioned on their score
Sampleclassifier-free guidance toward a high score, with a context taken from the data filled in; a proxy keeps the best final designs

The condition

A trajectory carries a direction — from where the data are toward where they are not. Its steps are steps of improvement, not time and not denoising steps.

The proxy

It only filters the final candidates; it never steers the sampler. Still G3.

From the paper: the model is trained on log⁡pθ(τ∣y(τ))\log p_\theta(\tau\mid y(\tau)) with y(τ)y(\tau) the sum of the trajectory's scores; at sampling, "inpainting the C context data points of the trajectory with τctx\tau_{\text{ctx}}, a subtrajectory sampled from DtrajD_{\text{traj}}", with guidance weight ω=1.2\omega=1.2; "we select top-Q samples in terms of the predicted score from the proxy".

Lecture 6 · Data-Driven Design Optimization — Generative-Based44 / 78
ACT 4Condition and samplerLecture 6

Wind-farm layouts — condition, measure, repeatShin, Yun, Yun, Woo, Lim & Park · KDD 2025 · our lab

Traina conditional diffusion model of layouts, with a graph-network denoiser
Sampleat high annual energy production, for the wind scenario at hand
Repairlocal search fixes layouts that break the site's constraints
Evaluatescore the new layouts
G3 on a real design problem — with Lecture 4's loop closed: every round is measured and added to the data. Not strictly offline: the condition asks, the evaluation answers.
Lecture 6 · Data-Driven Design Optimization — Generative-Based45 / 78
ACT 4Condition and samplerLecture 6

What asking for a score buys, and what it costs

Assumption added

The generator generalises to the score you ask for — usually the best in the data, or beyond it.

What it solves

Many modes and diverse candidates from one model. MINs searches over (y,z)(y,z) through the generator and a forward model; DDOM generates without a separate forward surrogate.

What it leaves

Asking past the data is extrapolation in yy. MINs bounds it with a forward model and an optimisation over (y,z)(y,z); DDOM asks only for the best label seen; the wind-farm loop measures every round.

Lecture 6 · Data-Driven Design Optimization — Generative-Based46 / 78
ACT 4Condition and samplerLecture 6

A KL-regularised view of candidate distributions

The best average estimate f^\hat f that stays close to the generator pp — the KL averaged over the search:

q⋆=arg max⁡q ≪ p Eq[f^(x)]−1β DKL(q ∥ p)⟹q⋆(x)=p(x) eβf^(x)Zβ(β>0, Zβ<∞)q^\star=\argmax_{q\,\ll\,p}\ \E_q\big[\hat f(x)\big]-\frac1\beta\,D_{\mathrm{KL}}(q\,\|\,p)\qquad\Longrightarrow\qquad \hl{q^\star(x)=\frac{p(x)\,e^{\beta\hat f(x)}}{Z_\beta}}\quad (\beta>0,\ Z_\beta<\infty)

Derivation.   Eq[f^]−1βDKL(q∥p)=1β Eq[log⁡p eβf^q]=1β[log⁡Zβ−DKL(q ∥ q⋆)]\;\E_q[\hat f]-\tfrac1\beta D_{\mathrm{KL}}(q\|p)=\tfrac1\beta\,\E_q\Big[\log\frac{p\,e^{\beta\hat f}}{q}\Big]=\tfrac1\beta\big[\log Z_\beta-D_{\mathrm{KL}}(q\,\|\,q^\star)\big], largest — uniquely — at q=q⋆q=q^\star.   ■\;\blacksquare

The two ends of β\beta

β→0\beta\to0: q⋆q^\star is the generator itself. As β\beta grows, q⋆q^\star concentrates — under suitable conditions — near the maximisers of f^\hat f on the generator's support. That is a statement about the target distribution, not about what a search algorithm returns.

What it regularises

Where we search, not what we believe: Lecture 5's COMs changed the predictor; this KL limits how far the candidate distribution moves from the generator.

For this optimisation problem the exponential tilt is the exact optimum. Next: which methods it helps to read — and which it does not.
Lecture 6 · Data-Driven Design Optimization — Generative-Based47 / 78
ACT 4Condition and samplerLecture 6

How the KL view relates to the methods

method relation to q⋆∝p eβf^q^\star\propto p\,e^{\beta\hat f}
CbAS (G2) its target is exactly q⋆q^\star with f^=log⁡P(S∣x)\hat f=\log P(S\mid x) and β=1\beta=1 — fitted with the other KL direction
guidance (G3, G4) each noise level's score is that of a tilted ptp_t; the full run need not sample the final tilted distribution, even with exact scores
DiBO (ICML 2025, our lab) p⋅eβ UCBp\cdot e^{\beta\,\mathrm{UCB}}, sampled by a fine-tuned diffusion model — online, Lecture 4
MINs, GFlowNet, BootGen not this problem: a constrained query over (y,z)(y,z); a reward-proportional sampler with no generator prior; rank weights and bootstrapping
A useful lens for some methods, not a common objective for all. And deriving q⋆q^\star exactly does not mean a sampler draws from it exactly. Tilting by a conservative or pessimistic estimate combines this view with Lecture 5.
Lecture 6 · Data-Driven Design Optimization — Generative-Based48 / 78
ACT 4Condition and samplerLecture 6

Too hard a tilt, and Lecture 5 returns

Lecture 5's fifteen designs and hidden ff; the generator is a smoothed copy of the data with a thin tail. Drag β; press → to tilt by Lecture 5's pessimistic μ−2σ\mu-2\sigma instead.
The KL is an average, not a leash: it does not bound how far any one design moves. Where the generator has a thin tail and the estimate is wrong, a hard tilt moves the mass there.

By the surrogate: average true value 1.521.52 at β=0\beta=0 (the data's average), 1.901.90 at β=2\beta=2, 1.491.49 at β=3\beta=3 when the mode jumps to x=10x=10, 0.150.15 at β=5\beta=5. By μ−2σ\mu-2\sigma: 2.002.00 at β=2\beta=2 and 2.432.43 at β=8\beta=8 — the best design in the data — with under 1%1\% of the mass past x=6x=6 at every β\beta; the price is that it cannot go past the data either. The generator: a Gaussian kernel density of the designs (bandwidth 0.50.5) with 0.5%0.5\% of its mass spread uniformly.

Lecture 6 · Data-Driven Design Optimization — Generative-Based49 / 78
ACT 4Condition and samplerLecture 6

G4 — use performance information to steer the samplerclassifier guidance · Dhariwal & Nichol · NeurIPS 2021

Classifier guidance — Bayes' rule for density scores (gradients of log-densities, not performance scores), with a classifier pϕp_\phi trained on noisy inputs and its term scaled by ss:

∇xtlog⁡pt(xt∣y)=∇xtlog⁡pt(xt)+∇xtlog⁡pϕ(y∣xt)\nabla_{x_t}\log p_t(x_t\mid y)=\nabla_{x_t}\log p_t(x_t)+\nabla_{x_t}\log p_\phi(y\mid x_t)

With a regression surrogate, define the candidate distribution at each noise level directly, qt(x)∝pt(x) esf^t(x)q_t(x)\propto p_t(x)\,e^{s\hat f_t(x)}, so that

∇log⁡qt=∇log⁡pt⏟the generator’s pull+  s ∇f^t⏟the surrogate’s push\nabla\log q_t=\underbrace{\nabla\log p_t}_{\text{the generator's pull}}+\;s\,\underbrace{\nabla\hat f_t}_{\text{the surrogate's push}}

A score at each noise level — not a guarantee that the whole denoising run samples the final tilted distribution.

This direct form is gradient ascent on the surrogate, interleaved with denoising — Lecture 5's optimiser inside the sampler. RGD, later, is G4's other form: the proxy sets only the guidance strength.
Classifier guidance on one class, scale 1 (left) and 10 (right). {p}(Dhariwal & Nichol, 2021)
Classifier guidance on one class, scale 1 (left) and 10 (right). (Dhariwal & Nichol, 2021)

Source alignment: Part 1 pp. 38–39 (controllable generation; the paper's Figure 3, FID 33.0 at scale 1 and 12.0 at scale 10). The surrogate f^t\hat f_t must be trained on noisy designs xtx_t. Defining qtq_t directly avoids treating ef^e^{\hat f} as a normalised p(y∣x)p(y\mid x). Even with exact scores, the guided run need not sample the final tilted distribution (Bradley & Nakkiran, 2024). G4 in general: performance information modifies the sampling process — by a gradient on the sample (classifier or energy guidance) or by setting the guidance strength (RGD).

Lecture 6 · Data-Driven Design Optimization — Generative-Based50 / 78
ACT 4Condition and samplerLecture 6

Too much guidance leaves the valid set

The thin-curve problem of Session 1, now sampled with direct guidance. Drag ss: on the valid samples prediction and truth stay close — the failure here is validity: the surrogate's gradient off the curve, where it has no data, pulls samples out.

Numbers behind the picture (150 samples; averages over the valid samples only, predicted / true): s=0s=0: 87% valid, 0.68/0.680.68/0.68 — the data's; s=1s=1: 83%, 0.91/0.900.91/0.90; s=1.5s=1.5: 74%, 0.97/0.960.97/0.96; s=2s=2: 63%, 1.00/0.991.00/0.99; s=3s=3: 37%, 0.95/0.980.95/0.98; s=4s=4: 19%, 0.83/0.980.83/0.98. In this example the surrogate is accurate on the curve near the data; what grows with ss is the share of samples that are not designs. The sampler stands in for a diffusion model: annealed Langevin dynamics on a kernel density of the data, whose noise-conditional score is exact, plus s∇f^s\nabla\hat f.

Lecture 6 · Data-Driven Design Optimization — Generative-Based51 / 78
ACT 4Condition and samplerLecture 6

RGD — give the proxy one knobRobust Guided Diffusion · Chen, Beckham, Liu, Liu & Pal · TMLR 2024

Proxy-enhanced sampling

Sample with classifier-free guidance — no proxy gradient on xx — and let the proxy choose only the guidance strength ω\omega, adjusted so the denoised sample scores higher.

Diffusion-based proxy refinement

Refine the proxy toward the diffusion model's own p(y∣x)p(y\mid x), so that the proxy and the generator agree where the proxy would otherwise over-promise.

G4's second mechanism. Lecture 5 says the proxy is unreliable off the data; RGD shrinks what it controls — one scalar per step instead of every coordinate of xx — and corrects it with the generator. That reduces the exposure; it does not make the proxy right.

The strength update is the paper's Eq. 8: ω\omega is optimised by gradient ascent on the proxy's score of the predicted clean sample. It builds on proxy-free conditional diffusion of the DDOM kind. RGD reports state-of-the-art results across Design-Bench tasks.

Lecture 6 · Data-Driven Design Optimization — Generative-Based52 / 78
ACT 4Condition and samplerLecture 6

Discrete designs — guided discrete diffusion

NOS / LaMBO-2 (Gruver et al., NeurIPS 2023)

Protein sequences have no gradient in xx, so NOS guides the denoiser's continuous hidden states. LaMBO-2 uses it inside a Bayesian-optimisation procedure with laboratory rounds — an application, not offline MBO.

Road-network layouts (Yun et al., WSDM 2026 · our lab)

A city's road layout is a discrete structure; a guided discrete diffusion model generates layouts for a given traffic pattern.

G4 travels to sequences and graphs: the same intervention point, wherever there is a continuous place to put the gradient.
Lecture 6 · Data-Driven Design Optimization — Generative-Based53 / 78
ACT 4Condition and samplerLecture 6

What steering the sampler buys, and what it costs

Assumption added

The performance information is right where the sampler goes — a gradient at every noise level, or a proxy's ranking for the strength.

What it solves

One trained generator, any objective at sampling time, and a dial for how hard to push.

What it leaves

Exploitation and invalid samples that grow with the strength. Limiting the proxy (RGD) or guiding by a pessimistic estimate (Lecture 5) can reduce the exposure; neither removes it, and both depend on their own estimates.

Lecture 6 · Data-Driven Design Optimization — Generative-Based54 / 78
ACT 4Condition and samplerLecture 6

Check — which intervention point?

DDOM samples a conditional diffusion model with classifier-free guidance at the dataset's best score. RGD lets a proxy choose the guidance strength. Which statement is right?

Classifier-free guidance strengthens a condition the model learned from labels — a G3 intervention. G4 means a separate model of performance steers the sampler; RGD lets the proxy set only the strength ω\omega, far less than a gradient on every coordinate of xx. DDOM's samples still need evaluation: strong guidance toward the best label it has seen can still leave the data.

Lecture 6 · Data-Driven Design Optimization — Generative-Based55 / 78
CLOSING

Closing

Four places for a performance signal to get in — and Lecture 5's tools in each.
CLOSINGLecture 6

Four interventions, side by side

G1 · coordinates G2 · distribution G3 · condition G4 · sampler
in depth latent search CbAS MINs, DDOM classifier guidance, RGD
the signal enters the search over codes the refit's weights the requested yy the sampling process
plausibility from the decoder the prior ratio the generator the generator's pull
what it leaves invalid decodes; an exploitable predictor follows the success model's errors extrapolation in yy invalid samples and exploitation that grow with strength
Lecture 5's tool there conservative latent predictor (LCOMs) pessimistic P(S∣x)P(S\mid x) a forward check (MINs), or measure a limited proxy (RGD), or pessimistic guidance
our lab — BootGen · Genetic GFN GTG · wind farms road networks
Not rivals: BootGen is G2 and G3, RGD is G3 with a G4 knob, and the KL view connects several of them — a lens, not their common objective.
Lecture 6 · Data-Driven Design Optimization — Generative-Based57 / 78
CLOSINGLecture 6

Performance, validity, reliability — the two lectures together

Lecture 5 closed on these three questions; here is where each lecture stands.

Performance

Is the design good? — the objective, in both lectures

Validity

Is it possible at all? — this lecture: plausibility from the generator; feasibility from a decoder that enforces it, or a check

Reliability

Can the prediction be trusted? — Lecture 5: conservatism, uncertainty, knowledge — wherever a surrogate enters

Generate to stay plausible, score with care, measure to settle: a generator proposes, a conservative or uncertainty-aware model ranks, and only an evaluation decides. Two settings: BootGen selects with a proxy and is evaluated afterwards; the wind-farm loop adds evaluations to the data while it optimises.
Lecture 6 · Data-Driven Design Optimization — Generative-Based58 / 78
CLOSINGLecture 6

Diffusion Policy — generate a sequence of actionsChi et al. · RSS 2023 · from design generation to robot behaviour

Replace a static design with an action sequence, conditioned on the robot's recent observations; train on demonstrations with the diffusion noise-prediction loss. Execute a few actions, observe again, generate anew.

The observation-conditioned generate-and-execute loop. Denoising steps happen inside one decision; they are not physical time steps.
The observation-conditioned generate-and-execute loop. Denoising steps happen inside one decision; they are not physical time steps.
A sequence keeps different demonstrated ways around an obstacle as different modes. This is imitation, not reward-based RL: the objective is the demonstrator's actions. Executing a little and re-planning anticipates feedback and MPC (Lectures 9–11). Paper
Lecture 6 · Data-Driven Design Optimization — Generative-Based59 / 78
CLOSINGLecture 6

Where we are — a performance model and a generator

Lecture 5 Lecture 6
the model a performance model f^(x)\hat f(x) a generator p(x)p(x) or p(x∣y)p(x\mid y), plus a performance model
the decision search f^\hat f — carefully search, refit, condition or steer the generator
what it secures grounds for trusting a prediction plausible, diverse candidates

A model of how good — value

Lectures 7–8 learn how good each decision is, Q(s,a)Q(s,a), and act by maximising it — and meet Lecture 5's hazard: the policy exploits QQ where QQ is wrong.

A model of what to propose — policy

Lecture 10 learns a distribution over actions, πθ(a∣s)\pi_\theta(a\mid s), and samples it. A comparison worth keeping, not the same algorithm: the two are usually trained together.

So far: static design choices, experiments that buy information, and inference over time (Lectures 3–4). From Lecture 7 on, the current action changes the next state and the future reward — and the decision is optimised over that sequence.

Lecture 6 · Data-Driven Design Optimization — Generative-Based60 / 78
A generator decides where to look; a performance model decides what to trust.
The signal can enter at four places — and wherever a surrogate enters, Lecture 5 comes with it. A measurement still has the last word.
Questions?
Two things to carry out of here. A generator buys plausibility, not evidence — the performance signal still has to be trusted, as in Lecture 5. And ask where the signal enters — coordinates, distribution, condition or sampler — because that is where it can be exploited.
APPENDIX

Backup slides

Complete derivations and reported numbers, kept out of the narrative.
APPENDIXBackup slidesLecture 6

Reading guide — Session 1 papers: generative models and G1

paper why we read it how it connects
VAE · Kingma & Welling, ICLR 2014 — in depth a generator with an explicit latent space the coordinates G1 searches
DDPM · Ho, Jain & Abbeel, NeurIPS 2020 — in depth the generator behind most current design methods steered by the two guidance papers
Classifier guidance · Dhariwal & Nichol, NeurIPS 2021 a separate model's gradient steers the sampler the mechanism of G4
Classifier-free guidance · Ho & Salimans, NeurIPS 2021 workshop steering without a separate model the sampler of G3
Latent-space search · Gómez-Bombarelli et al., ACS Cent. Sci. 2018 — in depth G1: search in learned coordinates also Lecture 4
JT-VAE · Jin, Barzilay & Jaakkola, ICML 2018 validity by construction fixes G1's decoder
LCOMs · Qi et al., ICLR 2023 workshop a conservative surrogate on latent codes Lecture 5 inside G1
Lecture 6 · Data-Driven Design Optimization — Generative-Based64 / 78
APPENDIXBackup slidesLecture 6

Reading guide, continued — Session 1 papers: G2

paper why we read it how it connects
CbAS · Brookes, Park & Listgarten, ICML 2019 — in depth G2: refit the generator toward success the tilt with β=1\beta=1
GFlowNet · Bengio et al., NeurIPS 2021 sample in proportion to reward needs reward calls or a proxy
BootGen · Kim, Berto, Ahn & Park, NeurIPS 2023 — our lab refit on its own proxy-scored samples G2 and G3
Genetic-guided GFlowNets · Kim, Kim, Choi & Park, NeurIPS 2024 — our lab GFlowNet training under a small oracle budget online, not offline
Lecture 6 · Data-Driven Design Optimization — Generative-Based65 / 78
APPENDIXBackup slidesLecture 6

Reading guide, continued — Session 2 papers: condition and sampler

paper why we read it how it connects
MINs · Kumar & Levine, NeurIPS 2020 — in depth G3: an inverse map asked for a score keeps a forward model as a check
DDOM · Krishnamoorthy, Mashkaria & Grover, ICML 2023 — in depth G3 with diffusion: every mode kept MINs' weights, no proxy
GTG · Yun, Yun, Lee & Park, NeurIPS 2024 — our lab condition a trajectory, not a point the proxy only filters
Wind-farm layouts · Shin et al., KDD 2025 — our lab G3 on a real design problem measured every round
DiBO · Yun, Om, Lee, Yun & Park, ICML 2025 — our lab the tilted target p eβ UCBp\,e^{\beta\,\mathrm{UCB}}, sampled by a fine-tuned diffusion model online; Lecture 4; a GFlowNet-family loss
RGD · Chen et al., TMLR 2024 — in depth G4 with the proxy limited to one knob classifier-free guidance + a proxy
NOS / LaMBO-2 · Gruver et al., NeurIPS 2023 guidance for discrete sequences a BO application
Road-network layouts · Yun et al., WSDM 2026 — our lab guided discrete diffusion for city design G4 on graphs

The lab's papers were checked against the KAIST SILAB list and the proceedings pages (2026-10-11); descriptions follow their abstracts.

Lecture 6 · Data-Driven Design Optimization — Generative-Based66 / 78
APPENDIXBackup slidesLecture 6

Backup 1 — the ELBO, derived, and the reparameterisation trick

We want log⁡pθ(x)\log p_\theta(x), which integrates over zz. Introduce qϕ(z∣x)q_\phi(z\mid x):

log⁡pθ(x)=Eqϕ[log⁡pθ(x,z)qϕ(z∣x)]⏟ELBO Lθ,ϕ(x)+KL(qϕ(z∣x) ∥ pθ(z∣x))⏟ ≥ 0\log p_\theta(x) = \underbrace{\E_{q_\phi}\Big[\log\tfrac{p_\theta(x,z)}{q_\phi(z\mid x)}\Big]}_{\text{ELBO } \mathcal{L}_{\theta,\phi}(x)} + \underbrace{\mathrm{KL}\big(q_\phi(z\mid x)\,\|\,p_\theta(z\mid x)\big)}_{\ \ge\ 0}

The KL is non-negative, so the ELBO is a lower bound — not generally the exact evidence; it is tight only when qϕq_\phi equals the true posterior. Expanding pθ(x,z)=pθ(x∣z)p(z)p_\theta(x,z)=p_\theta(x\mid z)p(z) gives the working form

Lθ,ϕ(x)=Eqϕ(z∣x)[log⁡pθ(x∣z)]−KL(qϕ(z∣x) ∥ p(z))■\mathcal{L}_{\theta,\phi}(x) = \E_{q_\phi(z\mid x)}[\log p_\theta(x\mid z)] - \mathrm{KL}(q_\phi(z\mid x)\,\|\,p(z)) \qquad\blacksquare

Reparameterisation. Write z=μϕ(x)+σϕ(x)⊙ϵz = \mu_\phi(x) + \sigma_\phi(x)\odot\epsilon with ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I): the randomness moves off the parameters and the gradient flows.

Why the KL term is the samplability term. With a linear-Gaussian decoder the β\beta-optimal encoder is μ=(W⊤W+βσ2I)−1W⊤(x−b)\mu = (W^\top W + \beta\sigma^2 I)^{-1}W^\top(x-b) and Σ=βσ2(W⊤W+βσ2I)−1\Sigma = \beta\sigma^2(W^\top W+\beta\sigma^2 I)^{-1}. Along an eigendirection of W⊤WW^\top W with eigenvalue ww the aggregate posterior variance is Var(μ)+Σ=(w2+σ2w+βσ2w+β2σ4)/(w+βσ2)2\mathrm{Var}(\mu)+\Sigma = \big(w^2+\sigma^2 w+\beta\sigma^2 w+\beta^2\sigma^4\big)/(w+\beta\sigma^2)^2, whose difference from 11 is σ2w(1−β)/(w+βσ2)2\sigma^2 w(1-\beta)/(w+\beta\sigma^2)^2 — zero at β=1\beta=1. The Turning the dial widget is that formula, plotted.

Lecture 6 · Data-Driven Design Optimization — Generative-Based67 / 78
APPENDIXBackup slidesLecture 6

Backup — one VAE calculation

For one latent coordinate, q(z∣x)=N(1,0.52)q(z\mid x)=\mathcal N(1,0.5^2) and p(z)=N(0,1)p(z)=\mathcal N(0,1):

KL(q∥p)=12(μ2+σ2−1−log⁡σ2)=12(1+0.25−1−log⁡0.25)≈0.818.\mathrm{KL}(q\|p)=\tfrac12\left(\mu^2+\sigma^2-1-\log\sigma^2\right)=\tfrac12(1+0.25-1-\log0.25)\approx0.818.

With an expected reconstruction log-likelihood of −2-2, the ELBO is −2−0.818=−2.818-2-0.818=\mathbf{-2.818}. One reparameterised draw with ϵ=−1\epsilon=-1: z=1+0.5(−1)=0.5z=1+0.5(-1)=0.5.

Reconstruction rewards explaining the input; KL charges for moving the code distribution away from the prior.
Lecture 6 · Data-Driven Design Optimization — Generative-Based68 / 78
APPENDIXBackup slidesLecture 6

Backup 2 — the four families of generative models

family training latent trade-off
VAE maximise the ELBO explicit, ∼N(0,I)\sim\mathcal{N}(0,I) stable; samples can be blurry
GAN adversarial, generator vs. discriminator implicit sharp; unstable, mode collapse
Normalising flow exact maximum likelihood invertible, same dimension exact density; architectural constraints
Diffusion denoising score matching fixed noise schedule high quality; slow sampling

Any of the four can serve as the prior p(x)p(x) of a generative design pipeline. The training objective of an inverse map can likewise be any divergence: Lp(D)=Ey∼p(y)[Div(pD(x∣y),pfθ−1(x∣y))]L_p(D)=\E_{y\sim p(y)}\big[\mathrm{Div}(p_D(x\mid y),p_{f^{-1}_\theta}(x\mid y))\big] — KL gives maximum likelihood, JS a GAN objective (Part 2 p. 21).

Lecture 6 · Data-Driven Design Optimization — Generative-Based69 / 78
APPENDIXBackup slidesLecture 6

Backup 2b — diffusion as a hierarchical VAE

Forward and reverse. q(x1:T∣x0)=∏t=1Tq(xt∣xt−1)q(x_{1:T}\mid x_0)=\prod_{t=1}^T q(x_t\mid x_{t-1}) and pθ(x0:T)=p(xT)∏t=1Tpθ(xt−1∣xt)p_\theta(x_{0:T}) = p(x_T)\prod_{t=1}^T p_\theta(x_{t-1}\mid x_t), and the same ELBO argument gives

L(θ)=Eq[KL(q(xT∣x0)∥pθ(xT))⏟LT+∑t=2TKL(q(xt−1∣xt,x0)∥pθ(xt−1∣xt))⏟Lt−1−log⁡pθ(x0∣x1)⏟−L0]\mathcal{L}(\theta) = \E_q\Big[\underbrace{\mathrm{KL}\big(q(x_T\mid x_0)\|p_\theta(x_T)\big)}_{L_T} + \sum_{t=2}^T\underbrace{\mathrm{KL}\big(q(x_{t-1}\mid x_t,x_0)\|p_\theta(x_{t-1}\mid x_t)\big)}_{L_{t-1}} - \underbrace{\log p_\theta(x_0\mid x_1)}_{-L_0}\Big]

Why Gaussian. The kernel is chosen so that q(xt∣x0)q(x_t\mid x_0) and q(xt−1∣xt,x0)q(x_{t-1}\mid x_t,x_0) have closed forms and q(xT∣x0)q(x_T\mid x_0) forgets x0x_0. With q(xt∣xt−1)=N(1−βtxt−1,βtI)q(x_t\mid x_{t-1})=\mathcal{N}(\sqrt{1-\beta_t}x_{t-1},\beta_t I), αt=1−βt\alpha_t = 1-\beta_t and αˉt=∏i≤tαi\bar\alpha_t=\prod_{i\le t}\alpha_i:

q(xt∣x0)=N(αˉt x0, (1−αˉt)I),q(xt−1∣xt,x0)=N(μ~t(xt,x0), β~tI)q(x_t\mid x_0) = \mathcal{N}\big(\sqrt{\bar\alpha_t}\,x_0,\ (1-\bar\alpha_t)I\big),\qquad q(x_{t-1}\mid x_t,x_0)=\mathcal{N}\big(\tilde\mu_t(x_t,x_0),\ \tilde\beta_t I\big)
μ~t=αˉt−1βt1−αˉtx0+αt(1−αˉt−1)1−αˉtxt,β~t=1−αˉt−11−αˉtβt,αˉt→0⇒q∞=N(0,I)\tilde\mu_t = \frac{\sqrt{\bar\alpha_{t-1}}\beta_t}{1-\bar\alpha_t}x_0 + \frac{\sqrt{\alpha_t}(1-\bar\alpha_{t-1})}{1-\bar\alpha_t}x_t,\qquad \tilde\beta_t = \frac{1-\bar\alpha_{t-1}}{1-\bar\alpha_t}\beta_t,\qquad \bar\alpha_t\to 0 \Rightarrow q_\infty=\mathcal{N}(0,I)
Lecture 6 · Data-Driven Design Optimization — Generative-Based70 / 78
APPENDIXBackup slidesLecture 6

Backup 3 — the DDPM objective collapses to one squared error

Both q(xt−1∣xt,x0)q(x_{t-1}\mid x_t,x_0) and pθ(xt−1∣xt)p_\theta(x_{t-1}\mid x_t) are Gaussian, so Lt−1L_{t-1} is an analytic KL:

Lt−1=Eq[12σt2∥μ~t(xt,x0)−μθ(xt,t)∥2]+CL_{t-1} = \E_q\Big[\tfrac{1}{2\sigma_t^2}\big\|\tilde\mu_t(x_t,x_0)-\mu_\theta(x_t,t)\big\|^2\Big] + C

Substituting xt=αˉtx0+1−αˉt ϵx_t=\sqrt{\bar\alpha_t}x_0+\sqrt{1-\bar\alpha_t}\,\epsilon suggests predicting the noise:

μθ  =def  1αt(xt−βt1−αˉt ϵθ(xt,t))  ⟹  Lt−1=Ex0,ϵ[βt22σt2αt(1−αˉt)∥ϵ−ϵθ(xt,t)∥2]\mu_\theta \;\overset{\text{def}}{=}\; \frac{1}{\sqrt{\alpha_t}}\Big(x_t - \frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\,\epsilon_\theta(x_t,t)\Big) \;\Longrightarrow\; L_{t-1}=\E_{x_0,\epsilon}\Big[\tfrac{\beta_t^2}{2\sigma_t^2\alpha_t(1-\bar\alpha_t)}\big\|\epsilon-\epsilon_\theta(x_t,t)\big\|^2\Big]

Dropping the weight — which the authors found works better — gives Lsimple=Et,x0,ϵ[∥ϵ−ϵθ(αˉtx0+1−αˉtϵ, t)∥2]\hl{\mathcal{L}_{\text{simple}} = \E_{t,x_0,\epsilon}\big[\|\epsilon - \epsilon_\theta(\sqrt{\bar\alpha_t}x_0+\sqrt{1-\bar\alpha_t}\epsilon,\ t)\|^2\big]}, a reweighted objective rather than the exact bound.

One training step, in numbers. x0=2x_0=2, αˉt=0.64\bar\alpha_t=0.64, ϵ=1\epsilon=1 give xt=0.8(2)+0.6(1)=2.2x_t=0.8(2)+0.6(1)=2.2; predicting ϵ^=0.5\hat\epsilon=0.5 costs (1−0.5)2=0.25(1-0.5)^2=0.25. (Ho, Jain & Abbeel, 2020)

Lecture 6 · Data-Driven Design Optimization — Generative-Based71 / 78
APPENDIXBackup slidesLecture 6

Backup 4 — score matching, Langevin dynamics, and the SDE that unifies them

The estimator. Fit sθ(x)≈∇xlog⁡p(x)s_\theta(x)\approx\nabla_x\log p(x) by the Fisher divergence, computable as Ep[tr(∇xsθ)+12∥sθ∥2]\E_{p}\big[\mathrm{tr}(\nabla_x s_\theta) + \tfrac12\|s_\theta\|^2\big] (Hyvärinen, 2005) or in the denoising form 12Eqσ(x~∣x)p(x)[∥∇x~log⁡qσ(x~∣x)−sθ(x~)∥2]\tfrac12\E_{q_\sigma(\tilde x\mid x)p(x)}\big[\|\nabla_{\tilde x}\log q_\sigma(\tilde x\mid x)-s_\theta(\tilde x)\|^2\big] (Vincent, 2011). For an energy model the partition function drops out of the score: ∇xlog⁡pθ(x)=−∇xεθ(x)\nabla_x\log p_\theta(x)=-\nabla_x\varepsilon_\theta(x).

The blind spot. The Fisher divergence weights the error by the data density, Ep(x)∥∇xlog⁡p(x)−sθ(x)∥2=∫p(x) ∥⋅∥2dx\E_{p(x)}\|\nabla_x\log p(x)-s_\theta(x)\|^2=\int p(x)\,\|\cdot\|^2dx, so the score is learned badly where data is thin — a learned object is unconstrained where there is no evidence, as Lecture 5's surrogate was. Its cure. Train on data perturbed at LL noise scales, pσi(x)=∫p(y)N(x;y,σi2I)dyp_{\sigma_i}(x)=\int p(y)\mathcal{N}(x;y,\sigma_i^2 I)dy, with a noise-conditional model sθ(x,i)s_\theta(x,i) and loss ∑iσi2 Epσi∥∇log⁡pσi−sθ(x,i)∥2\sum_i \sigma_i^2\,\E_{p_{\sigma_i}}\|\nabla\log p_{\sigma_i}-s_\theta(x,i)\|^2; sample by annealed Langevin dynamics, x~t=x~t−1+ϵ2∇xlog⁡p(x~t−1)+ϵ zt\tilde x_t = \tilde x_{t-1} + \tfrac{\epsilon}{2}\nabla_x\log p(\tilde x_{t-1}) + \sqrt{\epsilon}\,z_t, for i=L,…,1i = L,\dots,1.

The continuous limit. The perturbation becomes dx=f(x,t)dt+g(t)dwdx = f(x,t)dt + g(t)dw, whose reverse is

dx=[f(x,t)−g2(t) ∇xlog⁡pt(x)]dt+g(t) dwˉdx = \big[f(x,t) - g^2(t)\,\hl{\nabla_x\log p_t(x)}\big]dt + g(t)\,d\bar w

NCSN is the variance-exploding SDE; DDPM the variance-preserving one, dx=−12β(t)x dt+β(t) dwdx=-\tfrac12\beta(t)x\,dt+\sqrt{\beta(t)}\,dw.

Lecture 6 · Data-Driven Design Optimization — Generative-Based72 / 78
APPENDIXBackup slidesLecture 6

Backup — guidance, with and without a classifier

Train ϵθ(xt,t,y)\epsilon_\theta(x_t,t,y) and, by dropping yy at random, ϵθ(xt,t)\epsilon_\theta(x_t,t). Since ϵθ≈−1−αˉt ∇log⁡pt\epsilon_\theta\approx-\sqrt{1-\bar\alpha_t}\,\nabla\log p_t, the guided prediction ϵ^=(1+w)ϵθ(xt,t,y)−w ϵθ(xt,t)\hat\epsilon=(1+w)\epsilon_\theta(x_t,t,y)-w\,\epsilon_\theta(x_t,t) corresponds to the score

(1+w)∇log⁡pt(xt∣y)−w∇log⁡pt(xt)=∇log⁡pt(xt)+(1+w) ∇log⁡pt(y∣xt),(1+w)\nabla\log p_t(x_t\mid y)-w\nabla\log p_t(x_t)=\nabla\log p_t(x_t)+(1+w)\,\nabla\log p_t(y\mid x_t),

because p(y∣x)∝p(x∣y)/p(x)p(y\mid x)\propto p(x\mid y)/p(x). Classifier guidance adds s ∇log⁡pϕ(y∣xt)s\,\nabla\log p_\phi(y\mid x_t) with a separately trained pϕp_\phi instead. In both, each noise level's guided score is the score of a tilted distribution, pt(x) pt(y∣x)strengthp_t(x)\,p_t(y\mid x)^{\text{strength}} — but the full sampler need not draw from the final tilted distribution, even with exact scores: Bradley & Nakkiran (2024, "Classifier-Free Guidance is a Predictor-Corrector") show that CFG with DDPM or DDIM does not generate the γ\gamma-powered distribution.

Lecture 6 · Data-Driven Design Optimization — Generative-Based73 / 78
APPENDIXBackup slidesLecture 6

Backup — the inverse map, as the source motivates it

fθ:X→Y⟹fθ−1:Y→Xf_\theta : \mathcal{X}\to\mathcal{Y} \qquad\Longrightarrow\qquad \hl{f^{-1}_\theta : \mathcal{Y}\to\mathcal{X}}

The forward model answers "given xx, what is yy?". We wanted "given the yy I need, which xx delivers it?" — a map whose input is one number.

What the reversal fixes

The inverse map is not asked to search X\mathcal{X}; it is asked to emit a point of it.

What it does not fix

The output is still high-dimensional and still confined to a thin valid set. Reversing the arrow moves the difficulty; it does not remove it.

Source alignment: Part 2 p. 18 (MINs' motivation): "we can exploit various techniques from deep generative models to model high-dimensional data distribution" — the inverse map's output head has to be a generative model.

Lecture 6 · Data-Driven Design Optimization — Generative-Based74 / 78
APPENDIXBackup slidesLecture 6

Backup 5 — MINs and CbAS, in full

MINs — train, then query

Train. Lp(D)=Ey∼p(y)[Div(pD(x∣y),pfθ−1(x∣y))]L_p(D) = \E_{y\sim p(y)}\big[\mathrm{Div}(p_D(x\mid y), p_{f^{-1}_\theta}(x\mid y))\big], reweighted by wi=p(y)/pD(y)w_i = p(y)/p_D(y), p(y)∝NyNy+Key−y∗p(y)\propto \frac{N_y}{N_y+K}e^{y-y^*}.

The weight comes from a bias–variance bound with three terms: variance from thin data Ey[1/Ny]\E_y[1/N_y], distribution shift d2(p∥pD)/∣D∣d_2(p\|p_D)/|D|, and bias from the target DTV(p∗,p)2D_{\mathrm{TV}}(p^*,p)^2.

Query. x=fθ−1(z,ytarget)x = f^{-1}_\theta(z, y_{\text{target}}), with ytargety_{\text{target}} from Approx-Infer. Ablation on CIFAR-10 (uniform): MIN 77.1277.12, without Approx-Infer 76.3176.31, without reweighting 74.8774.87 — both components matter.

CbAS — the adaptive ladder

θ⁰ ← fit the prior p(x|θ⁰) to D
q ← p(·|θ⁰)
repeat:
  x₁..x_M ~ q                     ← propose
  γ ← min(target, Q-th pct of
          f̂(x₁..x_M))             ← relax
  wᵢ ← p(xᵢ|θ⁰)/q(xᵢ) · P(y≥γ|xᵢ)
  q ← weighted MLE on {xᵢ, wᵢ}    ← refit

The ratio uses the previous iterate as the proposal; finite variance additionally needs the proposal to cover the target. γ\gamma rises only as fast as the model can follow — condition (B).

The subtlety both share. Querying above the observed range is extrapolation — Lecture 5's hazard, moved from xx into yy. Practical systems temper ytargety_{\text{target}} and re-rank samples with a (conservative) forward model: generate to stay valid, score to select.

Lecture 6 · Data-Driven Design Optimization — Generative-Based75 / 78
APPENDIXBackup slidesLecture 6

What MINs does on real design problemsMINs, three task families · note the forward-map column

task dimension dataset avg dataset best forward map MIN
MNIST, thickest recognisable "3" 1,024 149.0 265.0 Invalid 276.3
MNIST, variant (b) 1,024 149.0 163.0 Invalid 234.3
Faces, youngest (≥15\ge 15) 12,288 38.7 −15.0-15.0 Invalid −12.2-12.2
Faces, youngest (≥25\ge 25) 12,288 41.5 −25.0-25.0 Invalid −23.9-23.9
HopperController reward 3,843 442.9 1915.5 93.1 1960.1
Pendulum reward 1,537 14.7 344.5 3.4 1000.0

The forward map is Lecture 5's naive route: on four of six tasks its answer is not a design at all; on Hopper it scores 9393 where the dataset already held 19151915. MIN beats the dataset's best on five of six. (Kumar & Levine, 2020)

Lecture 6 · Data-Driven Design Optimization — Generative-Based76 / 78
APPENDIXBackup slidesLecture 6

DDOM and BootGen — the reported numbersas the source deck reports them

DDOM (ICML 2023)

Reweighting ablation (normalised): D'Kitty 0.926→0.9300.926\to0.930, Ant 0.888→0.9600.888\to0.960, TFBind8 0.957→0.9710.957\to0.971 — improvements are task dependent.

Six Design-Bench tasks, budget 256: best mean rank 2.82.8, against COMs' 3.73.7 and gradient ascent's 3.53.5.

BootGen (NeurIPS 2023, our lab)

Six biological-sequence tasks, 100th percentile: average 0.8950.895 against the dataset's 0.3650.365 and the best baseline's 0.7920.792 — with the highest diversity and novelty of the methods compared.

Benchmark results on the papers' tasks, not guarantees for a new dataset: every returned design still needs evaluation.
Lecture 6 · Data-Driven Design Optimization — Generative-Based77 / 78
APPENDIXBackup slidesLecture 6

BootGen — generated labels do not become new measurementsrank weighting and iterative augmentation

Step A, rank-weighted training of a score-conditioned generator; Step B, bootstrapping: sample at a high score, label with a proxy, keep the
Step A, rank-weighted training of a score-conditioned generator; Step B, bootstrapping: sample at a high score, label with a proxy, keep the top KK, retrain. (Kim et al., 2023, Fig. 2.1)

Weights ∝[k∣D∣+rank⁡(y,D)]−1\propto[k|D|+\operatorname{rank}(y,D)]^{-1}, normalised: higher-ranked designs weigh more, and the scale of yy does not matter.

The training pool grows without new oracle evaluations: it gains proxy-labelled synthetic samples, not new performance measurements, so proxy errors can be reinforced as the loop repeats. This is pseudo-labelling and augmentation — not resampling the data to build a bootstrap ensemble.
Lecture 6 · Data-Driven Design Optimization — Generative-Based78 / 78

Keys

→ / space
next reveal, then next slide
←
back
↑ ↓
skip a whole slide
R
reset this slide's widgets and quizzes
M
slide index
P
print → save as PDF (1280×720)
F
fullscreen