Optimisation de fonctions partiellement

Transcription

Introduction
Formalism
Solve
Example
Conclusion
Optimisation de fonctions partiellement modélisées
avec requêtes contraintes à la vérité de terrain
Frédéric Dambreville
Lab-STICC UMR CNRS 6285, ENSTA Bretagne
DGA MI, Bruz
France
Email: [email protected]
PFMC 2013 – BREST – October 2013
1
Introduction
Formalism
Solve
Example
Conclusion
Introduction
Main problem of interest
Optimize a partially modelled function
Assess the computed solution by actual evaluation
Actual evaluation implies costly subprocess calls
← Subprocess may be a local optimization, a simulation. . .
Each actual evaluation improves the function model
2
Introduction
Formalism
Solve
Example
Conclusion
Introduction
How to combine model-based optimization and costly assessment ?
2
Introduction
Formalism
Solve
Example
Conclusion
Introduction
Application context :
Plan an Information collection by a team of sensors
→ Plan Manager only knows partial models
⇒ Optimize sensor-to-mission allocation and sensor route
Full model for specific sensor is known by local teams
→ Plan sensor locally, but cannot plan a team of sensor
⇒ Assessment of specific sensor plan
2
Introduction
Formalism
Solve
Example
Conclusion
Introduction
Application context :
Plan an Information collection by a team of sensors
→ Plan Manager only knows partial models
⇒ Optimize sensor-to-mission allocation and sensor route
Full model for specific sensor is known by local teams
→ Plan sensor locally, but cannot plan a team of sensor
⇒ Assessment of specific sensor plan
Issue : Cooperation cost between manager and local teams
2
Introduction
Formalism
Solve
Example
Conclusion
Sensor-to-mission, simplified scenario
Requests m = 1 : M located at x[m]
Sensors k = 1 : K with autonomy γ[k] and initial location y [k]
3
Introduction
Formalism
Solve
Example
Conclusion
A noisy map µ implying cost c[x, y ; µ] for moving from x to y
A prior law pµ of µ is known by the plan manager
The true map µ
b is unknown by the plan manager
The true map µ
b is known by the Local teams
3
Introduction
Formalism
Solve
Example
Conclusion
valid trip τ [k] = τ [k|ik ] = y [k]x[m1k ] · · · x[mikk ]y [k] of k
3
Introduction
Formalism
Solve
Example
Conclusion
Cost of valid trip τ [k] = τ [k|ik ] = y [k]x[m1k ] · · · x[mikk ]y [k] of k is
C (τ [k]; µ) = c[y [k], x[m1 ]; µ] + c[x[m1 ], x[m2 ]; µ] · · · + c[x[mj ], y [k]; µ]
3
Introduction
Formalism
Solve
Example
Conclusion
Optimize a mission assignment
k 7→ τ [k] which :
P
maximize : G [τ ; µ] − ǫ k=1:K C (τ [k]; µ) where 0 < ǫ ≪ 1
and G [τ ; µ] = card ({m = 1 : M /∃k = 1 : K , m ∈ τ [k] })
in compliance with : C (τ [k]; µ) ≤ γ[k] for k = 1 : K
3
Introduction
Formalism
Solve
Example
Conclusion
P
Solution assessed by local teams :
C (τ [k]; µ
b) > γ[k] ⇒ truncated to maximal valid trip
τb[k] = τ [k|bik ] where C (τ [k|bik ]; µ
b) ≤ γ[k] < C (τ [k|bik + 1]; µ
b)
P
Solution is evaluated by G [b
τ; µ
b] − ǫ k=1:K C (b
τ; µ
b)
3
Introduction
Formalism
Solve
Example
Conclusion
Maximize a function with model noise
Is given :
f : (x, ν) ∈ X × N 7→ f (x, ν)
where :
x ∈ X is a parameter to be optimized
ν ∈ N is a model noise
pν ∈ P (N) is a known probabilistic noise prior
and an unknown actual model noise :
νb ∈ N is the unknown actual value of the model noise
4
Introduction
Formalism
Solve
Example
Conclusion
Maximize a function with model noise
Is given :
f : (x, ν) ∈ X × N 7→ f (x, ν)
where :
x ∈ X is a parameter to be optimized
ν ∈ N is a model noise
pν ∈ P (N) is a known probabilistic noise prior
and an unknown actual model noise :
νb ∈ N is the unknown actual value of the model noise
Issue : How to optimize f (·|b
ν ) when each evaluation of f (·|b
ν ) is
obtained by a costly request ?
4
Introduction
Formalism
Solve
Example
Conclusion
Reference work
Efficient Global Optimization by Experiments Design [Jones, Schonlau,
Welch, Efficient Global Optimization of Expensive Black-Box Functions, JoGO’98]
Inspiration : Experiments Design
Objective : maximize model-noised function f (x, ν) = g (x) + ν(x)
Where ν is a functional Gaussian noise prior
Correlation of ν(x1 ) and ν(x2 ) depends on d(x1 , x2 )
5
Introduction
Formalism
Solve
Example
Conclusion
Reference work
Optimizing f on the basis of iterated experiments
Each measure f (b
xk ) of function f is costly
Optimize each measure
→ enhance the knowledge of ν around optimal values of f
5
Introduction
Formalism
Solve
Example
Conclusion
Reference work
Method : iterated measures maximizing Expected Improvement
b
xk+1 ∈ arg max EI (x) ,
x
xi ) f (b
x1:k ) .
with : EI (x) = E max 0, f (x) − max f (b
i=1:k
Additive Gaussian ⇒ computation of the conditional law
5
Introduction
Formalism
Solve
Example
Conclusion
Reference work
Method : iterated measures maximizing Expected Improvement
b
xk+1 ∈ arg max EI (x) ,
x
xi ) f (b
x1:k ) .
with : EI (x) = E max 0, f (x) − max f (b
i=1:k
Additive Gaussian ⇒ computation of the conditional law
Our concern : model noise is not additive Gaussian
Our approach : rare event simulation of the conditional law
5
Introduction
Formalism
Solve
Example
Conclusion
Generalization
The EGO generalizes easily to our problem :
[Expected Improvement Maximization (EIM)]
[Purpose : Optimize actual criterion f (·, ν
b)]
1
Set n = 0 ,
2
Repeat :
1
Compute b
xn+1 , the next candidate for an actual evaluation :
b
xn+1 ∈ arg max
x∈X
Z
pν [n](ν)f [n](x , ν) dν ,
ν∈N
where :
2
3
xk , ν) = b
yk and f [n](x , ν) = max
pν [n](ν) = pν ν ∀k = 1 : n, f (b
Request the actual evaluation of b
xn+1 : b
yn+1 = f (b
xn+1 , ν
b)
f (x , ν), max b
yk
k=1:n
Set n ← n + 1 ,
y1:n ) is sufficient.
x1:n , b
until the convergence of (b
[Output :]
The sequence (b
x1:n , b
y1:n ) ,
The model noise estimation pν [n] .
Issue : how to estimate the conditional probability ?
6
Introduction
Formalism
Solve
Example
Conclusion
Rare event simulation & conditional law
The conditional law is approximated by :
xk , ν) = b
yk = pν ν φ(ν) ≥ −ǫ)
pν ν ∀k = 1 : n, f (b
X
xk , ν), ybk and 0 < ǫ ≪ 1
where :φ(ν) = −
d f (b
k
Simulation of rare event φ−1 (] − ǫ, 0]) is used in order to
estimate the conditional probability
The cross-entropy method is considered
De Boer, Kroese, Mannor, Rubinstein, A Tutorial on the Cross-Entropy
Method
for the simulations
for the optimization
7
Introduction
Formalism
Solve
Example
Conclusion
Cross Entropy for simulation
[CE simulation]
1
2
[Purpose : Build an Importance sampler for the rare event φ−1 ([γ, +∞[)]
Set t = 0 and θ0 = θ o such that p = π(·|θ0 )
Repeat :
1 Generate the samples ωti ∈ Ω, i ∈ {1 : Nt }, according to π(·|θt )
2 Compute the evaluations φ(ωti ) of the samples
3
Compute the selective parameters :
ρt [0] = 1 − αt and ρt [i ] = αt P
Rt (φ(ωti ))
i
i =1:Nt Rt (φ(ωt ))
×
π(ωti |θ o )
π(ωti |θt )
, for all i ∈ {1 : Nt }
(1)
// Rt is a selection map ; eg. Rt selects a quantile of best solution
4
[Update] Update the importance sampler by maximizing the cross-entropy with the
selected samples :
!
Z
X
i
θt+1 ∈ arg max
ρt [0]π(ω|θt ) +
ρt [i ]δ[ω = ωt ] log (π(ω|θ)) dω
θ∈Θ
ω∈Ω
i =1:Nt
5 Set t ← t + 1
until the convergence is sufficient, ie. the importance sampling of φ−1 ([γ, +∞[) is sufficient
p
(ω)
ω
[Output :] The importance sampler π(·|θt ) and the likelihood ratio W (ω|θt ) = π(ω|θ
t)
8
(2)
Introduction
Formalism
Solve
Example
Conclusion
Cross Entropy for simulation
[CE simulation]
1
2
[Purpose : Build an Importance sampler for the rare event φ−1 ([γ, +∞[)]
Set t = 0 and θ0 = θ o such that p = π(·|θ0 )
Repeat :
1 Generate the samples ωti ∈ Ω, i ∈ {1 : Nt }, according to π(·|θt )
2 Compute the evaluations φ(ωti ) of the samples
3
ρt [0] = 1 − αt and ρt [i ] = αt P
Rt (φ(ωti ))
i
×
π(ωti |θ o )
π(ωti |θt )
, for all i ∈ {1 : Nt }
(1)
4
selected samples :
!
Z
X
i
θt+1 ∈ arg max
ρt [0]π(ω|θt ) +
ρt [i ]δ[ω = ωt ] log (π(ω|θ)) dω
θ∈Θ
ω∈Ω
i =1:Nt
5 Set t ← t + 1
until the convergence is sufficient, ie. the importance sampling of φ−1 ([γ, +∞[) is sufficient
p
(ω)
ω
[Output :] The importance sampler π(·|θt ) and the likelihood ratio W (ω|θt ) = π(ω|θ
t)
8
(2)
Introduction
Formalism
Solve
Example
Conclusion
Cross Entropy for optimization
[CE optimization]
[Purpose : Find a maximizer of φ]
1
Set t = 0 and θ0 = θ o
2
Repeat :
1
2
3
Generate the samples ωti ∈ Ω, i ∈ {1 : Nt }, according to π(·|θt )
Compute the evaluations φ(ωti ) of the samples
Rt (φ(ωti ))
,
i
ρt [0] = 1 − αt and ρt [i ] = αt P
for all i ∈ {1 : Nt }
(3)
4
selected samples :
θt+1 ∈ arg max
θ∈Θ
Z
ρt [0]π(ω|θt ) +
ω∈Ω
X
i =1:Nt
i
!
ρt [i ]δ[ω = ωt ]
log (π(ω|θ)) dω
5 Set t ← t + 1
until the convergence is sufficient
[Output :] An estimation of the maximizers by means of the importance sampler π(·|θt )
9
(4)
Introduction
Formalism
Solve
Example
Conclusion
Back to sensor-to-mission scenario
Cost C (τ [k]; µ) = c[y [k], x[m1 ]; µ] + · · · + c[x[mj ], y [k]; µ]
P
Solution assessed by local teams :
C (τ [k]; µ
b) > γ[k]
⇒ truncated to trip τb[k] = τ [k|bik ]
Solution is evaluated by G [b
τ; µ
b] − ǫ
P
k=1:K
C (b
τ; µ
b)
10
Introduction
Formalism
Solve
Example
Conclusion
Setings
Sensors : (1, 1), (1, 1), (9, 1), (9, 1), (5, 1)
Autonomy : 1, 2, 1, 2, 2
Missions : 20 missions chosen uniformly on [1, 10] × [1, 10]
Map of threats :
Theoretical threat position, with noise νi ∼ N(0, diag(2, 2)) :
µ = (2, 3) + ν1 , (5, 4) + ν2 , (4, 7) + ν3 , (2, 3) + ν4 }
Actual threat position : µ
b = (1, 1), (4, 6), (3, 7), (1, 4)}
11
Introduction
Formalism
Solve
Example
Conclusion
Setings
Sensors : (1, 1), (1, 1), (9, 1), (9, 1), (5, 1)
Autonomy : 1, 2, 1, 2, 2
Missions : 20 missions chosen uniformly on [1, 10] × [1, 10]
Map of threats :
Theoretical threat position, with noise νi ∼ N(0, diag(2, 2)) :
µ = (2, 3) + ν1 , (5, 4) + ν2 , (4, 7) + ν3 , (2, 3) + ν4 }
Actual threat position : µ
b = (1, 1), (4, 6), (3, 7), (1, 4)}
Inferred cost :
Local cost c[x] = 1/(1 + d(x, T )2 ) decrease with dist. to threats
R
Cost of a path x → y is C [x, y ; µ] = ω∈[x,y ] c[ω] dω
NB : complex path costs are possible, by choosing optimal paths
11
Introduction
Formalism
Solve
Example
Conclusion
Sampling family for simulation and optimization
The map : vector → Gaussian sampling laws
The plan : The choice of the sensor-to-mission assignment and the
choice of a mission rank is encoded by vector :
⇒ The plan sampling is obtained by sampling a vector by a
Gaussian law and by discretizing it into a plan
The CE-update of Gaussian family is a well known computation
→ empirical means and covariance
12
Introduction
Formalism
Solve
Example
Conclusion
A sequence of run
Optimization based on the true :
iter
R opt
samp
1
20
40
60
80 100 120
11.6 13.3 14.3 14.8 15.1 15.5 16
100 2K
4K
6K
8K 10K 12K
140 160 180 200
16.6 17.3 17.7 17.9
14K 16K 18K 20K
A sampled solution : 0 7→ {3} , 1 7→ {2, 10, 12, 16} ,
2 7→ {4, 0, 18} , 3 7→ {13, 6, 5, 15, 7, 8} , 4 7→ {17, 1, 9, 19, 14, 11}
Runs of the EGO :
0.8
2.6
ǫ NaN 0.07 0.96 0.72
y 15.7 17 17.7 17.7 17.1 NaN
b
y 14.9 13.9 11.9 15.92 15.93 NaN
Quality of the conditional estimation ← ǫ
Difficulty : multi-modal conditional law Vs Gaussian sampler
13
Introduction
Formalism
Solve
Example
Conclusion
Conclusion
Main Problem : Optimization with request to costly subprocesses
Related work : Efficient Global Optimization (EGO)
[expected improvement maximization algorithm]
14
Introduction
Formalism
Solve
Example
Conclusion
Conclusion
Generalization proposed to non additive Gaussian noise
CE method used for simulating
CE method used for optimizing
14
Introduction
Formalism
Solve
Example
Conclusion
Conclusion
Perspectives
Multi-modal law for conditional sampling
Performance comparison with pure EGO
Comparison with general POMDP approaches
14
Introduction
Formalism
Solve
Example
Conclusion
Conclusion
Perspectives
Multi-modal law for conditional sampling
Performance comparison with pure EGO
Comparison with general POMDP approaches
Questions ?
14

Optimisation de fonctions partiellement

Transcription

Documents pareils

Introductory Course : signal processing, modulations, coding

Endur`After-Taff (Torrent Ayard)