Case Study: Relative Entropy Minimization
Back to main pageCase study background and problem formulations
PROBLEM: problem_entropyr
Minimize Entropyr (minimizing Relative Entropy)nsubject to
Linear = 0 (constraint on sum of decision variables (probabilities)) A * x = b (linear equality constraint) Box constraints (non-negative lower bound for probabilities) ——————————————————————– Entropyr = Relative Entropy
A = matrix in the linear equality constraintnb = vector in the right hand side of the linear equality constraint
Box constraints = constraints on individual decision variables ——————————————————————– Data and solution in Run-File Environmentn
Linear = 0 (constraint on sum of decision variables (probabilities)) A * x = b (linear equality constraint) Box constraints (non-negative lower bound for probabilities) ——————————————————————– Entropyr = Relative Entropy
A = matrix in the linear equality constraintnb = vector in the right hand side of the linear equality constraint
Box constraints = constraints on individual decision variables ——————————————————————– Data and solution in Run-File Environmentn
| Problem Datasets | # of Variables | # of Scenarios | Objective Value | Solving Time, PC 2.66GHz (sec) | |||
|---|---|---|---|---|---|---|---|
| Dataset1 | Problem Statement | Data | Solution | 250 | 3 | 2.167273 | <0.01 |
| Dataset2 | Problem Statement | Data | Solution | 100,000 | 3 | 1.923068 | 1.38 |
Data and solution in MATLAB Environmentn
| Problem Datasets | # of Variables | # of Scenarios | Objective Value | Solving Time, PC 2.66GHz (sec) | |||
|---|---|---|---|---|---|---|---|
| Dataset1 | Matlab code | Data | Solution | 250 | 3 | 2.167273 | <0.01 |
| Dataset2 | Matlab code | Data | Solution | 100,000 | 3 | 1.92307 | 1.1 |
Data and solution in R Environmentn
| Problem Datasets | # of Variables | # of Scenarios | Objective Value | Solving Time, PC 2.66GHz (sec) | ||
|---|---|---|---|---|---|---|
| Dataset1 | R code | Data | 250 | 3 | 2.167273 | <0.01 |
| Dataset2 | R code | Data | 100,000 | 3 | 1.92307 | 1.1 |
CASE STUDY SUMMARYn
This case study demonstrates an optimization problem for minimizing Relative Entropy with linear constraints. Relative Entropy is used to find a probability distribution which is the most close to some "prior" probability distribution subject to available information about the distribution. For instance, moments of a distribution can be known and we want to find the “best” distribution accounting for this information.
Problem is solved for 250 probability atoms of a discrete probability distribution (250 decision variables), and for 100,000 probability atoms of a discrete probability distribution (100,000 decision variables).
Problem is solved using 2 datasets:n• Dataset1 for 250 decision variables;n• Dataset2 for 100,000 decision variables.