Simulation of correlated variables: Solution by DJCM and Iain Murray and Ed Snelson

x <- r -> y

AIM:

Simulate P(x,y) = ( (1-f)/2 , f/2 ; f/2 , (1-f)/2 ) using
sum_r P(r) P(x|r) P(y|r)
such that H(R) < 1 bit.

SOLUTION

SOLUTION:

r = 0 / 1 with probabilities p0 , p1, where p1 = 1/(2(1-f))

P(x|r=0) = { 1 , 0 }
P(y|r=0) = { 1 , 0 }

P(x|r=1) = { f , (1-f) }
P(y|r=1) = { f , (1-f) }

H(R) = H_2(p1) = H_2( 1/(2(1-f)) )

We confirmed, by brute-force plotting of the functions

 s(a,b,f) = -1.0/2.0*(2*f-1.0)/(-f+1.0-b-a+2*a*b)
 d(a,b,f) = (a-1+f)/(2.0*a-1)
 c(a,b,f) = (f-1+b)/(-1.0+2*b),
that this is the optimal solution using a binary latent variable "r" to generate a single pair (x,y).

QUESTIONS


David MacKay
Last modified: Thu Jul 15 12:54:42 2004