Suppose we have a Markov Random Field P(X1,...,Xn) on graph G. Suppose we know P(Xi,Xj) for every edge (i,j). Can we recover P(X1,...,Xn)?

If G is a tree, then there's a formula for joint (product of edge marginals divided by product of node marginals). Is there a nice formula that works for some non-tree graphs?

Edit: this is essentially equivalent to the following problem - given an exponential family, how do you write the joint in terms of mean parameters? There's a closed form solution when sufficient statistics are 2 variable functions defined on (Xi,Xj) pairs where (i,j) are edges in some tree graph, is there a closed form solution for other graphs?

Motivation: given an approximate marginalization method, can you fit parameters of a distribution by maximizing joint likelihood of the data under the model "implied" by this marginalization method?

Each variable could be the XOR (sum modulo 2) of the other two variables (i.e. functionally dependent)

The probability of each outcome could be 3/16 if 3 or 1 vertices are 1's, and 1/16 if 2 or 0 are 1's.

So what do you require to be able to uniquely determine the joint distribution? If the graph is decomposable (aka chordal or triangulated), you require the joint distribution for each clique (maximal complete subset) of the graph. Then the joint density is then:

If the graph is not decomposable, then the problem is a bit trickier: the only result I know of is Lauritzen (1996), Lemma 3.14. Basically, given the clique marginal distributions, uniqueness is determined when the sample space is finite, and each clique marginal density is the limit of a sequence of positive densities. I suspect this result could be made stronger in some way, but I am not aware of any efforts to do so.

Right, distr. over 3 binary variables has 7 degrees of freedom, while fixing edge/node marginals only constrains 6. So I'm looking for a formula of a joint, any joint, that's consistent with given marginals, in terms of those marginals. One choice might be to take the highest entropy distr consistent with marginals, but even for 3 variable distr, formula for such joint in terms of marginal parameters is quite large (mathurl.com/324o4a3) Testing if given set of marginals is consistent can takes exp time, so a formula must not provide an easy way to check that to be short – Yaroslav
–
Yaroslav BulatovJul 29 '10 at 18:15

@simon's link appears to be broken, but I believe they are referring to this paper: Graphical Models, Exponential Families, and Variational Inference dx.doi.org/10.1561/2200000001
–
Yoav KallusMay 20 at 14:26

the method for trees should generalize to graphs of bounded treewidth, where the formula might get exponentially long in the treewidth itself. Also, do you care how (algorithmically) complicated the formula is, because it's probably possible to write the general formula as some kind of sum over spanning trees of the graph.

I can see how it generalizes if you are given distributions over maximal cliques and separators, but how do you go from edge marginals to distributions over maximal cliques?
–
Yaroslav BulatovJul 24 '10 at 3:41

For uniform potentials, it doesn't get a lot nicer. For instance take Ising model with uniform potentials on a loop of size n, then probability of all spins being +1 can be written in terms of marginal m = P(x1=x2) as

$exp(n j)/Z$
where
Z=$\lambda_1^n + \lambda_2^n$, $\lambda_1=e^j+e^{-j}$, $\lambda_2=e^j-e^{-j}$ and j is the solution of