Abstract. If biology is the study of self-replicating entities, and we want to understand the role of information, it makes sense to see how information theory is connected to the ‘replicator equation’—a simple model of population dynamics for self-replicating entities. The relevant concept of information turns out to be the information of one probability distribution relative to another, also known as the Kullback–Liebler divergence. Using this we can see evolution as a learning process, and give a clean general formulation of Fisher’s fundamental theorem of natural selection.

]]>
By: Relative Entropy in Evolutionary Dynamics | Azimuth https://johncarlosbaez.wordpress.com/2012/06/07/information-geometry-part-11/#comment-35489
Wed, 22 Jan 2014 11:53:06 +0000http://johncarlosbaez.wordpress.com/?p=10114#comment-35489Suppose is fixed, while evolves in time according to the replicator equation. Then

Abstract. The equations of evolutionary change by natural selection are commonly expressed in statistical terms. Fisher’s fundamental theorem emphasizes the variance in fitness. Quantitative genetics expresses selection with covariances and regressions. Population genetic equations depend on genetic variances. How can we read those statistical expressions with respect to the meaning of natural selection? One possibility is to relate the statistical expressions to the amount of information that populations accumulate by selection. However, the connection between selection and information theory has never been compelling. Here, I show the correct relations between statistical expressions for selection and information theory expressions for selection. Those relations link selection to the fundamental concepts of entropy and information in the theories of physics, statistics, and communication. We can now read the equations of selection in terms of their natural meaning. Selection causes populations to accumulate information about the environment.

]]>
By: Tobias Fritz https://johncarlosbaez.wordpress.com/2012/06/07/information-geometry-part-11/#comment-17176
Thu, 26 Jul 2012 09:09:18 +0000http://johncarlosbaez.wordpress.com/?p=10114#comment-17176Thank you for the corrections and the lucid explanation! I hope to get back to this sort of stuff in some more detail in the intermediate future…
]]>
By: John Baez https://johncarlosbaez.wordpress.com/2012/06/07/information-geometry-part-11/#comment-17169
Thu, 26 Jul 2012 03:31:57 +0000http://johncarlosbaez.wordpress.com/?p=10114#comment-17169Tobias wrote:

One nice aspect of category theory, in contrast to real life, is that it’s pretty clear what’s evil and what’s not ;)

Well, a lot of people have trouble understanding the concept of ‘evil’ in category and n-category theory… and I’m afraid maybe you do, too!

Elements of sets are not evil, nor are objects of categories.

What’s ‘evil’, in the technical sense of that word, is to make a definition or state a theorem asserting some property of objects in an n-category that holds for one object but fails to hold for some equivalent object.

This is fairly precise, but click the link to read more subtleties.

For example, it’s evil to define a function

to be an inclusion when it’s 1-1 and is a subset of …

…if we are thinking of functions as objects in the arrow category of Set. The reason is that the second clause, “ is a subset of “, can easily be true for some function

but not true for some function

that’s isomorphic in the arrow category.

To fix this, we can make the non-evil definition: a function

is an injection when it’s 1-1.

And so on…

One reason for avoiding ‘evil’ definitions and theorems is just that then all the facts we prove are guaranteed to be invariant under equivalence, which means we don’t need to check a lot of annoying fine print before applying them. But some category theorists get very annoyed that I use the term ‘evil’ in this way… because they want to do things that are evil in this sense, and don’t like being called ‘evil’.

This is off topic, but I wonder if that means that even the concept of ‘morphism’ is evil: a morphism is an element of the set of morphisms between two objects, and elements are evil! A similar “objection” applies to the ‘objects’ in a category. Does higher category theory offer a solution to this problem?

I’m a bit puzzled by the term “subcategory” here; you rather get FinSet and FinStoch as a category of algebras of a PROP (or something like that) in FHilb, right?

Yeah, I shouldn’t have called it a “subcategory”—I was trying to show off, but I got carried away.

Among category theorists, it’s common to show off by saying X is a subset of Y even if it’s not, as long as X is equipped with a monomorphism from X to Y. This is, after all, the ‘non-evil’ variant of the notion of subset, as used in structural set theory. Being a subset is then not a property, but a structure.

The obvious functor from FinSet (or FinStoch) to finite-dimensional real Hilbert spaces is faithful, but not full. In this situation, it’s bound to be confusing to think of this functor as making FinSet a ‘subcategory’ of FinHilb, since it’s like saying the category of groups is a subcategory of Set, which sounds really stupid.

Actually I was thinking about the monoidal structure on FinStoch that restricts to the product in FinSet […]

Ah, darn! I can see why this makes more sense in the context of Bayesian networks.

It means the slice category [1]/FinStoch is quite interesting.

Yes, it’s like FinProb, with functions generalized to stochastic maps. You had brought this up before!

But the reason I’m not entirely satisfied with your characterization of FinSet inside FinStoch is […] that you’re describing morphisms that generate this subcatgory, whereas I was hoping there’s some property that holds precisely for the morphisms in this category.

Yes, I know! That’s what I meant by “highbrow” ;)

But I think I made some progress with Jamie and Brendan from a slightly different direction: starting from the category of finite-dimensional real Hilbert spaces, then picking out FinSet as the subcategory of special commutative dagger-Frobenius algebras with morphisms preserving the comultiplication, and then using that to also pick out FinStoch as a subcategory.

Great to see you’re making progress! I’m a bit puzzled by the term “subcategory” here; you rather get FinSet and FinStoch as a category of algebras of a PROP (or something like that) in FHilb, right?

]]>
By: John Baez https://johncarlosbaez.wordpress.com/2012/06/07/information-geometry-part-11/#comment-17145
Wed, 25 Jul 2012 10:18:50 +0000http://johncarlosbaez.wordpress.com/?p=10114#comment-17145Actually I was thinking about the monoidal structure on FinStoch that restricts to the product in FinSet, since a random variable taking values in is a stochastic map

and if we have two random variables

the pair of them can be considered a random variable in its own right

By the way, I learned this idea of treating random variables as stochastic maps from Jamie Vicary. It means the slice category [1]/FinStoch is quite interesting.

But the reason I’m not entirely satisfied with your characterization of FinSet inside FinStoch is not this, and it’s not that it’s insufficiently ‘highbrow’: it’s that you’re describing morphisms that generate this subcatgory, whereas I was hoping there’s some property that holds precisely for the morphisms in this category.

But I think I made some progress with Jamie and Brendan from a slightly different direction: starting from the category of finite-dimensional real Hilbert spaces, then picking out FinSet as the subcategory of special commutative dagger-Frobenius algebras with morphisms preserving the comultiplication, and then using that to also pick out FinStoch as a subcategory. This seems like a decent way to go about things, especially because ‘infinitesimal stochastic’ maps, i.e. generators of 1-parameter stochastic groups, are also present in the original big category.