Though... it occurs to me that one could construct something that acted like a "square root of a delta", which would then make an orthonormal basis (though still not part of the hilbert space).

I'm not sure what you're trying to construct, but note that one can only multiply distributions under rather restrictive conditions. There are some even more abstract classes of distributions which permit an associative multiplication (Colombeau algebras, generalized Gevrey classes of ultradistributions, and so on) but they're neither terribly common nor fun to work with.

then define integral B(x)f(x) dx as the limit as k->0+ of integral b(x,k)f(x) dx

I was thinking that then integral (B(x))^2 f(x) dx would be like integral delta(x)f(x) dx.

Now that I think about it more carefully, especially in light of your comment, perhaps that was naive and that wouldn't actually work. (Yeah, I can see now my reasoning wasn't actually valid there. Whoops.)

Good question! The Dirac delta distributions are a basis in a certain sense, but not in the sense that I was talking about in my previous comment (which is the sense in which mathematicians and physicists say that "the Hilbert space of quantum mechanics has a countable basis"). I realize now that I should have been more clear about what kind of basis I was talking about, which is an orthonormal basis - each element of the basis is a unit vector, and the lines spanned by distinct basis elements meet at right angles. Implicit in this formulation is the assumption that elements of the basis will be elements of Hilbert space. This is why the Dirac delta distributions are not a basis in this sense - they are not elements of Hilbert space; in fact they are not even functions but are rather generalized functions). Physicists also like to say that they are "nonrenormalizable" in the sense that "no scalar multiple of a delta function is a unit vector" - illustrating failure of the criterion of orthonormality in a more direct way.

The sense in which the Dirac delta distributions are a basis is that any element of Hilbert space can be written as a integral combination of them:

(Both sides of this equation are considered in the distributional sense, so what this formula really means is that for any function ,

which is a tautology.) This is of course a very different statement from the notion of orthonormal basis discussed above.

So what are some differences between these two notions of bases?

Orthonormal bases have the advantage that any two orthonormal bases have the same cardinality, allowing dimension to be defined consistently. By contrast, if one applies a Fourier transform to Hilbert space on [0,1], one gets Hilbert space on the integers; but the former has an uncountable basis of Dirac delta functions while the latter has a countable basis of Dirac delta functions. The Fourier transform is a unitary transformation, so intuitively that means it shouldn't change the dimension (or other properties) of the Hilbert space. So the size of the Dirac delta basis is not a good way of talking about dimension.

Orthonormal bases take the point of view that Hilbert space is an abstract geometric object, whose properties are determined only by its elements and the distances between them as defined by the distance function I described in my previous comment. By contrast, Dirac delta bases only make sense when you go back and think of the elements of Hilbert space as functions again. Both these points of view can be useful. A big advantage of the abstract approach is that it means that unitary transformations will automatically preserve all relevant properties (e.g. Fourier transform preserving dimension as noted above).

So to summarize, both bases are useful, but the orthonormal basis is the right basis with respect with which to ask and answer the question "What is the dimension of Hilbert space?"

I'd thought the Hilbert space was uncountably dimensional because the number of functions of a real line is uncountable. But in QM it's countable... because everything comes in multiples of Planck's constant, perhaps? Though I haven't seen the actual reason stated, and perhaps it's something beyond my current grasp.

Ahh... here's something I can help with. To see why Hilbert space has a countable basis, let's first define Hilbert space. So let

= the set of all functions such that the integral of is finite, and let

= the set of all functions such that the integral of is zero. This includes for example the Dirichlet function which is one on rational numbers but zero on irrational numbers. So it's actually a pretty big space.

Hilbert space is defined to be the quotient space . To see that it has a countable basis, it suffices to show that it contains a countable dense set. Then the Gram-Schmidt orthogonalization process can turn that set into a basis. What does it mean to say that a set is dense? Well, the metric on Hilbert space is given by the formula

,

so a sequence is dense if for every element of Hilbert space, you can find a sequence such that . Now we can see why we needed to mod out by -- any two points of are considered to have distance zero from each other!

So what's a countable dense sequence? One sequence that works is the sequence of all piecewise-linear continuous functions with finitely many pieces whose vertices are rational numbers. This class includes for example the function defined by the following equations:

for all

for all

for all

for all

Note that I don't need to specify what does if I plug in a number in the finite set , since any function which is zero outside of that set is an element of , so would represent the same element of Hilbert space as .

So to summarize:

The uncountable set that you would intuitively think is a basis for Hilbert space, namely the set of functions which are zero except at a single value where they are one, is in fact not even a sequence of distinct elements of Hilbert space, since all these functions are elements of , and are therefore considered to be equivalent to the zero function.

The actual countable basis for Hilbert space will look much different, and the Gram-Schmidt process I alluded to above doesn't really let you say exactly what the basis looks like. For Hilbert space over the unit interval, there is a convenient way to get around this, namely Parseval's theorem, which states that the sequences
and form a basis for Hilbert space. For Hilbert space over the entire real line, there are some known bases but they aren't as elegant, and in practice we rarely need an explicit countable basis.

Finally, the philosophical aspect: Having a countable basis means that elements of Hilbert space can be approximated arbitrarily well by elements which take only a finite amount of information to describe*, much like real numbers can be approximated by rational numbers. This means that an infinite set atheist should be much more comfortable with countable-basis Hilbert space than with uncountable-basis Hilbert space, where such approximation is impossible.

* The general rule is:

Elements of a finite set require a finite and bounded amount of information to describe.

Elements of a countable set require a finite but unbounded amount of information to describe.

Elements of an uncountable set (of the cardinality of the continuum) require a countable amount of information to describe.

Meant to reply to this a bit back, this is probably a stupid question, but...

The uncountable set that you would intuitively think is a basis for Hilbert space, namely the set of functions which are zero except at a single value where they are one, is in fact not even a sequence of distinct elements of Hilbert space, since all these functions are elements of , and are therefore considered to be equivalent to the zero function.

What about the semi intuitive notion of having the dirac delta distributions as a basis? ie, a basis delta(X - R) parameterized by the vector R? How does that fit into all this?

Actually, come to think about it, even specifying the desired behavior would be tricky. Like if the agent assigned a probability of 1/2 to the proposition that tomorrow they'd transition from v to w, or some other form of mixed hypothesis re possible future transitions, what rules should an ideal moral-learning reasoner follow today?

I'm not even sure what it should be doing. mix over normalized versions of v and w? what if at least one is unbounded? Yeah, on reflection, I'm not sure what the Right Way for a "conserves expected moral evidence" agent is. There're some special cases that seem to be well specified, but I'm not sure how I'd want it to behave in the general case.

Really interesting, but I'm a bit confused about something. Unless I misunderstand, you're claiming this has the property of conservation of moral evidence... But near as I can tell, it doesn't.

Conservation of moral evidence would imply that if it expected that tomorrow it would transition from v to w, then right now it would be acting on w rather than v (except for being indifferent as to whether or not it actually transitions to w), but what you have here would, if I understood what you said correctly, will act on v until that moment it transitions to w, even though it knew in advance it was going to transition to w.

Hey. You might have had this question answered already but just in case: they don't have housing or dorms. But they do have room and allow you to put up a cot or inflatable mattress and sleep there for the duration.

if I have two hypotheses that both explain an "absence of evidence" occurrence equally well, then that occurrence does not give me reason to favor either hypothesis and is not "evidence of absence."

This statement is technically true, but not in the way you're using it.

Suppose Vibrams had been around for a thousand years. For a thousand years, people had been challenging their claims to health benefits in court. For a thousand years, time and again, Vibrams had been unable to credibly defend their claims. Would that make you any more skeptical of the claims in question, at least a little bit?
If the answer is "yes", you are agreeing that some very large number of such events constitutes evidence against Vibrams. I don't see any way around concluding, from there, that at least one individual instance provides some nonzero amount of evidence - perhaps very small, but not zero.

"Vibrams work, but the effect is small and/or the experiment was shoddy" and "Vibrams don't work" explain the outcome nearly equally well. They cannot explain it precisely equally well: the first hypothesis would assign a higher P(claims defended) than the second, because even small effects are sometimes correctly detected, and even shoddy experiments sometimes aren't fatally flawed. So the second necessarily has a higher P(~claims defended) than the first. This difference is precisely the thing that makes (~claims defended) evidence for the second hypothesis.

Evidence is not proof. Depending on the ratios involved, it may constitute very weak evidence, sometimes weak enough that it's not even worth tracking for mere humans: a .0001% shift is lost in the noise when people aren't even calibrated to the nearest 10%.

If you have two hypotheses that both explain an "absence of evidence" precisely equally well, then you're looking at something completely uncorrelated: trying to deduce the existence of a Fifth Column from the result of a coin flip. And if they explain it only nearly, but not exactly equally well, then you have evidence of absence - although maybe not very much, and maybe not enough to actually push you into the other camp.

Chris covered a lot of things. Re getting accepted, I think you'll be okay. You're ahead of where I was and I can tell you're smart. Do the prep work they give you, do some project Euler problems. I don't think you have to do the challenges in Ruby, but knowing at least one language well will help.

If you are accepted I strongly recommend a) Going to SF, not NY. The job market is better and I suspect the instruction is as well. B) If you don't mind too much: stay at App Academy (2016 edit: they no longer allow this). It isn't comfortable but you'll greatly benefit from being around other people learning web development all the time and it will keep you from slacking off. Remember that this isn't college. You don't get a certificate or degree. So the point isn't to get through the program. The point is to learn as much as you possibly can while you're there.

Also, If you're still on the edge about doing it, I strongly recommend it. App Academy easily had a bigger beneficial impact on my life than anything else I've done. Let me know if you have any specific questions.

Hey there, I'm mid application process. (They're having me do the prep work as part of the application). Anyways,,,

B) If you don't mind too much: stay at App Academy. It isn't comfortable but you'll greatly benefit from being around other people learning web development all the time and it will keep you from slacking off.

I'm confused about that. App Academy has housing/dorms? I didn't see anything about that. Or did I misunderstand what you meant?