Trying to understand Dark Energy

Hi, I wonder if anyone can help me understand the concept of dark energy better.

I understand the very basic concept that the universe is expanding and that the rate of expansion may be speeding up (I've read somewhere that some don't think the rate is actually increasing) but how does that lead to there being some sort of dark energy?

From my very elementary understanding of this, I imagined that dark energy was actually part of space, where as dark matter and visible matter are things that exist in space. Is that correct?

Also, as the universe is expanding, then does this mean the amount of dark energy per cubic meter of space becomes more dilute or is the rate of dark energy per cubic meter of space a constant? (Or increasing?) If so how is that possible? Doesn't that contradict conservation of energy?

Another thing that confuses me is how are things like the Higgs field effected by the expanding universe? Does the Higgs field expand with the universe, so for any particles effected by the Higgs field, their mass remains constant? If so then how is that possible? Also does dark mater affect the Higgs field?

Or is there any line of thought that says as the universe expands, then the Higgs field becomes more dilute and thus has less of an effect on matter meaning matter doesn't attract as much over larger distances and therefore that could contribute to expansion speeding up?

Sorry for the multiple questions! And please excuse my terminology, I have no background in physics at all it just fascinates me :)

From my very elementary understanding of this, I imagined that dark energy was actually part of space, where as dark matter and visible matter are things that exist in space. Is that correct?

'A part of space' is not a well-defined term. I find it more helpful to just think of it as a feature of spacetime. If we start thinking of it as some sort of 'stuff' we can end up getting quite confused.

Also, as the universe is expanding, then does this mean the amount of dark energy per cubic meter of space becomes more dilute or is the rate of dark energy per cubic meter of space a constant? (Or increasing?) If so how is that possible? Doesn't that contradict conservation of energy?

It remains constant. It doesn't contradict conservation of energy because

(1) it's not really energy in the same way that heat and work are ('Dark Energy' is just a fancy name to make Einstein's Cosmological Constant (which is what it really is) sound cool); and

(2) Conservation of energy is a local phenomenon that does not apply in General Relativity, because energy is a frame-dependent measure.

'A part of space' is not a well-defined term. I find it more helpful to just think of it as a feature of spacetime. If we start thinking of it as some sort of 'stuff' we can end up getting quite confused.

Ah ok, that make sense. I guess my logic was that galaxies can recede from each other at speeds greater than c, as they are not moving through spacetime. So it is the fabric of space that is actually expanding. So as it isn't a 'force' that is pushing the galaxies apart through space, it must be a part of whatever 'space' is. If that makes sense.

I just had another question pop into my head, sorry! Is it correct to say all matter can warp spacetime, but Dark Energy doesn't warp spacetime or counteract matter from warping spacetime?

It remains constant. It doesn't contradict conservation of energy because

(1) it's not really energy in the same way that heat and work are ('Dark Energy' is just a fancy name to make Einstein's Cosmological Constant (which is what it really is) sound cool); and

(2) Conservation of energy is a local phenomenon that does not apply in General Relativity, because energy is a frame-dependent measure.

Ah ok, I see thanks. But as I understand it, the Cosmological Constant was a fudge factor to keep a static universe. So there must be some work being done to counteract gravity? Also if space is expanding, there must be some sort energy or work being done somewhere to generate the extra space? So I am finding it difficult to understand where all this extra 'Dark Energy' is coming from.

I also assumed that as the amount of matter isn't growing, the percentage that dark energy makes up of the total universe, which is 68% ish now I think, would be increasing with time.

the Cosmological Constant was a fudge factor to keep a static universe.

It can be used for that but, from what I have read, that was not the main reason for its original inclusion. My understanding is that it's more like a constant of integration. In developing his gravitation equation, Einstein was reasoning that the curvature of spacetime, which is a 4 x 4 tensor (the Einstein Tensor) must vary according to the density of mass-energy, which is represented by the 4 x 4 stress-energy tensor. But you can add a constant (wrt mass-energy) term to that and it will still keep that dependency on mass-energy. That constant term is ##\Lambda##, the cosmological constant multiplied by ##g^{\alpha\beta}## the metric tensor (another 4 x 4 tensor).

I have read that Einstein later regretted including it, because there was no evidence for its being nonzero, and it made the equation 50% messier (three terms instead of two). He even called it a 'mistake'. He didn't live long enough to see himself vindicated by the work of Brian Schmidt et al that demonstrated it was nonzero.

By the way, the term 'accelerating expansion' is ambiguous and can be misleading. As I recall, what that means is that the rate at which any given object recedes from us will increase over time. However, I think it is believed that the recession rate of objects at a fixed distance is actually decreasing. That is, the recession rate of a (co-moving) object currently 100 parsecs away is greater than the expected recession rate, a billion years from now, of another co-moving object that will be 100 parsecs away at that time. So the recession rate is increasing if we are fixing on an object, and decreasing if we are fixing on a distance.

It can be used for that but, from what I have read, that was not the main reason for its original inclusion. My understanding is that it's more like a constant of integration. In developing his gravitation equation, Einstein was reasoning that the curvature of spacetime, which is a 4 x 4 tensor (the Einstein Tensor) must vary according to the density of mass-energy, which is represented by the 4 x 4 stress-energy tensor. But you can add a constant (wrt mass-energy) term to that and it will still keep that dependency on mass-energy. That constant term is Λ \Lambda, the cosmological constant multiplied by gαβ g^{\alpha\beta} the metric tensor (another 4 x 4 tensor).

Thanks for the information, but to be honest that's a bit too advanced for me to understand. I am guessing it has something to do with gravity and creating a flat universe?

However, I think it is believed that the recession rate of objects at a fixed distance is actually decreasing. That is, the recession rate of a (co-moving) object currently 100 parsecs away is greater than the expected recession rate, a billion years from now, of another co-moving object that will be 100 parsecs away at that time. So the recession rate is increasing if we are fixing on an object, and decreasing if we are fixing on a distance.

That confused me a little. I was reading the Balloon Analogy that is posted in phinds signature. The section on the acceleration of expansion says

it was found that not only is the rate of expansion NOT slowing down, it is ACCELERATING.

Which suggests that the rate at which an object of a fixed distance (eg 100 parsecs) is moving away is increasing not decreasing. Or have I misinterpreted that?

By the way, the term 'accelerating expansion' is ambiguous and can be misleading. As I recall, what that means is that the rate at which any given object recedes from us will increase over time. However, I think it is believed that the recession rate of objects at a fixed distance is actually decreasing. That is, the recession rate of a (co-moving) object currently 100 parsecs away is greater than the expected recession rate, a billion years from now, of another co-moving object that will be 100 parsecs away at that time. So the recession rate is increasing if we are fixing on an object, and decreasing if we are fixing on a distance.

Yes, that's correct, but it's very confusing to bring the concept of a changing Hubble Constant into the conversation this way. You can see the confusion you've caused in @rede96.

Which suggests that the rate at which an object of a fixed distance (eg 100 parsecs) is moving away is increasing not decreasing. Or have I misinterpreted that?

Your understanding is correct. Andrew is bringing in a whole new concept, which is based on the fact that the rate of acceleration is very slowly decreasing over time. It IS still accelerating just not quite as fast. He is talking about an object that is a fixed distance from us now and an object that is the same distance away from us far in the future. The are both receding away from us at an ever increasing rate but the rate of that acceleration is just slightly less in the future.

Your understanding is correct. Andrew is bringing in a whole new concept, which is based on the fact that the rate of acceleration is very slowly decreasing over time. It IS still accelerating just not quite as fast. He is talking about an object that is a fixed distance from us now and an object that is the same distance away from us far in the future. The are both receding away from us at an ever increasing rate but the rate of that acceleration is just slightly less in the future.

Ah ok, I think I understand now. So when we talk about the ACCELERATION of expansion, then what that really means is that objects further away are moving away at a faster and faster rate. But that 'rate' of that acceleration is decreasing over time as measured for objects of the same distance now, and objects of the same distance in the future.

Ah ok, I think I understand now. So when we talk about the ACCELERATION of expansion, then what that really means is that objects further away are moving away at a faster and faster rate. But that 'rate' of that acceleration is decreasing over time as measured for objects of the same distance now, and objects of the same distance in the future.

There's a good discussion in the link I gave you about the three aspects of expansioy in cosmology (inflation, expansion, and acceleration of expansion).

I've read the links you mentions but I'm still a little confused about how that leads to some sort of dark energy being present. Couldn't the acceleration observed simply be caused by the energy produced from the Big Bang / inflation?

I've read the links you mentions but I'm still a little confused about how that leads to some sort of dark energy being present. Couldn't the acceleration observed simply be caused by the energy produced from the Big Bang / inflation?

No, that energy caused the expansion we see, but it requires an ongoing process, not something happened billions of years ago, to cause the acceleration of the expansion. Cosmologists 30 years ago firmly believed that the expansion would be found to be slowing down and would keep slowing down or would actually reverse (the "big crunch") but to everyone's surprise, when the first measurements were made, it was found to be accelerating. Thus dark energy was posited, and "dark energy" is really just a place-holder name that means "we know what's happening but we don't know why, so we're going to call the reason for it dark energy until we figure it out".

No, that energy caused the expansion we see, but it requires an ongoing process, not something happened billions of years ago, to cause the acceleration of the expansion. Cosmologists 30 years ago firmly believed that the expansion would be found to be slowing down and would keep slowing down or would actually reverse (the "big crunch") but to everyone's surprise, when the first measurements were made, it was found to be accelerating.

As I understand it, 'space' itself is expanding, which leads to more distant objects moving away faster (accelerating). So as I am understanding what is observed was simply that. The more distant the object the faster it moved away. This situation would be the same for any rate of expansion (Hubble constant)

So I was wondering why it needed to be an ongoing process? I suppose I imagined it like motion, e.g. Every object in a state of uniform motion tends to remain in that state of motion unless an external force is applied to it. So I saw the initial inflation or the expansion of space as a sort of motion. There was in initial energy to cause space to expand and without any other forces acting on it space would continue to expand. This causes more distance objects to move away faster. And it would make sense (to me anyway!) that gravity would be responsible for the gradual decrease of the Hubble constant.

So could you help me understand why it needs to be an ongoing process?

There was in initial energy to cause space to expand and without any other forces acting on it space would continue to expand. This causes more distance objects to move away faster. And it would make sense (to me anyway!) that gravity would be responsible for the gradual decrease of the Hubble constant.

So could you help me understand why it needs to be an ongoing process?

If the expansion were simply an inertial phenomenon, whereby things keep moving away from one another because of an impetus given them long ago, an object that is currently receding at 10 m/s would continue to recede at that rate if space were flat and hence infinite. One way to visualise that is that the gravitational pull of objects nearer to us than the object pull it towards us and retard its recession, but they are balanced by gravitational forces of objects farther from us pulling it away.

Alternatively, if space were elliptical (bounded), the recession would slow because of gravitational forces pulling things back together.

The real answer is in the gravitational equation and the Friedman equation. But that simplified visualisation works for me.

If the expansion were simply an inertial phenomenon, whereby things keep moving away from one another because of an impetus given them long ago, an object that is currently receding at 10 m/s would continue to recede at that rate

Sure, but I was referring to space expanding not objects moving through space. Again, as I understand it, the reason galaxies recede from each other is because of the space between them expanding, not the actual galaxies moving through space. Hence why we can have recession velocities greater than c.

So I was looking at space expanding having inertia (if that is the right term) if expanding space had inertia and expanding space is responsible for galaxies receding, then it makes sense that the further a galaxy was away from us, the faster it would be moving. So in that case we wouldn't necessarily need dark energy.

If galaxies moving away from each other is not due to expanding space, but rather some unknown 'energy' that permeates all space which is pushing against all matter in all directions, then I would understand that to be dark energy.

There is no inertia involved in metric expansion. If there were nothing but expansion, then as andrew said, the rate of recession of a distant galaxy would be constant. It is not constant. When it is X light years from us it is receding at R. When it is 2X LY from us, it is receding at 2.1R. When it is 3X LY from us, it is receding at 2.4R (I'm making these numbers up but they illustrate the point). This cannot be simple ongoing expansion of space. Something is causing the expansion to accelerate.

The real answer is in the gravitational equation and the Friedman equation...

Andrew I liked very much your posts 2 and 7 in this thread. I liked your ability to look at things in different ways.
One form of the Friedman equation for the spatially flat case is
H2 - H∞2 = const ρ

where rho ρ is the combined density of ordinary and dark matter. (In energy density terms 0.24 nanojoule per m3) There is no contribution from the cosmological constant since that is on the left side.
It's just a fact that the cosmological constant is 3H∞2 so it appears in the equation that way, similarly to how it does in the GR equation ("gravitational equation") you referred to in your post just now.

One way to look at this is that the square of the expansion rate is a kind of spacetime curvature and the constant gives a measure of the flexibility of spacetime geometry (or conversely it lets you know how stiff geometry is--how much matter density you have to put to get a given amount of curvature.)

The bigger rho is, the bigger the squared expansion rate.
Then as space expands the combined matter density goes to zero, so the lefthand side goes to zero so that H converges to H∞. Matter density DOES cause the expansion rate to decline, but it doesn't have to decline all the way to zero! There is a residual expansion rate given by the cosmo constant.
Some refer to it as "vacuum curvature"---it just complicates things to imagine it caused by some imagined "dark energy"---it behaves like a constant intrinsic feature of geometry.

I realize this is extremely vague and handwavy but maybe I can illustrate with some real quantities given by actual numbers

The point about the rate tailing off to a constant is that to the extent that we have a constant expansion rate we have exponential distance growth at a constant percentage growth rate. So if you look at a specific distance between two essentially stationary galaxies, that distance is growing exponentially. The speed of distance growth, for that particular distance you are tracking, is of course increasing and naturally it is not limited by c because this is geometry change, not ordinary motion. Nobody gets anywhere by it, everybody just becomes farther apart. Not good to think of it as ordinary relative motion (which is limited by c.)