I Normal modes using representation theory

Hello! I am reading some representation theory (the book is Lie Algebra in Particle Physics, by Georgi, part 1.17) and the author solves a problem of 3 bodies connected by springs forming a triangle, aiming to find the normal modes. He builds a 6 dimensional vector formed of the 3 particles and their x and y coordinates and a 6 dimensional representation of S3, as the symmetry of the system. Firstly, can someone explain to me why does the system has a 6D S3 symmetry. I understand you can permute the 3 balls, but I am not sure I understand why changing the x direction motion of one particle with the y direction of another one, is still a symmetry of the system (while keeping everything else the same). Then, he finds the projector operators onto the subspaces on which the irreducible representations act on. The 1D representations appear only once, so finding the normal modes associated to them are easy to find. However the 2D representation appears 2 times and he tries to find the normal modes associated to that. He calculates the projector - P2 - associated to this (P2a.png) and he takes the x and y translations as already known modes and subtract them from P2 and then in order to find the other 2 modes he does the stuff in P2b.png. Can someone explain that to me, please? Why does he pick that vector and why does he rotates it by $2\pi/3$. I assume it is something trivial, but I can't seem to figure it out. Thank you!

Firstly, can someone explain to me why does the system has a 6D S3 symmetry.

It is the system itself that has an S3 symmetry. You can exchange any two of the bodies and end up with the same system assuming that you transform the displacements of the bodies accordingly. For example you can exchange bodies 1 and 2 by a reflection in the vertical line through the position of body 3. This reflection inverts the x-component of all displacements and exchanges the displacements of bodies 1 and 2, which tells you how this symmetry acts on your 6D representation. In effect, you have three displacement vectors that each transform according to an ##O(2)## representation of ##S_3## (let us call it ##\rho_2##) and that are exchanged according to the natural three-dimensional representation of ##S_3## (let us call it ##\rho##), meaning that your 6D representation is given by ##\rho \otimes \rho_2##.

Now, you know that the 6D representation contains overall translations and rotations (as those can be easily constructed by having the displacements of all bodies be the same and by having the displacements being those given by a rotation, respectively). Therefore, your 6D representation will also contain those degrees of freedom. If you are only interested in the vibrational modes of the system, you will need to remove the irreps corresponding to those transformations.

but I am not sure I understand why changing the x direction motion of one particle with the y direction of another one, is still a symmetry of the system (while keeping everything else the same).

It isn't. The representation of the elements of ##S_3## on this 6D space are very particular transformations. You cannot just choose any transformation of the 6D space that you would like. If that was the case there would be no point in looking for irreps ...

It is the system itself that has an S3 symmetry. You can exchange any two of the bodies and end up with the same system assuming that you transform the displacements of the bodies accordingly. For example you can exchange bodies 1 and 2 by a reflection in the vertical line through the position of body 3. This reflection inverts the x-component of all displacements and exchanges the displacements of bodies 1 and 2, which tells you how this symmetry acts on your 6D representation. In effect, you have three displacement vectors that each transform according to an ##O(2)## representation of ##S_3## (let us call it ##\rho_2##) and that are exchanged according to the natural three-dimensional representation of ##S_3## (let us call it ##\rho##), meaning that your 6D representation is given by ##\rho \otimes \rho_2##.

Now, you know that the 6D representation contains overall translations and rotations (as those can be easily constructed by having the displacements of all bodies be the same and by having the displacements being those given by a rotation, respectively). Therefore, your 6D representation will also contain those degrees of freedom. If you are only interested in the vibrational modes of the system, you will need to remove the irreps corresponding to those transformations.

It isn't. The representation of the elements of ##S_3## on this 6D space are very particular transformations. You cannot just choose any transformation of the 6D space that you would like. If that was the case there would be no point in looking for irreps ...

Thank you for your reply. I am still a bit confused about a 2D representation of S3. I think of S3 as some operators that permutes 3 elements, so acting on a 2 dim vector space, would permute just 2 elements. How does this actually works?

Thank you for your reply. I am still a bit confused about a 2D representation of S3. I think of S3 as some operators that permutes 3 elements, so acting on a 2 dim vector space, would permute just 2 elements. How does this actually works?

A representation of a group on a vector space is a map from the group itself to linear operators on the vector space. The S3 permutations of the three bodies can be easily implemented as reflections and rotations in 2D. For example, a reflection in the vertical axis would be representing the permutation of bodies 1 and 2 and the corresponding ##O(2)## matrix would be
$$
\begin{pmatrix}
-1 & 0 \\ 0 & 1
\end{pmatrix}.
$$
I suggest you try to figure out which ##O(2)## matrix represents the permutation ##(123)## on your own for practice. Once you have those you can generate the entire representation as ##(12)## and ##(123)## together generate ##S_3##.