judging the "artistic" program components

Most of the criteria for the Performance/Execution, Choreography, and Interpretation components draw on concepts central to the performing arts, especially music and dance.

Many people who like to watch figure skating primarily for its aesthetic qualities, including a number of knowledgeable fans here at FSU, also have backgrounds in the arts that give them insight into effective ways to evaluate performances.

I believe that the people who wrote the PCS criteria for these components also had outside knowledge beyond just their skating backgrounds. But many judges do not, so all they have learned is what the judge training provides them.

Can we draw on the expertise of posters here to brainstorm improvements we'd like to see in the judging of these components?

*How could the criteria be rewritten or expanded on to help judges focus on the important aspects of performance, choreography, and interpretation without too much overlap with each other and without tieing these components directly to the skating skills?

Even if only hypothetically, just for fun let's imagine that the ISU decided to create a separate "artistic judging appointment" only for these three components, which would require a separate training track from the technical judging track that allows one to judge GOEs and Skating Skills and Transitions (or a general judging track for lower-level events without split panels). (The logistics of how this could happen affordably worldwide is another question we don't need to worry about for now.)

*What kind of music and dance knowledge should a skating judge, or skater who wishes to become a judge, already possess before being accepted into the artistic judge training?

*What kind of skating knowledge should a nonskater with outside arts expertise already possess before being accepted into the artistic judge training?

*What might this training consist of, to get everyone on the same page as much as possible, with as high a level of shared knowledge and standards as possible?

My main wish would be that the PCS criteria, having indeed borrowed the terminology from the performing arts, would not be trying to reinvent the wheel. The judges should adopt the most accepted definitions of what musical interpretation, performance quality, quality of execusion and choreography mean; and rather than creating more and more complex descriptions, start training the judges in deeper understanding of what these artistic criteria represent. Separate panels would be splendid; they would allow judges to truly pay attention to what they are seeing on ice at any given moment. As was posted in an article in one of the similar discussions here a while ago, a human brain is not capable of multitasking to the extend that current judging criteria demands. In order to to be able to pay close attention to musical interpretation and performance, they should be the point of concentration. The added benefit would be a relative independence of the panels. There might be strong discrepancies between the technical and artistic marks, and I think it would add excitement to events, and also help the audience to appreciate skating as the blend of sport and art.
A question about the necessary qualifications is an interesting one. I think anybody who judges skating ( even if just IN/PE/CH) should be required to have some knowlegde of elements and moves in the field, the conventions of program construction; be able to recognise transitions in and out of elements, and have a good understanding of what blade can and cannot do. I know when I put videos for non- skating dancers, they are quite critical of how dance elements translate to ice ( "There is little Latin in this Latin"- I've heard that a lot).
The reverse is harder to come up with. Clearly one needs to know dance styles; ballet, modern dance, ballroom ( swing hustle)) The requirements for music are harder to define and to test... I'll sleep on it.

My main wish would be that the PCS criteria, having indeed borrowed the terminology from the performing arts, would not be trying to reinvent the wheel. The judges should adopt the most accepted definitions of what musical interpretation, performance quality, quality of execusion and choreography mean; and rather than creating more and more complex descriptions, start training the judges in deeper understanding of what these artistic criteria represent.

Click to expand...

Should that mean rewording the existing explanations of the criteria? Adding more written explanations? Or should the additional training happen in videos, seminars, and trial judging discussions?

Written criteria and videos could be made available publicly for skaters, coaches, journalists, and interested fans also to get an idea of what is being judged. But live discussions analyzing live examples would be both necessary and hard to share publicly. And not appropriate to share where the examples are currently competing, not necessarily elite skaters.

I have some ideas about basic music knowledge I would want a skating judge to have before entering an advanced artistic-judging training, and basic skating knowledge I would want an outside arts expert to have before entering skating-(artistic) judge training, just so that everybody could be using the same vocabulary. I'll give some examples below.

I'm less clear about what prior dance knowledge would be important. Maybe none -- the important points would come up during the training itself. I don't think it's necessary for skating judges, from a skating or music or visual arts or theatre background, to know ballet terminology or anything about the specific technical demands of off-ice dance. What they do need to know is principles of evaluating movement and how to apply that to movement on ice.

A question about the necessary qualifications is an interesting one. I think anybody who judges skating ( even if just IN/PE/CH) should be required to have some knowlegde of elements and moves in the field, the conventions of program construction; be able to recognise transitions in and out of elements, and have a good understanding of what blade can and cannot do.

Click to expand...

Agreed.
For entry-level knowledge, I would expect would-be judges to be able to identify edges, jump takeoffs, basic spin positions and forward vs. backspins (this would be an easy concept for dancers), easy vs. difficult and one-foot vs. change-foot turns, general categories of pair moves. They don't need to be able to judge the technique, but they need to be able to participate in discussions using the correct terms. They don't need to be able to distinguish a counter from a rocker or bracket, but they do need to recognize when they just saw one of those turns, that exits in a somewhat surprising direction, rather than a three turn.

I'm imagining a multiple choice test -- watch a short video clip of a skating move that is very clear in what it is, no gray areas or trick questions, and identify it by name.

This test would be a no-brainer for existing judges and anyone who had been a mid- or high-level skater. Low-level skaters and serious fans might also be able to pass it with no further study. People who are coming in from the arts but who had only ever watched skating on TV or the occasional ice show to enjoy the aesthetics would need to do some studying first; a video course should be made available.

Then there could also be a few questions to test knowledge of the short program and well-balanced program requirements (elements and program lengths) for whatever levels and disciplines they're training to judge.

For skating judges to demonstrate some knowledge of music basics, there could be audio clips and questions asking people to identify obvious musical concepts, no gray areas or trick questions. It would be a no-brainer for anyone who has studied music for a few years, and for many dancers, but might require some study for people who are coming into the process with a skating background but little formal music training.

Again, I don't think it's that important to memorize the technical terms from an outside art if there are colloquial ways to express the same concept, but the concepts can be important. So questions could be something like

This excerpt is an example of
a) increase in volume b) decrease in volume c) increase in tempo d) decrease in tempo e) key change

[each clip should only include one of the above]

This passage is characterized by its use of
a) dissonance b) legato c) minor key d) staccato e) syncopation

My main wish would be that the PCS criteria, having indeed borrowed the terminology from the performing arts, would not be trying to reinvent the wheel. The judges should adopt the most accepted definitions of what musical interpretation, performance quality, quality of execution and choreography mean; and rather than creating more and more complex descriptions, start training the judges in deeper understanding of what these artistic criteria represent.

Click to expand...

I agree!

Separate panels would be splendid; they would allow judges to truly pay attention to what they are seeing on ice at any given moment. As was posted in an article in one of the similar discussions here a while ago, a human brain is not capable of multitasking to the extend that current judging criteria demands. In order to to be able to pay close attention to musical interpretation and performance, they should be the point of concentration. The added benefit would be a relative independence of the panels.

Click to expand...

Totally agree!

I think anybody who judges skating ( even if just IN/PE/CH) should be required to have some knowlegde of elements and moves in the field, the conventions of program construction; be able to recognise transitions in and out of elements, and have a good understanding of what blade can and cannot do. I know when I put videos for non-skating dancers, they are quite critical of how dance elements translate to ice ( "There is little Latin in this Latin"- I've heard that a lot).
The reverse is harder to come up with. Clearly one needs to know dance styles; ballet, modern dance, ballroom ( swing hustle)) The requirements for music are harder to define and to test... I'll sleep on it.

Click to expand...

Yes, I would prefer to have the "artistic panel" be skating judges well educated in performance arts (not limited to dance) rather than performance-oriented judges educated in skating. The center and foundation of all this must still be the principles of what constitute "good and difficult skating." It is more difficult to skate to the music than to skate without regards to music. It is more difficult and pleasing to extend the free leg and arms far out while skating. It is more (mentally instead of physically?) difficult to skate an original and creative program than to copy 90% of commonly used music/themes/choreography.

There might be strong discrepancies between the technical and artistic marks, and I think it would add excitement to events, and also help the audience to appreciate skating as the blend of sport and art.

Click to expand...

Herein lies the fun and difficulty. Just imagine a skater with slightly less technical advantage win over others because of an edge in the artistic mark. Imagine the outcry! The horror! The petition to change the scores and results!

One issue to consider is cultural difference and preconceived notion (self-regarded cultural superiority) about what is "better" and "more pleasing," even though aesthetics is 99% universal. I would guess that everyone around the world would agree what is good ballet or ballroom dance, but judges from Russia, France, Canada/US, and East Asia would likely disagree what is good or superior in a program using Japanese/Chinese/Filipino ethnic music and dance vocabulary. I am all for diversity and adventures, but such sentiment is hardly unanimous in the figure skating community. There are plenty of traditionalists. This is one reason I was very sorry to see folk dance abolished from ice dancing.

If you want this specific background, it seems like figure skating might need to make judges paid officials...

Already, I think skating will have problems finding judges in years to come. Becoming a judge is a very expensive endeavour.

Personally I think continuing to refer to program components as 'the artistic mark' is one of the reason they are so misunderstood. Although they do encompass artistic elements, it is not the same mark it used to be.

I do think judges need to be trained to know their PCS scores do not all need to be within a point of each other. Someone might have fabulous interpretation but shit skating skills. It shouldn't be everything or nothing like it is.

I would like to see Interpertation and Choreography factored higher than the other components and to have them judged more carefully. For instance, if SS/ TR/ PE were factored at 2.0 I'd like to see CH & IN factored at 3.0 (might have to be like 1.7 and 2.5 to balance out scores or something). Knowing this, both judges and skaters would pay more attention to the artistic side, making programs theoretically more pleasing. I know that skating skills are important, but I feel that a good portion of the top 20 in the world are missing some artistry that I'd like to see.

Separate panels would be splendid; they would allow judges to truly pay attention to what they are seeing on ice at any given moment.

Click to expand...

I would also love to see the judges split up so one set is judging the elements and the other is judging the PCS. I find it hard to believe they can do an accurate job with all they have to do and the short amount of time they have to do it.

I think only then will you see a bigger difference in someone who is very pretty but makes mistakes. I've seen many beautifully choreographed programs where they skater really had an off night where the score really should have been 8.00 for CH and 4.5 for PE.

I also think a replay of all the elements should be mandatory before scores are imputed.

If you want this specific background, it seems like figure skating might need to make judges paid officials...

Click to expand...

Maybe a topic for a different thread, but I see some difficulties with going this route:

At what levels of competition would judges be paid? How would the judging needs at lower level events that can't afford professional judges be met? What would be the path for lower-level volunteer judges to enter the professional judging corps? Would some events be paying some of their judges and not others?

Already, I think skating will have problems finding judges in years to come. Becoming a judge is a very expensive endeavour.

Click to expand...

Yes, that is true. Video training can reduce the need to travel, but some travel will still be necessary. Which puts potential judges at a disadvantage who live in areas with fewer local events and that don't or rarely bid to host national or international events.

Would professionalizing judging make it even harder for a judge from a country with a small skating population to join the international judging ranks?

Splitting the judging panels, of course, would require greater total numbers of judges, which is one argument against it.

Could the ISU and member federations do more to recruit judges not only from the ranks of former skaters (not necessarily elite) and adult skaters, but also encourage interested outsiders (e.g., fans) to learn everything they need to know to judge skating effectively -- including technical fine points?

Personally I think continuing to refer to program components as 'the artistic mark' is one of the reason they are so misunderstood. Although they do encompass artistic elements, it is not the same mark it used to be.

Click to expand...

I'm not sure it ever was what "it used to be" in the sense of really being an "artistic" mark.

There was a time when the second mark was named "Artistic Impression." But that mark goes back to a time before theme programs were common, when medleys of many unrelated pieces of music were common, when simple good posture and simple well-aligned body line but not extreme extension were valued. It really meant more "Does the skater have sufficient control of the skating technique so that the body lines are pleasant to watch, the movement is smooth and not jerky, the steps and arm movements match rhythm and style of the music?" It didn't originally mean "Is this performance artistically coherent?"

And by the time that such coherence did become more common, in the 1990s, the name of the second mark in free programs was changed to Presentation, to match the term used in short programs -- specifically, I think, to counteract any sense that "artistry," whatever that is, is a prime element that is being judged.

At what levels of competition would judges be paid? How would the judging needs at lower level events that can't afford professional judges be met? What would be the path for lower-level volunteer judges to enter the professional judging corps? Would some events be paying some of their judges and not others?

Click to expand...

I really don't know- but a friend used to referree kindergarten basketball (where more often the ref would stop and instruct, rather than call a foul) and she got paid... My BIL and FIL are working their way up through football referreeing, first elementary/middle school, then you get to do exciting high school games, and on the very rare occasion are called to college, and they get paid. If you prove to be a good ref, you get into the college games more. So in a way, the low level games are paid trial judging.

I don't know where the money would come from, or how low it goes. But if you expect the credentials being addressed, it seems a lot to ask of a volunteer, who already has to put signifigant time and money into the already existing trial judge system (at least in the US) where they are expected to show they understand appropriate application of the scoring.

But live discussions analyzing live examples would be both necessary and hard to share publicly. And not appropriate to share where the examples are currently competing, not necessarily elite skaters.

Click to expand...

OT, but is there not enough footage of retired skaters to cover the bases? Unless one could guarantee that the judges in question would never be called upon to judge the skaters being used as examples, it doesn't seem appropriate to use current competitors (whether elite or not) as examples.

In trial judging situations, the trainees are judging actual competitions that are taking place right there. And get to experience the skating live and up close (although often not from the same angle as the official panel). I don't think it's possible to train effectively as a judge while never training with real live skating. It's those discussions that are not appropriate to make public.

It's also not possible to train judges to evaluate intermediate-level skaters only by watching videos of past champions. They need to be familiar with what to expect at intermediate levels.

Yes, I would prefer to have the "artistic panel" be skating judges well educated in performance arts (not limited to dance) rather than performance-oriented judges educated in skating. The center and foundation of all this must still be the principles of what constitute "good and difficult skating." It is more difficult to skate to the music than to skate without regards to music. It is more difficult and pleasing to extend the free leg and arms far out while skating. It is more (mentally instead of physically?) difficult to skate an original and creative program than to copy 90% of commonly used music/themes/choreography.

One issue to consider is cultural difference and preconceived notion (self-regarded cultural superiority) about what is "better" and "more pleasing," even though aesthetics is 99% universal. I would guess that everyone around the world would agree what is good ballet or ballroom dance, but judges from Russia, France, Canada/US, and East Asia would likely disagree what is good or superior in a program using Japanese/Chinese/Filipino ethnic music and dance vocabulary. I am all for diversity and adventures, but such sentiment is hardly unanimous in the figure skating community. There are plenty of traditionalists. This is one reason I was very sorry to see folk dance abolished from ice dancing.

Click to expand...

The cultural difference is a very interesting point, and undeniable. There are definite cultural preferences even in the most developed skating countries; and the more countries participate skating, the more issues might arise in judging. I do not have an answer, just acknowledging you are right. Though I think it is less of a problem for Japan which is dominant in skating and has culture that has reached world- wide. The issue remains.
As to: skating judges who are well versed in arts vs artists who understand skating; I keep going back and forth. The sad truth is, if one has a deep knowledge of one art, one probably does not have the same deep knowledge of another ( am thinking dancing and music here). Both disciplines have a point of view. But if one has neither, then they would have mostly superficial knowledge in both ( exceptions apply). And I still am not sure how to teach musical understanding to judges who just don't have the ear for it. It seems to me it cannot be done... anything involving rhythm requires visceral absorption. The very basics can be explained, but not the complexities of musical language.

Personally I think continuing to refer to program components as 'the artistic mark' is one of the reason they are so misunderstood. Although they do encompass artistic elements, it is not the same mark it used to be.

I do think judges need to be trained to know their PCS scores do not all need to be within a point of each other. Someone might have fabulous interpretation but shit skating skills. It shouldn't be everything or nothing like it is.

Click to expand...

I know that PCS is not the "artistic mark". However 3 out of 5 PCS criteria have artistic demands. IN- interpretation of music; music is art, and so is its interpretation, so it's a double whammy. It's a fact of life that art is studied and absorbed over many years. One cannot get discussion points and descriptions and be qualified. Thus, either judging on IN is currectly fake in many cases, or we need to accept that Expression of the music&#8217;s rhythm, character, content and style
&#61623; Use of finesse1 to reflect the nuances of the music in the rules are just words which do not mean what they say.
To me, one of the pinnacles of IN is Kozuka's Take Five. http://youtu.be/NmT0AY29CX8
Regardless of any other mark or one's opinion of the skater, the IN here should be getting 9.5-10. It is pretty much a perfect example of timing the blade work to a complex musical piece.

And I still am not sure how to teach musical understanding to judges who just don't have the ear for it. It seems to me it cannot be done... anything involving rhythm requires visceral absorption. The very basics can be explained, but not the complexities of musical language.

Click to expand...

IF there were a separate appointment just for PE/CH/IN, then judges who aren't interested in studying music or who have tried and just don't get it would stick to the technical judging side.

But for those who have some basic or intermediate musical training or some love for listening and are willing to start analyzing as adults, as skating judges, what do they need to learn?

Ice dance judges already need to know how to count and need at least a general familiarity with characteristic movements of ballroom/social dance styles.

I think freestyle judges should also be able to count, recognize downbeats and measures as well as whole phrases, recognize typical rhythmic patterns and common variations, identify characteristic styles of dance music and the types of movement associated with them.

Not all freestyle skaters will use the details -- for that matter, not all ice dancers use them in their free dances -- but judges should be able to recognize when the skaters do and reward how well they succeed.

There should also be some familiarity with typical rhythmic patterns and variations in commonly used genres of classical music, jazz, rock, etc.

Outside of what's now the Short Dance, skaters aren't required to imitate off-ice dancing. But when they do, judges should be able to recognize and evaluate the attempt.

When the skaters use non-dance music or use dance music in abstract, skating-specific ways, then the judges have to evaluate the direct match between the music and the skating movement.

Sometimes you can just feel that the skater is doing a good job -- or is ignoring the music completely -- without any analytical tools. But it would help to be able to mentally identify what the music is doing and what the skater is doing rhythmically to evaluate how well they match. It's easier to identify something if you have a name for it, although for judges whose first language is neither English nor Italian it could be extra challenging to learn the Italian terms while being instructed in English.

The ISU videos, in the concepts they do address, use terms like "downbeat," "accents," "tempo," "intensity," etc. That's why I think it would be good if all judges entering this training would have to show that they are already familiar with this vocabulary.

Basically, I feel like the ISU videos for IN have focused on just a few concepts, which are useful, but only partial. There's a lot more that could be broken down that would be helpful for judges who want to do a good job evaluating the interpretation but don't already have the music and dance knowledge to analyze it. I know I would want more instruction.

And maybe more detailed written criteria that are less vague. E.g., not only Does each movement phrase begin and end with the musical phrases, but also Does the underlying rhythm of the movement within that phrase match the underlying rhythm of the music? How many beats per measure? Does the movement acknowledge the downbeats, backbeat, syncopation, etc.? Do quick steps to quick music or slow lyrical moves to slow lyrical music actually match the beats of the music, or is the connection only the general concept of "quickness" or "slow and lyrical"?

As to: skating judges who are well versed in arts vs artists who understand skating; I keep going back and forth. The sad truth is, if one has a deep knowledge of one art, one probably does not have the same deep knowledge of another ( am thinking dancing and music here). Both disciplines have a point of view. But if one has neither, then they would have mostly superficial knowledge in both ( exceptions apply).

Click to expand...

Just a quick note, because I don't quite agree. Or, rather, I agree that quite many people with a musical education/knowledge don't have that much knowledge about dance, but I also think that the opposite is not completely true in most cases, as dancing without music is very rare, and it's actually very important in dance. Dancers of some specific genre/people who have knowledge of that specific genre/form of dance might not have vast knowledge about music that is not used in their genre, but they at least have some understanding of rhythm, phrasing, and so on.

It would be nice that a certain genre is chosen for SP like OD. (* he said "original dance") Then the differences of expression between skaters will appear more clearly. And you can choose anything freely for FP. How about this rule? I propose it!

Just a quick note, because I don't quite agree. Or, rather, I agree that quite many people with a musical education/knowledge don't have that much knowledge about dance, but I also think that the opposite is not completely true in most cases, as dancing without music is very rare, and it's actually very important in dance. Dancers of some specific genre/people who have knowledge of that specific genre/form of dance might not have vast knowledge about music that is not used in their genre, but they at least have some understanding of rhythm, phrasing, and so on.

Click to expand...

Oh, for sure. I agree- to a degree. I've known some very fine dancers who knew music of their genre very well, but did not know or care to know any other. Usually a dancer would first pay attention to form, style, line, authenticity ( am again generalizing here) and a musician- to rhythm and emotion; I think it just comes from different emphasis in training.

And maybe more detailed written criteria that are less vague. E.g., not only Does each movement phrase begin and end with the musical phrases, but also Does the underlying rhythm of the movement within that phrase match the underlying rhythm of the music? How many beats per measure? Does the movement acknowledge the downbeats, backbeat, syncopation, etc.? Do quick steps to quick music or slow lyrical moves to slow lyrical music actually match the beats of the music, or is the connection only the general concept of "quickness" or "slow and lyrical"?

Click to expand...

I think this is a good start. Though of course that has to be done in training, not asking all these questions while the skater is performing. By then that evaluation has to be quick and instinctive.
The stuff I would recommend paying attention to:
- musical form ( being able to hear and identify the underlying musical structure. Nothing too complex, but skating standard is ABA'; however, there are many options). That shows whether the musical piece and the basic movement development follow the same arch.
- phrasing; starting with the broadest and breaking down into the smaller segments until the shortest phrase within the big phrase can be identified. It can be done both by ear and by following the score. Watching dance examples where choreographers follow different phrasing. Because there are many ways to approach it. Movement can follow the rhythm within a phrase or choose the broadest arch. The ability to identify both is absolutely crucial. As an example of following the rhythmic complexity is Kozuka's Take 5; Abbott's 8 Seasons. The example of following the broader arch to a great effect is the last 30 seconds of Asada's 2013 LP. Then there is the whole issue of timing- feeling the length of the phrase and elongating the movement to match the beginning and the end ( finishing too abruptly, too soon is a very common occurrence, or giving a sluggish or languid gesture on a strong accent ). But then again, the beat can be strong, but the build- up to the beat slow, so sometimes a slow gesture with a "pointed" ending is just as good as the abrupt one... ( Kostner SP 2012 is a perfect example of how one "grows" gestures into accents. Amazing stuff, that SP http://youtu.be/IutFxNBUujU- watch her approach starting from about 2:45 -3;03, opening up into a big accent, and then reinterpreting the same phrase by two different and equally precise ways. That is 10 in IN- it simply cannot be done better. Differently, sure, but not better) Argh... one cannot describe all options! It's not only studying, it's feeling and then being able justify the movement that need to go hand in hand in training.
The main difficulty with this theoretical training is that everybody starts from such a different point... which is why it would be so helpful for everybody who judges IN/CH/P/E to have certain basics.