Judging it!
This discussion has an associated proposal. View Proposal Details here.Comments about this discussion:
Started
Sorry about the time I took to put this down, but here it is! Let's talk about a totally different way to judge this discipline.
Why?
There's in my opinion two basic thing/concept in the current rules that doesn't "force" the rider to show us a work of art:
1) The separation of the jury into technical and presentation/performance/art.
2) The amount of things we ask a judge to look at.
1, why?
What we want (I think) is a "50/50 feeling" in the judging between technic and art. We can't have that if we split the jury. Because it's impossible not to be influenced by the thing you don't have to judge. Because technic and art comes together. For instance, a japanese doing a stand glide or going to stand glide (with the little, very hard, jump they have the secret of) or even just riding that beautifully, precisely, with that much speed, conscience of body is SO harder than just do the trick. Expressing something with a trick or while doing it is also MUCH harder than just doing the trick. "Changing" the way to doing a trick for the "routine's need" is also usually harder. Etc. Those two things are really linked. That's why I think we shouldn't divide the judging job among different judges; everyone should look at everything.
An additional good thing about having 10 judges giving the same "type" of results is that you can erase the "most generous" and the "most miserly" score (which doesn't really make sense with only 5 judges). This is something done in a lot of judging system because it prevents "corruption" of the results.
2, why?
It's pretty obvious that the judging can't be good if the judge isn't able to look at the whole performance. While taking notes, it's pretty hard to still be focus on what's happening. I think that judges should be able to do their "judging job" after each performance, not during it. Of course, they should look at the rider with a judging eye, but should still "enjoy" the show, look at him/her/them "like the audience", which is: doing nothing except looking. In order to allow them this freedom of mind during the performance they should have less (thus more general) criteria to fill. There is a need to take note during the routine if and only if there is a ask for a precise "report" of what happened in it. If not, the judge can be much wiser, fairer, because of this liberty of mind. Moreover, I think the more we set criteria or/and complex rules about it, the more we condition both the riders and judges minds to show us how well they understood those rules/criteria. But that's not what we want to see; at least I don't want to see cloned routines, routines that are made to fit boxes. I'd say that we need to stay vague (and start trusting the humans behind riders and judges), if we want to see art.
I will present you 3 alternative way to judge it differently. I don't love them all, but they should all "force" riders to think a lot more about what they do in their routines, about how they behave on stage and about "what the rules expect". And the more one think (or even become aware) of what he's doing, the more he is in an artistic process and, hence, will eventually show us a work of art.
Comment
First alternative: (what's said above doesn't fully apply to this one...)
Each judge give five positive score and one negative score, each score being on the same scale (0-10, for instance):
+ Unicycle skills
+ Presence
+ Space use, composition/choreography, overall coherence, costume
+ Body and face expressions/awareness
+ Music choice and use
– General ugliness and/or boringness, falls
Second alternative:
Each judge give two positive score on the same scale (0-10, for instance):
+ Skills
+ Artistry
Third alternative:
Each judge give three positive score: (always on same scale)
+ Skills
+ Artistry
+ General impression
Quick history of those systems:
1) Inspired by the skating rules posted on this forum. All judges give five scores. When I asked circus artists "name five criteria for a good performance", they were pretty unanimous on what's written above, everyone insisting on the second point: presence (for its fundamentalism).
2) It is the answer (of people I've asked) to the question: "describe the ideal system".
3) It's the judging sheet I managed to find for a circus festival (competition).
I think it's wiser to let you react to those different systems before giving my own opinion on them, so after you...
PS: Try to put yourself in the mind of a competitor who has to prepare a routine which will be judged by those criteria.
Comment
YAAY! And now there are at least two of us. I'm not sure if anyone else is also of the opinion that every judge must judge the complete performance. The excuses of why we do not do this (anymore) do not supersede the fact that this is true. Dividing the judging makes all results less accurate; less complete.
The reason it's nearly impossible to judge the entire performance currently is that the judging system is too complex. The scoring system requires a lot of work to see which example score fits what the rider did. I have tried to judge both sides at once (NZ); it ended with me being removed from the table because I was unable to keep up. I will not judge half a performance. The only way to get me back to the judging table is to fix this problem.
To make the above ideas work will require a much more detailed proposal, but this is where we should be going. For it to work, whichever labels we use for the different scoring sections must be very well defined. All judges must be on the same page as to what is meant by each label. Examples, pictures and even videos will be helpful in the process of explaining what is meant in each case.
In years past, before we split the judging, judges gave two scores, for Difficulty and for Presentation. The criteria for each of these scores was divided up into multiple areas of what was meant by each score. In the above suggestions, I would prefer no more than three different scores to be entered by the judges. Then our goal would be to explain what each of those labels is all about. The name alone will mean very different things to different people, so we will need good descriptions and support materials.
Thanks Thomas, I'm glad you also see it as I do!
Comment
When attending a workshop for a judge in UNICON17, I felt that a judging system of presentation was very complicated.
Many judges can't have a clear confidence in themselves by a complicated judging system.
It's important for all judges to learn a basis of a dance and a performance. But that's an ideal to the end and is difficult realistically.
While it was a workshop for about 1 hour of Montreal, we just sat down on a chair.
After that we had to judge while being alarmed in the respective original interpretation about a judging system.
Though a show was the one which bet long time and trained, some judges underwent almost no discipline. That was a very regrettable affair.
But it's impossible to gather judges of all over the world before holding of UNICON to train. So it's necessary to change the judging system to something more concise and plain.
A performance of a unicycle includes art, but I think it's important to be made for audience.
I want you to put the angle of the audience in a judging system for the person who isn't also a performer by a dancer to do a judge easily.
I think and everyone is able to do a judge fairly with a confidence by inserting a factor as the angle of the audience, not an original sense of values of the judge who doesn't get education of art.
For example my idea of a judging system of presentation is the following.
1. Do you think a challenge and success to the unicycle skill by which the whole performance is difficult or is correct and looks pleasant raised the value of the performance, and bigger excitement was given to audience, and made them hold a sense of respect? (20%)
2. Could audience feel one of them of gracefulness, prettiness, forcefulness, brightness, the humor and the charisma or some by the choreography in the performance and acting? Do you think and charm of a performer was handed down to audience sufficiently? (10%)
3. Do you think the performance (as whole music, costume, tools and work including a theme and a story) evoked a deep impression inside the audience clearly by one of beauty, the pleasure, a thrill, surprise and a laugh or the some, and could attract them? (20%)
The remaining 50% is technical.
"Audience" here are defined as "other IUF members". I thought it was better to set the person's angle with the knowledge about a unicycle and the affection to a unicycle as a factor.
"Audience" aren't a real audience, it's a supposed factor to the end. Real audience should enjoy a show. Only judges judge. It's possible to exclude that a performance of one country or club gets a high score by this.
And I think of this system as judge possible fairly about both of ballet-like and zombie-like.
It's necessary to consider demerit mark system separately about dismounting, but please remark on my idea.
Comment
Thank you, Suzuki san, for two things. Japan needs a voice in these discussions, and often does not get represented. Also, for being in a similar mind to Thomas and myself:
- Judging system is too complicated
- Our judges come from very different backgrounds
- Audience is an important factor in judging since it's a form of entertainment
- Being aware that our typical Unicon audience is not a "normal" audience. But in the future we may have opportunities to attract more general public if there is enough room, and a reasonable schedule
Comment
Yes, the range between zombie or ballerina makes it hard to have neutral and fair comparisons (as personal taste may kick in). What I found important in the artistic part is the energy exchange between unicyclist and audience, or better; the ability of the performer to shape that. The ability to deal with failures or other improvisation can also show someone is a better artist then the other. But I found it difficult to rank such qualities in the current system.
#1 - I do like the negative approach, as it brings a -to me welcome- balance, and allows to weight negative aspects that otherwise can't be addressed, so bringing more accurate divides.
#3 - I think that's the most practical, in combination with a cheat-sheet conform option #1
What also would be possible with scenario 3 is to still have judges that do either tech or art plus a 50% general, and thus still allowing the possibility for artistic judges from outside the unicycle-sphere.
Initially I found it a hard question if I considered a good thing, seeing the art/tech seperation among judges being removed.
But then I simply reversed that dilemma: would I consider it bad if it was.
Then my conclusion is, that IMHO it's not an bad idea.
I don't expect it's going to bring significant different rankings, but do expect score becomes more accurate, which is something we should strive for.
Comment
I agree with most of what's being said in this discussion. However, I am also a realist and have a difficult time in seeing how this system could be implemented, mainly when it comes to the Experts. I think most capable, trained judges could handle judging both halves of a routine when it comes to the younger, lower skilled riders. My concern is when we start getting to the Jr. Experts, 15+, and Expert riders. I just don't see how this could be feasibly done in a timely matter. We have to find a balance between giving adequate time to the judges without extending the freestyle competition to be much longer than it already is. I also think it is important that the judges write down the tricks that the riders do. In a close competition this can be really helpful in determining the results. I do agree with what Thomas said about how a trick is harder to do if it's done in an elegant way but the trick itself is also worth points regardless.
Comment
Patricia wrote: "My concern is when we start getting to the Jr. Experts, 15+, and Expert riders. I just don't see how this could be feasibly done in a timely matter."
Under the current system, you would be absolutely right. I've tried it. That would have to be part of the solution; to "un-screw" the complexity of what is now required from the judges. To me, it adds so many layers onto the performance that it's hard to remember what the performance was, or be true to it, in splitting out all those scores.
What's needed is a more wholistic approach. On the Presentation side, we mostly need to watch and "feel" the performance. This should provide the information we need to rank one competitor over another. It doesn't need four different scores to reach that, as long as the appropriate criteria are still applied to the score that is given.
On the Technical side, yes you have to pay more attention because there is more detail. You don't have to count every trick and certainly you don't have to write them down. You can, if that's what works for you, but do your notes take into account how well each trick was performed, or how long/far? Or how they blended from one to the next? Bah. All of that is important, not just trick names. A rider doing a sloppy trick for 3 revolutions is not equal to another rider doing it effortlessly for 10. What that rider loses in time, she gains in demonstrating mastery, control and confidence.
Judging Freestyle wasn't much easier back in the early days of Unicon. The main difference was that there were fewer competitors, but they were still doing a wide mixture of styles, variations and new tricks. The best performances represented the state of the art at that time. Each judge scored the entire performance, and it worked.
Oh, and it worked an awful lot faster.
Comment
John, what were the rules (if any) that the judges in the early days of Unicon used?
Comment
Ooh. Short question, but with a long answer. I didn't think I'd have anything electronic with the right info, but my documents went back farther than I thought! Some of my earliest rulebook documents are in an unknown format (no file extensions and Word couldn't open them). But then I ran through what I had to see when the IUF went to the "half judging" method for Freestyle. This was in 2006, for Unicon XIII in Switzerland. For the 12 Unicons before that, each judge watched and scored the complete performance.
I've posted links to two examples of "whole judging":
- The oldest "presentable" copy I found is the 1994 Rulebook, in .doc format, for Unicon VII (Minneapolis). It's much shorter than the later editions; they grew with each iteration.
https://www.dropbox.com/s/6mivn21nepfcpcr/Iufr0794.doc?dl=0 - The other example I'm posting is a much nicer looking PDF version of the 2004 IUF Rulebook, for Unicon 12 in Tokyo. The formatting is undamaged, and much cleaner! Notice how much more content there is 10 years after the previous set, even though many of today's events are still not present.
https://www.dropbox.com/s/ltd2e01wvmnhcaf/iuf_rules_2004_final.pdf?dl=0
The vast majority of the original IUF competition rules were derived directly from what the Unicycling Society of America was using. They have remained largely parallel ever since. I have copies of the rules that were used all the way back to Unicon I (which was called "International Unicycle Convention"), but all of that is on paper. Someday it would be a nice project to do...
Comment
From reading through these (correct me if I'm wrong, John) it seems like the judges used to give a score between 1 and 10 for Difficulty and Presentation. In determining this score they would use similar sub-categories to what we have today. Do you remember the reasoning behind the sub-categories becoming given their own scores that then added up to determine the whole category score?
Sorry if this seems like I'm trying to re-talk through what I'm sure was discussed many moons ago in the rulebook committees. I just want to understand how and why we went from a simplified judging system to a more complicated one. I have a feeling that I'm going to agree with some of the reasons for the change.
Comment
Judges each gave two scores of 1-10, one for Difficulty and one for Presentation.
We went from a simpler to a more complicated system because I think rules tend to evolve that way. Compare other parts of the old Rulebooks to what we have today and you may see more examples. Most of what we add is to make things clear, mostly based on past experiences, but sometimes it becomes too much; better to let the officials make logical judgement calls and the rest of us not have to wade through so much content.
Then there's the question of why we went to the partial-judging method that we're using now. I think this may have come from a perceived need for more judges, based on having lots of people say that they didn't feel qualified to judge the Technical/Difficulty side. But this led to a group of judges that is very Technical-focused and perhaps not paying enough attention to the big picture, and Presentation judges that have very mixed levels of experience with various forms of entertainment. I think we did better when we were a little less anal about being able to tell the difference between every trick, and a little more concerned with finding judges that could watch an entire performance.
Comment
I guess where my concern comes in is for the Expert competitors. I think that we are seeing a level of competition that would be very hard to judge on the whole without keeping detailed notes of the performance. And honestly, this can be an issue for some of our lower level competitors too, where they are all performing the same skills. We also can't really afford to add any more time to the judging because it takes long enough as is and the competitions barely fit into their allotted days.
So I'm not really opposed to the theoretical idea of judges judging the whole performance, I just don't see a realistic application.
Comment
I apologize for the lateness of this post.
As there is not much time left and after reading what's been said, here is what I think the rules should be:
(erase all 5c section of current rules)
There are 10 judges, each giving 3 scores from 0 to 10: technic, artistic and fault. Technic and artistic are counted positively. Fault are counted negatively, so a high "fault score" would eventually result in a bad ranking. Judges give their score right after the rider's routine. For each score, the chief judge should remove the highest and lowest one. The total score is then computed this way: (sum of all remaining technic score) + (sum of all remaining artistic score) - (sum of all remaining fault score) = total score (this number is allowed to be negative)
In everything that follows, "audience" is to understand as a fictive audience, not the actual one watching the competition.
Technic:
This score is counted positively and should reflect the rider's unicycling skills.
In order to give this score, judges should ask themselves the following questions:
- Do you think the rider took big risks in doing the tricks/transitions he made? Were the tricks/transitions difficult?
- Where the tricks executed easily or was the rider searching for his balance all the time? Did the rider showed mastery, control?
- Did the rider showed any original tricks, original variations of a tricks, original transitions, original use of the unicycle?
- Where the tricks executed: fast or slow? Huge or tiny? Long time or short time? Did the rider took risks in the realization of his tricks, making it big, last longer?
- Did the rider showed a very precise routine where every tricks is at the place it should be or were the tricks disorganize and kind of randomly placed in space and time?
- Was the trick choice coherent with the overall routine? Were the tricks "usefull"?
Artistic:
This score is counted positively and should reflect the rider's ability to entertain an audience.
In order to give this score, judges should ask themselves the following questions:
- Did the rider looked like being on stage, in front of an audience or was he just training in a stage situation? Was the rider aware of the audience, present on stage with his whole body or lost in the middle of the gym?
- Was the rider able to transmit a sense of beauty, gracefulness, humor, poetry, story, theme or some of the kind to the audience? Would an audience be attracted by the routine?
- Was the music choice coherent with the costume and the overall routine? Was the music well used or used at all?
- Was the rider aware of his body and face expressions and used it to express some beauty, humor, poetry or some of the kind?
- While doing tricks, was the rider in a totally different "body mood" than in the rest of the routine or did he kept acting, dancing, singing or whatever?
- If props were used: were they essential to the routine? What happens if you take them away?
Fault:
This score is counted negatively and should reflect two things: dismounts (max 5 points) and boringness (max 5 points). (boringness also includes redundant choice of tricks)
In order to give those score, judges should ask themselves the following questions:
- How much and how visibly did the rider fell?
- How did the rider handled his dismounts?
- Would an audience feel like seeing always the same trick?
- Would an audience be bored by the routine?
Please tell me what you think is wrong or incomplete with those rules. I already know that we should add some special questions for pair and group. The questions are of course not definitive and I'm very open to any suggestion or new questions (better english...). I'm turning this into a proposal soon. (only because I have to...)
Patricia:
The beauty of humanity is that we're all different, we all think differently. Thus, by asking judges to ask themselves questions and not telling them to fill labeled boxes with numbers, we use this diversity present in our mind to generate scores which really correspond to what the judge (as an individual) think and by doing the mean of all those different opinions, we get, I think, the most honest ranking possible. So, if you can't separate two riders, maybe someone else can (and it may even be obvious for him/her).
You may be right in some way: what do we do in case of tie? I think we should define an order (between technic, artistic, fault) and look who wins in the first one; if tie, the second one; if tie the third one. If every score is a tie, the chief judge decide (or we keep the tie, I don't know).
Comment
I like Thomas's opinion.
Because it's felt that there is a posture which tries to catch unicycle Freestyle by the category of [music/performance] in the root. That collateralizes an infinite possibility in a unicycle Freestyle performance.
But I also wish that unicycle Freestyle stays at the category of [sport].
So I'd like to add several judging points.
It was divided into [technical judge] and [presentation judge] up to now. I say that judges are divided into [total judge] and [basic skill judge] by my new idea.
[Total judge] uses the one done based on Thomas's judging system. Perfection is 100 about one judge for a mark.
[Basic skill judge] the quality judges about [5 components].
[5 components] are the concept used in figure skating.
[5 components]
(For example. Running trick, spin trick, standing trick, hopping trick and mount trick. Someone propose something good, please.)
It's possible to give a guideline of a needed skill to Freestyle riders by this. I think that's indispensable for development of Freestyle as a sport.
About each component, [Difficulty] 10 points. [Quantity] 10 points. So the total of one judge is perfect, 100.
The number of people ratio of [total judge] and [basic skill judge] is made 4 to 1.
(The score to the skill is included during [total judge], so the ratio of the number of people of [basic skill judge] is small.)
Therefore a total of the number of people of the judge is always a multiple of 5. (For example. A local competition is 5. A large competition is 10.)
The one which didn't cut up and down and totaled the score of all judges will be last score.
(I'd like to think of Freestyle by the category of [music/performance] basically, so I think a balance sense like the upper and lower cut isn't necessary.)
I think innovative reformation is necessary to a judging system of Freestyle.
An idea of Thomas and me is the 1st step.
I think argument about a technical judge becomes too complicated now. When pursuing more detailed and correct description, much person's understanding doesn't probably catch up. (I'm sorry. But I feel so.)
It's probably better for us to change an outline of a judging system to "something more sensuous tolerantly" first.
It's better to begin argument about how a judge does the raised angle of the foot, the skill speed and etc after a new outline is decided.
My idea is rough incompletely.
And if unpleasantness was given to you by my unskillful English long passage, I apologize.
Comment
Thomas concept is very interessting, I love the deductions for the boringness the most. I also have my prototype ready. I think that's something to invest in post this committee, to compare different prototypes and doing research on it. Also we also need to get the education going for all those future committee members (I'm already working on it).
Comment
I love Thomas' ideas. Best thing about it is the fresh approach. Also the idea that each judge is responsible to the entire performance. For the details, we can still use much of the text we already have, with modifications where necessary.
The area of Faults is probably where we need to do the most work (the other two are similar to Technical and Presentation). Another name for that part could also be Negatives. Some things might not be actual faults, but are things that subtract from the performance.
- How visibly did the rider fall? - That's a good one, which gets more to the core of the issue. Dismounts are not all equal!
- Would an audience feel like always seeing the same trick? - Of course not, but judges must be aware of variations of the same trick or logical reasons for some repeats if there is a theme involved. Or a comedy performance where trying to do a specific trick is central to the story.
- Would an audience be bored? - This is a good thought for a negative (or fault). But are there other negatives we may also want to ask? "Boringness", to use Gossi's word means different things to different people. This is okay, since Freestyle judging is subjective. In this same area I might ask if the performer used the space well, vs. riding in the back half of the space, etc.
Can this be turned into a polished proposal in time? Probably not. But I will help anyone that wants to work on it for the next round of rule changes (or this one).
Comment
I like a lot of this proposal but I feel like it's not quite ready to be implemented. There are still things that need to be developed and fixed in the wording. I think that we should continue developing this proposal after the committee is finished and get it more polished, perhaps even test it out. I am not comfortable with this big of a change being implemented until it is more polished.
In the meantime, since we know the current Presentation rules don't work, I would suggest trying out the skating rules. These seem to develop what we already have and most people seem to agree that they are better than what we are currently working with.
Comment
While this proposal is a little "bare-bones", it has simplicity. With a bit of tweaking we could use this proposal and it could work. We could then embellish it with support materials that explain all the definitions of what is being discussed, explain the differences, etc. I like this proposal a lot.
In her earlier comments Patricia indicated worry about being able to judge Jr. Expert and Experts with this method. Those groups are always the hardest ones to judge, so of course it will be harder. But because the system is (a lot) less complex for the judge, I think judges would come up with their scores *faster*, not slower than the current system. When considering judging rules I seldom think outside of the realm of Expert riders. If it works for them, it will work for all lesser performances.
Let's get back to a Freestyle competition where the spaces between the performances aren't longer than the performances themselves! :-)