Trying to get a true 50/50 balance in judging (Closed for comments)
Comments about this discussion:
Started
Hi everybody,
I'm going to throw this out there again, and see if it gains any traction, or if people even think this is a real problem. Rather than try to rewrite the rules, let's start with what we think needs to be different.
Problem:
Freestyle results appear to favor technical riding over performance/showmanship. Every Unicon, some really excellent performances get passed over in favor of really hard tricks, or a lot of tricks, or a wide range of tricks, in a performance that isn't so great.
Intent:
If we still want a 50/50 weighting of Technical vs. Presentation, we're having a problem. Tricks/skills, transitions, etc. are much easier to count and quantify than "performance elements", which is part of why they tend to dominate. Also, many excellent Freestylers and judges are not necessarily familiar with the performing arts, in terms of what makes a "good" performance.
How to address this?:
Definitely not by changing the 50/50 relationship. Somehow we need to give more recognition to that other half of the routine. In general, the audience seems to know which are the better performances. While a segment of that audience are very technical-focused, most are not watching with such a critical eye to that, and have an easier time seeing the performance as a whole. While we have Technical and Presentation judges, most Freestyle judges are very familiar with the technical aspects of unicycling, but not as many have that performance or theater background. It is a weakness based on the lack of a unicycles-as-entertainment world. We have to watch other forms of sports/entertainment to get examples. Or be performers ourselves. There just isn't as much of that.
If anyone else sees this as an issue, do you have any ideas of how to better-balance the judging of Freestyle? The other suggestion I can make is that the judging as a whole is way too complicated. This complexity takes away from the judges' ability to watch the performance, especially when they are only trying to see certain aspects of it. Each judge should evaluate the entire performance, and none should try to separate out any part.
Comment
As for myself, as a Performance judge at NAUCC, I had a picture in my mind of routines I had seen at previous competitions that stood out from a performance perspective. All routines that day were then judged against those outstanding performance examples in my mind. I tried to balance that with "did I enjoy it, would I like to see it again, did it tell a story". BTW: I would not be good technical judge, besides not being technically competent I find myself too busy watching the performance. Maybe it's my history in theater and such but since "performance" is very subjective this was my process.
Someone I talked to summed up the performance aspect as "without a strong performance, it's just standard skills to music".
Comment
Hi John,
as you mentioned above freestyle judging appears to you to favour technical riding over the performance. Thats hard to discuss, since it might just be a feeling.
To prove it we need to look at the results in particular, which we can't since such results are not public. I looked at the results of our last competition and could not find a prove for your statement.
However, I think judging presentation is much easier than judging the technic and the judging sheets shows what you have to judge: - Mistakes,- Choreography + Style, Showmanship Originality and Interpretation and all splitt in the details a judge has to look for.
Very important is that the judges are fully trained and that's were we should start doing things different.
A possible way could be to train the presentation judges in a seperate workshop and the technical judges in an other workshop. It would help, if only the judges, who will be judging at the UNICON are trained and instead of having a crowded room full with people, who won't judge during the event. The workshops should use and practise the judging with video examples. Not just reading the rules, judging needs to be practised.
Comment
"without a strong performance, it's just standard skills to music" -- I like that. Or perhaps it's just X-Style, but I missed the comp and still don't know that X-Style is. Those events are all about the tricks, but Freestyle is the concept of incorporating the tricks into a performance.
In a perfect world, each judge would judge the entire performance because, as I have argued in the past, it's impossible to split it in half without gaps or duplications. Usually the counter-argument for that is that many of our potential judges don't feel comfortable on the technical side. But this is really no different from good technical judges that don't have much background on the theater side. They might not even be aware of it. So are we better off splitting the judging? I think not but that may be separate from the topic of the 50/50.
I agree with Kirsten about this being hard to discuss (in specifics) because some of it is feelings, or personal tastes or even emotions, perhaps. All of those must come into play when judging art. The art contains technical elements which, to a certain extent, can be quantified, added and compared. It's harder to do this with the Presentation elements. But even on the Technical side, we must remember, tricks are not performed in a vacuum. Beyond the mastery component of executing each one, there are other aspects, such as attitude, style, fitting to the rest of the performance, how they are joined to what comes before and after, how the rider recovers from mistakes, etc. In other words, the Technical cannot be entirely separated from the Presentation either.
I also agree with Kirsten in that training is where we must concentrate our efforts. It doesn't matter what the Rulebook says, if judges all approach their jobs with different interpretations of those writings. We need examples. Beyond that, we need to agree upon the examples we use, as to which are better than each other, and why.
I have written of this in the past also, but never had the time to devote to putting something together. We need to gather videos of Freestyle performances, at all levels, and start discussing them in terms of why they are good, bad or whatever, which are our own favorites, which may have been judged inaccurately, etc. We will probably learn the most from the performances that we can't agree on. From those we will find areas of the "philosophy" of judging that may need re-thinking, or better definitions.
Comment
Hey John,
thanks for putting this up, because this is a huge problem. Most of the time I see routines, I end up with "nice tricks, but where was the routine?", so it's more like a X-Style with costume and music.
There are two problems:
First one is the rulebook, because just from the rulebook you don't know what is good art and how to differentiate it. Second is education of judges.
Oh and yes, these two are connected together, because the education uses the rulebook to educate judges and hence the rulebook even contains falsy definitions and more. That's a devicious circle. To make this problem visible, let routines be judged by unicycle judges and externals, like professional dancers just by their quality of artistics. I think this will show the discrepency.
The other point here, is that you judge poor and a brilliant artistics, at the moment the judging system doesn't give you the opportunity to validly insert the correct values. Instead a porr and brilliant routine are judged with a narrow margin instead of the huge margin, which it is in reallity (e.g. compare one of the wonderful japenese routines and a poor western-style like routine from a beginner, to highlight two extremes - from the judging system, there is not much difference between them, which is very, very sad, because riders don't get their respectful quantification of their capabilities)
To adress these:
We must throw the current judging system over board and start from scratch, which also requires a lot of research and second, to install a proper education system - My wish would be that artistic judges have a dance education.
P.S. Unfortunately you couldn't be at the workshop I gave at unicon about a future freestyle judging system. Here are the slides: https://speakerdeck.com/gossi/unidancing-judging-system-prototype-2014 which also adress the problems (p. 11-14) and have a prototype of a new judging system included. But that is work that can start after this rulebook committee is finished and probably can take up to two years to work it out, but is a start.
Comment
If anyone is into the idea of throwing out the current judging system and starting from scratch, I want to work with them!
While the *idea* of combining sport and art in Freestyle is a simple one, putting it into practice is not so simple. Maybe our initial focus should be on the training side, for both judges and future performers. Getting a collection of videos together, and starting a discussion about them, would be a good starting point.
Thomas, I'm going to take a look at your workshop notes--I'm very curious as to where you are with that! But if it's a long-term project, maybe we can get something going on the training side in the short term.
Comment
To create a new judging System needs really more time than we have for this rulebook revision.
I prefer the separation of the technical part from the presentation part. I don't think we will ever have the sufficient amount of judges who fulfill the requirements "perfectly" in both parts (for me personally, it would not be possilble).
I also agree in that training is where we must concentrate our efforts. It doesn't matter what the rulebook says, if judges all approach their jobs with different interpretations of those writings. We need examples. Beyond that, we need to agree upon the examples we use, as to which are better than each other, and why.
Comment
Ingrid, you are absolutely right, the timeframe of this comittee isn't even big enough to get the basic research done. It is planned to have a committee after the rulebook to work on new rules, which could take 2 years+.
Concentrating on practice is a good idea to better educated riders, judges and trainers with good examples. However, if an ugly routine will become good scores, riders will continue to ride ugly routines (which by the way frustrates the good riders that put a lot of effort on creating these beautiful routines of high quality and I don't feel very comfortable with that on all sides). I just wanted to say the rules are one constraint for training.
Comment
I fully agree that there are issues with the current judging system and that it might be most beneficial to just start from scratch. However, I also agree that we are looking at a time limit for this rulebook committee that is too short to really develop something entirely new.
I propose that for now we stick with the 50/50 Technical/Presentation split as is. Then we create two new discussions: one to develop Technical and one to develop Presentation. While longterm we can think of using Gossi's system of looking at data and constantly improving the system, I think we also need a short term fix. There were so many issues with the judging results at Unicon and we cannot leave the system as is for the next Unicon while we work on a completely new system. We need something at least slightly better in the meantime.
Comment
Patricia wrote: "We need something at least slightly better in the meantime."
Perhaps we should focus on the training side first. This would aid any future discussion of rule changes as well; having more of a consensus on what's "good" in a Freestyle performance and what isn't. For example, Gossi mentions "ugly" routines getting good scores. "Ugly" is open to interpretation of course, but one thing that comes to my mind is Ryan Woessner. While I would never label his performances as ugly, if you compare his riding style to the common Japanese model, his *riding* was pretty ugly. He always seemed a little shaky, and like he might be about to fall. He just didn't, which raised up his scores.
And I liked Ryan's performances. Were they great theater? No. they were an overwhelming amount of hard tricks, presented with a constant smile and a fun attitude. Also Ryan's (Unicon) performances are not necessarily great examples because their technical levels were so high, even by today's system they would likely have won fairly. He just didn't ride "pretty".
What I mean by all of that, is that a discussion of actual performances, based on having a shared collection of videos from various competitions, would help us to find a level playing field of what we know we like in a Freestyle performance, what we don't like, what we want more of, and what we want less of. Knowing this information will help all future discussions of judging, and also rides for their future performances. This is something that can be done starting right now, and doesn't require rule changes to happen.
Comment
Sounds like a good idea John and will also be useful for further development. Things to do:
- Getting examples for good and bad routines for either tricks and artistics (sorry, I stick with my terms - as I have defined them - hehe)
- Fix issues with the current judging system (patricia, do you have a list?)
- I will really want to run a committee post this one, that develops a new system (based on my prototype) - because this cannot be done in this committee.
Comment
John, I really like the idea of watching routines and coming up with ideas about what we like to see versus what we don't like. Simply put, if we could judge these routines simply based on our gut instinct, who would win and why?
Gossi, I am in full support of a committee continuing after this one to develop a system. However, in the meantime I think we still need to have a better system than we have now by the end of this committee. We need to have something to go into the next rulebook.
I am going to close this discussion and create a discussion for technical and one for presentation. There we can develop each half of the system independently.