Generic Contest Discussion

[Akassin]

10 hours ago, gyenesvi said:

Skip it if you don't want to read it. You have a choice. And let others discuss on a discussion forum ;)

I knew there would be a misunderstanding somewhere :) In my system, you'd only have to score the entries you like, 8-10 in this case and ignore the rest. The ones that don't excite you automatically get a score of 0, without mentioning. In case of the top 6 ranking, while you select the 6 out of the 8-10 ones you like, you are effectively scoring or comparing those ones in some way. That's why scoring the ones you like seems easier than ranking the final 6, because to arrive at the final 6, you probably already had to consider all the ones you like. And my system simplifies the opposite case as well: if you only have 3 contenders to start with, you only have to score those 3, and don't need to think hard to pick another 3 uninteresting models to get a total of 6. Does that sound simpler?

 

If I understood your idea correctly, then one participant can vote for three works (for example, give 10 points), and another participant can vote for 10 works (give, for example, 25 points). Then it will distort the results and violate the very foundations of any vote - all voters must have the same conditions.

In fact, this is the first time I see such a heated debate about the voting system, although I have participated in many competitions before in my other hobbies.
Let me express my opinion on this situation. It only makes sense to introduce a more detailed voting system (with points for each of the criteria, with penalties for non-compliance, etc.) if we completely abandon public voting and only jury voting remains (and we still need to make sure that even the jury would be willing to use such a complex system and spend so much time on it).
  It will never work in a public vote. You can believe me or not, agree with my opinion or not, but this is how our world works: the vast majority of people will simply choose the model that they like the most, just by personal feelings. Sometimes they can't even explain why they like this particular model more than another. Most people are not ready to spend a lot of time on this and will not make huge tables where they will count the points for each of the voting criteria. Come on guys, soon there will be a generation here (which is now growing up) for which everything that is more difficult than clicking "like" will be unacceptable.
The few people who are really willing to do a detailed analysis (especially for a competition like TC20 with almost 50 participants) will be in such a minority that it will not affect the overall results.
In addition, a very complex voting system will definitely scare away many who were potentially ready to vote (even now, with a fairly simple F1 system, the number of those who voted is comparable to the number of participants)
All I would like to change is to make voting more flexible. I liked the idea to leave the total points around 25-30 (it must be used in full), but allow it to be distributed among any number of participants (at the same time, you cannot give more than 5 points for one participant). This would allow choosing more than 6 works (for those cases when the number of quality works is very large and I would like to award more participants)

Edited September 2, 2023 by Akassin

[Toastie]

3 minutes ago, Akassin said:

Come on guys, soon there will be a generation here (which is now growing up) for which everything that is more difficult than clicking "like" will be unacceptable.

I boldly agree to your assessment, without >any< harsh feelings on this now "growing up generation" - they acquire other and more relevant skills in the world developing on their growing up.

Every poll is - as you said - naturally "skewed". You can come up with as many rules as you want; it will remain skewed. More or less. Homo sapiens (sapiens) sounds good, but after all, the time to develop educated skills for an "unbiased poll", was - and will be for a long time - a little short.

Best wishes,
Thorsten

[SaperPL]

@Akassin What could make sense (in context of people not taking into account actual criteria when voting) is the approach of game jams that only contestants vote, so there's a higher chance that the voting people are deep into the rules. This doesn't guarantee though they will actually vote according to the spirit of the competition, as well as we want to let people participate in voting even if they didn't have time to make models for the contest.

For me the ideal situation would be vetting stage where jury is checking all entries against criteria first and disqualifying those who didn't go according to the spirit of the competition (with some room for discussing this) and then public vote, but I know that it would take significantly more time and this it's not going to work here if there's limited time frame for handling the contest.

So I think it makes sense for two types of contests - one is where the criteria are a bit more specific and so the jury is voting, but again, it should be pretty clear how the criteria will be evaluated, and it would be good to get scores for all criteria instead of the order, because I think this is the part where it takes time to figure out what is better than what, and the second one which the topic is really simple and clear like it was with TC18 (make a car matching specific size) and it's about public vote. But since fleshing out the criteria description seems to be too much, the second approach with specific size/part limit with simple fairly open topic and public vote is a better idea. And I think that we could even re-do some of the contests with specific size simply because we have new parts and new people to participate in them.

[gyenesvi]

14 hours ago, Akassin said:

In fact, this is the first time I see such a heated debate about the voting system, although I have participated in many competitions before in my other hobbies.

I don't think about this as a heated debate, just discussing options and pros and cons for various voting systems, and the ways people think about them, their thought processes when actually doing the voting. So it is great that you also express your view.

14 hours ago, Akassin said:

If I understood your idea correctly, then one participant can vote for three works (for example, give 10 points), and another participant can vote for 10 works (give, for example, 25 points). Then it will distort the results and violate the very foundations of any vote - all voters must have the same conditions.

Yes, it could easily be the case that one voter votes for 3 entries and the other for 10. And I think it is not a problem, rather the contrary, it is natural. All voters have the same opportunity to vote, but we don't need to force them to vote for equal number of entries.

I believe this should be viewed from the point of entries, not from the point of voters. From the point of entries, the name of the game is who can attract more 'likes'. As if for every entry, we asked: who likes this one, hands up people! That sounds like a completely legitimate voting system for me. My system is just a generalization of simple likes (+1), in the sense that there are 'stronger likes' (+2 / +3) as well. For simplicity let's just consider simple likes (+1), and imagine that we force each voter to vote for exactly 5 entries. But it is very unlikely that a voter likes exactly 5 entries, so either of two cases will happen. If a voter likes less than 5 entries, he will have to give a vote for something that he does not really like. On the other hand, if a voter likes more than 5 entries, he cannot give a vote for some entries that he did like. Those cases do distort the opinions of the voters and hence the results, as they cannot freely express which ones they like! That is why I am against having a problem with max number of possible votes.

14 hours ago, Akassin said:

Let me express my opinion on this situation. It only makes sense to introduce a more detailed voting system (with points for each of the criteria, with penalties for non-compliance, etc.) if we completely abandon public voting and only jury voting remains (and we still need to make sure that even the jury would be willing to use such a complex system and spend so much time on it).
  It will never work in a public vote. You can believe me or not, agree with my opinion or not, but this is how our world works

I completely agree with you on this one, that is why I also expressed previously that I think contests with jury voting and public voting should be more different in terms of voting criteria. The simple system of max 3 points for each entry I am proposing now is for the public voting case. For the jury voting case, I'd be in favor of more elaborate criteria and in that case I think either a more elaborate scoring or a point distribution scheme could work better, because the jury's task is to consider all entries in more detail, not just the ones they happen to like by quickly flicking through them. I think this is the key difference between the two cases.

[howitzer]

Some contests have had a huge number of entries which can become overwhelming for voters, especially if you're a busy adult having a lot of adult-things to do in your life. In that case votes get cast pretty randomly if at all, and the weight of looks is exaggerated as it's much easier to decide if something is good-looking than if it has nice and interesting functions.

In the recreate a studful set contest we had a popular vote for the top ten and jury selecting the winner and runner-ups, but I'd like to propose an inversion of this: jury to select the top ten and then a popular vote for the winner. Of course it's a lot of work for the jury but so it is every time jury selection is used, and this would make it easier for voters to do an actual comparison of the quality of each entry, functions and all.

[SaperPL]

13 hours ago, howitzer said:

In the recreate a studful set contest we had a popular vote for the top ten and jury selecting the winner and runner-ups, but I'd like to propose an inversion of this: jury to select the top ten and then a popular vote for the winner. Of course it's a lot of work for the jury but so it is every time jury selection is used, and this would make it easier for voters to do an actual comparison of the quality of each entry, functions and all.

As shown in that exact contest, there may be a huge difference between what jury values (what is the interpretation of the rules) versus what community votes for mostly. If jury pre-selects limited amount of entries in different manner then what the community would vote for, you may end up with a situation where something that you liked the most, or you'd put in 2nd or 3rd place is not there in the pool because jury didn't pick that. Unless it's really clear for the contestants at the beginning of contest how things are supposed to be judged by the jury, it's a slippery slope either way (before or after popular vote) where jury has significant power to override what the community would choose.

The only clean situation is where jury goes precisely through a checklist of precise criteria to make initial scores i then it's voting for top 10 of those who scored highest, were penalised the least. But again - even just proposing for the jury/admins to first vet and disqualify those who went against the terms of the contest before the voting is seemingly going to take too long for the time frames that Jim has when it comes to organising the prizes and their shipping. So any multi-stage winners selection might be out of the question.

[howitzer]

22 hours ago, SaperPL said:

As shown in that exact contest, there may be a huge difference between what jury values (what is the interpretation of the rules) versus what community votes for mostly. If jury pre-selects limited amount of entries in different manner then what the community would vote for, you may end up with a situation where something that you liked the most, or you'd put in 2nd or 3rd place is not there in the pool because jury didn't pick that. Unless it's really clear for the contestants at the beginning of contest how things are supposed to be judged by the jury, it's a slippery slope either way (before or after popular vote) where jury has significant power to override what the community would choose.

The only clean situation is where jury goes precisely through a checklist of precise criteria to make initial scores i then it's voting for top 10 of those who scored highest, were penalised the least. But again - even just proposing for the jury/admins to first vet and disqualify those who went against the terms of the contest before the voting is seemingly going to take too long for the time frames that Jim has when it comes to organising the prizes and their shipping. So any multi-stage winners selection might be out of the question.

All systems have their upsides and their downsides and they have been discussed exhaustively in this very topic. Whatever the system, there will always be a possibility of different results if different system were used - and that's ok, because contests are supposed to be fun and not about winning no matter what.

Now the time required for multistage process is indeed a valid argument against the system I proposed, it wouldn't have been possible with the latest contest for example. On the other hand, in the studful recreation contest similar system was used successfully, so I guess it all depends on the timeframe of the contest and arrangements for the prizes and so on.

[Berthil]

A lot of discussion here and I shall try to make it even more complicated :)

When I look at myself voting I follow progress topcis and when looking at the entry topic, I already know what my favorites are. The personal reasoning behind these favorites is not carved in stone by rules, it is based on many factors and each contest different.

What if public voting would not be on entries but on contest rules (or category) and you only have to vote what categories you find important?
The jury will rate contest entries on agreed categories and cross reference them with public category voting

Simplified example:

1. For whatever contest jury has determined categories are complexity of functions and model creativity (real contest would have more categories).
2. Participants know what to design because of clear categories.
3. Jury rates final entries on categories and winner is the one with highest total score for the two mentioned categories. Point 1 to 3 is pretty much how TC23 went if I'm not mistaken but now it gets interesting with public voting on top of the jury voting.
4. Public votes what they find their most important categories and these can be different from the jury categories. In the this example I would give creativity 10 points and functions 5. Normally may be there are 8 rules and you can give 8 to 1 point on what category you find important.
5. Your points go to the jury winner of every category. So if model A has the highest jury score on creativity your points for creativity go to that entry. In my example 10 points. Winner of functions category gets my 5 points. So voters do not even need to look at all the entries in detail, the jury has done that and public points are distributed accordingly.

This way jury determines category winners and public voting can still influence the end outcome by what category they find important in the contest. It also rules out any form of rigging. Both jury and public voting run in parallel.

I know this suggestion will never make it as it would largely rule out the earlier mentioned personal reasoning for personal favorites but this would be a system of combined and unambiguous jury AND public voting! It would even be possible to create one list 10 categories the same for EVERY contest and rate them for every contest differently by the jury. So participants will know upfront what the jury finds important in the contest. Public voters can have a saved list of of these 10 categories they update for every contest voting round which would not be much work. If 10 is too limited jury could add 'wildcard' categories which public voters would give points in their voting next to the 10 fixed ones.

Edited September 7, 2023 by Berthil
typos, fixed 10 categories added.

[SaperPL]

@Berthil the problem with this approach is that whenever you introduce separate categories and just a single place on the podium for each category, there is a chance of situation where two awesome models fight for one place in one category while significantly worse/lower quality model has it easy for that other podium if in that criteria he's just slightly ahead of what those two superior models have.

Also even if you give people points to distribute for each criteria/category, how can you be sure they won't just dump it on a single entry that they like? Of course a lot of us will vote properly, but there will be ones that'll just pick few they like the most without looking at the categories and filtering their decision through that.

It's either having specific criteria being specifically judged by jury based on specific rules, or a streamlined contest where the only thing that is rated is awesomeness factor because everything else is equal and mandatory.

[Berthil]

@SaperPL It is unfortunate you always see problems and not the benefits as this comes closest to what is practically possible and what you are after without creating that 100 page lawyer document.

3 hours ago, SaperPL said:

Also even if you give people points to distribute for each criteria/category, how can you be sure they won't just dump it on a single entry that they like? Of course a lot of us will vote properly, but there will be ones that'll just pick few they like the most without looking at the categories and filtering their decision through that.

You may be have misunderstood, people are not voting for entries but for categories within the contest without knowing what entry has won per category as jury and public voting run in parallel. So they cannot dump on one entry. This system gives the jury the ability to compose the top entry list with models they want and public voting can change the final order of entries and vote without much effort but personal entry preference is lost.

3 hours ago, SaperPL said:

or a streamlined contest where the only thing that is rated is awesomeness factor because everything else is equal and mandatory.

The proposed system with voting for your most important categories is NOT making everything else equal.

Again an example. I find creativity important so that would be my category to get 10 points. So the most creative model according to the jury would get my 10 points. But if I find that in a next contest I find aesthetics more important, I give that category 10 points and creativity 9. Very flexible, jury driven and not much work to vote.

This proposed system is the first tangible voting system I have seen in the whole discussion to be honest. All other remarks seem to convey personal opinion and preference but for the most without a proposal for a voting system that could work for jury and public voters. This voting system enables public voters to vote for WHAT they find important and not why they find a specific model important to win for whatever reason.

As a test it would be possible to add public voting to TC25 WITHOUT changing the current outcome, just as a test.
The jury will let me know their categories and their voting (I don't need to know who is behind what vote), I setup an Excel and we start public voting according to the jury categories and I enter in the Excel. No extra work for @Jim or jury. We can discuss the outcome and even use the Excel for future contests if that would become the new voting system. If there are not many public category votes we can also conclude this is not the way forward.

Contest participants can also vote without problems of course as they don't know the final jury entry list.

Edited September 7, 2023 by Berthil
typos, voting test added.

[Alex Ilea]

I am not invested in this "how to vote" conversation, but why not keep it simple? For public voting, give each build a score out of 100, any way you like. You are more into aestetics, fine. More into functions, fine. Entry 1: 90/100, entry 2: 55/100 and so on... Obviously, if someone votes 100 for 1 build and 0 for the rest that shouldn't count.

[SaperPL]

1 hour ago, Berthil said:

@SaperPL It is unfortunate you always see problems and not the benefits as this comes closest to what is practically possible and what you are after without creating that 100 page lawyer document.

Specific rules are a way to have clean and fair competition. And again - I never said that I want 100 page lawyer document. As I said before we have literally one sentence explaining criteria specific to the specific contest, within a page of general rules that don't really explain how the criteria would be judged. I'm not postulating 100 pages. I'm asking for one paragraph for each of 4-6 criteria specific to the contest. That would be additional one page maybe. The fact that people are asking for explanation over and over again in each contest is a solid proof that contest criteria are not ironed out.

Now go and draw yourself an axis with natural numbers to see how many natural numbers you can fit between 0 and 100 (if you want to count in pages) - don't extrapolate to ridiculous numbers just so you can prove your point. You're simply wrong and over blowing numbers just to prove a point where the thing we're talking about isn't something that is bound to get there (to infinity), is simply not an argument. We get no explanation of criteria outside of single word for most of the criteria specified in each contest. That's zero explanation on what will be valued. Quality criteria as noted by many of us can be valued differently between the looks, the number of mechanisms, the rigidity of the build, the uniqueness of building techniques and so on.

Fair point is that I misunderstood what you meant by giving points to categories. With that said, your system seems really convoluted. It would be clear if everyone was to distribute points between categories and then those would affect the jury scores through multiplication. But from what I understood in your system, it's just for determining the order of winners between categories? What happens if the same entry wins two categories that are less valued by a hair? Does it mean that this one will score those points on two categories and win the first place even if there's another winner of a more popular category that was really close with points on those other two? Or does your system assume that all winners get their scores multiplied by category popularity points in each category?

1 hour ago, Berthil said:

This proposed system is the first tangible voting system I have seen in the whole discussion to be honest. All other remarks seem to convey personal opinion and preference but for the most without a proposal for a voting system that could work for jury and public voters. This voting system enables public voters to vote for WHAT they find important and not why they find a specific model important to win for whatever reason.

That's just your personal opinion... I wouldn't call your system tangible. And still if you want it to make sense, the jury should vote on specific criteria and if you want that to make sense, the criteria should be specifically explained and followed by jury, otherwise jury can have different interpretation of criteria than the community and it doesn't matter that people like one category or quality of a model the most if jury sees it differently and will end up scoring models completely differently than the community would (if they were to vote properly) and you'd end up with different first places in each category.

What I would say is a out of the question for a contest in your idea is that decision what is valued more in the contest happens after the entries were already finished. This is something that should be clear at the beginning of the contest, otherwise it's either a lottery if you made a good decision or not, or we could end up with contest after a contest being dominated by category that was most popular in the first contest of this type.

 

All the problems of this discussion about scoring boils down to not wanting to precisely score on all criteria because it's a lot of work for all entries, while also it poses a risk of having multiple entries with exactly same amount of points, so jury doesn't want to do it because it would mean second stage for resolving such conflicts. But such conflicts of two or more entries with same amount of points also happened with public voting through places from 1st to 6th, and also each time there are a lot of quality entries we get back to same statement that judging will be hard - and it will be hard if you're supposed to decide between two entries that are both really good. Judging through criteria with specific explanation what will be penalised is a semi-automated work where there's just need of few judges to even things out a little. Unless of course you want to make a contest where community votes, but then it's either simple or you end up with weird contraptions like you've proposed here.

 

5 minutes ago, Alex Ilea said:

I am not invested in this "how to vote" conversation, but why not keep it simple? For public voting, give each build a score out of 100, any way you like. You are more into aestetics, fine. More into functions, fine. Entry 1: 90/100, entry 2: 55/100 and so on... Obviously, if someone votes 100 for 1 build and 0 for the rest that shouldn't count.

Because that doesn't solve the problem of awesomeness factor dominating when it comes to public voting where most of people will just pick few entries that they like in a contest that is more than just awesomeness and is about doing a specific thing. That is why there are two types of contest - the ones with jury and the ones with public votes and in general this is a good idea so we can have a contest with more interesting goal and terms where jury needs to rate entries and we can have more streamlined contest where it's obvious what are they about and it's just about awesomeness factor in the voting. Current system for public voting is good. The problem is that it's not going to fit more complex topics where awesomeness factor shouldn't be the only one.

[Alex Ilea]

11 minutes ago, SaperPL said:

Because that doesn't solve the problem of awesomeness factor dominating when it comes to public voting where most of people will just pick few entries that they like in a contest that is more than just awesomeness and is about doing a specific thing. That is why there are two types of contest - the ones with jury and the ones with public votes.

But if an entry is not awesome it shouldn't win. It doesn't matter if you make a superbly engineered build, if it is ugly it shouldn't win. Same for the opposite. And yes, double voting is the way to go, but again, keep jt simple.

[SaperPL]

1 minute ago, Alex Ilea said:

But if an entry is not awesome it shouldn't win. It doesn't matter if you make a superbly engineered build, if it is ugly it shouldn't win. Same for the opposite. And yes, double voting is the way to go, but again, keep jt simple.

But if there is a requirement for the contest to make something specific and someone goes against rules or makes a model that is just superficial and it looks good on the photos, but doesn't really do a good job on the other requirements, it can still win.

I'm not saying that we should specifically promote ugly builds just because they are rigid, but I think there should be a penalty for compromising build quality for just the looks for example. Unless the contests are supposed to be just about the looks, then if that's true, then it should be okay to cheat on the inside and build things with modified and 3d printed pieces, glue and ungodly building techniques.

The good example of that is again TC20 where one of two mostly voted entries was something that was really not what the contest was supposed to be about. It was awesome and that's why it was voted for, but that's one example what will happen if you want simple voting system and some specific topic of the contest that can be ignored in popular vote.

TC18 on the other side was a really good example of a contest where it was pretty clear what needs to be done and everyone was making something in similar size and similar functionality, so the awesomeness factor was a good enough criteria for public vote.

[dhc6twinotter]

1 hour ago, Alex Ilea said:

But if an entry is not awesome it shouldn't win. It doesn't matter if you make a superbly engineered build, if it is ugly it shouldn't win. Same for the opposite. And yes, double voting is the way to go, but again, keep jt simple.

I disagree with this.  This is a Technic.  Technic is supposed to be about the technical functions of a build.  An entry that has complicated functions and is well constructed is more awesome to me than something that is poorly built but looks good.  
 

When I vote, I vote based on technical complexity and how well something is constructed.  There is a certain amount of skill that is required to make an entry look good with Technic, and I do account for that when I vote, but technical complexity is far more important to me. 
 

If future entries were voted on by only people who mostly frequent the Technic forum rather than Eurobricks wide voting, I think our results would be very different.  

[Berthil]

1 hour ago, SaperPL said:

What happens if the same entry wins two categories that are less valued by a hair? Does it mean that this one will score those points on two categories and win the first place even if there's another winner of a more popular category that was really close with points on those other two? Or does your system assume that all winners get their scores multiplied by category popularity points in each category?

If a model wins two categories by jury it probably deserves to win so it wil get the points by category by public voters of two categories. I'm just trying to make the voting less biased and easier for everyone with this proposal.

1 hour ago, SaperPL said:

That's just your personal opinion

Sure it is my personal opinion until adapted by everyone. Seems like you also are giving your (big) share of personal opinion so I don't know why this is suddenly a 'just ' when I have one.

1 hour ago, SaperPL said:

end up with weird contraptions like you've proposed here

Thanks! Seems like a compliment coming from you.

As a final statement for me in this discussion, in the end Jim and Jury decide what will happen in the future (or with the TC25 test as I proposed to add public voting to TC25). I'm glad to help if that is what the contest and Jim would help time wise and meanwhile will be looking forward to the next competition.

Edited September 7, 2023 by Berthil
Final statement added.

[Berthil]

1 hour ago, SaperPL said:

don't extrapolate to ridiculous numbers just so you can prove your point.

If I remember well 500 pages was mentioned by someone else so I'm not extrapolating, I'm downsizing. 

1 hour ago, SaperPL said:

I never said that I want 100 page lawyer document

I never said you want a 100 page lawyer document

Edited September 7, 2023 by Berthil
can't get it in the same reply

[SaperPL]

25 minutes ago, Berthil said:

If a model wins two categories by jury it probably deserves to win so it wil get the points by category by public voters of two categories. I'm just trying to make the voting less biased and easier for everyone with this proposal.

But my point was something like this: you get two entries like so:

• Best in category A, second in category B by 95% of the top score, second in category C by 95% of the top score, second in category D by 95% of the top score.
• Last in category A, first in category B, first in category C, last in category D.

If I understood correctly your point distribution it would be that we only distribute points for winning in a category and the amount of points based on how people voted. If that's correct, then we get something like this for the entries above:

• Entry getting just points for winning category A despite being almost perfect in all other categories and not compromising anywhere
• Entry getting points for winning in categories B and C while completely compromising on everything else

This could go weird both ways, either if you end up in this specific situation and someone wins by winning in two lesser categories despite the entry being really poor in other categories, or you could have one category that gets most of the points from public vote and even entries winning two or three other categories taking the second place if over 50% of people vote for one category. The idea to rate how important categories are might be good as long as all entries get all of their scores in specific criteria multiplied by those criteria categories, but my understanding is that this isn't what you're proposing here.

25 minutes ago, Berthil said:

Sure it is my personal opinion until adapted by everyone. Seems like you also are giving your (big) share of personal opinion so I don't know why this is suddenly a 'just ' when I have one.

You explicitly picked on others stating their opinion to as if to glorify yours being the best one and only reasonable one. That is why I picked on that it's still just another opinion.

25 minutes ago, Berthil said:

Thanks! Seems like a compliment coming from you.

Well, it's not. Others are trying to figure out something simple that will be acceptable, yours is really weird, but again, that's just my opinion.

9 minutes ago, Berthil said:

If I remember 500 pages was mentioned by someone else so I'm not extrapolating, I'm downsizing. 

No, what you're doing is effectively following the same thing that others are doing - making a straw man to defeat easily - instead of addressing actually my arguments. It's easier to defeat an argument by blowing it out of proportion to show how outrageous it would be if blown to those proportions, but that is only correct if it's about a mechanism that actually works this way (there is some growth defined in itself) on it's own, for example if I proposed that we keep increasing the amount of criteria in each contest - then extrapolating it would be valid argument. Otherwise - don't hide behind "other's said so", just address my arguments if you want to counter them.

What I agree with is referring to previous contests and showing examples of how things could be scored, but also I don't remember anyone tried doing so fully.

Edited September 7, 2023 by SaperPL

[Berthil]

22 minutes ago, SaperPL said:

This could go weird both ways, either if you end up in this specific situation and someone wins by winning in two lesser categories despite the entry being really poor in other categories, or you could have one category that gets most of the points from public vote and even entries winning two or three other categories taking the second place if over 50% of people vote for one category. The idea to rate how important categories are might be good as long as all entries get all of their scores in specific criteria multiplied by those criteria categories, but my understanding is that this isn't what you're proposing here.

I wanted to keep it less work for the jury by proposing a winner by jury in each category but it would indeed be better if the jury would score each entry for each category like you propose. With a multiplying factor all public votes can be applied to all entry categories resulting in applying 100% voting for all entries in all categories with jury voting for all entries as guidance by jury multiplying factor.

So if a model gets 10 in a creativity category through jury voting this is multiplied by public voting for creativity. So when one public vote has 8 points for creativity, voter would hand out eighty points to that entry in creativity category. This can be applied to all entries and all categories and all votes and jury only has to vote all (self defined) categories for all entries and voters will only give their category vote. I don't think there will be many entries with the same amount of points in this system, of course with enough voters. But since voters do not have to look at every entry in detail (the jury does), and everybody knows the categories they like (and don't need to tweak a lot for every contest), I hope there will be more voters than usual. Everybody happy, may be even you.

If not clear I could work out a tangible contest example in more detail. If nobody is interested I save myself the work and await the next contest.

Edited September 7, 2023 by Berthil
typos

[SaperPL]

But then we get back one of my core points - jury should score on specific criteria/categories and not in order who's which place. And also if you want your public vote system to actually affect those scores fairly, those scores from jurors should also follow some rules that are clear at the beginning of the contest before you enter so you can decide how to approach it. If jury can also score those categories however they are interpreting the rules, you can have someone score 25% on specific criteria for example simply because of interpretation being different than what the contestant assumed is okay, and even if the category is worth a lot due to how community votes, the damage was already done at the stage of jury vote and also everyone will get score for this criteria multiplied.

What I'm opposing in general is any system that makes it so that you don't know the final system of scoring at the beginning of the competition. If we want it to be a fair competition or even challenge, the rules should be clear so everyone can figure out how to approach the challenge within the rules. If there's a lot left for interpretation that jury doesn't want to explain beforehand, then it's not a fair competition. It also includes the scaling of how valued the criteria/categories are as in your proposition, because it will effectively mean how important each of the criteria will be and how much it'll affect overall score. If for example amount of functions were to be the highest valued criteria but the quality of mechanism were to be lowest, you can approach it by making some junky contraption with dozens of functions being implemented poorly but being there, so having 0 points for buld quality of those mechanisms won't hurt that much when you got max score for how much mechanisms you managed to fit.

But if you have some kind of view of how the requirements work and what you can and cannot do and make a decision based on limited info that you have because you don't get answers or simply you wouldn't even think about asking specific question, and then either when you're already in the middle of building or at voting stage contest shows that was okay to go with something that you assumed would be not accepted, it means that you didn't really know what is acceptable or not so you couldn't make a properly informed choice.

Not explaining/establishing the rules to a reasonable degree at the beginning of the competition, is equivalent of a game master running the RPG game with a specific scenario in mind where specific classes or races are going to have a disadvantage while the players don't know that and some of them will pick those. Even in a game that is not about rewards once you realise that for the whole game you're unlucky because you've picked wrong, it's not a fun experience.

[gyenesvi]

 

Quote

If not clear I could work out a tangible contest example in more detail. If nobody is interested I save myself the work and await the next contest.

Honestly, I agree with @SaperPL that your system sounds quite convoluted, furhermore that it isn’t even sound, in the sense that what the competition is all about would only become decided after building the entries, because it would basically be decided by the public voting. That sounds very counterintuitive.

Also agree with many generic principles put forth by @SaperPL throughout the whole discussion for the case of jury voting. For example that without some (short) specification and prioritization of criteria it just becomes a gamble. Which could be acceptable for public voting, but not so much expected for jury voting.

On 9/7/2023 at 6:15 PM, Berthil said:

@SaperPL It is unfortunate you always see problems and not the benefits as this comes closest to what is practically possible and what you are after without creating that 100 page lawyer document.

I’d say that my max 3 points per entry proposal (for public voting) sounds practically much more possible to me.

Edited September 11, 2023 by gyenesvi

[Berthil]

14 hours ago, gyenesvi said:

furhermore that it isn’t even sound, in the sense that what the competition is all about would only become decided after building the entries, because it would basically be decided by the public voting. That sounds very counterintuitive.

Probably also a misunderstanding. The contest categories are fixed by the jury at the start of the contest and not the end. Probably as it is now but a bit more detailed via the categories. Public voting would be in those categories and how many points is set by the jury per model and category after closure of the contest. So the categories are known upfront but of course how many points per model per category after the models are finished. Even a third multiplier could be introduced by category order of the contest (preferably defined at the start). With the three multipliers there should be enough point differentiation along the predefined contest categories.

Let me do a full scenario for a contest with three distinctive categories and three model entries to keep it simple, because I feel I'm not good at explaining the system.

TC26 Technic Powerboat Contest with categories in order of jury importance (multiplier in brackets):

Jury Categories:

1. Speed on water (3)
2. Maneuverability on water (2)
3. Looks (1)

It is clear at the start of the contest that the jury is looking for fast models that turn well, looks are less important. So you shouldn't make a Technic Titanic that looks great but is slow and can't turn if you want to win by jury. Of course if that is what you want to make to have fun it is no problem, you can make what you like.

End of contest and models A, B and C are finished;

1. Model A, moderate speed, high maneuverability, looks are okay
2. Model B, clearly highest speed, maneuverability acceptable, looks totally absent
3. Model C, low speed, moderate maneuverability, looks 10 out of 10 with a mini Technic Titanic.

4 jury members score 1 to 10 in the categories for the three models using their own interpretation. This could mean one jury could give model B 10 points for speed but the other jury 9 points because that jury thinks it is still not fast enough. I created the above matrix in an Excel with the three models, three categories and 4 fictional jury votes from 1 to 10 for each model in each category. I won't bother you with the details but each jury vote fluctuaties between the categories because of personal preference and interpretation. The jury totals for each model and category are (where list number is category);

1. A=22 , B=39, C=3
2. A=39, B=16, C=16
3. A=18, B=1, C=40

Totals for each model; A=79, B=56, C=59
WITHOUT the category multiplier Model A wins, Model C 2nd, Model B third. This was probably the jury voting method in the last contest
But now we multiply with category multiplier in this simple example, probably we would have 10 categories in a real contest. Calculation could stop here and end of contest.

1. multiplier 3; A=66, B=117, C=9
2. multiplier 2; A=16, B=32, C=32
3. multiplier 1; A=18, B=1, C=40

Totals for each model; A=162, B=150, C=81 (looks has multiplier 1 which is no multiplier, probably avoid category with multiplier 1).
WITH multiplier Model B moved to 2nd place because speed has highest multiplier as it was most important in the competition, but not enough to win from model A which had the best mix of speed and maneuverability. Voting could stop here and this would be the end result without public voting.

Now it gets interesting, public voting was in parallel with jury voting so model order was not known and also doesn't matter and each voter rates the categories they find important. We have 5 voters with category number and score between 1 and 10. They look at the overal competition with all entries and decide what they find important, list number is the voter

1. voter finds straight line speed very important because voter is a drag racer; 1=10, 2=1, 3=6
2. voter is a creative designer and find looks very important; 1=4, 2=3, 3=10
3. voter looks at technical solutions and knows maneuverability in water is hardest to achieve but also finds looks important; 1=2; 2=10, 3=8
4. voter just agrees with the jury and should be rated accordingly; 1=10; 2=9, 3=8
5. voter finds everything technical not important and just looks at how it looks; 1=1; 2=1; 3=10

These category votes can be used as percentage or multiplier on jury votes with or without the jury category multiplier.
If jury wants more effect on the outcome take scores with jury category multiplier. In this example I will take voter percentage of each jury category vote without jury multiplier for each model where 10 points is 100%, 9 points is 90% etc. So model A had 22 jury points for speed, voter 1 has 10 points for speed meaning voter 1 awards 100% of jury votes to speed meaning 22 points for model A. Model C had 40 points for looks and voter 4 has an 8 for looks so Model C gets 80% of 40 points = 32 points. Etcetera for all voter categorie percentages and models.

Outcome after public voting where list number is voter with model total according to category voting (because of percentage we are getting decimals now);

1. A=36.7, B=41.2, C=28.6
2. A=38.5, B=21.4, C=46.0
3. A=57.8, B=24.6, C=48.6
4. A=71.5, B=54.2, C=49.4
5. A=24.1, B=6.5, C=41.9

Totals (rounded): A= 229, B=148, C=215

So after public voting Model C has moved to 2nd place because there were voters that found looks important. Still model A is 1st. Now add jury votes to public votes, again this can be done with multiplied jury votes or not. Without jury multiplier order is A-C-B, with jury multiplier it is still A-B-C with a very close tie between B and C.

Contest organizer and/or jury can decide to use different multipliers, or even a jury/public voting ratio, but voting system would be the same. I created an Excel that did all the calculations and most of the time was typing this, the Excel was quickly created. In the end there would be categories with a multiplier rating by jury, jury category voting per model and category, public voting per category and an Excel that brings it all together including decisions on multipliers and jury/public voting ratio.

Edited September 11, 2023 by Berthil
typos

[SaperPL]

1 hour ago, Berthil said:

Probably also a misunderstanding. The contest categories are fixed by the jury at the start of the contest and not the end. Probably as it is now but a bit more detailed via the categories. Public voting would be in those categories and how many points is set by the jury per model and category after closure of the contest. So the categories are known upfront but of course how many points per model per category after the models are finished. Even a third multiplier could be introduced by category order of the contest (preferably defined at the start). With the three multipliers there should be enough point differentiation along the predefined contest categories.

This is something that you seem to not get - if you make it so that after the models are built there is public vote on which category should be valued the most, it means that we don't have those values when we're deciding what to build at the beginning of the contest. That's why for us, me and @gyenesvi this makes it into a gamble when deciding on what to build and how to approach the contest criteria. I don't know how else can we explain this that if you want to make the challenge feel fair and reasonable, you need a solid criteria for the scoring system that don't change after you've made your entry. 

I get it that you want to make jury and public vote in parallel, but it shouldn't affect the foundations of decision making for those who enter the contest.

Instead of voting on criteria, community could vote for awesomeness factor and this score could be either added to overall jury score with a ratio predefined at the beginning of the contest between criteria score and awesomeness factor, or the awesomeness factor would be something that would decide which entry should win between two with the same criteria score. But obviously this would mean that jury shouldn't score awesomeness, just the build quality itself (and other criteria). And in this case again making solid rules on how points will be given, what will be penalised with negative points in a category should be precisely defined and followed by jury, otherwise the community vote will be meaningless if jury vote already decides the order of entries. But it could solve the problem that always brought up if someone proposes point scoring of criteria instead of F1 places that we would end up with multiple stages of scoring because of that, but at the same time there is a risk that community vote will not be used if criteria is not precise enough for enough of the entries to be near to max score.

[Berthil]

22 minutes ago, SaperPL said:

This is something that you seem to not get - if you make it so that after the models are built there is public vote on which category should be valued the most, it means that we don't have those values when we're deciding what to build at the beginning of the contest. That's why for us, me and @gyenesvi this makes it into a gamble when deciding on what to build and how to approach the contest criteria. I don't know how else can we explain this that if you want to make the challenge feel fair and reasonable, you need a solid criteria for the scoring system that don't change after you've made your entry. 

Jury categories are clear at the start but as a builder you can deviate from that to have more fun building because you like more what you make. The reward for a having fun can come afterwards when public voting choses their categories they find important and if they are the same as your fun categories, you will move up the order depending on the jury/public voting ratio (preferably set at the start). Sticking 100% to the top categories might bring you glory but could be less fun as you are building in categories you find less challenging and fun. Models might also look alike a lot. You want the last and I want the first so don't tell me I don't get it. In the latest competition there was no public voting and I have the feeling there won't be in the future. There was also not much transparency in the jury voting. With the category system we can have transparency upfront and public voting without much extra effort and keep having fun.

I guess it would be possible to replace the category voting by your 'awesomeness' voting for every model. But when 50 models, this would be a lot of work for public voters or we go to the previous system where they can pick their top 5 or top 10. In the category system they would only have to rate categories once and the result is tied to the jury voting.

I'm not participating in the discussion anymore as I have made my point. I will wait what the next contest will bring, it is not in our hands but in the hands of Jim and Jury.

Edited September 11, 2023 by Berthil
typos

[gyenesvi]

2 hours ago, Berthil said:

Probably also a misunderstanding. The contest categories are fixed by the jury at the start of the contest and not the end.

Yes, I get that the categories would be fixed in the beginning.

Quote

So the categories are known upfront but of course how many points per model per category after the models are finished.

As @SaperPL notes, this is why it is still not sound, because the weighting would change afterwards. And that can change everything.

Another thing I find strange about your idea is that if the public votes for the criteria, they could actually do that without even looking at the models. After all, I can decide whether I value looks or functions more, without having any models in mind; it's a generic preference. This does not make sense to me as a public voting, where people want to participate in making a decision about the entries.

8 minutes ago, Berthil said:

as a builder you can deviate from that [jury categories] to have more fun building because you like more what you make. The reward for a having fun can come afterwards when public voting choses their categories they find important and if they are the same as your fun categories, you will move up the order

That sounds like a weird twisted contest; hoping that the public has the same value system in terms of categories as I do, which is rarely the case for many very technical builders I guess.