Generic Contest Discussion

[Jundis]

I hope the voting systems won't be more complex than the rules :D

[artemisovsky]

In my opinion current public voting system (old Formula 1 style) is probably the best with less than 15-20 entries. With more entries (over 20), it is impossible to select only 6 cool models (like in TC25 contest) so it would be a good idea change the rules voting to "+1 any you like" or "distribute X points among all entries as you wish" but only when there are more participants.

For me, jury vote will always be fairer than an open vote. Even than, in the future competitions with only jury voting it would be nice to make an additional audience award (even without a physical reward) from rest of the entries.

Is there an indicative date when the next competition TC26 can start?

[Jim]

1 minute ago, artemisovsky said:

With more entries (over 20), it is impossible to select only 6 cool models

That is a valid point and could be the reason to change the voting scheme.

2 minutes ago, artemisovsky said:

Is there an indicative date when the next competition TC26 can start?

Probably in a year or so. I'm currently writing an elaborate list of requirements and voting criteria. We need to think of every little detail. Otherwise it's not clear for people what to build and how to build it. I will discuss the results in a focal group, have some Teams meeting to iron things out. After that, I need to run it by legal to see if there are any issues. This process will be repeated a couple of times until we have a bullet proof list of requirements and voting criteria so we can all enjoy the contest. We can't just simply wing it start a contest now can we 

[Mr Jos]

17 minutes ago, Jim said:

That is a valid point and could be the reason to change the voting scheme.

Probably in a year or so. I'm currently writing an elaborate list of requirements and voting criteria. We need to think of every little detail. Otherwise it's not clear for people what to build and how to build it. I will discuss the results in a focal group, have some Teams meeting to iron things out. After that, I need to run it by legal to see if there are any issues. This process will be repeated a couple of times until we have a bullet proof list of requirements and voting criteria so we can all enjoy the contest. We can't just simply wing it start a contest now can we 

Don't forget to invite all minority groups, to see if there are no insults being made to any one of them by any of the rules. Then check if all potential participants have a sufficient amount of bricks to build what they want, else an other theme will need to be chosen for the contest. Lastly have some mystery guests going to everyone's home to check their computer files if there are no pre-preparation builds made yet, else, you guessed it, chose another theme, and start over the process from step 1. And don't forget to put all timelines together when everyone is back from holiday to know the optimal start-date.

With all of this, having an AI take over all these tasks doesn't seem so unreal to keep our flesh-MOD's healthy and give them some breathing time.

[Jim]

3 minutes ago, Mr Jos said:

Don't forget to invite all minority groups, to see if there are no insults being made to any one of them by any of the rules. Then check if all potential participants have a sufficient amount of bricks to build what they want, else an other theme will need to be chosen for the contest. Lastly have some mystery guests going to everyone's home to check their computer files if there are no pre-preparation builds made yet, else, you guessed it, chose another theme, and start over the process from step 1. And don't forget to put all timelines together when everyone is back from holiday to know the optimal start-date.

I knew there were some things I forgot. Thanks for the heads-up. We do need to know how many parts everybody has. Otherwise it won't be fair.

[artemisovsky]

52 minutes ago, Jim said:

Probably in a year or so. I'm currently writing an elaborate list of requirements and voting criteria. We need to think of every little detail. Otherwise it's not clear for people what to build and how to build it. I will discuss the results in a focal group, have some Teams meeting to iron things out. After that, I need to run it by legal to see if there are any issues. This process will be repeated a couple of times until we have a bullet proof list of requirements and voting criteria so we can all enjoy the contest. We can't just simply wing it start a contest now can we 

Absolutely not!

21 minutes ago, Jim said:

I knew there were some things I forgot. Thanks for the heads-up. We do need to know how many parts everybody has. Otherwise it won't be fair.

It should also be checked whether each participant has appropriate camera and completed a photography course (because better photos / film give an unhealthy advantage). This is not acceptable. Ideally, prohibit photos and videos at all.

[Jim]

32 minutes ago, artemisovsky said:

It should also be checked whether each participant has appropriate camera and completed a photography course (because better photos / film give an unhealthy advantage). This is not acceptable. Ideally, prohibit photos and videos at all.

Exactly. 

And come to think of it; do all members the same sticker supplier?!

[gyenesvi]

It's nice to hear others' way of thinking about rules/voting :) Really, I do learn from it. I can accept that when voting is open to the community, everybody votes the way they want, no elaborate criteria are required, and that's intentional. There's only one thing left that I don't quite get: if the same applies to jury voting, then how is it different from public voting (except for being limited to a few people)? What is the purpose/advantage? I thought it is to better keep the spirit of the contest, but if the criteria are only vaguely defined and can be interpreted differently by each jury, then that can't be the difference/purpose.

4 hours ago, Erik Leppen said:

With all this discussion about voting and point systems, I wonder: has there been a case where there was consensus that a contest had "the wrong winners"? As a rather long-time reader on this forum, I can't remember any such case. I could be wrong, but I believe the current system works in that sense that it generates good results.

That's not the reason why the discussion started, rather (on my behalf) to better understand how the criteria should be interpreted (now I understand better: quite loosely, all right), and to make the voting/point distribution system more easy on the voters.

As for the point distribution system, when distributing a fixed amount of points / among a fixed amount of entries, my issue with that is those numbers (points, number of entries) will always inherently be quite arbitrary, they may need to depend on the number of participants, furthermore, most importantly, the votes inherently depend on each other, so it is hard to consider each entry on its own (everything needs to be compared to everything). The reason I proposed the scoring system, is that the scores have an intuitive meaning. If you can give 1-10 points for something, then 1 means - I don't like it, and 10 means - I like it very much. That's simple to grasp.

2 hours ago, Jim said:

Agreed. Max 10 sounds logical.

Not sure about the minimum number of entries. We could do 6 like usual. Otherwise 10, 10, 5 would be an option.

So now that I get that we don't want fleshed out criteria and weighting among them, my next thought was to simplify the scoring idea and just allow people to give a certain max amount of points to each entry, which is getting similar to what you guys are proposing here. Maybe even a 0-10 scale is too fine grained. I'd say max 3 points to each entry could be enough, that way a single person cannot bias an entry. It is kind of a generalization of number of 'likes' (to something like +1, +2, +3). And most importantly, I would not put a min/max bound on the number of entries to vote on or number of points to distribute in total.

What do you think about that? I think it can't get any more simple than that, easy to interpret / execute. Do you see any dangers / disadvantages? What I did think of is whether it's a problem if somebody votes to a lot of entries (as could have easily been the case in the shrinking contest). I don't think so, because then all such entries benefit equally (no biasing happens).

Edited August 31, 2023 by gyenesvi

[Jim]

3 minutes ago, gyenesvi said:

It's nice to hear others' way of thinking about rules/voting :) Really, I do learn from it. I can accept that when voting is open to the community, everybody votes the way they want, no elaborate criteria are required, and that's intentional. There's only one thing left that I don't quite get: if the same applies to jury voting, then how is it different from public voting (except for being limited to a few people)?

Public voting can be more biased. Voting for the helicopter because they like helicopters. Voting for yellow, because they don't like red. And even people voting for certain members or members from certain countries (yes, this can be a thing). A jury is supposed to be more neutral when it comes to these things. Not saying that is always the case, but they are less biased than (part of the) public voters. 

[gyenesvi]

4 minutes ago, Jim said:

Public voting can be more biased. Voting for the helicopter because they like helicopters. Voting for yellow, because they don't like red. And even people voting for certain members or members from certain countries (yes, this can be a thing). A jury is supposed to be more neutral when it comes to these things. Not saying that is always the case, but they are less biased than (part of the) public voters. 

Thanks for the explanation. So do you have a rule of thumb when it's okay to allow for public voting and when to resort to jury voting? I was thinking something like when entries are more similar in theme/size then there's less chance for bias, but I guess that still does not rule out the bias with certain members/countries.

What do you think about the point system I proposed above?

[Jim]

8 minutes ago, gyenesvi said:

Thanks for the explanation. So do you have a rule of thumb when it's okay to allow for public voting and when to resort to jury voting?

No not really. We will probably do public voting again next time.

28 minutes ago, gyenesvi said:

So now that I get that we don't want fleshed out criteria and weighting among them, my next thought was to simplify the scoring idea and just allow people to give a certain max amount of points to each entry, which is getting similar to what you guys are proposing here. Maybe even a 0-10 scale is too fine grained. I'd say max 3 points to each entry could be enough, that way a single person cannot bias an entry. It is kind of a generalization of number of 'likes' (to something like +1, +2, +3). And most importantly, I would not put a min/max bound on the number of entries to vote on or number of points to distribute in total.

What do you think about that? I think it can't get any more simple than that, easy to interpret / execute. Do you see any dangers / disadvantages? What I did think of is whether it's a problem if somebody votes to a lot of entries (as could have easily been the case in the shrinking contest). I don't think so, because then all such entries benefit equally (no biasing happens).

I kinda like this idea. But with 3 point per entry max, do you still have 25 points in total?

Maybe 3 is on the low side, but a total points of 25 with a limit of 5 per entry could work.

[Seasider]

@Jim - @gyenesvi was saying everyone scores every model out of a score 0-3 or that’s how I read it. But with this does that mean you don’t pick a first place model as you could score more than one model with 3 points ?

I think people generally like creating their own 1st 2nd 3rd and seeing how it compares to the final overall positions form all those who voted

[SaperPL]

4 hours ago, Erik Leppen said:

With all this discussion about voting and point systems, I wonder: has there been a case where there was consensus that a contest had "the wrong winners"? As a rather long-time reader on this forum, I can't remember any such case. I could be wrong, but I believe the current system works in that sense that it generates good results.

If you take a look at TC20, which had public vote first and jury afterwards is an interesting case of community choice vs what jury "disqualified" and thus so what ended up on podium.

The main voting criteria was replicating studded sets as close as possible with studless Lego Technic, so just from the rules you could figure out that the point of it is to recreate those frame/edge built models with holes, just with liftarms, which effectively could be boring replacement of beams with liftarms. But once the discussion and questions about the rules went on, you could figure out that you are supposed to go with modern approach with panels and rule of the thumb is just that the model should be instantly recognizable.

Because of this ambiguity, quite a few people went on to build models in modern standard where the holes and gaps are supposed to be closed, and the model winning by popular vote by most amount of points, was removed from podium because of the modern approach with panels, but once you'd check it against others who used panels, it's like it was disqualified for using panels in just one spot on both the sides of the tow truck.

So one thing is that criteria not being fleshed out could mean completely different perception of what the contest is supposed to be about between contestants and jury, second thing is that contest shows that people may also vote completely different than jury.

Finally note that since there are different outcomes between jury vote and public vote, if the method is decided at the beginning, your approach to picking out what to do and how to do it may differ. And therefore changing the method of choosing who ends up on the podium after the beginning of contest may not be right if there was ambiguity in the criteria, and such change happened when jury vote was added on top of public vote.

If popular vote winners not being on the podium because of how jury voted isn't an example of jury picking "wrong winners" as you said it in context of what majority of voters picked, then I don't know what is...

[Erik Leppen]

1 hour ago, Jim said:

Public voting can be more biased. Voting for the helicopter because they like helicopters. Voting for yellow, because they don't like red. [...]

The more voters there are, the more things like this are averaged out.

Unless, of course, there is a community-wide dislike for red 

[kolbjha]

And another suggestion for public voting :

Keep it simple, every voter makes a list ranking all entries from best to least best... Best entry on a voter's list is given points corresponding to number of entries, next is given the same points minus one, etc. For example, if there are 12 entries, the points 12, 11, 10, ..., 1 are distributed from each voter. This will also address one of the main complaints about the currently used formula one system: voters wish they could give points to more than 6 entries.

Edited August 31, 2023 by kolbjha

[gyenesvi]

1 hour ago, Jim said:

I kinda like this idea. But with 3 point per entry max, do you still have 25 points in total?

Maybe 3 is on the low side, but a total points of 25 with a limit of 5 per entry could work.

As I noted, I would not put a min/max bound on total points or on number of entries. The whole idea is to make scoring entries independent of each other. The bounds introduce dependencies, and I think they are not necessary, they just complicate things. Or can you give a case that could potentially become problematic without the bounds?

1 hour ago, Seasider said:

@Jim - @gyenesvi was saying everyone scores every model out of a score 0-3 or that’s how I read it. But with this does that mean you don’t pick a first place model as you could score more than one model with 3 points ?

Exactly, that would be allowed. If you want to pick a single winner, then don't score multiple ones with 3, give them 1 or 2 :) In the end it's the whole community that picks the first place model with the accumulation of points, not the individuals. But I often have the problem the other way around: I can't pick a single winner, I see multiple ones as roughly equally good, and would like to express that.

Quote

I think people generally like creating their own 1st 2nd 3rd and seeing how it compares to the final overall positions form all those who voted

Maybe, but I also have the impression that people often have difficulty picking a single first/second/third place and having to decide in a strict way. But if that's a real problem, then we could allow half points, so that one could better differentiate between their favorite ones.

4 minutes ago, kolbjha said:

For example, if there are 12 entries, the points 12, 11, 10, ..., 1 are distributed from each voter. 

That does not differentiate entries properly. Furthermore, if there are 30 entries and I like 10 of them, why do I have to rank all the other 20 that I don't like. That is pretty difficult.

[marcus2388]

3 hours ago, Jim said:

Public voting can be more biased. Voting for the helicopter because they like helicopters. Voting for yellow, because they don't like red. And even people voting for certain members or members from certain countries (yes, this can be a thing). A jury is supposed to be more neutral when it comes to these things. Not saying that is always the case, but they are less biased than (part of the) public voters. 

So… if I vote for the entry which I like most… and there is a good chance this ‘liking’ is based on a brilliant colour scheme of the model – that’s a bad thing? And if the majority of forum members likes this exact model also best… that’s still not a valid reason to win the contest for the best model?

Even though it managed to obey to all the contest rules.

This seems like the perfect setup to lose sight of the “trending Zeitgeist” – especially if it is directly ignored. I do not understand why this seems to be a recurring motif here… for instance: why are you annoyed about “soooo many cars” – instead of celebrating it as the major lifeline which kept Lego (and the interest in it) alive?

I am aware of the principle of qualified Jury voting… but I believe this can hardly be applied to judging Lego Technic Models. There are simply too many different approaches to “what makes a great Technic Model” of which a Jury is not able to fully incorporate.

Like/Dislike Electronics

Styling VS Function VS True Scale

Playability / Robustness / Exotic Building techniques

…

 

As an example: I do not believe a great Lego Technic Model of a real life car should have a gearbox the size of a hot tub (if converted to real size)… especially if it’s always the same gearbox design. It’s a much bigger (and from the perspective of MODEL building: important-) job to focus und true-scale appearance.

I accept that there are (very) different opinions to the one I just voiced – hence I would not feel comfortable seeing my decision as the sole reason for naming a winner of a Lego Technic contest… But I would love to see how the Zeitgeist develops… what’s currently appealing to the Lego-Technic-loving majority and what’s not. And if there is bias - which there will be - I would rather try to understand it (& use it) instead of trying to ignore it. 

[Jim]

2 hours ago, kolbjha said:

And another suggestion for public voting :

Keep it simple, every voter makes a list ranking all entries from best to least best... Best entry on a voter's list is given points corresponding to number of entries, next is given the same points minus one, etc. For example, if there are 12 entries, the points 12, 11, 10, ..., 1 are distributed from each voter. This will also address one of the main complaints about the currently used formula one system: voters wish they could give points to more than 6 entries.

This is exactly what the jury did with last contest  

[allanp]

8 minutes ago, Jim said:

This is exactly what the jury did with last contest  

Appears to have worked quite well!

[gyenesvi]

51 minutes ago, Jim said:

This is exactly what the jury did with last contest  

42 minutes ago, allanp said:

Appears to have worked quite well!

I think that is a fundamental difference between jury voting and public voting. The jury's 'job' is to consider each entry, so for them, ranking all of them is not an extra burden. But in case of public voting, I guess people might not want to consider all entries, only the ones they like, and just say nothing about the rest.

[kolbjha]

5 hours ago, gyenesvi said:

 

5 hours ago, kolbjha said:

For example, if there are 12 entries, the points 12, 11, 10, ..., 1 are distributed from each voter. 

That does not differentiate entries properly. Furthermore, if there are 30 entries and I like 10 of them, why do I have to rank all the other 20 that I don't like. That is pretty difficult.

I can not see how distributing up to 3 (or 5) points on individual entries, with a total of X points, would differentiate more. And if you don't want to rank more than say 10 out of 20 entries, then just rank the 10 best of your choice. They would then get 20, 19, ..., 11 points from you, the other entries zero. Then we would not have to worry too much about format giving the votes either, any list of entry numbers would do, comma separated, whitespace separated, one entry on each line etc. would do. And @Jim and @Milan could easily validate each list by removing any duplicate or invalid entry numbers. And if two or more entries should end up with the same total score, differentiate them by finding the highest individual score each of them got, and if that number is the same, then count numbers of that score for each of the entries. If still same score, check the second highest individual score and so on. Relatively easy to make a script for that (for example in Excel).

Of course, if you have two entries you consider exactly equal, my suggestion would not give you the opportunity to reflect that in your voting. However, myself, when I have voted in previous competiotions, I have ranked all the entries, and given the points to the top 6 entries accordingly. And some times this ranking has been hard to do, for TC24 I ended up "bubble sorting" the entries; (1) I made an initial list of the entries, (2) then go through the list (3) comparing two at the time (which one of these two do I like best), (3) swapped these two if needed, continue with the same to the bottom of the list, repeat going through the list until no more swapping was necessary. I know this sounds like a lot of work (and it is). I acknowledge that others find it hard to rank, they simply wish to give the score of each entry independant from the other entries.

Saying all this, I am just shooting out an idea, I am not even sure if I think this is the best voting system myself. I also like your suggestion a lot, especially the fact that each voter can focus on evaluating each entry independant from the other entries (except that you will still have a limited total number of points to distribute), and the beauty of that there (maybe) is no ranking from each individual voter; the ranking is formed by the community as a whole.

Edited August 31, 2023 by kolbjha

[2GodBDGlory]

For what it's worth, I'm perfectly happy with the standard F1 voting system we've used before! It doesn't take too much thought on the voter's part, and it seems to work!

[gyenesvi]

4 hours ago, kolbjha said:

I can not see how distributing up to 3 (or 5) points on individual entries, with a total of X points, would differentiate more. And if you don't want to rank more than say 10 out of 20 entries, then just rank the 10 best of your choice. They would then get 20, 19, ..., 11 points from you, the other entries zero.

The difference I see is that when you give 20, 19, 18, ... points, then you essentially only get to decide on the ranking, but their relative scores are fixed, and linear. So if you think about 1st = 20 points as 100%, then 2nd = 19 points is 95% and 3rd = 18 points is 90% and you cannot change that. But what if you think 3rd is only 50% of 1st? You can't express that, and the decay is typically not linear. That's why the F1 scoring exists, it's a non-linear decay, so it's one step better in this respect, but even if that's used, you still don't get to decide the actual relative scores, only the ranking.

Quote

And some times this ranking has been hard to do, for TC24 I ended up "bubble sorting" the entries

Wow, cudos for doing that, that indeed sounds like a lot of work! That's exactly what I'd hope to eliminate the need for.

Quote

I am not even sure if I think this is the best voting system myself

I am not sure that such a voting system exists! :)

Quote

and the beauty of that there (maybe) is no ranking from each individual voter; the ranking is formed by the community as a whole.

Exactly, I don't think there is a need for a ranking from each individual, since we cannot accumulate rankings (meaningless) we can only accumulate scores.

2 hours ago, 2GodBDGlory said:

For what it's worth, I'm perfectly happy with the standard F1 voting system we've used before! It doesn't take too much thought on the voter's part, and it seems to work!

I think the F1 system is a good system for converting a ranking to scores, so that they can be accumulated, when the ranking is already given by the race itself. But if the ranking is not naturally given, why not provide the scores directly instead of going through the ranking to just get the scores?

Edited September 1, 2023 by gyenesvi

[2GodBDGlory]

1 hour ago, gyenesvi said:

I think the F1 system is a good system for converting a ranking to scores, so that they can be accumulated, when the ranking is already given by the race itself. But if the ranking is not naturally given, why not provide the scores directly instead of going through the ranking to just get the scores?

Well, that just seems like it takes a decent bit of thought! I think it's easier to just put my favorite six in order and call it a day...

Doesn't mean it's the best way, but easier would mean more participation (though maybe votes aren't as meaningful if people think less about them)

Anyways, I hadn't meant to get into this discussion, since I can't say I really care, but those are my thoughts! :)

[MangaNOID]

8 hours ago, allanp said:

Appears to have worked quite well!

 

The more there are who disagree, the more there will be an agreeable outcome.

 

7 hours ago, gyenesvi said:

I think that is a fundamental difference between jury voting and public voting. The jury's 'job' is to consider each entry,

 

I think the public consider each entry. Sometimes its pretty quick and easy to disregard and entry or two because of x or x that pops right out, especially from videos.

 

@SaperPL should be on any next Jury as he has about the best eye for detail that I've read from anyone.

 

And for what its worth:

 

General kind comments should be banned

Critique of build quality, function quality comments should be encouraged

A voting system where you vote for the worst instead of the best and least points win

Comments on why you voted the worst to encourage better builds

Small token prize for the worst.