Jump to content

SURVEY SAYS!!! GSB!!!


Rusty White

Recommended Posts

For those of you who don't know, the official survey conducted by IPMS/USA to determine membership's preference of judging systems at the nationals is in.  The results are 238 For G/S/B and 236 for 1/2/3.  To say the least, this comes as a surprise to me.  Not so much that GSB won, but that the vote was so close.  I was anticipating 1,2,3 winning, but as they say, the membership has spoken!  The preference for GSB probably won't generate significant change right off the bat, but it will certainly cause a LOT of talk by the NCC and the E-board and will go on record as the most accurate survey ever taken by IPMS/USA.  Hopefully the E-board and NCC will now seriously discuss and consider any future proposals for a GSB system at the national model contest.

Link to comment
Share on other sites

  • Replies 67
  • Created
  • Last Reply

This tally has all the drawbacks I and others anticipated.(1) Of a 4000+ member organization, only a little over 10% even bothered to vote. (2) Of those who did vote, there was no statistical difference in the result. Given these two facts, this hardly represents a clear and unequivocal mandate for change. Were I on the National Contest Committee, I would be saying something like: ‘Yes, this is interesting, but it is not a clear demand by the majority of the membership for G,S,B at the National Contest.’  Given the logistical and cultural upheaval such a change would represent, I do not see this feeble response and ‘flip of the coin’ result as making it either desirable or necessary. After all, one way of looking at this result is that, what: 6 or 7% of the membership want this transformation and that 94 or 95 % either do not or do not care enough to even vote on the matter?  The NCC and the E-Board would competently be exercising their fiduciary responsibilities as the leaders of this organization to say that the voice of members has been heard and that, practically speaking, it is “full of sound and fury, (but) signifying nothing.” 

Nick Filippone, Senior National Judge 

 

 

 

Link to comment
Share on other sites

1 hour ago, Nick Filippone said:

This tally has all the drawbacks I and others anticipated.(1) Of a 4000+ member organization, only a little over 10% even bothered to vote. (2) Of those who did vote, there was no statistical difference in the result. Given these two facts, this hardly represents a clear and unequivocal mandate for change. Were I on the National Contest Committee, I would be saying something like: ‘Yes, this is interesting, but it is not a clear demand by the majority of the membership for G,S,B at the National Contest.’  Given the logistical and cultural upheaval such a change would represent, I do not see this feeble response and ‘flip of the coin’ result as making it either desirable or necessary. After all, one way of looking at this result is that, what: 6 or 7% of the membership want this transformation and that 94 or 95 % either do not or do not care enough to even vote on the matter?  The NCC and the E-Board would competently be exercising their fiduciary responsibilities as the leaders of this organization to say that the voice of members has been heard and that, practically speaking, it is “full of sound and fury, (but) signifying nothing.” 

Nick Filippone, Senior National Judge 

Nick, I understand your view on this and respect it.  I will add this.  Yes, statistically you cannot say, the majority of IPMS members believe GSB is the way to go. The most significant number in any survey,  is margin of error.  Given a population of 4000 and a sample size of 450 the margin of error is about 5%.  From a purely statistical perspective, you can say that 45% to 55% of the members prefer GSB with 95% certainty.  

  Now that in no way indicates that the contests should be changed to reflect this.  What is says in real terms is that there is a large enough population of believers in that format of judging, that National leadership needs to consider that element and not cast it aside as if it were irrelevant. From my perspective, that is how this looks.  People digging their heels in and declaring, "That is the way we have always done it and that is how we will continue to do it as long as I am around."  

This survey calls for a change in attitude more than it calls for a change in judging format.   

 

 

 

Link to comment
Share on other sites

If I am interpreting this correctly, what this survey shows is that there is 95% certainty that this result is no better than a coin toss. Or there is a 95% chance that possibly only 45% of members prefer GSB. Or that possibly 55% of members actually prefer 123. Or that 55% prefer GSB, and so on.

If one reflects on the attitude of the NCC and E-Board about 10 years ago, when they refused to sanction any IPMS event at any level ( and thereby withheld insurance coverage) that was GSB, I would say that their attitude has changed significantly. 

But given the outstanding support that the membership gives to the National Contest in its present format, I think it would be reckless of the leadership to risk screwing it up based on this data. It might be said that while less than 500 people bothered to even render an opinion, every year 700 or more vote with their feet (and pocket books) and say we enjoy this show and contest in it’s current form. 

Nick

Link to comment
Share on other sites

Actually, while I haven't checked the math on the specific numbers, the purely statistical perspective rests on the assumption that the sample is randomly selected in a way that is representative of the population being sampled. Since those who voted self-selected rather than being randomly sampled and called up like in a poll, the idea of a "margin of error" doesn't really make statistical sense.

Since people self-selected, I suppose it is theoretically possible that there is a silent majority that did not vote but lean one way or another. Further, there are a number of people who may have an interest in the question but are not eligible to vote, such as IPMS Canada members that regularly attend the nationals, that given the tightness of the results could have swung things one way or another if they were asked their preference.

While one could argue that the razor-thin victory doesn't show a clear preference for either system, if the general attitude emanating from people in charge is one of "we hear what people have said and we don't care, lets shelf this whole GSB idea because we're never going to change it," that's not a good look either.

Link to comment
Share on other sites

Brian, Excellent point. I did not want to get into the lack of randomness, but you are quite correct. But then, this was a poll, so it is not like comparing randomly, say, the efficacy of a new antibiotic in a double blind study. 

When I proposed this survey last year about this time, I had suggested it be done as part of the model registration process at the Nationals for three years because I thought that it was National  Contest attendees whose opinion should matter most. By doing it for three years it might be a good cross- section of contestants. But that has bias also.

Of course, the debate will continue. The NCC and the E-Board have now heard the same thing we have all heard: There is still no compelling polling data to suggest that such a significant majority of the membership is so dissatisfied with our current judging system at the Nationals that it should be changed to GSB. 

Nick

 

 

 

 

Link to comment
Share on other sites

First, this was my comment on FB.....

Chillax....I was pleasantly surprised that GSB had (essentially) an equal number of people advocating it as 1-2-3, especially considering the legacy of decades of 1-2-3 shows. However, the ONLY way GSB becomes the predominate system at the Nats in the future is for it to FIRST become the predominate system at Locals and Regionals, and get all the bugs worked out there. IF, and when that's done, we can actually discuss changing the Nats. Until then, 1-2-3 WILL be the system for the Nats.

But, here's what the survey says to ME, after some further thought...

GSB should NOT have had 50% "support" in this survey. 1-2-3 is, for all practical purposes, THE contest system in IPMS. It's the system that IPMS shows have used for 50yrs. It's the system the majority of EVERYONE is most familiar with, whether or not they're IPMSUSA members or not. So, in my view, the survey results should have been 350-400 for 1-2-3 and 150 or less for GSB. No one would have been surprised or questioned the survey IF that had been the result. And IF it had, those same people saying "the survey is all sound and wind, signifying nothing", would be using the survey to say that the discussion is done and GSB has no support!

Instead, they say "only" 10% responded and THAT signifies nothing. Actually, since the history of IPMSUSA elections and surveys is one of having a VERY low response percent of the total membership, it actually says that the USUAL number of members responded; (the usual members who actually have an interest and care about the future of IPMSUSA) and thus the consensus of opinion among those who do the voting and keep IPMSUSA afloat is that there is room for discussion of GSB as a valid alternative to 1-2-3 as a contest system within IPMS.

Personally, I DO NOT want the Nats to have to change to GSB. I want the NCC to WANT to adopt it after GSB has become the "norm". I want them to go to successful Local and Regional GSB shows and have the people ask them why they're so slow to accept what most members truly want: Recognition of craftsmanship without the need to "beat" the model on the table next to it. The desire to win an award at NO ONE'S expense. And, the knowledge that if you do NOT get an award, YOU are the only one responsible for that failure.

Yes, there are logistical hurdles in GSB to be overcome, and THAT is why it's important to do that at the Local and Regional level. There are also ingrained, stubborn attitudes that have to be overcome before GSB could be accepted. They cannot be overcome by discussion or theories on line. They must be changed by experience with GSB, which is why (again) it must be first proven at the Local and Regional shows. And IF it cannot be done successfully at those levels, then GSB will either slowly die out, or it will merely be used as a valid system for smaller shows, and the NCC need never have to consider it for the Nats! Cheers!

 

GIL :smiley16:

Link to comment
Share on other sites

But if GSB is what members truly want, why were those caring 10% so evenly split?

When I proposed this survey, I did it primarily out of curiosity and a scientist’s desire to know the facts. Unfortunately, some involved in writing the wording for the survey  already had a desired result in mind. There was even speculation that, as a result of the survey, in 2020 we would be using GSB at the Nationals! Their disappointment may be understandable, but their lack of objectivity reflects poorly on their scientific open-mindedness. 

Whenever this discussion has taken place, those favouring GSB have frequently taken the position that GSB is what most members truly want. But this may not be so. The GSB supporters are more vocal, perhaps, but so what? The GSB supporters are convinced that GSB is better, but how many members agree? Hence, the survey-that says either 90% of members do not care OR of the 10% that do care, they are evenly split between GSB or 123. This was hardly a ‘to the barricades’ demand for action or change. 

Of course, the conversation can go on. But for the time being, the people have spoken and have made it clear there is not sufficient support for change at the National Contest.

Nick

 

 

 

 

 

 

 

Link to comment
Share on other sites

Jim, your question is a fair one.  In theory, a model that earns Gold at one AMPS show should earn Gold at all AMPS shows.  The system is presented to the judges that way, and the fact that judges' training is scripted and uniform aids in this.  AMPS still allows OJT shifts, but in order to become a certified judge, you must sit through the training session and do two shifts. 

As I said on Facebook, a tie vote is no indication of a "win", and that in and of itself means that nothing will change.  Maybe some will talk about it from time to time, but unless something drastic happens, this gets filed in the "Nice to Know" file.  If IPMS does a similar survey two years from now and the results are the same, nothing will happen.  The only way anything will change is if the preponderance to the votes indicate the membership wants change--and that change will take between 5 and 10 years to happen at the National level.  It will not be immediate.

A friend of mine made a comment that sparked some discussion.  At the local level, what do the Chapter members (both IPMS members and non-members) have to say?  At the National level, including non-members is a non-starter since you have to be a member to participate in the contest, but at the local level, I would hazard a guess that 80% of the people involved with a show are non-members, either through ignorance ("What's IPMS?", in which case the Chapter Contact isn't doing his or her job) or choice (don't want to, did it but got nothing from it, etc.).  It would be interesting if each Chapter Contact polled their membership and posted their findings.  That would become a planning tool for those clubs who host shows, too...

I can say this--at the Region 12 Show several weeks ago, this very topic dominated the business meeting.  There was a lot of interest shown by those in attendance...

Link to comment
Share on other sites

12 hours ago, JClark said:

 

   I was talking with a fellow judge today about all this and here is the fly in the ointment that we talked about. Lets assume the national goes to the Open style. In theory if an entrants model got a Gold in a regional or even a local then its entered in a National and doesn't get a Gold then what does that say about the whole system? As it is now You could get a gold in a local open and then go to Nats and place or not . Essentially the standard is the same in the Open system whether it's local/regional/ or National so the result would have to be the same right? And what happens if it isn't is the bigger question. I wonder how AMPS handles this ?

 

Jim

Jim, good point but that is a flaw of  all  judging systems as long as humans are doing the judging.  There have been models that won best of show at Nat's and didn't place at other major shows and visa versa.  In theory, that shouldn't happen when the criteria are the same, but it does happen for many reasons.  One judge(or group of judges) may not see a particular flaw or judge it less significant that another.  Nothing works 100% of the time and lord know we have tried.  GSB, 1-2-3, points systems, it doesn't matter.  Try as you may, you will never remove human bias from the equation.  

Link to comment
Share on other sites

Are we overlooking an important possible explanation for the lack of interest in changing to GSB at the National Contest? Maybe most members do not want such a change. Perhaps that explains the lack of enthusiasm for such a change. If we do the survey again in two years, and the result is the same, nothing will happen perhaps because nothing should happen. 

Nick

Link to comment
Share on other sites

If there's sincere interest to more fully explore the GSB system, I can appreciate the organization’s desire to do so with great deliberation. That said, I think there’s a middle ground — baby steps — that begins to address the interests of those who prefer GSB.

One of those interests is to recognize a greater number of entrants who exhibit high quality work. With the 123 system only three modelers in each category go home knowing where their models stood relative to all the other entries. Why not recognize the presumed 4th, 5th, 6th, etc. place entries by awarding something like a “Highly Commended” certificate? Allow the judges to award these on a discretionary basis based on quality of entries in a category.

I’ve judged at the Nats and know how close the models can be when narrowing down the final round. There’s often five or six models in contention for 1-2-3. Recognizing 4, 5, and 6 can only be beneficial to the convention and organization and, I believe, has no downside while IPMS considers a move to a GSB system.

Highly Commended awards won’t completely satisfy those who prefer GSB, but it’s a step in that direction. Even if we ultimately decide to retain the 123 system, it’s a good thing.

I'm eager to hear y'all's thoughts.

P.S. Implementing Highly Commended awards is limited to just the cost of printing 3-4 certificates per category. I can’t image that would be a budget buster.

Link to comment
Share on other sites

3 hours ago, Nick Filippone said:

Are we overlooking an important possible explanation for the lack of interest in changing to GSB at the National Contest? Maybe most members do not want such a change. Perhaps that explains the lack of enthusiasm for such a change. If we do the survey again in two years, and the result is the same, nothing will happen perhaps because nothing should happen. 

Nick

Exactly my thought too.  One answer we have gleaned from the survey is that interest in G/S/B has increased since the last survey 5 years ago.  We need to continue doing the survey every few years to monitor the membership's feeling on the subject.

Link to comment
Share on other sites

My first thought is that it should be considered that if a narrow victory for GSB isn't enough of a mandate to change, a narrow loss for 123 should also not be seen as enough of a mandate to completely shelve GSB and stick to 123 for the foreseeable future. 51% may not be a strong mandate, but neither is 49%.

Honestly, I think Gil is correct. Following a lot of the commentary on here and on facebook, I think a lot** of the objections are based on not much more than assumptions -- for example, people assuming that GSB has to be done the way AMPS does it and is therefore too much work, or assuming that is too expensive, and therefore can't be done. Correcting these assumptions is difficult to do through online discussions because people rarely change their minds because someone on the internet made a good point. It would take firsthand experience with GSB to change those assumptions.

**Not all, there are some that just prefer 123 or highly value direct competition. I personally disagree with that, but I feel like "I prefer 123 because X" is a more valid argument than "GSB can't be done because Y" where Y is an assumption that may or may not be based on reality.

3 hours ago, SoupDoctor said:

If there's sincere interest to more fully explore the GSB system, I can appreciate the organization’s desire to do so with great deliberation. That said, I think there’s a middle ground — baby steps — that begins to address the interests of those who prefer GSB.

One of those interests is to recognize a greater number of entrants who exhibit high quality work. With the 123 system only three modelers in each category go home knowing where their models stood relative to all the other entries. Why not recognize the presumed 4th, 5th, 6th, etc. place entries by awarding something like a “Highly Commended” certificate? Allow the judges to award these on a discretionary basis based on quality of entries in a category.

I’ve judged at the Nats and know how close the models can be when narrowing down the final round. There’s often five or six models in contention for 1-2-3. Recognizing 4, 5, and 6 can only be beneficial to the convention and organization and, I believe, has no downside while IPMS considers a move to a GSB system.

Highly Commended awards won’t completely satisfy those who prefer GSB, but it’s a step in that direction. Even if we ultimately decide to retain the 123 system, it’s a good thing.

I'm eager to hear y'all's thoughts.

P.S. Implementing Highly Commended awards is limited to just the cost of printing 3-4 certificates per category. I can’t image that would be a budget buster.

Personally, I think while this may be a good idea, I don't think it goes far enough to really address my fundamental concerns with 123 and how this style of competition ends up contributing to unhealthy attitudes towards the hobby. And to play devil's advocate, the people who like 123 because they dislike what they term "participation trophies" and need there to be losers so they feel good about winning probably won't like this either.

Link to comment
Share on other sites

There is nothing wrong with being a loser, because, having been one plenty of times, I know it feels soooooo good when you win!

There is also one other benefit of competition. It is a powerful incentive to actually finish something.

Nick

Link to comment
Share on other sites

43 minutes ago, Nick Filippone said:

There is nothing wrong with being a loser, because, having been one plenty of times, I know it feels soooooo good when you win!

There is also one other benefit of competition. It is a powerful incentive to actually finish something.

Nick

Funnily enough, sometimes I feel worse when I win because I feel bad when that means other people lose.

As for your second line... that's basically what I think is the biggest challenge and the fundamental question behind judging at any show. Finding a way to have the incentive of competition to encourage people to finish stuff and bring it out, but balancing that with the downsides that can come from being too focused on competition like drama, hard feelings, etc. That's what I think discussions should start with, because anything beyond that is just details.

Link to comment
Share on other sites

The key to the GSB consistency that everyone would hope to get and IPMS would need to have is the same as it is for 1-2-3: training!

GSB judging isn't really any different since judges are looking at EXACTLY the same basics as in 1-2-3. So, the judges don't need training on WHAT to look for. What they'll need is experience in changing their thought process from "elimination from competition" to "grading up or down" on what they see. Models that are easily eliminated in a 1-2-3 show will have obvious flaws that will keep them from getting a medal in GSB. Models that don't "make the cut", but take some observation and thought MIGHT get a bronze, or nothing after some debate among the team. Models that "make the cut" in 1-2-3 are almost certainly bronze/silver level models, especially in a competition as tough as the Nats. Oddly, gold medal models are the easiest, as their outstanding characteristics jump out at judges, just like any model you'd be thinking  "okay, who's taking 2nd and 3rd in this category, since this is the obvious front runner"...or you'd be leaning towards nominating for a "best of" in its genre.

Is GSB still subjective? Yes, no less so than 1-2-3. Judging in GSB will STILL be questioned, and the answers will STILL be the same...change the 3 judges judging and you'll likely get a somewhat different or slightly different result. BUT, the one thing that can be said about GSB is that NO ONE will "lose" to anyone else. Another plus is that no matter HOW many times you hear someone's name winning a medal, it will never keep anyone else from having their own shot to win theirs! So, that "the same guys always win" lament will fade away once everyone understands GSB judging.

The things I've cited above, and the change of the thought process for the judging is why it's so important for GSB to be mastered at the Local and Regional levels first. Not only does it give those judges experience, many of them are IPMSUSA judges and thus take that experience to form a "core" that can make GSB successful at the Nats, IF that ever happens.

GIL  :smiley16:

Link to comment
Share on other sites

I agree with Jim. And to take it a step further: if we achieved the consistency in judging ( which we must do under a GSB system, for it to be internally logical from local to National), then once a model has won Gold at a local show, the entrant would not even have to bring the entry to the National or even attend. He or she could simply submit proof of his local Gold to the NCC, and the award could be mailed. The postage expense to the National Organization would be offset by not requiring rental of all that display space.

Eventually, we could, in keeping up with the times, evolve into a virtual National. Everybody stays home in front of their computer. You would cruise the virtual vendor room- which would be limitless, and purchase items on line for home delivery. Seminars could stream from all over the world, with computer provided translations. A hologram of your entry could be transmitted to the virtual display room- but since you have already received IPMS Gold at the East Schnipetze, New York local show, even this effort would be unnecessary. And no more anxiety provoking transport of models. Holograms of Aris Pappas and Bill Devins could deliver their annual awards presentation which could be paused anytime to go to the bathroom or fast forwarded through the categories in which the viewer has no entries- even though you know you have already won. When one’s name is announced a computer link to your 3D printer would produce your award, which could be produced in the size, colour and material of the recipient’s choice. Simultaneously, a picture of the winner could be sent via his or her computer camera to be displayed for all to see. ( So please keep your clothes on!) No bar would be needed. One could get safely bombed in one’s living room without the need to drive or even walk anywhere. The attendee could prepare any meal he or she wants, or get something delivered- no more complaining about IPMS National Convention rubber chicken banquets. 

No question about it- eventually no one will need to subject one’s self to the stress of building the best model one can and then put his or her honour and reputation as a craftsman on the line by facing down strong competition and proudly displaying that model - win or lose.

The future indeed looks bright.

Nick

Link to comment
Share on other sites

Nick, your sarcasm overwhelms! 😄 I had the same thoughts on the subject, but why not have a higher standard for both nationals and regionals.  That is what happens now with a 123 but it is set by the standards of the contest.  A BOS and the 2019 show may not compare to a BOS from the 2018 show.  As you said it all depends on what shows up.  I don't see a diffuculty in having higher standard as you progress up the ladder.  I suspect the membership would understand that a Nat's gold was a higher standard that a Regional Gold or a local Gold.  In any event each of the shows has a different purpose.  By the nature of the audience, the local show is to encourage local modelers to engage with each other and to encourage non-modelers to join the fun.  The regional is more to encourage quality builders to sharpen their skills and Nationals is for the best to compete with the best.  

Link to comment
Share on other sites

My take:  I don't care about "winning" or "losing" at a model show--I've been on both ends of the spectrum many, many times.  That's not why I build models and put them on a show table--I put my models out there to show what I've been doing.  I welcome questions.  I value critique.  That's the only way to get better at something--have another set of eyes (or two, or two thousand) have a look and tell me what they see that maybe I could have done differently.  Note--this is different from merely collecting a trophy that says I'm King Styrene for a Day and assuming that I need to copy what I did on the "winning" model in order to keep raking in the tin pots.  TELL me what you see.  TELL me what I might do differently.  DISCUS different techniques.  Don't assume I will learn through osmosis.

That, that right there is why I prefer an Open Judging system that offers feedback.  When I put my models in an AMPS show, I value the feedback more than I value the medal that goes with the score I earned. 

But even more to the point, I enjoy contests that use Open Judgng because the people involved are more apt to share.  I've been to IPMS shows where the people who have entered models are stand-offish and tight lipped about which models they entered and how they did things--for some odd reason, they seem to think that if they share their techniques, they'll be giving away Government Secrets. 

One of the Facebook posts on this subject led to a sub-thread on Wonderfest.  George Seletas (Wonderfest Chairman) summed it up nicely: " The most important thing is that we are in that room for fun and brotherhood and not to bump puffed chests like it's the NFL."

Its supposed to be about the models, not the medals.

Link to comment
Share on other sites

Ralph, well said.  When I was younger I was all about winning.  As I have aged the trophies became a box a dusty relics. It took a while, but I, like you discovered that the fun of the contest was in sharing a room with a couple of hundred like minded people.  The biggest compliment I can get is "how did you do that?"   I have found that my attitude plays more into the "fun" factor than the judging method or how others act around me.   The more open and inquisitive I am, the more people respond to me in kind regardless of the judging method.  I agree that GSB gives everybody feedback and that is good.  You don't improve if you can't see the mistakes.  I also understand it takes longer to write comments on every entry.  It makes judging much more tedious, but it does what we purport on primary goal to be.  Encouraging everyone who shows up to improve and have fun.  123 basically crowns a king. 

Link to comment
Share on other sites

3 hours ago, Nick Filippone said:

Jim and Pete, That was mostly an exercise in creative writing with a little satire. Jim’s summary was much more appropriate and, dare I say it, Judicial? 

Nick

Yup.  That was a lot of words and effort to say; "I don't agree".  😁

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.


×
×
  • Create New...