Judging Discussion

[Alvis 3.1]

Who said anything aboot shutting it down? Certainly not me....

Now, as far as a mod editing out the stupid stuff, yeah, that's something I have grown VERY tired of.

Al P.

I agree. The subject is interesting and actually is model related so it's cool to get different people's perspective on what they consider a good way of judging a contest. Just because some can't carry on in a civil manner doesn't mean the whole conversation should be shut down. I know this subject has been discussed countless numbers of times before, but that makes it no less interesting to some people. Just my humble opinion.

For example I for one am interested in how the general population here feels about things such as FOD covers and tarps. I honestly never considered the "it's chickening out" thought once I first saw people use them. I always thought they added color contrast and realism so they were cool, but then I hear some people say it's just an easy way to hide bad intakes and/or exhaust. Things such as this are interesting to me. So what's the problem discussing?

Bill

[Alvis 3.1]

Zactoman, individuals should have the right to express their ideas, experiences, and opinions without insulting one another no matter what level of education, race and ethnicity they are or language they speak. It’s unfortunate but I will defend who I am with great passion and honor. I would like to hear more about your ideas and thoughts about our hobby. I have a few of your items you have created and I must say there awesome. Please continue the great work in the aftermarket area.

I concur, but when it descends into poo flinging, then it's time that the repercussions of anyone's actions will be seen. People acting responsibly will see a different result, of course.

Well, looks it's time some of the un-necessary posts were removed.

Al P.

ARC Moderation team and overall crabby dude.

[MoFo]

Y'know, after skimming this thread, I think I've been cured of any interest in on-line discussions about model contests. I'm completely over reading ludicrous, hypotheticals about supposedly-impossible-to-decide entries.

Let's pop the big myth here. There are no "perfect" entries. Ever. There are only "less flawed" ones.

Even Mr. Vojtech's much-vaunted Nats stuff had build flaws.

Myth #2: judges have to decide between accuracy/degree of detail/various other nebulous, hard to quantify factors. That's NOT the tough part about judging. Because at most contests, the decision never gets to that level. On a table with a dozen entries, half will have obvious, significant basic construction faults that will knock them out from the first look (streaky brush marks, seams completely un-filled, etc.). Half of the remaining will get bounced for faults that only appear on closer inspection (slight alignment errors, wonky canopy masking, small bits of putty that weren't sanded down). MOST of the time will be spent weighing the faults of the three remaining models - is one slightly mis-aligned tailplane better or worse than a slightly silvered stencil? Should the gold go to a model where 1mm of panel line got missed by the wash, or the one where the IFR stencil decal is 1/2mm off to one side. Should the model with one small bit of paint-bleed from the canopy mask get silver, or the model with the perfect canopy that was simply white-glued (not puttied) on? And the annual debate about whether missing the seam line through the gun access door on a FW-190 is an accuracy issue, or a basic construction issue.

Whether the paint is correct, or the outline is accurate, or the overall scenario is realistic, or whether the control stick matches the control surface position? Never going to be an issue.

Myth #3: judges know what they're doing. Judges are human. Some are good. Some are bad. Some (most) know the rules they should be applying and do a great job. Sometimes your model loses because it just flat-out sucks. But sometimes judges don't know the rules. Sometimes they'll make catastrophic errors. Sometimes they'll be terribly biased. Sometimes they'll flat out miss something. Sometimes they'll just be obstinate a-holes who, in spite of being told what the rules *actually* are, every year, for more than a decade, will try to judge things their own way, to their own tastes. It's less likely to be an issue the bigger a contest is, but when poor judging rears its ugly head, it'll over-ride every other consideration.

At a *well* run contest, head judges should ensure EVERYONE is on the same page, that they know the rules, and do their job to the best of their ability. That team leads are well informed and can keep their teams in line, in case any individual IS causing a problem. But not all contests ARE well run. Individuals have good days and bad days. Clubs can have "issues" with the rules, or problems finding the manpower to do the job well. Lighting can be poor. Tables can be low. There are dozens of variables that come together to create a particular decision on a particular day.

And a bonus point #4: wow factor is/should be favoured, or is/should be penalized. Current IPMS judging criteria has its issues. But wow-factor *is* awarded, in the forms of people's choice, and special awards (as Zacto said, if you're awarding a plaque, you can set the criteria). However, making it a part of category judging is very problematic, and opens up many potential problems. Because "wow factor" is ultimately a matter of personal taste. Some people love open panels. Some people hate seeing open panels with ordnance loaded. How do you judge that? And judge it consistently across a contest (with multiple judging teams - maybe it's okay with the 1/48 guys, but not the 1/72 team). Define "stunning paint job". Some people go nuts over pre-shading, others hate it with a passion. It's realism for some, caricature for others. So does pre-shading get bonus points for 'wow', or lose points for 'blech'? Then there's the issue of an aftermarket arms race. Throwing a ton of resin and etch at a model will make it "spectacular". Most people don't do it very well, though. How do you weight that, fairly? Just because I spent more on my model, should that absolve me of (some of) its failings? Should my entry get a break because I added lots of tat? Should YOURS be PENALIZED because you DIDN'T? Which then pushes people to add more and more stuff to their model, rather than trying to build them better. (this is already an issue, since lots of people think you need a ton of detail to be competitive) Degree of difficulty? Do you KNOW whether all the entries were easy to build, or hard to build? How many bonus points should a Hasegawa kit get vs. a Tamiya kit? What if it's an OLD Tamiya kit? What about a brand new Tamiya kit, but it was warped or short-shot and took more effort? How do you rank a Hobbycraft Fokker Triplane vs. a Roden Triplane (1/32)? If you haven't built the kit in question, do you just go by the manufacturer's reputation? What if you're wrong?

Soooo.... having said all of that, let's put this into practice. I'll use Greg's MiG, since it's an acknowledged winner, because it reinforces my points. And 'cause I know he won't take it as a personal attack.

First, it's almost impossible to accurately judge a model (to contest standards) based on photos on the internet. You can't check seams, decals, paint finish, etc. maybe it's not visible, maybe it's not close enough, maybe it's blurry, maybe it's obscured by JPEG compression. But going from what I can see using a judge's eye...

- there's an issue with the seam around the nose ring. Just above the sand/metal demarcation line, there's a small spot where either the seam wasn't re-scribed, or the wash got missed. (unless it's just lighting)

- the inboard wing fence. might be short shot, but it should have been repaired. **unless its supposed to be that way for some reason, in which case it should be documented on the entry form so the judges won't have to guess.

-it also looks like there's an un-filled seam line at the bottom/front of the windscreen; there's a line visible at the base of the U on the framing.

- the circles just in front of the tail weren't re-scribed properly: there's a slip/knick visible on the outer ring. Or else a wash/weathering fault (unless it's supposed to be a paint chip)

So yeah, it's a very nice build, and one that any modeller would be happy to call their own. But it's not "perfect". It won gold, but it had some minor flaws. Meaning the silver medal model had more/bigger flaws. etc. etc. And THAT is what a judge looks for (should be looking for...). Look past the paint scheme, past the details, past the subject matter, and focus on the fundamentals of construction. That's what makes it a difficult job.

[Dave Williams]

Y'know, after skimming this thread, I think I've been cured of any interest in on-line discussions about model contests. ......

Yet you've shown enough interest to join the thread by "judging" another member's model and highlighting supposed flaws based on what you think you see in pictures, just to make your point. That's pretty rotten in my book. Valid as the point is, you shouldn't throw someone else under the bus to make it.

Might I suggest you read the rules of the road, MoFo?

Public Critiquing of Models at ARC

If you notice an inaccuracy on one of the models displayed on ARC and you have some carefully worded constructive criticism to offer, then please contact the modeler in question privately by e-mail. The majority of modelers don't appreciate having inaccuracies in their models addressed in a public forum. Unsolicited critiques will be removed by the moderators. This also goes for discussions of models posted at other scale modeling websites. The exception to this is the Critiques Forum. Here, you can offer polite, constructive critiques on models when it is asked for. Rude or insulting posts will be deleted at the discretion of the moderators.

BTW, a number of MiG-15s have that cutout in the left inboard wing fence, although it may be only on Lim-2s, but I'm sure the kit came that way.

http://www.richard-seaman.com/Aircraft/Air...ig15Chino08.jpg

I've never seen it documented in any contest, and don't think it needs to be. Apparently, judges can figure it out.

Edited May 1, 2010 by Dave Williams

[Steve N]

a number of MiG-15s have that cutout in the left inboard wing fence, although it may be only on Lim-2s,

Several of us were puzzling over that while looking at the MiG in the Kalamazoo Air Zoo. We finally came to the conclusion that it was so the pilot could see the pop-up flap indicator on the wing.

As for the judging discussion....

SN

[TF51GREGWISE]

The rules for the up comming Orlando Show. A "shoulda woulda coulda" read.2010 modelpalooza

[is it windy yet?]

Time to wade into this one,

Firstly I've been a judge at local and regional shows. I judge the categories that I am not entered, which means I have less knowledge of the kits, and sometimes the subject.

Secondly I've made errors in judgment, and I will again, as humans make errors.

One time though error or lack of communication, an entire category was missed. I needed to judge it while the awards were being presented. It was 1:72 aircraft, or possibly jets. I assumed that 1:72 Helos were a separate category, they weren't. So one of the best models at the show, was never properly judged. It may have won best of show, and it received squat.

At the same show, I had no part of this, a 1:48 corsair didn't win anything in the aircraft side of things, but managed to pick up Best Paint, which an airbrush was the prize. It was an average paint job at best. But he was a friend of the head judge, and he managed to pick up a nice prize. I personally think best paint, is a car award myself. I also remember way back my best friend won best Tamiya vehicle with a Tamiya 1:32 F-4C/D. The award was from Tamiya Canada, and the rules were it had to be a Tamiya kit.

Most recently in an in store hobby contest, limit of two entries per scale. I won best aircraft, and should have gotten 2nd too and only got 2nd in the armour category. It was a hooray for everyone style of contest. It wasn't about the prizes, but I sure felt insulted not being recognized as the winner and second place in both. My models were clearly better. I found out today the person that beat me felt the same as I did, and he donated his prize money, $30.00 to my son, the winner of the kids side of the contest. Am I ever glad I kept my big mouth shut for change.

Judging is judging, sometimes you make the mistake, sometimes you win, and sometimes things work out in a way you never expected.

[sgt dutch]

Thanks for post the to 2010 modelpalooza judges guide. Just leaving the score with the model will give the person and idea of his errors. Most contest you have no clue why you didn't place. Or why your model didn't make the cut. That least with scores you can see where you need to improve. Also it might be good to place a couple of notes point it the big errors in the model.

[Liberator24]

Y'know, after skimming this thread, I think I've been cured of any interest in on-line discussions about model contests. I'm completely over reading ludicrous, hypotheticals about supposedly-impossible-to-decide entries.

Let's pop the big myth here. There are no "perfect" entries. Ever. There are only "less flawed" ones.

Even Mr. Vojtech's much-vaunted Nats stuff had build flaws.

Myth #2: judges have to decide between accuracy/degree of detail/various other nebulous, hard to quantify factors. That's NOT the tough part about judging. Because at most contests, the decision never gets to that level. On a table with a dozen entries, half will have obvious, significant basic construction faults that will knock them out from the first look (streaky brush marks, seams completely un-filled, etc.). Half of the remaining will get bounced for faults that only appear on closer inspection (slight alignment errors, wonky canopy masking, small bits of putty that weren't sanded down). MOST of the time will be spent weighing the faults of the three remaining models - is one slightly mis-aligned tailplane better or worse than a slightly silvered stencil? Should the gold go to a model where 1mm of panel line got missed by the wash, or the one where the IFR stencil decal is 1/2mm off to one side. Should the model with one small bit of paint-bleed from the canopy mask get silver, or the model with the perfect canopy that was simply white-glued (not puttied) on? And the annual debate about whether missing the seam line through the gun access door on a FW-190 is an accuracy issue, or a basic construction issue.

Whether the paint is correct, or the outline is accurate, or the overall scenario is realistic, or whether the control stick matches the control surface position? Never going to be an issue.

Myth #3: judges know what they're doing. Judges are human. Some are good. Some are bad. Some (most) know the rules they should be applying and do a great job. Sometimes your model loses because it just flat-out sucks. But sometimes judges don't know the rules. Sometimes they'll make catastrophic errors. Sometimes they'll be terribly biased. Sometimes they'll flat out miss something. Sometimes they'll just be obstinate a-holes who, in spite of being told what the rules *actually* are, every year, for more than a decade, will try to judge things their own way, to their own tastes. It's less likely to be an issue the bigger a contest is, but when poor judging rears its ugly head, it'll over-ride every other consideration.

At a *well* run contest, head judges should ensure EVERYONE is on the same page, that they know the rules, and do their job to the best of their ability. That team leads are well informed and can keep their teams in line, in case any individual IS causing a problem. But not all contests ARE well run. Individuals have good days and bad days. Clubs can have "issues" with the rules, or problems finding the manpower to do the job well. Lighting can be poor. Tables can be low. There are dozens of variables that come together to create a particular decision on a particular day.

And a bonus point #4: wow factor is/should be favoured, or is/should be penalized. Current IPMS judging criteria has its issues. But wow-factor *is* awarded, in the forms of people's choice, and special awards (as Zacto said, if you're awarding a plaque, you can set the criteria). However, making it a part of category judging is very problematic, and opens up many potential problems. Because "wow factor" is ultimately a matter of personal taste. Some people love open panels. Some people hate seeing open panels with ordnance loaded. How do you judge that? And judge it consistently across a contest (with multiple judging teams - maybe it's okay with the 1/48 guys, but not the 1/72 team). Define "stunning paint job". Some people go nuts over pre-shading, others hate it with a passion. It's realism for some, caricature for others. So does pre-shading get bonus points for 'wow', or lose points for 'blech'? Then there's the issue of an aftermarket arms race. Throwing a ton of resin and etch at a model will make it "spectacular". Most people don't do it very well, though. How do you weight that, fairly? Just because I spent more on my model, should that absolve me of (some of) its failings? Should my entry get a break because I added lots of tat? Should YOURS be PENALIZED because you DIDN'T? Which then pushes people to add more and more stuff to their model, rather than trying to build them better. (this is already an issue, since lots of people think you need a ton of detail to be competitive) Degree of difficulty? Do you KNOW whether all the entries were easy to build, or hard to build? How many bonus points should a Hasegawa kit get vs. a Tamiya kit? What if it's an OLD Tamiya kit? What about a brand new Tamiya kit, but it was warped or short-shot and took more effort? How do you rank a Hobbycraft Fokker Triplane vs. a Roden Triplane (1/32)? If you haven't built the kit in question, do you just go by the manufacturer's reputation? What if you're wrong?

[/quote/]

Much better.....

VOTE LIBERATOR24 for Moderator!

Thank you! I will be here for shows at 7:30 and 9:00. Enjoy the shrimp!!

-Jim

[GaryP]

Ya know... that makes a pile of sense.

Where I live, the biggest show of the year is judged by spectators. Anything with wings in 72nd that isn't a jetliner or big bomber has little or no WOW factor and never stands a chance at winning. However if your aircraft is a jet that's 32nd scale, you are a shoe in for a medal. Last year a not terribly well done Russian carrier won best of show because it was huge.

There is another contest a few hundred miles away I will be attending that uses the AMPS method. The thing I liked about that show isn't the awards. It's being able to learn how to improve my models. I will never be as good as some of the guys around here, but at least I can improve with good guidance.

These discussions about contest judging seem to come up on various forums relatively frequently, usually when someone feels like another model should have won. It seems to me that there are a few basic realities regarding this topic. Starting with the first one:

#1 – Someone is always going to be unhappy.

People can debate to the end of time about whether a particular judging is fair, or which system is better, but in the end someone is going to complain about judging. No system is perfect.

#2 – The debate about IPMS type judging vs. AMPS type (gold-silver-bronze) will always go on.

Whenever someone posts criticizing about judging at IMPS contests, people always reply touting about how much better the gold-silver-bronze type system is. Both types have their pros and cons, and this will never be settled. It just spawns more argument that has the potential for getting out of hand (I’ve seen a lot of threads on HS go bad and get pulled).

#3 – IPMS judging is not going to change, at least not as long as the same IPMS Head Judge is in place, and maybe not even afterwards.

It’s quite clear that the people (or person) in charge of IPMS judging are not interested in changing from their 1-2-3, basic constructions counts philosophy. Right or wrong, it’s not going to happen and the Head Judge has made that quite clear on a number of occasions. In fact, the President’s column in the latest IMPS Journal addressed this very topic and made clear that they believe the current system is the right way for IMPS contests.

Bottom line, this was an IMPS regional contest, hosted by an IMPS chapter, and so was judged under IPMS judging rules. If people don’t care for IMPS judging, then they shouldn’t enter IPMS contests. Not saying it’s right or wrong, just that it’s the way it is. Debating amongst ourselves in this forum may be entertaining, but I don’t think anything is going to change on how IPMS contests are judged.

Just my 2 cents.

[Pirata]

Let's pop the big myth here. There are no "perfect" entries. Ever. There are only "less flawed" ones.

Allow me to quote myself:

*Note: The term "perfect" is used to define a model with no easily detected flaws in the time allotted.

It's easy to find a flaw in one model when the judge has photos of it from several angles and can take as much time as he wants to look it over.

It's harder when you're on a deadline and are critiquing several models that appear to be virtually flawless. That is, after close inspection, in the time allotted, some of them appear to meet all the criteria that the judges are looking for. This is when the judging becomes very difficult. That is neither impossible nor hypothetical. Once all the basics are deemed to have been covered by multiple entries in a class, (Yes, this happens, though it's rare.) other factors, difficulty of build, etc.. will start to play in the judges deliberations. It probably shouldn't, but it does.

- the inboard wing fence. might be short shot, but it should have been repaired. **unless its supposed to be that way for some reason, in which case it should be documented on the entry form so the judges won't have to guess.

It doesn't need to be repaired. Some MiG-15s, or license built MiG-15s had those. Is it accurate for that particular aircraft? The answer is irrelevant. One does not judge accuracy at an IPMS contest. At least, one is not supposed to.

From the ARC walkarounds:

-it also looks like there's an un-filled seam line at the bottom/front of the windscreen; there's a line visible at the base of the U on the framing.

A judge must know the difference between a seam, a crack and a panel line. From the ARC walkarounds:

The line in front of the U on the framing:

Another angle:

The line that stretches around the windscreen:

Again, this is why judging accuracy no worky-worky. You claimed that in your post but didn't practice it in your judging example.

[DutyCat]

Well, I just looked over the Orlando judging guide, and quite frankly, it looks far too cumbersome and time consuming, especially the details part. Once you start assigning points, tabulating, using a computer to cross check results, etc, well...then it is just too much hassle for me to be involved with. I don't need all that to tell me what what the top three models are that I am looking at, or what is gold, silver, or bronze, for that matter. I am considering sitting this one out, or if I go, not being involved with the judging. I will not tabulate scores on a sheet, ever. Sorry.

[Pirata]

Well, I just looked over the Orlando judging guide, and quite frankly, it looks far too cumbersome and time consuming, especially the details part. Once you start assigning points, tabulating, using a computer to cross check results, etc, well...then it is just too much hassle for me to be involved with. I don't need all that to tell me what what the top three models are that I am looking at, or what is gold, silver, or bronze, for that matter. I am considering sitting this one out, or if I go, not being involved with the judging. I will not tabulate scores on a sheet, ever. Sorry.

I looked at it and it does look very different from what I'm used to. I'll try to keep an open mind. The only thing I don't understand is: What if two people get the same score? Who wins?

[geedubelyer]

The only thing I don't understand is: What if two people get the same score? Who wins?

Easy,........alphabetical

Just change your name to Aaron Aardvark....................

Edited May 2, 2010 by geedubelyer

[MoFo]

Look, my intention was not to rag on Greg's (very nice) model, to criticize it or in any way diminish it. It was simply to illustrate the way judges look at models, and Gregs MiG was a convenient and pertinent example. They don't look at how pretty or spectacular a model is. They don't look at how detailed it is. Or how accurate. When a judge is looking at a model on the contest table, they're looking along the panel lines for seams (un-filled, un-sanded), checking the paint for smooth application, decal silvering, alignment, glue smears, restored panel detail, etc. etc.

They may love a particular subject, or think a particular model looks gorgeous. But they set aside their personal tastes and opinions and coldly search out and tabulate construction flaws. Which is why judging takes practice. It's not difficult, but you have to learn what to look for, but just as importantly, *where* to look. You're not really scrutinizing every single square inch of every single entry - you don't have to - you're looking at the key areas where modellers are most likely to fail. Time isn't really an issue; you can knock out the obvious non-contenders in under a minute (obvious faults that take them out at first glance), another minute or two of closer examination will knock you down to the top three or four (knocking out the models with less-obvious-from-a-distance-but-still-major issue); the top few entries usually sort themselves out (these two are clearly the best, this one's obviously third), so it's just another couple of minutes picking nits and weighing fault to decide between entries that are close (these two are clearly the best, so which one is better?). If you're not figuring out 1/2/3 in a category inside of five minutes, either the standards are incredibly high, or the judging team is struggling. Granted, at a national competition things are going to be much tougher so things are a little different, but the same basic rules apply: it may be a longer process (though usually, you're looking at multi-day shows, closed judging, more judges and longer judging times), but what they're looking for and the way they look for it is pretty much the same.

Finally, regarding the issues I pointed out on Greg's model. I'm not going to debate whether anything I mentioned is or is not a 'fault'. That would only draw things out further, and is pretty much beside the point anyway - again, it wasn't about specific issues or a specific model, but trying to help people understand what judges look for, and how they look at entries.

However.

I did blow it on the wing fence.

Which is a great example of judges being human, and blowing things from time to time. (would anyone buy if it I suggested I *meant* to do that, just to help illustrate my point? ) It's not something I'd ever noticed on a MiG-15. If it had been on both wings, as a judge, I'd probably have assumed it was fine and given it a pass. On just the one wing, it stuck out and looked like an issue to me. Now, on the one hand, on a judging team, hopefully someone would know that it was fine, and the model wouldn't get bounced for it. On the other hand though, it's a good example that you should always note any odd, unusal, and potentially judgable peculiarities on your entry form. If it's something that you might possibly get dinged for, write it down. Judges don't really care whether you used a NeOmega or True Details ejection seat, but they DO want to know that the crooked or upside down insignia is the same as the aircraft you're depicting.

[Pirata]

MoFo, I think we can all agree that modeling is a nit-picky hobby. We nitpick kits, decals, resin, PE, manufacturers, retailers, websites, model forums, model contests, completed models, real airplanes, etc... It should be no surprise to anybody that we're going to end up critiquing critiques & judging judges. Perhaps, it's just force of habit. I'm sorry if it looked like we were ganging up on you. It wasn't intended.

Which is a great example of judges being human, and blowing things from time to time. (would anyone buy if it I suggested I *meant* to do that, just to help illustrate my point? :))

I'll accept that.

[frankv74]

Okay just to clear something up. Greg's Mig looks Beautyful and looks Exact. But i know some of you area looking at make something real. Okay well have you seen Russian Aircraft? They look like someone built them using a Sledge hammer and a Jack hammer to build them. The factory these aircraft came out of are not exactly State of the art. They look like something out of "Junkyard Wars". You can crank out the shotest looking Mig and it still would look Acurate.

Frank

[Mutant_X_109]

Miami Fun Snipers!

I like It!

I will offer the name change to our local South Florida modeling club during our next meeting. Our club is conveniently located between The Conch Republic and Alabama. Yes, we are the educated bunch in Miami.

I was at the Space Coast contest and was glad to be there, spending only 15 bucks on a Monogram 1/48 OA-4M. I should have it ready by the Orlando show in September.........perhaps.

Basics!!!

Construction!!!. I know some people that spend two months working on 8 models at the same time, hoping that quantity equals award(s).

A big NO NO!!!. Spend all that energy on ONE model and you might have great chances on winning something!

Miami Fun Snipers Club!

Where all languages and all people from all countries are welcome!

Happy Modeling!

MX109

Edited June 4, 2010 by Mutant_X_109

[frankv74]

Miami Fun Snipers!

I like It!

I will offer the name change to our local South Florida modeling club during our next meeting. Our club is conveniently located between The Conch Republic and Alabama. Yes, we are the educated bunch in Miami.

I was at the Space Coast contest and was glad to be there, spending only 15 bucks on a Monogram 1/48 OA-4M. I should have it ready by the Orlando show in September.........perhaps.

Basics!!!

Construction!!!. I know some people that spend two months working on 8 models at the same time, hoping that quantity equals award(s).

A big NO NO!!!. Spend all that energy on ONE model and you might have great chances on winning something!

Miami Fun Snipers Club!

Where all languages and all people from all countries are welcome!

Happy Modeling!

MX109

MX109

Miami Fun Snipers Club!

Where all languages and all people from all countries are welcome!

I LOVE IT !!!!!!!!!!!!!!!