One thing is certain. We are touching on an issue that is of importance to food bloggers. Judging by the length and passion of the responses, this is an important issue for you all. I believe in the openness of the blogosphere and I am glad that we can explore this together to make it better.
A lot of the feedback in the comments was certainly positive. Thanks to all of you who had kind words and encouragement. Before I dive into the more technical aspects of voting, I want to let you all know that I am heartened by what seems like the majority of you who feel like a winner regardless. You confirmed that these contests aren’t just about the prize. They are also about the challenge, the free stuff you get at the outset, the camaraderie, the exposure to new readers, and the fun that you have.
Thanks to everyone for your candor. Sometimes the hardest things to say are the most important things to say. I appreciate everyone’s candor and want to be sure to respond in kind. Certainly, I view this as an opportunity for everyone to understand each other better. Similarly, it is an opportunity for us to make our challenges work better.
I have considered mashing up all of your comments. While threads of consensus certainly marble through the dialogue, for every point that any of us made in this, someone else made a completely valid counterpoint. Might getting us all to agree be like herding cats? Probably. If it wasn’t my busiest two weeks of the year, I would do a mashup. But, alas, I’d like to see my wife tonight.
If one consensus could be drawn from the comments thread, it would be that we should have multiple methodologies to determine a winner. We had been moving toward that, but let me make it official. Going forward, we are going to have at least two equal winners and in some cases three. Due to costs, I will probably need to do away with runners-up prizes. But, if we use multiple methodologies to select multiple equal champions, that seems like the best of all worlds. Besides, let’s face it … in any challenge, especially this Iron Foodie, there are a lot of excellent submissions that deserve to win. So, if we provide for multiple channels to become winner, we will merely be acknowledging the good work of several deserving challengers, all to be considered equal winners. Next year, we won’t be selecting an Iron Foodie. We will be selecting a few Iron Foodies. Going forward, we will use at least 2, if not 3, of the following methodologies for every contest that includes an open poll. If we don’t have an open poll on a given contest, then we will use just one methodology.
1. Winner chosen by internal contestant vote.
2. Winner chosen by poll.
3. Winner chosen by us, a partner or independent judge(s)
Question: For those who think we should do away with the open voting, does it go far enough for you if we make the open voting winner equal to the winner selected by the challengers or a judge? Or, do you think we should make the open voting winner the runner-up?
Some Guidelines for me going forward. I had a few “a ha” moments reading through the comments:
1. Make sure that the challenge is as challenging as possible, so that everyone can flex their creative muscles. We did that with the Iron Foodie, but many of our previous challenges have been open ended and not conceptually challenging. What is clear from reading the comment thread is that many of the bloggers enter challenges for the challenge, not the prize.
2. Always ship the goods at the outset of the challenge (which we have always done anyway), so that everyone wins in a small way, at a minimum. This is important.
3. Make sure that our messaging is 100% consistent top-to-bottom. The Foodie Blogroll and I had extensive planning and implementation dialogue. The implementation part took an incredibly large amount of time and excruciating attention to detail. FBR and I did mess up one detail that turned out to be very important to you all … where the poll was to be hosted. FBR and I always intended for the poll to be on my site, but after a few of you pointed out the specific emails, it is clear that we did miscommunicate that to you. We are very sorry for the mixed signals.
4. We need to explore improved technologies for the polls in 2011. Some mentioned ISP blockers (something that is completely over my head) and that you can vote multiple times from your smart phone because the IP changes. I didn’t realize these things exist. Next year we will explore how to tighten up the technologies.
My Thoughts Different Methods of Voting
Open Voting
Most of the time, the “right” winner is selected. And, that is looking to be the case in this Iron Foodie challenge. As of this writing, La Buona Cucina is in the lead and, unless something changes quick, will win. The quality of submissions in this challenge was extremely impressive. La Buona Cucina is certainly a “right” winner. It wasn’t the recipe that I voted for, but it is an amazing recipe. I can’t imagine anyone arguing with a straight face that La Buona Cucina isn’t a plausible “right” winner. I do feel that while this doesn’t prove that the poll is fair, it does slightly redeem its status as a reasonably decent arbiter of best submission. I hear you loud and clear that it can be manipulated, but it is also not woefully flawed. It is somewhere in between.
After thinking about it for a bit, and even considering doing away with the open voting polls, I will be straight up in that the open voting poll is a very important part of the contest for us. For all the time and money that we put into the challenges, the poll is one of the important benefits to us. (to be clear, building relationships and trust with you all is the paramount goal for me, and that is why I want to make sure that we evolve so that this works best for all). While I will diminish the poll’s prominence, they do need to stick around. To be clear, we have never and will never do a poll where visitors are encouraged to come back daily or vote as many times as they want. I am committed to running contests that reward excellent work … and a repeat-visitor type poll is repugnant to me and completely off the table. We have never done that and never will.
Winner Selected by Marx Foods
We did this a fair bit in the beginning, but it is a lot of work and over our head. That’s why we have avoided it lately. But, maybe we need to suck it up. I know you are thinking: how difficult can it be? I can tell you with authority that it is agonizingly difficult. When a voter or a contestant judges the recipes, they can judge based on their own personal tastes or just on whim. Since we are the sponsor, we can’t do that if we want to maintain credibility. We have to pore over the entries. We need to weigh the complexity. We need to understand the cuisine, the technique, all of the ingredients. Frankly, none of us here are qualified to do that. There is no doubt that we know food really well. But, we principally know food from a production and logistics standpoint. Yes, everyone on my staff cooks. And, we understand the products. But sooooo many of you are sooo much more knowledgeable about cooking than we are. We aren’t professional chefs, expert judges or good food bloggers so we feel slightly awkward in the role of recipe judge.
Winner Selected by a non-Marx Foods judge
We do this from time to time and I think it works well. But, there are downsides to this as well. First of all, I would regard this as the least transparent method of judging. I completely trust the blogger judges that have helped me judge contests, but it is totally possible that they just threw the contest to their friend. No one would know. And, frankly, that would be their prerogative. I have considered having a panel of blogger judges, but the issue that I run into with that is the cost factor. Whenever we have a blogger judge, I always hook them up with something delicious from our store. The more judges the more expensive it gets … and eventually it could become cost-prohibitive. Am I wrong in assuming that I need to compensate judges for their time? Are any of you willing to judge contests for the prestige or maybe a token gift? Let me know. And, keep in mind that I am always looking for bloggers to partner with on contests. So, speak up if you want to do one.
A weighted average
We could create a weighted average, drawing in votes from many sources and assigning values to them. We have tried this and it worked in the past. My only concern is that it wouldn’t mitigate anyone’s concern about whether the open voting portion corrupts the results. For that reason, and the fact that weighted averages can get complex and detailed, I am inclined not to go that route.
Multi-round Contests
I think I want to get more into this next year. In fact, I already have a challenge like this planned. We would do a multi-round contest where a different methodology is used each round. So, in a three-round contest, the first group of challengers would be selected by open voting, then the next round would yield a smaller field via an independent judge and then I would choose the winner in the last round. Something like that. Certainly, we would not do multiple rounds of open-voting only contests.
In Conclusion
Everyone did certainly go all out for this challenge and developed some amazing recipes. And, despite the misgivings about the open poll, my reading of everyone’s comments and emails is that this was an overwhelmingly positive experience. I hope I am right. And, if I am not, well I hope that the changes that we make improve the collective positivity that these challenges are supposed to facilitate!
I hope you view this dialogue as ongoing. I do. I’d love to know what everyone’s thoughts are on the above. Please weigh-in again in the comments below. Or, reach out to me later. I am grateful for everyone’s candor, as it is helping us to be better. Your candor is the greatest gift you could give me. I am building this business for the long-haul, and I can’t do it without your honest feedback. So, please keep it coming.
I vote for 25 contests from 25 contestants – 2 a month for the year – and then 1 extra.
I could be open to that.
I agree that open voting has its place. I don’t think I would have gotten the traffic to my site that I did if it hadn’t been for people checking out all the entries.
I also agree that it shouldn’t be the entire contest. Votes from other contestants would be great, and I think that vote should happen before the general polls open so the votes aren’t swayed by the popular votes.
A third prize based on the sponsor or some third party would be great. To make it easier, focus on some particular criteria. Instead of trying to figure out what the best recipe is (and what does that mean, anyway?) make it for the recipe you’d most like to eat or the best use of the secret ingredients. Or the most unusual.
Or heck, come up with 5 criteria, five smaller prizes, and go for it. Your favorite photo, the funniest blog post. Whatever YOU want to reward. Maybe keep them all secret until prizes are announced so people don’t know what’s being rewarded until it’s all over.
As far as “paying” judges, yes, I think they should get something for what they do. How much might depend on how many entries they have to read and what you expect them to critique or evaluate. Half a cow is too much. A sample pack of something inexpensive is probably not enough. To begin with, the judges should at least get the same pre-contest goodies that the contestants get.
I can also imagine ways to create a second contest around something like this one. We all have the same ingredients, and now we all see each other’s recipes. An interesting challenge might be for people to make one of the other contestant’s recipes. Then send you a photo of the result. Whoever’s recipe gets made the most often could win something, and then the person whose result looks the closest to the original could win something as well. Not saying you should do that for this contest, but just that it would be kind of a fun add-on in the future, and popular vote wouldn’t have anything to do with it.
I can’t tell you how much I appreciate the amount of thought and effort is going into the entire process of hosting these challenges and opening this conversation up to all of us! I for one, love the opportunity to play with new ingredients more than anything. I would never have used fennel pollen or ginger salt before I received my samples. I was so excited to see fennel pollen used in a Top Chef episode this week (granted it was Stephen’s dish so ick! :P) and I thought, “Hey, I could do that!” The winter squash challenge pushed me to make something I never would have either, so I’m definitely stretching my food boundaries.
That personal growth is why I love the challenges so much. After seeing the level of other entries, I upped my own game. I don’t care how winners are chosen, though I can see the validity of some of the complaints. I also haven’t seen too much of an increase in blog traffic, but I’m not really one to pay attention to that anyway. For me, it’s all about the fun.
A very thoughtful and reasoned post – one that I have to read over again to fully absorb – but I hasten to comment because (I think I speak for everyone) we didn’t care where the contest was ‘hosted’ (in fact it should be on your site so you get the peripheral benefit), but that we thought it would only be open to FBR members. It was not a question of where but who.
I think Donna comes up with some good ideas to explore — and if La Buona Cucina wins, I’ll applaud. I commented on her blog when I visited that it was a dish I’d really like to make.
Thanks again for all the time and energy you’ve put into this dialogue. I think it’s done a lot for the food blogging community. It’s certainly stirred the pot! 🙂
One other idea, which is something we implement with The Royal Foodie Joust, is that results can not be viewed until AFTER you have voted. That way there is no swaying due to the popular vote. Justin, you are really to be commended here – you have really gone out of your way to listen and take to heart the concerns of the blogging community. I am listening too, and this has all been very helpful in terms of moving forward with contests we hold at FBR. There are some big things in the works for our members this coming year, and having an open dialogue is very important. Thanks everyone for your comments.
I think that trying to make open voting equal for everyone would be impossible. It is what it is. I think the best solution is one that was already implemented for this contest – have a separate vote by the contestants or by some other select group.
You’re still not necessarily getting the vote for the best recipe, since “best” is in the eye of the beholder. But it would be more fair for people who have a smaller fanbase.
Regardless of how it moves forward, this contest was so much fun and such an exciting thing to look forward to when the box arrived on the doorstep. When I dumped out the ingredients, I was in awe. What great variety! There was much that I’d never worked with, and I loved plotting out my plan.
I just want to reiterate what I said in the last comment thread, in that I read and re-read every entry in the contest, even if I didn’t comment on all of them. I’ve never been one to comment unless I have something solid to say; I don’t say comments like ‘Yum!’ or something ubiquitous like that. Everything looked delicious.
It was an excellent experience. Thank you Justin, and Jenn for the opportunity, and congrats to the winners. I look forward to hearing who won the internal vote!!
Justin and Leftover Queen/Foodie Blog Roll:
Thank you so much for everything! I went into this with the full intent of having fun, and I did! The ingredients were wonderful to work with, and I had a ball using exotic ingredients in my cooking, which is what a true chef should be doing. I really debated whether to participate in this contest-little old’ home chef me, felt I could not compete with the “big guns” of cooking. You gave me a chance, you boosted my confidence, you gave me the tools to grow as a chef.
Sure, all of us would like to have won, but I was thrilled to be part of the twenty-five (honestly!), and I was honored to be amongst such other wonderful chefs. To me this was all good.
I do like your ideas for next time. Perhaps have a popular vote as one prize-almost like the People’s Choice award, and then a judged vote for the 2nd prize. Or…why not break it down into categories: Offer smaller prizes spread out amongst four or five categories: Main dish, soups and stews, appetizer, breakfast dish, and dessert, or something like that. Whatever you decide will be fine, because after all, this was your contest, your ingredients, your ideas. I would hope that your contestants could/would be professional enough to realize that and not complain when things do not go their way.
Donna – I LOVE your idea of a second round contest.
Joanna / Kate – Stoked that you are having fun and taking it to the next level.
Joan / Jenn – It is an important dialogue and I thank you and everyone else for having it with me. It helps make us better.
Jules – You say that you are a small blogger … well we are a small business. You are in good company.
There’s an added benefit from this contest… I’m STILL getting people clicking through from the contest page to my site. It’s not like it was when voting was live, but I’m getting one or two an hour.
Awesome, Donna!
Justin:
I agree with Jenn, you are truly to be commended for your willingness to listen to the food blogging community and be prepared to change based on the commentary (even when it is not always positive.)
When we entered the Iron Foodie Challenge, it was for fun and to try something out of our usual comfort zone. We have had increased traffic and we have established new friends in the food blogging community which has been a wonderful side stream benefit! Regardless of the judging criteria, count us in for future contests. Marx Foods rocks!
NCAA bracket style!
Raej – that is an interesting idea. A series of faceoffs. Could be very interesting. Let me know if you can go any deeper into detail…
Denise – glad you had fun. I will always listen to the food blogging community’s perspective.
I have always had a lot of problems with open voting because I think in the end, it often does turn into a popularity contest. I felt that most acutely with Project Food Blog, where I would put my all into my posts but knew that there were bloggers out there with far more readers than I who were going to be chosen over me. That being said, I realize the irony of this given that I just won one of your contests that was based on an open vote and I’m pretty sure it was based at least partly on popularity (although I am eternally grateful and will be emailing you my address soon!). I really like your new method of having multiple ways that people can win. The internal vote is especially awesome because I think the contestants themselves know how much work goes into creating a recipe and so are really more likely to choose which one they think is the best versus one that their friend created.
For me, I really enjoy the process of creating a recipe for the contest and care more about that than the prize. I also think the exposure you get from just being in a contest is great, irregardless of whether you win or lose.
I would be a judge for nothin’! GREG
My primary idea was to make it possible/reasonable for judges to actually make the recipes and compare them that way.
Because, honestly, that’s the oddest part to me about most internet recipe contests: People vote on recipes that sound good. But they don’t know a) if they actually are good or b) if the recipe “works”.
So, say you start with 16 competitors and 4 judges. Each judge makes four dishes (which were assigned head-to-head based on…whatever) and declares 2 winners.
8 competitors advance. 4 additional judges make 2 dishes apiece.
4 competitors advance. Two of the 2nd round judges make those dishes.
Then you’re down to the final 2 and whatever combination of judges from the eight you already have on hand can make those two recipes and vote.
It’s a little complicated and provides a bit more commitment from the judges, but it would definitely be interesting.