Author: Justin

It has been awhile since we discussed voting methods and we have had a chance to try some new stuff. I’d love to know your thoughts on this.

The Ridiculously Delicious Challenge went very well. Do you agree? Since it was a multi-stage challenge, we were able to try

several voting methods and make sure that no particular method overwhelmed the results.

In the first round, we selected challengers. In round two, we did a neutered reader poll that was designed to prevent it from being a popularity contest … rather than the poll selecting the winner, it was used as a tool to advance 80% of the challengers. In round three, we let the challengers vote … they cast their vote by re-making one of the recipes submitted in the previous round. And, in the fourth and final round we did a weighted average that gave the challengers a little more than 50% of the votes, with the rest going to the Marx Foods staff.

In the next couple days we are going to announce our 3rd Annual Morel Recipe Challenge. Our initial inclination was to have two winners: one selected by reader poll and the other by an internal vote of challengers and Marx Foods staff only. 2 winners. But, we decided to do only the latter and instead let the top vote getter bestow the second prize on another challenger of their choice. Should be fun.

So … any thoughts or feedback on voting?

It’s been since August that we have had a proper photoshoot.  Life got in the way.  Our chef got married (cheers Chris & Lauren!), then we had the holiday craziness … and now we are finally getting back in the swing of things.  Next Monday & Tuesday, we’ll be in our photography studio (ahem, my little kitchen) shooting away.

Chris has been cooking for us for two years now … we get together with him about monthly.   Every shoot, I’ll send Chris a list of things that I want to work with.   Sometimes, the shoot list will be pretty straightforward: “Let’s do a dozen dishes with fresh truffles”.  Other times, I send him a challenging list of products … lot’s of uber-unusual items.  He always steps up to it. 

This month’s task list might just have been the craziest:

bianchetto truffles, dragonfruit, ground kangaroo meat, shishito, maple sugar, squid ink, kobe hot dogs, quince,  poulet rouge, truffle honey, halibut cheeks, ice cream sauces, alligator tenderloin, meyer lemon, shucked oyster meat, wild boar racks, uni, quail airline breasts, elk stew meat, and savory dishes using vanilla beans.

And, below is the menu that I received back.  Quite a wild menu.  Based on his track record, just about everything will be delicious.  Chris is the man!

Parsnip & vanilla soup with beet oil and fried capers
Kangaroo crepinette with kale, garlic and apple puree
Squid ink pasta with uni sabayon
Fresh pasta demo
Vanilla crème anglaise
Quince butter
Homemade caramel
Mint syrup
Sauteed shishito peppers with sea salt
Truffle & orange blossom honey roasted whole poulet rouge
Kobe dogs on brioche buns with kimchee and siracha aioli
Kobe dogs with yogurt, cucumber, celery, pepper bacon and celery salt
Roasted alligator t-loin with brussel sprouts and carrot puree
Grilled alligator with  grilled chicory salad
Halibut cheek  sandwich with butter lettuce and meyer lemon emulsion
Quinoa salad with tuna, preserved meyer lemon, parsley, shallots
Oyster chowder with bacon, potatoes, fennel and chives
Sous vide boar racks
Pan roasted quail over faro with pomegranate glaze and pearl onions
Quail with lettuces, radish, walnuts, lemon and an herb vinaigrette
Elk stew with red wine, mushrooms, gunciale, vegetables and olives
Elk stew with corona beans, leeks, parsnips and turnips
Vanilla infused olive oil
Kangaroo burger with camembert, onions and roasted garlic
Maple crème brulee
Dragonfruit pate de fruit

If any seattle-area bloggers want to experiment with some microgreens … we just got finished photographing about 40 varieties of new microgreens and have plenty of leftover.  if you want some, shoot me an email at justin at marxfoods dot com.  You’d need to swing through this Friday (tomorrow) between 12 & 3PM to our Lower Queen Anne office to pick them up.  If more than one blogger wants in, we’ll divide them up … but no more than the first 4 bloggers, so that everyone still gets at least 10.

We announced our ridiculously delicious challenge back in October and it has been a work-in-progress since.  Anyone who won or judged a contest from mid-October through the New Year is eligible … the count is currently around 30 challengers.  We will be adding an additional 10 challengers at our whim (and will likely give special consideration to anyone who has been and continues to be part of this dialogue about voting, hint hint)

The concept for this challenge is still crystallizing, so, please check it out below and let me know if you have any feedback/suggestions.  It is going to be a multiple-stage challenge with a number of hurdles to clear and a big prize at the end.  And, we are going to try our hardest to make it live up to its name: the Ridiculously Delicious Challenge.

Stage One – Entry Post – Marx Foods Selects Top 30

To start some time in January.  We will ask challengers to pick three items out of a list of 10-20 items (kobe burgers, smoked salmon, meyer lemons, saffron, edible flowers, etc, etc. … big stuff … not just a little bottle of truffle oil).  Challengers must answer this question on their blog (or ours if they don’t have one): What ridiculously delicious thing would you do with your chosen three items?  Note: Challengers should be sure to pick the right three items because the person who wins the Ridiculously Delicious Challenge will receive the three they picked in stage one as their prize, shipped to them on the day of their choosing. 

We pick the top 30 and send them a surprise box of 8 pantry ingredients.

Stage Two – Develop an Original Recipe – Reader Poll Selects Top 20

Challengers use 2 of the 8 ingredients to make an original recipe to post on their blog (or ours if they don’t have one).  The top 20 challengers advance via a reader poll. 

Stage Three – Makeover a Recipe – Challenger’s Choice Selects Top 10

Challengers use 3 of the remaining 6 ingredients to makeover a dish made by one of the challengers in Stage 2.  The creators of the 10 most madeover dishes will advance, even if the creator has already been eliminated in Stage 2.  If there is a tie for 10^th place, then a volunteer blogger (tbd) will break the tie.

Stage Four – Second Shipment – Mixed Selection Methods to Get Final 5

We ship two perishable ingredients, such as crabmeat, kobe steak, heirloom potato or microgreens to the remaining 10 challengers.  Challengers use 1 of their remaining 3 pantry ingredients and one of the perishable ingredients.  5 Challengers advance.  A reader poll selects two.  An internal challengers’ poll of the remaining challengers selects two (before the poll ends).  Marx Foods selects one.

Stage Five – The Final Recipe – Winner Chosen by a Panel of Bloggers

Challengers use both of the remaining pantry ingredients and the other perishable ingredient to develop an original recipe.  2-5 Blogger Volunteers choose the winner. 

Thoughts?

Is this too much?  Too complicated?  Too many stages?  Fun or boring?  Delicious or not?  Do you like the voting methods?  This is still very much not set in stone, so please let me know if you have any thoughts.

One thing is certain.  We are touching on an issue that is of importance to food bloggers.  Judging by the length and passion of the responses, this is an important issue for you all.  I believe in the openness of the blogosphere and I am glad that we can explore this together to make it better.

A lot of the feedback in the comments was certainly positive.  Thanks to all of you who had kind words and encouragement.  Before I dive into the more technical aspects of voting, I want to let you all know that I am heartened by what seems like the majority of you who feel like a winner regardless.  You confirmed that these contests aren’t just about the prize.  They are also about the challenge, the free stuff you get at the outset, the camaraderie, the exposure to new readers, and the fun that you have. 

Thanks to everyone for your candor.  Sometimes the hardest things to say are the most important things to say.  I appreciate everyone’s candor and want to be sure to respond in kind.   Certainly, I view this as an opportunity for everyone to understand each other better.  Similarly, it is an opportunity for us to make our challenges work better.

I have considered mashing up all of your comments.  While threads of consensus certainly marble through the dialogue, for every point that any of us made in this, someone else made a completely valid counterpoint.  Might getting us all to agree be like herding cats?  Probably.  If it wasn’t my busiest two weeks of the year, I would do a mashup.  But, alas, I’d like to see my wife tonight.

If one consensus could be drawn from the comments thread, it would be that we should have multiple methodologies to determine a winner.  We had been moving toward that, but let me make it official.  Going forward, we are going to have at least two equal winners and in some cases three.   Due to costs, I will probably need to do away with runners-up prizes.  But, if we use multiple methodologies to select multiple equal champions, that seems like the best of all worlds.  Besides, let’s face it … in any challenge, especially this Iron Foodie, there are a lot of excellent submissions that deserve to win.  So, if we provide for multiple channels to become winner, we will merely be acknowledging the good work of several deserving challengers, all to be considered equal winners.  Next year, we won’t be selecting an Iron Foodie.  We will be selecting a few Iron Foodies.  Going forward, we will use at least 2, if not 3, of the following methodologies for every contest that includes an open poll.  If we don’t have an open poll on a given contest, then we will use just one methodology.

1. Winner chosen by internal contestant vote.

2. Winner chosen by poll.

3. Winner chosen by us, a partner or independent judge(s)

Question: For those who think we should do away with the open voting, does it go far enough for you if we make the open voting winner equal to the winner selected by the challengers or a judge?  Or, do you think we should make the open voting winner the runner-up?

Some Guidelines for me going forward.  I had a few “a ha” moments reading through the comments:

1.  Make sure that the challenge is as challenging as possible, so that everyone can flex their creative muscles.  We did that with the Iron Foodie, but many of our previous challenges have been open ended and not conceptually challenging.  What is clear from reading the comment thread is that many of the bloggers enter challenges for the challenge, not the prize. 

2.  Always ship the goods at the outset of the challenge (which we have always done anyway), so that everyone wins in a small way, at a minimum.  This is important.

3.  Make sure that our messaging is 100% consistent top-to-bottom.  The Foodie Blogroll and I had extensive planning and implementation dialogue.  The implementation part took an incredibly large amount of time and excruciating attention to detail.  FBR and I did mess up one detail that turned out to be very important to you all … where the poll was to be hosted.  FBR and I always intended for the poll to be on my site, but after a few of you pointed out the specific emails, it is clear that we did miscommunicate that to you.  We are very sorry for the mixed signals.

4.  We need to explore improved technologies for the polls in 2011.  Some mentioned ISP blockers (something that is completely over my head) and that you can vote multiple times from your smart phone because the IP changes.  I didn’t realize these things exist.  Next year we will explore how to tighten up the technologies.    

My Thoughts Different Methods of Voting

Open Voting

Most of the time, the “right” winner is selected.  And, that is looking to be the case in this Iron Foodie challenge.  As of this writing, La Buona Cucina is in the lead and, unless something changes quick, will win.  The quality of submissions in this challenge was extremely impressive.  La Buona Cucina is certainly a “right” winner.  It wasn’t the recipe that I voted for, but it is an amazing recipe.  I can’t imagine anyone arguing with a straight face that La Buona Cucina isn’t a plausible “right” winner.   I do feel that while this doesn’t prove that the poll is fair, it does slightly redeem its status as a reasonably decent arbiter of best submission.  I hear you loud and clear that it can be manipulated, but it is also not woefully flawed.  It is somewhere in between.

After thinking about it for a bit, and even considering doing away with the open voting polls, I will be straight up in that the open voting poll is a very important part of the contest for us.  For all the time and money that we put into the challenges, the poll is one of the important benefits to us. (to be clear, building relationships and trust with you all is the paramount goal for me, and that is why I want to make sure that we evolve so that this works best for all). While I will diminish the poll’s prominence, they do need to stick around.  To be clear, we have never and will never do a poll where visitors are encouraged to come back daily or vote as many times as they want.  I am committed to running contests that reward excellent work … and a repeat-visitor type poll is repugnant to me and completely off the table.  We have never done that and never will.

Winner Selected by Marx Foods

We did this a fair bit in the beginning, but it is a lot of work and over our head.  That’s why we have avoided it lately.  But, maybe we need to suck it up.  I know you are thinking: how difficult can it be?  I can tell you with authority that it is agonizingly difficult.  When a voter or a contestant judges the recipes, they can judge based on their own personal tastes or just on whim.  Since we are the sponsor, we can’t do that if we want to maintain credibility.  We have to pore over the entries.  We need to weigh the complexity.  We need to understand the cuisine, the technique, all of the ingredients.  Frankly, none of us here are qualified to do that.  There is no doubt that we know food really well.  But, we principally know food from a production and logistics standpoint.  Yes, everyone on my staff cooks.  And, we understand the products.  But sooooo many of you are sooo much more knowledgeable about cooking than we are.  We aren’t professional chefs, expert judges or good food bloggers so we feel slightly awkward in the role of recipe judge.

Winner Selected by a non-Marx Foods judge

We do this from time to time and I think it works well.  But, there are downsides to this as well.  First of all, I would regard this as the least transparent method of judging.  I completely trust the blogger judges that have helped me judge contests, but it is totally possible that they just threw the contest to their friend.  No one would know.  And, frankly, that would be their prerogative.  I have considered having a panel of blogger judges, but the issue that I run into with that is the cost factor.  Whenever we have a blogger judge, I always hook them up with something delicious from our store.  The more judges the more expensive it gets … and eventually it could become cost-prohibitive.  Am I wrong in assuming that I need to compensate judges for their time?  Are any of you willing to judge contests for the prestige or maybe a token gift?  Let me know.  And, keep in mind that I am always looking for bloggers to partner with on contests.  So, speak up if you want to do one.

A weighted average

We could create a weighted average, drawing in votes from many sources and assigning values to them.  We have tried this and it worked in the past.  My only concern is that it wouldn’t mitigate anyone’s concern about whether the open voting portion corrupts the results.  For that reason, and the fact that weighted averages can get complex and detailed, I am inclined not to go that route.

Multi-round Contests

I think I want to get more into this next year.  In fact, I already have a challenge like this planned.  We would do a multi-round contest where a different methodology is used each round.  So, in a three-round contest, the first group of challengers would be selected by open voting, then the next round would yield a smaller field via an independent judge and then I would choose the winner in the last round.  Something like that.  Certainly, we would not do multiple rounds of open-voting only contests.

In Conclusion

Everyone did certainly go all out for this challenge and developed some amazing recipes.  And, despite the misgivings about the open poll, my reading of everyone’s comments and emails is that this was an overwhelmingly positive experience.  I hope I am right.  And, if I am not, well I hope that the changes that we make improve the collective positivity that these challenges are supposed to facilitate!

I hope you view this dialogue as ongoing.  I do.  I’d love to know what everyone’s thoughts are on the above.  Please weigh-in again in the comments below.  Or, reach out to me later.  I am grateful for everyone’s candor, as it is helping us to be better.  Your candor is the greatest gift you could give me.  I am building this business for the long-haul, and I can’t do it without your honest feedback.  So, please keep it coming.

Two bloggers reached out to me to express frustration with the voting structure of the Iron Foodie Challenge. They were concerned that the winner would be chosen not by merit, but by the breadth of their social network. It is definitely a valid concern.

After organizing several recipe challenges, my view is that the far majority of the polls produce the “right” winner. When I say “right” I simply mean that it is plausible that the winner produced the best recipe in the group.

I certainly have seen recipe challenges slip into a popularity contest. I does not happen often, but when it does … it just kind of spoils the party. I remember an instance where the worst submission won the contest. So, we awarded an identical prize to the runner-up.

Lately we have been sending free samples at the outset of the challenge so everyone wins in a way … And, we have been in the habit of creating many prizes, sort of as a hedge against a runaway popularity contest. For example, in the Iron Foodie Challenge, we initially announced that there would be only one winner … but then we added three runners-up prizes. And at the last minute, because the possibility of a popularity contest was actually in the back of my head, we added a second grand-prize winner to be chosen by the 25 recipe challengers. (I have only gotten about 15 votes back, so if you haven’t emailed me your pick, please do so).

We set up our recipe challenges based on some basic presumptions … and I’d love any feedback to confirm or debunk them. The first presumption is that recipe challengers benefit whether they win or not. We do get lots of positive feedback … many bloggers have told me that their blog had record visitors during a recipe challenge. I also presume that the recipe challenges act like a little social network of themselves…challengers make new relationships with their competitors and readership nudges up. If my presumptions are correct, then wouldn’t we all want to drive maximum traffic to the voting regardless of who wins?  Because if we didn’t do voting, then there would be only 3 judges instead of the 1,400 that have already weighed in. In most cases, in theory at least, a higher number of voters should lead to a more accurate result.

I am SOOOOOOO open to ideas regarding how we can structure these challenges. We keep trying various things to balance the popularity factor, but I am actually considering doing away with voting altogether. Of course, we like the voting because it drives traffic to our site. And, I am not just saying that in terms of numbers. It helps us start relationships with more bloggers and potential customers, which is one of the main goals of our blog. But we also want you all to feel that these contests have been fun, challenging, and as fair as possible … whether you win or not.

I just kind of wanted to bring this discussion into the open. I am immensely grateful for the two bloggers who reached out to me. And, I hope that you all will share your sentiments either in the comments below or via private email to me.