The case against dot voting

10 min read

Why we're failing in our efforts to democratise inputs(+ Five alternative ideas)

Picture the scenario: you want to gather ideas from your users. You set up a focus group and give the participants five minutes to write ideas on sticky notes. You ask the group to dot vote on the ideas they like the best. Lastly, you translate these ideas into actions and everyone goes home. Hopefully you recognise this as a bad way to conduct research and reach a conclusion. So why has this, in many teams, become the go-to method of research with internal users?

Dot votes are one-sided

Dot voting doesn't tell us who strongly believes in an idea. It doesn't tell us who voted for an idea just because they had to place a dot somewhere. It doesn't tell us whether everyone who voted for the same idea understood it to be the same thing.

We don't even know who had conviction in their idea when they first wrote a post-it. Perhaps you wrote something just because you had nothing better to think of, and you had one minute left to put something — anything — on the board.

Crucially, dot votes don't tell us the opinions or strength of feeling about the things that individuals did not vote for.

Soliciting only positive votes leaves us with one-sided views. If a participant disagrees with an idea, they have to decide whether to keep quiet or risk being seen as negative by speaking up. It leaves them on the defensive.

‘There's no such thing as a bad idea!’

As designers, we crit each others' work. Having your work critiqued can sometimes feel uncomfortable, but it almost always improve outcomes. In workshops, we want to keep a positive, inclusive mood. This means we don't often leave space to critique ideas or discuss why an idea might not be great.

This omission is perpelexing given the proliferation of 'Design Thinking', a process that aims to help non-designers think and work through a design process as a designer would.

Dot voting can be a quick activity and a way to gain active participation. It may work well to prioritise a list of similar, low-stakes items, but too often it's used in medium–to–long term decision making. Let's be mindful of optimising for outcomes over efficient workshops.

Timers add unnecessary stress

Not all people are able to come up — or thoughtfully vote between — great ideas on the spot. For some (including myself), the act of performing against a timer can be a stressful, un-natural and unneccessary way to perform creative work. This stress can be exasperated when seeing what others are coming up with.

In these settings, there's often little time for meaningful discussion, clarification, and exploration of ideas as we power through the schedule.

Timers are an efficient way to manage time in a workshop, but we need to remember to optimise for better outcomes, not 'time on task'. PS. Asking 'does anyone need more time?' after the countdown has ended does not solve this problem!

It's easy to skew the results

A typical round of dot-voting gives each person 3 dots. Unless explicitly mentioned, you're usually able to:

✔ Vote for your own ideas
✔ Place all your votes on the same idea
✔ Give yourself extra votes (if you're sneaky and using an online tool like Miro)

Bias and power dynamics can skew results...

We're all influenced by each other. When we see how others are voting, we're more likely to strategically align our votes. When you see how your boss is voting, how might that influence you?

...but anonymity can skew results too

Let's imagine that 8 people have 5 minutes to create as many ideas as they can. When the time's up, each person casts 3 votes. Let's look at some possible scenarios:

Possible scenario 1:

In this scenario, 7 participants spread their votes for 1 idea each. 1 participant places all their votes on a single idea (which might even be their own idea). We have a winner! But this person has forced their idea to the top:

Possible scenario 2:

If the facilitator decides to cancel the duplicate votes, we're still no further ahead as 24 of the 40 ideas all have 1 vote each:

Possible scenario 3:

In this third scenario, 2 participants with the same agenda vote for 1 idea. A single idea now has 6 votes, but from only a quarter of the group. Without interrogation, this looks like the winning idea, but actually the idea with 3 votes from 3 people is the most democratic choice.

These scenarios can be mitigated, or at least be made visible, by using different coloured dots for each participant. It would now be obvious in the following two outcomes that the results are skewed:

Grouping themes before voting: a slight improvement?

A common way to avoid the overwhelm of many ideas is to quickly identify, and vote on themes. This seems sensible, but the danger here is that we only identify the most obvious ideas, not the best ideas. We dismiss the majority of ideas before we even have the chance to consider them!

Here, only 5 out of 29 ideas have the chance to be voted on.

After we've voted on our themes, we might end up with this scenario. Theme 3 is our winner! The workshop is a success!

At first glance, this looks like alignment. But it's possible that the winning vote was decided by a minority of the group. We’ve designed a process with the aim of democracy, and unintentionally ignored the opinions of the majority. Without an additional step, 5 of the 8 participants could be leaving the session thinking ‘I completely disagree with this approach, and I didn’t have the opportunity to state my opinion.’

What are some other options?

💡1. Use Miro's built-in voting feature

At the very least, using this feature gives facilitators the option to only allow one vote per idea. This is a simple way to mitigate one participant's voting from skewing the results.

💡2. Include downvotes in your dot voting

A simple way to add more balance to a dot voting session is to give participants 'upvotes' and 'downvotes'. One downvote cancels out an upvote. While this doesn't address the aforementioned issues of thinking against the clock, it provides a more balanced view of participants' views.

💡3. Solicit ideas asynchronously

Another option is to adopt part of the method Annie Jukes advocates for. Rather than using workshops to Discover, Discuss and Decide, we can 'Discover' asynchronously:

A week before the workshop, give participants 48 hours to submit their 3 best ideas. These can be submitted in a private online form or direct message to the facilitator. This allows us time to consider and submit only our best ideas without the stress of 'thinking on the spot' and without being influenced by other participants.

One or two days before the workshop, we can require participants to rank each idea against what good looks like. For example, rank each idea from 'most to least likely to achieve the stated goal', or 'rate each idea from highest to lowest priority'. These options could be ranked manually in a form or DM, or by using an online tool like Menti.

💡4. Run a live poll

We might run a survey or poll to test an assumption, gauge sentiment, or to rank each participants' self-selected best ideas that were submitted before the workshop. We may improve the quality of these votes by adding a text field for respondents to explain their thinking, and use these as prompts for discussion. This doesn't have to take a long time. Tools like Menti allow us to do this on the fly.

Tip: Also consider if it's best to only share the top 3 voted ideas to prevent any individuals feeling bad about coming last.

💡5. Rank suggestions

Rather than giving each participant 3 votes each, ask participants to rank each idea from best to worst. Again, tools like Menti allow us to create quick ranking polls on the fly. This won't work if there are dozens of ideas to vote on, so having participants submit their best one, two or three ideas (depending on your group size) will keep things manageable. The benefit of this approach is that we get a balanced view of each idea. This can prompt valuable follow-up discussions to delve into differing views.

One size doesn't fit all

Maybe dot voting is the perfect activity if we're trying to decide on a low-stakes problem, such as what game to play at the next social event. However, if our objective is to prioritise solutions that the team will spend time developing, or will lead to more work for some team members, we should consider spending a little longer on understanding, critiquing and testing the merits of each possibility.

Some questions that might be useful to consider:

Might the results of the voting lead to more work for any team members?
Are the outcomes likely to influence short-term, or long-term decisions?
What might the quietest / most difficult / newest / oldest team member be thinking?