Russian Babushka or Matryoshka Doll inside the other dolls.

When is Discovery done?

Russian Babushka or Matryoshka Doll inside the other dolls.

When is Discovery done?

It’s uncomfortable to be in the Discovery phase of a large project without an end in sight. One unknown leads to another, like a set of diabolical Russian dolls, and you become circumspect, even though you know your job is to keep digging.   Still, I’ve had managers ask me the legitimate question: “How will you know when you are done?”

A thorough Discovery will reveal:

    • who all your users are
    • their context, goals and pain points
    • how frequently they interact with the current system
    • their level of expertise
    • their workarounds
    • the other systems they use and how those systems interact (or don’t)
    • how the system is designed to work

The Russian doll part here often involves figuring out how the system is designed to work, and why that is not happening.  Users can become so accustomed to dysfunctional or nonfunctional  systems, that they cease questioning them, like a tree forming a protective gall over an irritant.

For example, users might face frequent character count errors if a form is unable to calculate running character totals.  They will then have to click “Submit” to become aware of the overrun.

Why?  – “Because this part of the interface was deprioritized due to higher-visibility fixes.”

Why not fix it anyway? – “Because the code base is antiquated.”

Why are you not updating it? –  “Because it has to integrate with a different legacy system.”

Why not replace that system?  “I don’t know, they were thinking about that a few years ago.”

Meanwhile your boss is yelling: “Hey you!! ! Is that Discovery done yet!??”

You could shorten your discovery time by simply listing “form fields with character counts are subject to frequent errors”.  In that case you are just a scribe, jotting down symptoms.  If you want to provide value, you’ve got to discover the mechanism that causes user pain.  Only then you can intelligently suggest solutions.

There are other approaches to “done-ness”, one is saying that you are never done, the concept of Continuous Discovery .  Teresa Torres talks about doing Discovery with a focus on opportunities rather than problems.  She has created an Opportunity Solution Tree model to do this.   This adds a dimension of complexity, because driving towards an opportunity’s outcome requires a theory – a theory of how things could be.  To prove a theory you need run experiments, to answer questions like:

  • Which elements of the existing system will be useful?
  • Which broken ones do you fix?
  • When do you add in new ones?
  • Will an improvement degrade another area of functionality?

One can see how this goes beyond traditional Discovery, which is essentially mapping  the territory of “what is”.   As such,  it requires continuous adjustment and effort.

Jeff Gothelf, in his article “When is Amazon Done?” suggests that the way to stay focused when moving this many levers is to keep the customer in mind at all times.  Eliminate opportunities that don’t directly help the end-customer.

Amazon was built on this.  Although their discovery method (not “methodology” – pet peeve) may be though of as continuous, the way they think about adding value is discrete.  It’s not through gradual improvement that Amazon created Prime next-day shipping.  They didn’t say “Lets get it to around 46 hours per shipment and see how people react. It’s more likely they said to themselves “What’s the one biggest thing we can provide that will create customer loyalty and what do we have to do to make that happen?”  It’s risky to bet so much on one opportunity, but less so when customer value is the North Star.

On a much simpler level, I’ve found that Discovery starts to “feel” done, when your team can start answering their own questions.  As part of a shared discovery team, for a UX’er to hear the tech lead say something like: “Almost correct, Roy.  Actually a regional manager can re-assign an account, but only a district supervisor can delete one…..” is a sweet sound, and means you are close to reaching the shared understanding we all seek in the Discovery phase.


abstract computer users with one highlighted with a red bullseye

The Paradox of the Power User

abstract computer users with one highlighted with a red bullseye

The Paradox of the Power User

If you’ve been doing enterprise-level UX for awhile you may run into the phenomenon of finding a subset of users who LOVE a system that almost all other users can’t stand and can barely use. Behold the power user. The blessed frequent users who use the system the most, who are the most adept and knowledgable.

Why on earth wouldn’t you listen to these angels, when the vast majority of your interview and testing subjects are discouraged, angry, or even worse: resigned? The answer is that in hard-to-use systems, adept users have created a mental model that has skewed from the norm.

A mental model, roughly, is the internal representation people create about how things are structured and how they work. I know that sounds like “everything ever”, so let’s use a specific example: a modal window vs. a browser interface. Because we have a mental model about how a modal window works – it displays ancillary information and then is dismissed or goes away – we are comfortable clicking on the little “x” (sometimes it is very little) and closing it. A person who had never used a browser might treat those windows equally, they might be afraid to close the modal – after all closing a browser shuts down the entire program.

Systems that seem just baffling, and I’ve worked on a few, go beyond simple usability hurdles such as being slow, not showing status, or being visually cluttered. Instead they can present a “sinister” mental model.

I recently worked on a website that turned the idea of user registration on its head. The site enabled customers to look up stuff they had purchased and buy additional options. This system required a username, password, and a unique token — a number that was printed on paper included in the shipping materials. Multiple items could have this same number if included in the same shipment, or a number might apply to a single item.

It gets worse: a number could apply to one user, or be shared by multiple users. Other users could “borrow” numbers.  For each function in this system, the user was re-prompted to enter the number. Compare this to the standard mental model of username + password = show me my stuff. For most users, the only way to be successful was to start the process by calling support.

A good number of power users had no issues. They were on the site all day, every day and had their own spreadsheets and order of operations.  They knew workarounds, and people to call in the company outside of standard support. By doing specialized tasks over and over, they had normalized a sinister mental model.   And some did not want change, because their normalization was hard-won, and gave them a sense of job security. You tend to see this sometimes with SME’s, admins, and coders on old legacy systems.

As a UX’er you can still get a great deal of value from these users if you study their workarounds, their “cowpaths”, their cheat sheets and secret methods.   It’s as if power users are doing continuous user testing for you. It’s smart to leverage their knowledge in order to help the next version of the product embody a more user-friendly mental model.