If you have never tried what Mark suggests, I (as well) suggest you try it. However, I think an important point must be made to put the article in context.
A Listening Lab is a very useful tool in terms of design and usability. Mark notes that this is research, but equates a Listening Lab mostly to usability. Not that usability isn’t research, of course, but it is the type of information he is getting/looking for that makes me see it as more than just usability.
I see two things happening in a Listening Lab.
Current Task Analysis (CTA). What do users do now with the site/product? However, a Listening Lab is CTA out of context to how the user actually uses the site/product. Honestly, I don’t know if this is a good or bad thing. I prefer, as you might guess to do CTA in the context of the user.
Exploratory Usability. There are four main categories (as well as 3 main methods) of usability (as of today anyway): Exploratory, Assessment, Validation, and Comparative. Generally speaking they fit into a hierarchy or range:
do-before-designing to do-after-users-are-using-your-product-for-a-while.
Mark is in the first category with a Listening Lab. It involves a high-level of interaction between the facilitator and the participant. It is done early in the design process. It usually involves getting more than just the generic (yet important) demographic information.
Exploratory testing is absolutely a must and yet is often skipped by most people (for many, often valid reasons). Most people do either Assessment of Validation. Well, most good designers will do at least Assessment, and most everyone else only does Validation, if anything.
For Assessment and Validation testing you should absolutely have tasks set up that match your Future Task Analysis. You’ve designed how people will use the new product, you should test to see how users will fare. For Comparative testing I can see value for using a Listening Lab method or a scenario-based method.
My question though is when is it suggested to do a Listening Lab during a project, and how many users is enough?
I have an answer for the first question, but I don’t see it answered in Mark’s article. I do not have an answer for the second question. If you’ve been in this business for more than two days, it’s an argument that is best left to researchers with more time and talent than me.