Part I: All Jury Consultants Are Not Created Equal: Do you have the Right Consultant for Your Research?
Valid and reliable jury research is more than just running focus groups. In this and the next issue of Insights, we discuss the importance of choosing a jury research consultant whose methodological strengths match the goals for your research and the questions you need answered. We will also explore additional considerations for your consultant search to ensure you get answers to your questions and that you can have confidence in those answers.
The Dilemma: Choosing the Right Jury Consultant
Your client has asked you to secure three bids for conducting a mock trial. The proposals seem similar in design, description of the work product and cost. So which company do you choose – the one who is local, the one who claims more experience in this type of litigation or the one who is lowest in price? While all of these may be valid decision criteria depending upon your goals, what matters most in this scenario are the results of your research. Only if you work with a consultant who has the background and expertise to properly design a methodologically sound study will you be able to have confidence in the results.
With more consultants hanging up their shingles, how do you decide if you’re choosing the right consultant? There are critical differences between consultants’ qualifications, philosophy regarding proper methodology and the level of analysis provided. You should expect that the methodology and analysis used will provide you reliable answers to your research questions – i.e., that you will learn what you hope to learn.
It is critical to keep in mind that different methods answer different kinds of questions. If you’re using the wrong kind of method for the questions you’re trying to answer, you can’t obtain reliable results for your research. How each method is designed and executed makes a huge difference in whether you can have confidence in your research results. Therefore when you are choosing a consultant, you are also choosing a methodology that may or may not meet your goals.
Prioritize What You Want to Learn
Why do you or your client want to conduct jury research? There are many answers – e.g., to test one specific issue, test your causation theory, test your liability case, determine the range of potential damages, see if the evidence is such that jurors want to award punitive damages and/or understand the characteristics of jurors you want to deselect at trial. You may have several things you want to know.
Your consultant’s priority should be to help you formulate what you really want to learn. If you are not clear about what you want to learn, then any result will get you there – but you may not like where “there” is. It’s the same as the old adage: “If you don’t know where you are going, any route will take you there.” Research is less beneficial if you select the “throw-up-against-wall-and-see-what-sticks” approach.
Instead, your consultant should be helping you answer questions related to what you want to learn, what your goals are and what research questions you want jurors to answer. It may take some conversation to formulate and capture the essence of what you want to learn, but your consultant has to have the expertise to design a test that is methodologically appropriate for gaining insight into the answers to your questions.
- Example #1. If you want to learn about the various ways jury-eligibles might react to a particular set of facts in your case, conducting a survey of that population would be a poor methodological choice for this objective. Why? Because surveys can only give you information about what you already know. It cannot give you insights about new or additional reactions you hadn’t already anticipated. However, if your goal is to learn about the percentage of the jury-eligible population who believes one particular way or another about a certain set of facts, a survey could well be an excellent methodological choice to answer those questions. In contrast, methodologies such as focus or deliberation groups are well-suited to explore the various ways people might react to key facts in your case. They offer opportunities to think about issues in an entirely new way.
- Example #2. Suppose your primary goal is to explore the range of damages you are likely to face at trial, but you have a limited budget. You might think having only one or two focus groups would be the best choice to curb costs. In this scenario, an LI consultant would advise you of the importance of having a third group as part of the design (e.g., having three groups of eight jurors, rather than two groups of 12), which ensures the project stays within budget, ensures a reliability measure is in place and avoids a reliance on “outliers.”
Well-designed jury research can help you better understand your case and to do so from the perspectives of the people who will actually decide the case. Well-designed jury research can give you insight into how people similar to jury-eligibles may react to your case’s fact pattern and to the success of your themes (or your opponent’s anticipated themes).
Jury Research is Not a Crystal Ball
Jury research is not designed to measure the likelihood of your victory at trial. One of the cardinal rules of any jury research is that it is not designed to be predictive of the ultimate trial outcome. From a methodological standpoint, it’s just not possible to design a study that replicates all of the variables that go into a trial; there are just too many things that happen between the test and the trial (e.g., judicial rulings that change the direction of the case, plaintiffs change their strategy, etc.). So be very skeptical of any consultant who claims he or she can predict your trial outcome from jury research. The same goes for damage awards, as jury research can only provide you information on a range of damages that these mock jurors find under this fact pattern, tried under these conditions/theories, and most importantly, their rationale for doing so. But, for the reasons we’ve just outlined, it is not designed to predict actual damages figures at trial.
Education and Experience Matter
In addition to evaluating a consultant’s ability to help you identify your research goals, look for someone who has the background, expertise and training to design research that is suited to addressing those goals and therefore answer your most important questions. You should ask about the educational backgrounds of the consultants running your project and about the level of field experience they hold. Do they have an advanced social science degree, a law degree or both? How many projects have they managed or assisted with?
Why do the answers to these questions matter? Rigorous educational and professional training enables consultants to bring to their analysis the psychological and communication insights that go beyond raw data on a page. They have the knowledge and experience to design and conduct research you and your clients can rely on. At Litigation Insights, our consultants hold Ph.D. or Master’s level graduate degrees in the social sciences, communication and/or the law, plus extensive field experience.
You should expect a consultant to design the research in a way that is best suited to your goals. If you merely hire consultants to execute a project, then you will overpay for event managers, rather than achieve valuable results from reliable research.
When you hire a jury research company, you are looking for a consultant team who can design, conduct and analyze applied research. Jury research is more than just running a project – it is learning your case and connecting you with insights from the people who matter most – representative mock jurors.