Part II: All Jury Consultants Are Not Created Equal: Do you have the Right Consultant for Your Research?
All clients want to avoid a false positive result as they adversely affect counsels’ confidence in case strategies going forward. How can you ensure your consultant’s methodology minimizes the likelihood of such results? The following are some of the key questions you should ask prospective consultants to increase the likelihood that you will receive reliable results,* subject to the caveats that limit all jury research (e.g., the theory of the case may change, interim judicial rulings can change the scope of the research, etc.), as well as the employment of an appropriate level of analysis to address key research goals.
What Method(s) Is the Jury Consultant Using to Customize Your Project?
- Why This Is Important: As we discussed in the March 2012 issue of Insights, different methodologies answer different kinds of questions. Accordingly, it is important for you to know what type of data (quantitative and/or qualitative) will be collected and how the data will be analyzed to address your research questions. Regardless of the type of data collected, the conclusions drawn from the research should be grounded in the research data. You should be able to tell where conclusions come from. During each stage of the research, confirm that the type of data being collected will give you information to address your research goals.
- Consequences of Poor Practices: Selection of the wrong methodology, or improper analysis of collected data, means you cannot trust your results. As they say, “You can’t use a hammer to tighten a screw.”
- Questions to Ask: Will the consultant be solely using qualitative methods (narrative data)? Will the consultant employ quantitative methods (numerical data)? Both? Which methods will be used to answer each particular research question?
- Why This Is Important: Your panel of mock jurors should reflect the demographics and sensibilities of jurors you would see in your actual jurisdiction, anticipating as accurately as possible the effects of hardship excuses and cause strikes on jury composition. This is critical because the more representative your sample of mock jurors, the more confidence you will have generalizing your results to the larger pool of jury eligibles.
- Consequences of Poor Practices: While they may be less expensive, to the extent your consultant uses recruiting methods that yield a non-representative distribution (e.g., administrative assistants from the law firm, unemployed individuals from an employment agency, people who respond to a newspaper/Craigslist classified ad, etc.), these methods make it difficult to generalize from your results.
- Question to Ask: What methods will the consultant use to recruit your mock jurors? The answer to this question should reassure you that the consultant employs sampling techniques that result in as representative a sample as possible of virgin** mock jurors, which mirrors the composition of a jury in the actual trial venue.
Number of Jury Panels
- Why This Is Important: If your goals include exploring a potential range of damages, developing a juror profile and/or testing themes in an adversarial format, you want to ensure that you have an appropriate number of jury panels as a key reliability measure for receiving feedback you can trust.
- Consequences of Poor Practices: While cost is always a consideration, using only one jury panel to deliberate doesn’t provide the reliability measure necessary to control for the unique characteristics of any one particular group. Instead, it is essential to have a large enough sample of jurors (and groups) to diminish the influence of a particular group’s idiosyncrasies.
- Question to Ask: Given your goals, how many groups does the consultant recommend you use to provide a reliability measure for your results?
Number of Consultants Used
- Why This Is Important: To maximize your investment in obtaining feedback from mock jurors, you should know who will be supervising your jury panels. Each panel should be monitored and debriefed by a qualified consultant to ensure they obtain information that is responsive to the research goals, as well as to provide the opportunity to ask more specific questions. Having a consultant per group to moderate a panel minimizes the risk for missing data.
- Consequences: When one consultant runs multiple groups (which some consultants do to reduce costs), the trade-off is that all the groups have to finish deliberating before the jurors can be reconvened into one room for a full group debriefing. When this happens, the group that finishes early sits idle until all groups are done.
- Question to Ask: How will the consultant monitor and moderate each jury panel?
- Why This Is Important: It is important to protect the attorney work product privilege of the jury research. While no consultant can control what each juror does after he/she leaves the research facility, your consultant of choice should take key steps to protect the confidentiality of your project to the full extent possible.
- Consequences of Poor Practices: The most obvious consequence is a breach of confidentiality by a juror. The prevalence of mobile phones means jurors could surreptitiously record presentations or take photos of exhibits.
- Questions to Ask: Does the consultant ask mock jurors to, on arrival, sign stringent confidentiality agreements, then review their terms in orientation and reaffirm them after deliberations? What does the consultant’s confidentiality agreement specifically say? As a potential client, you have the right to review the agreement and to suggest additional language as needed. What is the consultant’s practice if a protective order is in place? Lastly, how does the consultant handle mock jurors’ possession or use of mobile phones during presentations and deliberations?
What Type and Level of Analysis Will Be Used by the Jury Consultant?
Data Analysis and Recommendations
- Why This Is Important: What are you looking for in a jury consultant? Are you interested in a listing of the research results – a report of what happened at your project? Or, are you looking for a consultant to explain not only what happened during the research (reporting of results), but also what the data means and how this information can inform trial strategy going forward? Particularly if you are interested in assistance beyond just a reporting of results, it is critical to understand the level of analysis and synthesis of the data you will receive. You also should know if the consultant will provide recommendations for enhancing your communication with jurors.It is not difficult to report results. And sometimes, that is all that is needed. What is more difficult is to provide a sophisticated level of analysis that clearly explains the vulnerabilities and strengths of your case, from the jurors’ perspectives.
- Question to Ask: What type of analysis will you receive from your qualitative and quantitative data? That is, does the final report include an analysis and synthesis of the information provided or is it simply a listing of the frequency data? Does the analysis also include a statistical analysis? With regard to quantitative results, if you hope to determine the types of jurors you want to deselect at trial, ask consultants what statistics they use to develop juror profiles for jury deselection.*** Again, depending on your goals, it is critical to learn if the report will go beyond a mere listing of results.
- Why This Is Important: The key challenges and strengths facing your case have been identified. Where do you go from here? Generally, these are not purely “legal” issues, but rather issues of effectively communicating and connecting with jurors. You can have all the “facts” lined up and on your side, but if the jurors don’t understand them or reject your framing (your story) of the case, they are likely to reject your case. Creating case themes and strategies that leverage psychological and communication principles can help to maximize your case story and strengths while minimizing your vulnerabilities.
- Question to Ask: Does the consultant’s analysis include thematic/story recommendations; does the consultant provide this level of thematic recommendations? If you aren’t sure, feel free to ask for a redacted sample report. If this isn’t what you are looking for from your consulting company, then a basic listing of results may be appropriate for your goals. Either way, it is important to know what to expect to ensure your goals are being met.
What should you be mindful of when shopping for a jury research consultant? Beware of prices too low or too high or too ambiguous. You should know what you are receiving and how much it is going to cost you. Therefore, when seeking three proposals, make sure you receive answers to the questions noted above and that the answers you receive meet your satisfaction. As a result, you will identify any red flags that may not have been detectable in the review of any one proposal.
Let’s face it, jury research is expensive and you want to get what you pay for. As an upfront investment, quality, well-designed research has benefits multiple times over. It reduces uncertainty by gauging jurors’ responses to your case before it’s too late to do something about it. Therefore, while not all consultants (and methods) are created equal, be sure you are receiving the value you expect based on the goals you have identified.
* One thing to note: these questions matter especially if one of your goals depends on running a statistical analysis – e.g., developing a juror profile, etc.
** It is important to recruit jurors who haven’t previously participated in legal research to avoid a response bias. That is, the consequence of having a “veteran” juror means you could be receiving feedback that the juror thinks you want to hear rather than what you need to hear. “Virgin” respondents are not familiar with how the process works and, therefore, provide feedback not influenced by response bias.
*** There are two schools of thought within the trial consulting community – i.e., descriptive statistics (e.g., averages, frequencies) are as far as one should go with mock trial data if using a small sample size versus those who use more sophisticated techniques such as a quantitative data analysis to determine the trends that distinguish plaintiff jurors from their defense counterparts. We are of the latter mindset, assuming we have the conditions present to conduct such analysis and are using the analyses appropriate to the sample size typically found in jury research. It is important to know if you have the conditions present to run such an analysis and the reliability of the techniques being used.