You have questions. We have answers.

Frequently Asked Questions

Psychological Assessment in Business

How are Siena Consulting assessments different from traditional assessments? Why do they result in less bias?

There was a long-held assumption that intelligence tests will always result in race differences to the detriment of certain groups such as African Americans and Latinos and in some circumstances women.  Siena researchers feel this outcome does not reflect reality and is instead based on traditional ways of measuring intelligence that don’t properly capture the construct and are contaminated by factors that have nothing to do with intelligence. In particular, these traditional tests use items that depend on a test taker’s past experience and education. Siena takes a modern approach to test design that does not use traditional test items to measure intelligence. Siena’s modern approach focuses on having test takers respond to novel, unfamiliar problems so an individual's prior education, work, and life experience are not used to solve these problems. Siena researchers believe that the best leaders and employees are the ones who can solve unique problems; not familiar problems that rely more on book smarts and memorization. By taking this approach, Siena’s tests dramatically reduce race and ethnic differences (as well as the gender differences that sometimes are observed) thus providing a test that is valid and fair when assessing a diverse, global population.  

What does it mean that assessments from Siena Consulting are based on science?

The Siena team are experts and have PhD’s in the field of Industrial and Organizational Psychology (also called I/O Psychology).  This specialty area of psychology is characterized by the scientific study of human behavior in organizations and the workplace. I/O psychology focuses on deriving principles of individual, group, and organizational behavior and applying this knowledge to the solution of problems at work. In fact, the field of I/O Psychology is responsible for most of the research and work in measuring intelligence and personality to predict performance in the workplace over the past 4 decades.  

Siena Consulting uniquely leverages research from I/O psychology and blends it with the research and findings from other critical fields including Developmental Psychology, Neuropsychology, Cognition, and Social Psychology.  This interdisciplinary strategy has led to the development of Siena’s innovative ways to measure intelligence, personality, and motivation.

Tell me more about Siena Consulting and the Siena Reasoning Test (SRT).

Across a wide range of sectors and industries, Siena Consulting develops world-class assessment tools that identify talented, diverse candidates for selection and manager & executive development. They develop innovative ways of measuring the intelligence and personality capabilities that drive performance in the workplace. The award-winning Siena Reasoning Test (SRT) has been developed as the optimal way to measure the multi-dimensional elements of intelligence as used in a work environment. The SRT can be customized to best fit those elements of intelligence that are situationally more relevant to a wide range of jobs, levels, and organizations. Versions of the SRT are used today in Fortune 500 companies and a wide variety of government agencies.

What is the Player Assessment Test (PAT) and how has it been used?

The Player Assessment Test (PAT) is a customized test of intelligence, personality, and motivation that has been administered to all participants at the NFL Combine since 2013. Results from this Player Assessment Test (PAT) are distributed to all NFL clubs for use in player selection and/or coaching individual athletes.  Alternative tailored versions of the PAT are used in other professional sports as well as with college athletes. 

  • Input into draft strategy meetings to set the draft board
  • Detailed individual report generation and report interpretation
  • Interviews with top athletes being considered for high draft picks
  • Training of Interviewers (like coaches & scouts)
  • Guidance in modifying coaching approach with individual players
How long are these tests? How much administrative work does this take?

The short answer is “it depends”; but our tests range from anywhere from 15 to 60 minutes.  Ultimately, different businesses and sports teams choose different options customized to their existing evaluation process and the breadth and depth of the insights they seek.  
Siena’s tests can be administered on a range of devices including computers, tablets, and cell phones. Typically, test results are available immediately for use by the organization.

Psychological Assessment in Sports

Different sports don’t require the same skills and abilities. How can this comprehensive assessment help with all sports?

We agree that different sports require different contributions from athletes. Fluid sports, like basketball and soccer, allow less time for coaching interaction and play calling to optimize a team’s chance for offensive or defensive success vs. less fluid sports like football and baseball.

Fortunately, the simple answer is (for the most common sports like basketball, soccer, baseball, and football) there is a core set of dimensions – that is a great universal model and a great starting point. However, within each of these dimensions, we have “tweaked” scoring and evaluation based on judgment and prioritization.  For example, you might choose to measure aspects of intelligence for sports like Basketball and Soccer (where decision making is on-the-fly and there are fewer coaching interventions) with a greater emphasis on visual processing and the ability to think ahead as play evolves. In each case, we fully evaluate the requirements and desired culture of an organization to best meet the needs of that organization. 

How long are these tests? How much administrative work does this take?

The short is “it depends”; but the range is anywhere from 15 minutes up to an hour. Ultimately, different teams choose different options customized to how it best fits into their existing evaluation process and the breadth and depth of the insights they seek. Here are some of the biggest variables:

  1. Test taking. The full PAT takes approximately 60 minutes for each candidate; but truncated (as short as 15 minutes) are available and we are happy to discuss lengths and the trade-offs for shorter tests.
  2. Test Taking. Standardized reports are highly confidential and must be delivered in a highly controlled environment – ideally on-site and proctored by trained professionals (like a licensed psychologist).
  3. Test Results for Individuals. Once the process has been established, individual test results are available right away after the candidate has completed the test.
  4. Test Results for full Teams. Comprehensive team results are available approximately 2 weeks after the complete data is made available. Complete data includes end-of-season coaches’ ratings. This is generally a scheduled end-of-year review shortly after the season is finished to inform the next recruiting cycle.
  5. Test Interpretation. In our experience, most of the time stems from test interpretation and comparing across candidates. However, most teams already budget time for scouts, coaches, and recruiters to do this and the PAT becomes part of this process; often adding clarity and confidence to decision making.
How does the PAT scoring work? How are results interpreted?

The scores provided are based on the percentile achieved by the individual. The percentile is calculated by comparing the individual to the sample of athletes who have taken the test to date.

Scores range from low (1) to high (10) based on the percentile achieved. The scores are generated based on the responses of the individual relative to the relevant sample of athletes who have taken the test to date. Scores of 1 to 3 are considered low, scores of 4 to 7 are considered moderate, and scores of 8 to 10 are considered high.

An overall score for each athlete is provided, as well as summary scores for Intelligence and Personality.  As described above, these scores are all on a 10-point scale with 10 being the highest and one the lowest. Each dimension also has a 1-10 score. For each 1-10 score, it compares the individual athlete relative to the sample of athletes who have taken the test.

To get a complete picture of the athlete, it is recommended that the information provided by the PAT be used in combination with other data that teams collect on players via other sources – like the combine and scouting operations. The information from this assessment should be seen as an additional indicator of the potential capabilities of a player. Using this information in combination with other valid indicators can help a team gain insight into a player’s capabilities and how that individual could perform in their specific sport (like the NFL).

What is the difference between the NFL Player Assessment Test and other Intelligence & Personality Tests like the Wonderlic and the Myers Briggs? How have they been applied to sports?

Long standing intelligence and personality tests like the Wonderlic and the Myers Briggs have rich data that goes back many years and across many cultures. They can share a wide range of benchmarking data to allow for comparing and contrasting and have large enough samples to examine subsets of the populations vs. new test takers. Most of this work has been outside of sports but is more broadly familiar and understood by mainstream society and especially in US business.

Newer tests (like the PAT) use different ways to measure and assess some of the same attributes (arguably) with more precision and (inarguably) with less bias. Because of new ways of asking questions and new methods for item development, the PAT has the advantage of removing bias as well as being more closely focused on sports-related behaviors and competencies.This is especially critical with measuring intelligence which has been a challenge across several decades with intelligence testing. It also has the benefit of more data and statistical evidence for being highly correlated to performance and outcome measures in sports. 

Scouts already collect a lot of personality and “character” assessment information based on observations and conversations with prior head coaches. What is the role of a scout or a recruiter and the PAT when they seem to be measuring the same things?

Based on our experience, we believe that when you add the PAT to player evaluation, the assessment becomes the best friend of the scout or recruiter. We don’t recommend the PAT be the sole solution. But it does provide an infrastructure for scouts and recruiters to compare across candidates. In our experience, a single scout or recruiter does not have the bandwidth or expert psychological assessment capabilities to dive deep into each individual player – they instead need to compare notes across scouts and across coaches. The PAT allows them to do this. And the PAT provide the predictive variables whereas scouts and recruiters need to connect these assessments to output data – like on-field performance, leadership behaviors, team behaviors, and classroom performance. The PAT also directs conversations towards what parts of a player's character needs to be most closely examined and thus focuses conversation to decisions can be made based on facts. 

The PAT is not a magic wand. But it does provide material information to explain player on-the-field and off-the-field performance. It allows teams to understand if one-off incidents are red flags or if they are just that; one-off exceptions. 

How much can these tests really help in sports? Isn’t it really all about athletic ability?

Based on interviews with coaches and former players, we believe tests like the PAT are becoming more important. To quote Teddy Bruschi, “Your players are only as good as their ability to execute your game plan.” Most would argue that sports strategy and game plans are becoming more and more complicated. If you agree with this, the PAT gives teams a running start into fully evaluating the individual beyond their athletic ability.

The other factor that has emerged in interviews is being a good player “off-the-field”. The emergence of social media has amplified mistakes being made by young adults.  In college, it’s going to class, staying eligible, and staying out of negative social media. Professional athletes need to worry about social media and also carry the added burden of the media and the paparazzi waiting to catch them with an ill-advised quote or a regretted tweet.

In either or both cases, getting a comprehensive picture of an individual is predictive of outcome variables both on and off-the-field; above and beyond athletic ability. You can watch more about how the PAT predicts beyond athletic ability when Ken presents on this topic at the MIT Sports Analytics Conference.

Title 3

Lorem ipsum dolor?

Lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor.

Lorem ipsum dolor?

Lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor.

Lorem ipsum dolor?

Lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor.

Lorem ipsum dolor?

Lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor.