The McNamara Fallacy in Fraternity/Sorority Life

McNamara Fallacy – Those things which can be easily measured will be given greater priority over those things which can not be easily measured.

I recently watched the Ken Burns documentary on the Vietnam War on Netflix (which I highly recommend). One of the key themes that emerges during the series is the priority that Secretary of Defense Robert McNamara placed on quantitative metrics, specifically body count, during the war. McNamara was obsessed with statistics and, given the Pentagon’s new computing technology, he devised a series of metrics related to the success of the war effort it Vietnam. 

In the documentary, Ken Burns recounts a conversation that McNamara had with his generals in the early stages of the war, after he had put together the metrics for success. He gave the generals a printout of all of the metrics he wanted to track, and asked them if he had missed anything. The generals poured over his materials, and then one of the generals stated, “as far as I can tell, you’re only missing one thing…the hearts and minds of the Vietnamese people.”

“Well I’m not sure how we can measure that,” McNamara is said to have replied.

Thus was born the McNamara Fallacy. Those things that can be easily measured receive priority over those things which can’t be easily measured. And things that can’t be easily measured are deemed unimportant.

As I pondered the McNamara Fallacy and its broad implications, I realized that we operate under our own version of the McNamara Fallacy in the fraternity and sorority world. We rely too heavily upon data points that are easy to measure, often neglecting those metrics that are a bit harder to measure but are actually much more important in understanding what is really going on in our chapters. We have seen this problem magnified in the last year, as more and more campuses are moving towards the publication of chapter "report cards" showing metrics which tell us little, if anything, about chapter culture.

What are some of those easy-to-measure metrics that we have grown to rely on but which do not really tell the story of what is happening in our communities and in our chapters? Like McNamara’s body counts, what are those data points we are using to try to tell our story but that don’t tell the full story, and what metrics should we be using instead?

Here are four metrics upon which we have become overly reliant, followed by four metrics that are a bit harder to gather, but that tell a much more robust story about the culture of our chapters.

1. GPA – Grade point average is not a bad metric. In fact, GPA can tell us some things about a chapter culture – namely, how smart the people in a chapter are, and how much time they spend studying. But there is an over-reliance in our industry on GPA as a proxy for chapter culture. I would suggest that we rely WAY too much on GPA as an indicator of chapter quality. This is especially true in sororities. In our research, we see a strong correlation between relative recruiting strength and GPA in Panhellenic communities. The stronger-recruiting sororities on any given campus tend to recruit women with stronger high school GPA’s. System strugglers and bottom-third chapters in terms of recruiting strength tend to have lower GPA’s because they tend to not get the top scholastic performers out of high school, most of whom get bids to top-tier sororities. They end up with a larger portion of members with questionable high school grades - members that the top-tier chapters pass over. Does this mean those chapters have a bad culture? Are their members partying all the time instead of studying? Are they distracting their members from their academic pursuits just because their GPA is a little lower than the top chapters on campus? Not likely. The GPA disparity in sorority world likely has much more to do with the quality of members that chapters are able to recruit and very little, if anything, to do with the chapter culture.

On the fraternity side, we generally see a bell-curve associated with GPA. Again, we often see top-tier chapters selecting those members with stronger high school achievements, but we also see newer chapters selecting academically talented students who may not socially fit in with “top-tier” groups (i.e. the nerdy fraternities). We also generally see a bell-curve, with a lot of chapters tightly bunched around the center and only a few outliers on either side. At the campus level, those low-GPA outliers almost certainly have cultural challenges, but outside of those few groups, GPA tells us very little. On a campus with an all-fraternity average of 3.0, with a few fraternities around a 3.2 and another few fraternities at a 2.6, and everyone else between a 2.8 and a 3.1, what is the real difference between and among those groups bunched up in the middle, and how does GPA account for the differences in their culture? And at the national level, comparing between chapters on different campuses is not an apples-to-apples comparison. A 2.8 at Middle Tennessee State is not the same as a 2.8 just up the road at Vanderbilt. Understanding grades within the context of the natural academic ability of the students on that campus is important.

I would offer that, for both fraternities and sororities, chapter GPA would be a much more reliable predictor of chapter culture if we looked at chapter GPA after controlling for the high school GPA’s of the members of that chapter. This data would tell us if a chapter is over/under-performing in relation to their members’ natural academic abilities. We also need to look at trends over time - is a group consistent in terms of their GPA, with only minor fluctuations between semesters, or is there a trend in one direction or another? A steady downward trend should be a red flag. Short of that, GPA tells us very little about chapter culture. In the national data set compiled through our research at Dyad Strategies, GPA has a very weak correlation with any of the measures related to chapter culture, including hazing, sexual assault and alcohol use. We see chapters at all ends of the GPA spectrum struggle with these and other cultural issues. Our industry places far too much emphasis on GPA because it is the easiest data point for us to gather, but outside of the extreme fraternity outliers, it actually tells us very little about chapter culture.

2. Recruitment Stats – Both campus and headquarters-based professionals place far too much emphasis on recruitment stats. On the national side, many sorority headquarters do not allow chapters that fail to make quota on their respective campuses to be eligible for national awards. On the fraternity side, we see national organizations regularly touting the recruitment prowess of their top chapters, often placing that metric at or near the top of their national recognition. I’ve written before of the folly of placing too much emphasis on recruitment stats as a measure of chapter quality, but we continue to see organizations prioritize the number of new members over the quality of the experience those new members are going to be a part of. If you recruit 100 new members into a toxic culture, what have you accomplished? Is that something to be celebrated? I would suggest the opposite, yet we see both campus and organizational leaders investing much more heavily in recruitment training than in any other areas connected to chapter culture. 

On the campus side, I regularly see campus-based fraternity and sorority advisors taking WAYYYY too much credit for the growth in their communities. I feel like I am uniquely qualified to comment on this. During my five years as Director of Greek Life at Alabama, our fraternity/sorority community nearly doubled in size, and we became the largest community in the country. It would be easy for me to pound my chest and take a lot of credit for that growth, but the fact of the matter is that I had almost nothing to do with it. The University was growing. We were recruiting more and more out-of-state students, many of whom wanted to be in fraternities/sororities. The University seriously invested in fraternity/sorority housing. This was all the doing of the University President and the folks in the admissions office. While I will take some credit for boosting recruitment retention and placement rates during my time there, I can not take any credit at all for growth in the community, because almost all of it had to do with factors beyond my control. I would venture to guess that the same could be said of 90 percent of colleges in the country that experienced growth in their communities in the last decade. Growing or declining communities are less a sign of an effective or ineffective fraternity/sorority advisor and more an indicator of the demographics of an institution’s student body. 

3. Community Service Hours/Philanthropy Dollars Raised – Like GPA, Community service and philanthropic contributions are not a bad metric. They tell us something about a chapter – specifically, how much community service/philanthropy in which the chapter is engaged. But we see a lot of people use service/philanthropy statistics as a proxy for chapter culture, and there are a few problems associated with reading too much into these numbers.

First, the data are over-inflated. How many national organizations or campuses are calling up community partners to verify the monetary contributions or hours of service reported by chapters? My guess is that this rarely, if ever, happens. Chapters may raise a lot of money at an event, but when they report those totals, are they subtracting out the costs associated with that event? Are they reporting gross revenues or net revenues? I suspect that, if we took a really hard look at the actual philanthropic activity of our chapters, the numbers would not be nearly as impressive as what we often report.

Secondly, the fact is that service is a core value for some of our organizations, but not for all. I think we should be encouraging our chapters to pursue and live out their espoused values and to reward them if and when they do that. If service to others is an espoused value of an organization, then we should reward those chapters for engaging in that service. If it is not an espoused value of that organization, then we should be rewarding them for doing activities that support their own mission and values and not superimposing our own values onto them. And I've seen no data that convinces me of any connection between service/philanthropy and other areas of chapter culture.

4. Involvement in Other Student Orgs – Almost every campus or organization has a requirement (or at least a strong suggestion) that all members of fraternities and sororities be involved in at least one other campus organization. Inevitably, this leads to chapters submitting a chapter roster, listing off each member and the names of the other organizations of which they are a part. When I worked at Alabama, approximately 90 percent of the entire Greek community was in the College Republicans. I assume the College Republicans held their meetings in the basketball arena, because according to my count they had at least 10,000 active members.  Using involvement in other orgs as a success metric is easy because chapters can just tell us how many different organizations their members are a part of, but the data are absolutely meaningless. They tell us nothing. Most of us are not verifying membership rosters. We are not examining the quality (Time on Task) of their involvement in these other groups. We are simply grabbing a data point that is easy to capture and then reporting some asinine statistic like “93 percent of fraternity/sorority members are involved in another organization on campus,” and we report it knowing that the number is a complete and total lie.

So, if over-reliance on the metrics above is problematic, then what harder-to-measure metrics should we be focused on gathering and reporting? How do we accurately capture the “hearts and minds” of the fraternity sorority members under our purview? What attitudes and beliefs are really at the heart of understanding the culture of an organization and the experience that students are having as part of a fraternity or sorority chapter? Here I offer the four measures that I think show the most promise.

1. Sense of Belonging – So much of a student’s experience in a fraternity or sorority is predicated upon their sense of belonging. In the research we are doing at Dyad Strategies, we find that Belonging as a function of brother/sisterhood is the single most important predictor of so many other outcomes, including organizational commitment, satisfaction, retention, and organizational accountability. The depth and the quality of the relationships between and among members is the single biggest driver of chapter culture. The fact is that some chapters do an incredible job creating a sense of belonging in their members, and some chapters do an awful job. Understanding which chapters are and are not creating a sense of belonging – a feeling of mattering – is the single most important metric for both campuses and national organizations to have in order to understand what is really going on in a chapter.

2. Social Status Importance – I’ve previously written about the campus social hierarchy and how chapters can fall victim to their own success. Understanding how much chapters care about that social hierarchy – how much they are motivated by achieving or maintaining social clout – is an important metric to understand. Chapters with high social status motivation are willing to take incredible risks in order to achieve/maintain that social status, make poor decisions in recruitment (will this person boost our social clout vs. will this person be a good, contributing member), and lose focus on the brother/sisterhood they are creating as they become more and more motivated by factors external to their chapters. Seeing this construct modeled over time can help us predict when a chapter may begin making bad decisions, as it is the driver behind regression to the mean as newer chapters begin assimilating into their respective campus cultures (more on that here).

3. Hazing Motivation – Groups haze for different reasons, and not all reasons are created equally. Researcher Aldo Cimino has explored hazing through a sociological lens and offered multiple motivations for hazing of newcomers in groups. At Dyad Strategies, we have expanded on Cimino’s work, building an instrument that measures the various motivations of hazing. Are groups trying to build solidarity or teach new members group-relevant skills through their hazing? If so, these behaviors are somewhat altruistic and fairly easy to redirect. Are groups trying to reinforce the social hierarchy within the group or ensure that new members are properly committed to the group through their hazing? If so, these motivations are likely to be emblematic of much deeper cultural problems in a chapter which are much more difficult to root out through education alone. Understanding what it is groups are trying to accomplish through their treatment of new members is crucial in any efforts to prevent or redirect hazing behaviors.

4. Motivation to Join – What is it that students joining fraternities and sororities are looking to gain from the experience? Are they merely looking for a social outlet, or are they interested in leadership, networking, or a home away from home? Motivated by our research findings indicating tremendous spikes in recent years related to the social aspect of brother/sisterhood, our team at Dyad Strategies has now constructed a scale measuring students’ motivation for joining fraternities and sororities. Any campus seeking to understand how changes in recruitment, education for potential members, or overall changes to the social culture in a community is having an impact, then understanding the motivations of the students seeking to join that community is of the utmost importance. If would-be fraternity and sorority members continue to see the fraternity/sorority experience chiefly as a social experience in spite of changes, then we know those changes are not having the desired impact. But if we see less of a social motivation and increases in motivation related to belonging, networking, or campus involvement, then it can be said that those changes are having a positive impact.

As I began writing this post, I did not intend for it to be an advertisement for the research we are doing at Dyad Strategies. But, as it turns out, we have spent a great deal of time in the last few years really trying to grasp what we need to know about our students’ beliefs, attitudes and behaviors in order to really understand chapter culture. The things we measure at Dyad Strategies paint a much more robust picture about what is really going on in a chapter than the standard metrics upon which we see many campuses and organizations rely. If your campus or organization is serious about understanding and measuring the impact of the changes you are implementing, then I invite you to contact us to find out how we may be able to help.

Like the hearts and minds of people in Vietnam, the important concepts that I have laid out here are not easy to measure. But it IS possible to measure these concepts. With the scrutiny that fraternities and sororities are under today, we can no longer rely on simple counts or self-reported activities. We have to survey our students in order to understand the attitudes and beliefs that are underlying their behaviors. Once we understand those attitudes and beliefs, then we can be much more surgical in our approach to winning the war that must be won – moving fraternity/sorority beyond a social experience towards the personal development experience that it was intended by our founders to be.