HSR's Alumni Engagement Blog

As I mentioned in my last blog post, we do not always obtain a three-factor solution when we analyze our alumni attitude rating items, and when we do obtain a three-factor solution, the same items do not always define the alumni engagement factors we obtain.  Some of these differences result from the fact that we sometimes use slightly different rating items when evaluating the attitudes of any single alumni community.  Schools differ in the academic, campus and alumni outreach experiences that they offer students and alumni, and these differences sometimes warrant that items be added to our survey.  This usually does not result in different factors being identified by our factor analyses.  Usually, the same factors are obtained, but one or two different items might define these factors. 

In other studies, these differences occur because different alumni samples respond to the same items differently.  The same rating items may have been used, but different alumni samples respond to these items differently when describing their attitudes or feelings of engagement.  As such, the factor structures we obtain are slightly different.  Still, the majority of the time, we obtain the same three factors; however, alumni may define these factors using slightly different items or behaviors to define these variables. 

While this discussion may sound somewhat academic to most advancement professionals, it is important because it directly addresses the objective measurement of alumni engagement.  Engagement is a construct that many alumni advancement professionals use to describe the feelings and behaviors of their alumni community.  However, there is little agreement regarding the specific behaviors or attitudes (psychological states) that engaged alumni can be expected to exhibit.  Do engaged alumni attend events?  Is giving the best indicator of alumni engagement?  What role do alumni communications play in maintaining an engaged alumni community?  What causes alumni to become engaged?  Are there any specific strategies that are successful at building more engaged alumni communities?  Understanding what is important to alumni when they talk about their attitudes toward their alma mater, will help us develop a more objective measure of alumni engagement.   

What I want to do now is present the results of the factor analyses we performed on a few of the Alumni Attitude Surveys we have conducted at different schools.  This will begin to give you an idea of the specific Likert-type rating items that we use to measure alumni engagement. 

Table 1 presents a three-factor solution we obtained on an alumni sample.  We identified three factors - quality of education, focus on students and sense of belonging.  However, some items behaved differently in how they related to factors.  For instance, the item - “I feel a strong sense of belonging to the XYZ community”, related more strongly to the first factor - “Quality of Education” than it did to the third factor - “Sense of Belonging”.  However, on the campus of this small liberal arts college, faculty not only played a major role in determining the quality of the education students received, but also were an integral part of campus life.  Alumni wrote about faculty being available for help outside the classroom, the mentoring and one-on-one instruction faculty provided students, the interest faculty showed in the success and well-being of individual students and the friendships students developed with faculty.  Students frequently interacted with faculty outside the classroom.  Faculty were actively involved in campus life and played very important roles in both the classroom and campus community.  They were an integral part of the campus community. This affected how alumni responded to individual rating items and how these items related to the three factors.

The second factor evaluated the school’s “focus on students”.  Some of the items that defined this factor contained content that addressed specific administrative functions or administrators at the College, as opposed to the more generic behavioral statements that we have used on subsequent surveys. This factor, “focus on students”, was defined largely by the quality of the interactions that students had with College administrators and staff. 

 In this study, the third factor – “sense of belonging” was only beginning to be defined.  It appears that alumni who were actively involved in campus life as students felt more a sense of belonging as alumni. 

Table 2 presents a one-factor solution that was obtained on a small, single-gender liberal arts college.  In this study, the purpose of the alumnae survey was to update alumni contact information and Raiser's Edge data records.  The collection of alumni attitude data was not the primary purpose of the survey and fewer items were used.  On this sample, a single-factor solution was obtained for our measure of engagement.  Thirteen items related to this single factor.  When you look at the content of the items that define this factor, you will note that this content describes things alumni can do and say in support of their alma mater, their satisfaction with the quality of the education they received and the sense of belonging alumni feel.  The obtained solution did not include items that evaluated the school’s focus on students, but many of these items had been left off the survey.

Tables 3 and 4 present three-factor solutions that were obtained on two samples from HSR’s National Alumni Engagement Study.  This was a national study in which we used several social media platforms and numerous list serves to collect alumni attitude survey data.  A single survey item asked respondents if they worked in alumni advancement, and we used this item to divide our sample into two groups – individuals who worked in alumni advancement and those who did not.  Table 3 presents the three-factor solution we obtained on alumni who did not work in advancement. 

Table 4 presents the three-factor solution we obtained on alumni who worked in advancement.  For both samples, three factors were obtained – quality of education, focus on students and sense of belonging.  There was some shifting in the ordering of items for the first two factors and minor differences in how many items related to individual factors.  However, the factor structure generally remained the same for these two different samples.  The third factor – sense of belonging, was only partially defined for both samples.  Only two items related to this factor.

Table 5 presents a three-factor solution that we obtained on alumni from a Career University.  This sample was somewhat different from the other samples on which we had obtained alumni attitude survey data.  While the school did offer traditional liberal arts programs and degrees, it was much better known for its culinary and hospitality programs.  More importantly, the University had never made a systematic effort to ask alumni for gifts and the school had no history of a consistent, systematic outreach to alumni.  The University did host large alumni galas at trade shows, however, these events were more focused on highlighting their programs and students to employers than they were directed at engaging alumni.  Still, a three-factor solution was obtained on this sample as well. 

These are the results from just a few of the alumni attitude survey studies we have conducted.  So far, it seems that three variable are important to alumni when then recall their student experience and talk about their attitudes toward their alma mater.  These variables include alumni perceptions of the quality of the education they received, the extent to which the school was focused on student success and well-being and the sense of belonging alumni feel.  However, much more research needs to be performed.  It is quite possible that additional variables relate to alumni attitudes, and additional items need to be included on our measure of alumni engagement.  We are continuously developing new items based on the comments alumni write on our attitude surveys or the discussions among alumni in the focus groups we conduct.

It is possible that the quality of alumni outreach, including alumni communication, alumni events, alumni services and the quality of alumni interactions with administrators and staff does affect alumni attitudes. However, to date, our analyses have not obtained this factor. In response to the open-ended questions we ask, alumni frequently write about how much they like being kept informed about classmates, the campus and upcoming alumni events.  They also write about the way they have been treated as alumni.  Younger alumni frequently write about the need for career services and jobs. 

Alumni use three variables, quality of education, focus on students and sense of belonging, to describe their attitudes toward their alma mater.  However, alumni engagement is also defined by a set of behaviors that all alumni can potentially perform. These behaviors typically involve saying and doing good things in support of their alma mater, such as recommending the school to family and friends, speaking well of their alma mater to others or telling people that they would do it all over again if they had the chance.  An alumnus's willingness to perform these behaviors is an indication of their engagement level.  To date, these items have loaded on the first factor, quality of education.  They have not formed a separate factor.  Perhaps, alumni only recommend and say good things about their alma mater when they feel they received a quality education. 

Identifying the variables and items that alumni use to describe their attitudes and feelings of engagement is a continuous process.  We are constantly evaluating the factor structure of our Alumni Engagement Scale (AES), as well as reviewing the comments alumni write to see if there are any additional items we need to include on our Alumni Attitudes Survey.  This is part of the AES’s ongoing validation process.

As part of this process, Harford Survey Research is currently conducting an Alumni Attitudes Study that is directed at the validation of our Alumni Engagement Scale (AES).  We are answering some of the questions that were raised above.  To encourage participation we are offering schools extraordinary discounts off the cost of our Alumni Attitudes Survey.  If you are considering conducting an Alumni Attitudes Survey in the near future, you should call or email us to find out just how inexpensive your next alumni survey can be.

Comments (0) | Trackbacks (0) | Permalink

Today, I want to talk about the theoretical models that guide our research related to the measurement of alumni attitudes and engagement.  Macey and Schneider (2007) discussed the need for engagement research to objectively measure the psychological states and behaviors implied by this construct. They also underscored the need for engagement research to be able to identify and describe the antecedents, moderators and consequences of engagement. Macey and Schneider were talking about employee engagement, but the implications for the measurement of alumni engagement are pretty clear.  Research must be able to describe the things that engaged alumni do, say, think and feel. They must be able to identify the variables that cause alumni to become engaged, as well as identify those variables that moderate alumni feelings of engagement.   Any comprehensive model of engagement must be able to objectively describe the antecedents, moderators and consequences of alumni engagement.

So what are the behaviors that engaged alumni perform? We can start building this list by first looking at the things that schools want alumni to do - attend events, visit campus, stay in touch, say good things about their alma mater and do good things for the school. Schools want their alumni to value the education they received, think highly of the degree they received and participate in giving. These are all behaviors that engaged alumni can be expected to perform. They provide a basis for the development of Likert-type rating items that evaluate behaviors which engaged alumni perform.

Feelings of engagement also relate to what alumni experienced as students - the quality of classroom instruction, the availability of faculty, the internships and career services that were offered and how students were treated by administrators, staff and the school. When alumni talk about their student experience, they frequently talk about the friendships they made, the favorite faculty who were both mentors and friends and the student organizations, clubs and athletic teams to which they belonged.. All of these experiences relate to alumni feelings of engagement and provide much content for item development.

Alumni attitudes and feelings of engagement are also affected by what alumni have experienced since graduation - the quality of alumni interactions with the school, their satisfaction with alumni outreach programs and what they perceive to be the value of their degree. Schools will differ in terms of the campus experiences they can provide, the amount of alumni outreach they perform, what they ask alumni to do and how their degree is valued by employers. All of this has an effect on alumni feelings of engagement and play important roles in the measurement of alumni engagement.

Figure 1 (above) offers a graphical representation of some of the behaviors we think are characteristic of many alumni communities. It is not comprehensive in the behaviors it describes, but it does provide a working model to begin talking about alumni engagement and the behaviors that relate to this construct. As depicted in figure 1, most alumni communities have a small group of not satisfied alumni. Sometimes this is because of something the school did; other times, it is just due to individual differences – values, beliefs or personalities that are specific to the individual alum. Many schools have a relatively large number of alumni who hold generally positive attitudes toward their alma mater, but do not attend events or participate in giving. A smaller group might be a bit more involved, and it is typically a much smaller group that regularly attends events and participates in giving.

Schools expend a considerable amount of time and resources encouraging alumni to attend events, participate in programs and make financial gifts to their alma mater. Alumni boards, leaders and volunteers play key roles in building an engaged alumni community. Through visits, events and continuous campus updates, schools strive to engage increasingly larger numbers of alumni.  They recruit alumni leaders from their more engaged alumni, and use this team of alumni leaders to accomplish advancement objectives, engage alumni and foster giving. The focus is developing an engaged alumni community that supports its alma mater with gifts. 

Figure 1 offers a theoretical framework for talking about the measurement of alumni engagement and designing strategies for building more engaged alumni communities. Any measure of alumni engagement should be sensitive to the behaviors that engaged alumni are likely to perform, the things that engaged alumni are likely to do, say, think or feel about their alma mater. It should include the variables that are important to alumni when they talk about their alma mater. Figure 1 provides a theoretical framework for thinking about what these behaviors might be.  These behaviors will likely comprise any objective measure of alumni engagement.

Macey and Schneider (2007) also wrote about the need for any model of engagement to address the antecedents, moderators and consequences of engagement – the engagement process. Figure 2 (above) describes the variables that our research has shown to relate alumni engagement - the antecedents, moderators and consequences of alumni engagement. Student experiences appear to be major determinants of alumni attitudes and engagement. The variables that appear to be most important to alumni when they talk about their student experience include the quality of education alumni received, the extent to which the school was focused on student success and well-being and the degree to which alumni feel a sense of belonging to their alma mater. Institutional values and campus community affect the quality of the student experience, as well as moderate alumni attitudes and feelings of engagement. Alumni outreach programs are designed to engage alumni and enhance the alumni community. Alumni engagement is affected by the student experience, campus and alumni communities, alumni outreach programs and institutional values.

As shown in figure 2, alumni engagement levels have a direct affect on alumni decisions to make a gift . Engaged alumni are much more likely to make gifts to their alma mater than are alumni who are not engaged or have less than positive attitudes toward their alma mater.  One of the most basic findings of our research is that satisfied students are much more likely to become engaged alumni, and engaged alumni are much more likely to make financial gifts to their alma mater.

We use the models presented in figures 1 and 2 to talk about alumni engagement, our Alumni Attitudes Survey, survey results, giving and the strategies we use to build more engaged alumni communities and increase giving. They also provide a good theoretical framework for thinking about the measurement of engagement.  In my next blog, I will share some of the Likert-type rating items we use to measure engagement, and talk about the methods we use to determine the variables and items that are important to alumni.

Comments (0) | Trackbacks (0) | Permalink