HSR's Alumni Engagement Blog

As I mentioned in my last blog post, we do not always obtain a three-factor solution when we analyze our alumni attitude rating items, and when we do obtain a three-factor solution, the same items do not always define the alumni engagement factors we obtain.  Some of these differences result from the fact that we sometimes use slightly different rating items when evaluating the attitudes of any single alumni community.  Schools differ in the academic, campus and alumni outreach experiences that they offer students and alumni, and these differences sometimes warrant that items be added to our survey.  This usually does not result in different factors being identified by our factor analyses.  Usually, the same factors are obtained, but one or two different items might define these factors. 

In other studies, these differences occur because different alumni samples respond to the same items differently.  The same rating items may have been used, but different alumni samples respond to these items differently when describing their attitudes or feelings of engagement.  As such, the factor structures we obtain are slightly different.  Still, the majority of the time, we obtain the same three factors; however, alumni may define these factors using slightly different items or behaviors to define these variables. 

While this discussion may sound somewhat academic to most advancement professionals, it is important because it directly addresses the objective measurement of alumni engagement.  Engagement is a construct that many alumni advancement professionals use to describe the feelings and behaviors of their alumni community.  However, there is little agreement regarding the specific behaviors or attitudes (psychological states) that engaged alumni can be expected to exhibit.  Do engaged alumni attend events?  Is giving the best indicator of alumni engagement?  What role do alumni communications play in maintaining an engaged alumni community?  What causes alumni to become engaged?  Are there any specific strategies that are successful at building more engaged alumni communities?  Understanding what is important to alumni when they talk about their attitudes toward their alma mater, will help us develop a more objective measure of alumni engagement.   

What I want to do now is present the results of the factor analyses we performed on a few of the Alumni Attitude Surveys we have conducted at different schools.  This will begin to give you an idea of the specific Likert-type rating items that we use to measure alumni engagement. 

Table 1 presents a three-factor solution we obtained on an alumni sample.  We identified three factors - quality of education, focus on students and sense of belonging.  However, some items behaved differently in how they related to factors.  For instance, the item - “I feel a strong sense of belonging to the XYZ community”, related more strongly to the first factor - “Quality of Education” than it did to the third factor - “Sense of Belonging”.  However, on the campus of this small liberal arts college, faculty not only played a major role in determining the quality of the education students received, but also were an integral part of campus life.  Alumni wrote about faculty being available for help outside the classroom, the mentoring and one-on-one instruction faculty provided students, the interest faculty showed in the success and well-being of individual students and the friendships students developed with faculty.  Students frequently interacted with faculty outside the classroom.  Faculty were actively involved in campus life and played very important roles in both the classroom and campus community.  They were an integral part of the campus community. This affected how alumni responded to individual rating items and how these items related to the three factors.

The second factor evaluated the school’s “focus on students”.  Some of the items that defined this factor contained content that addressed specific administrative functions or administrators at the College, as opposed to the more generic behavioral statements that we have used on subsequent surveys. This factor, “focus on students”, was defined largely by the quality of the interactions that students had with College administrators and staff. 

 In this study, the third factor – “sense of belonging” was only beginning to be defined.  It appears that alumni who were actively involved in campus life as students felt more a sense of belonging as alumni. 

Table 2 presents a one-factor solution that was obtained on a small, single-gender liberal arts college.  In this study, the purpose of the alumnae survey was to update alumni contact information and Raiser's Edge data records.  The collection of alumni attitude data was not the primary purpose of the survey and fewer items were used.  On this sample, a single-factor solution was obtained for our measure of engagement.  Thirteen items related to this single factor.  When you look at the content of the items that define this factor, you will note that this content describes things alumni can do and say in support of their alma mater, their satisfaction with the quality of the education they received and the sense of belonging alumni feel.  The obtained solution did not include items that evaluated the school’s focus on students, but many of these items had been left off the survey.

Tables 3 and 4 present three-factor solutions that were obtained on two samples from HSR’s National Alumni Engagement Study.  This was a national study in which we used several social media platforms and numerous list serves to collect alumni attitude survey data.  A single survey item asked respondents if they worked in alumni advancement, and we used this item to divide our sample into two groups – individuals who worked in alumni advancement and those who did not.  Table 3 presents the three-factor solution we obtained on alumni who did not work in advancement. 

Table 4 presents the three-factor solution we obtained on alumni who worked in advancement.  For both samples, three factors were obtained – quality of education, focus on students and sense of belonging.  There was some shifting in the ordering of items for the first two factors and minor differences in how many items related to individual factors.  However, the factor structure generally remained the same for these two different samples.  The third factor – sense of belonging, was only partially defined for both samples.  Only two items related to this factor.

Table 5 presents a three-factor solution that we obtained on alumni from a Career University.  This sample was somewhat different from the other samples on which we had obtained alumni attitude survey data.  While the school did offer traditional liberal arts programs and degrees, it was much better known for its culinary and hospitality programs.  More importantly, the University had never made a systematic effort to ask alumni for gifts and the school had no history of a consistent, systematic outreach to alumni.  The University did host large alumni galas at trade shows, however, these events were more focused on highlighting their programs and students to employers than they were directed at engaging alumni.  Still, a three-factor solution was obtained on this sample as well. 

These are the results from just a few of the alumni attitude survey studies we have conducted.  So far, it seems that three variable are important to alumni when then recall their student experience and talk about their attitudes toward their alma mater.  These variables include alumni perceptions of the quality of the education they received, the extent to which the school was focused on student success and well-being and the sense of belonging alumni feel.  However, much more research needs to be performed.  It is quite possible that additional variables relate to alumni attitudes, and additional items need to be included on our measure of alumni engagement.  We are continuously developing new items based on the comments alumni write on our attitude surveys or the discussions among alumni in the focus groups we conduct.

It is possible that the quality of alumni outreach, including alumni communication, alumni events, alumni services and the quality of alumni interactions with administrators and staff does affect alumni attitudes. However, to date, our analyses have not obtained this factor. In response to the open-ended questions we ask, alumni frequently write about how much they like being kept informed about classmates, the campus and upcoming alumni events.  They also write about the way they have been treated as alumni.  Younger alumni frequently write about the need for career services and jobs. 

Alumni use three variables, quality of education, focus on students and sense of belonging, to describe their attitudes toward their alma mater.  However, alumni engagement is also defined by a set of behaviors that all alumni can potentially perform. These behaviors typically involve saying and doing good things in support of their alma mater, such as recommending the school to family and friends, speaking well of their alma mater to others or telling people that they would do it all over again if they had the chance.  An alumnus's willingness to perform these behaviors is an indication of their engagement level.  To date, these items have loaded on the first factor, quality of education.  They have not formed a separate factor.  Perhaps, alumni only recommend and say good things about their alma mater when they feel they received a quality education. 

Identifying the variables and items that alumni use to describe their attitudes and feelings of engagement is a continuous process.  We are constantly evaluating the factor structure of our Alumni Engagement Scale (AES), as well as reviewing the comments alumni write to see if there are any additional items we need to include on our Alumni Attitudes Survey.  This is part of the AES’s ongoing validation process.

As part of this process, Harford Survey Research is currently conducting an Alumni Attitudes Study that is directed at the validation of our Alumni Engagement Scale (AES).  We are answering some of the questions that were raised above.  To encourage participation we are offering schools extraordinary discounts off the cost of our Alumni Attitudes Survey.  If you are considering conducting an Alumni Attitudes Survey in the near future, you should call or email us to find out just how inexpensive your next alumni survey can be.

Comments (0) | Trackbacks (0) | Permalink