Harford Survey Research
Engage

HSR's Alumni Engagement Blog

Jim Flynn, Ph.D.Engage is HSR’s Alumni Engagement Blog. Dr. Flynn writes about the measurement of alumni attitudes and engagement, best practices for conducting alumni surveys, alumni survey results and strategies for building more engaged alumni communities and increasing alumni giving. If you want to think more strategically about your alumni outreach programs, follow our blog. We want to hear from you, so please feel free to comment on what we write.

Measuring Alumni Engagement – Survey Questions and Items

Wednesday, December 07, 2011

We are often asked by alumni advancement professionals if we can share our Alumni Attitudes Survey. They want to see the individual items we use and the questions we ask.  Today, I will do a little sharing.  I want to talk about our approach to evaluating alumni attitudes and measuring alumni engagement.  I will discuss HSR's Alumni Attitudes Survey, the Alumni Engagement Scale (AES), the questions we ask, the items we use and how we identify the variables that are most important to alumni.

Harford Survey Research’s approach to the measurement of alumni engagement is quite similar to the methods advocated by Macey and Schneider (2007). We use Likert-type rating items to evaluate the behaviors and attitudes (psychological states) that engaged alumni can be expected to exhibit. The rating items evaluate things that engaged alumni can be expected to do, say, think, believe or feel. They evaluate both the behaviors and psychological states that relate to feelings of engagement. The rating items we use are based on items we gleamed from measures of employee engagement (where there are many similarities in how engagement is defined); the behavioral examples we obtained from the interviews and focus groups we have conducted with alumni, faculty and staff and the behavioral examples we have obtained from the comments alumni write in response to the open-ended survey questions we have asked. We have developed a comprehensive pool of Likert-type rating items that evaluate alumni engagement.

Our Alumni Attitudes Survey also includes a set of open-ended questions that obtain written feedback about alumni educational and campus experiences, their attitudes toward the school and their attitudes toward giving. These open-ended questions are a very good source of information for developing additional Likert-type rating items. As we collect survey data on different types of schools, we are continuously analyzing the comments alumni write to insure that our Alumni Attitudes Survey is comprehensive in the behaviors it evaluates. Some of the questions we ask on our Alumni Attitudes Survey include:  

  • What does the school do well? 
  • Where can improvements be made? 
  • Why do alumni attend or not attend events? 
  • Why do alumni make or not make financial gifts to their alma mater?
  • How can your alma mater improve its alumni outreach?

We frequently obtain a considerable amount of information from the comments alumni write in response to these open-ended questions. These comments oftentimes relate to the variables measured by our alumni engagement scale, the AES. However, because academic institutions differ in many important ways, we are continuously reviewing these comments to determine if our rating items are comprehensive in the behaviors and experiences they evaluate. Our open-ended questions are a rich source of information that provides schools with additional insight into alumni attitudes and provide us with the behavioral examples we need to evaluate the validity of the AES. 

The validation of the AES is an ongoing process. We are continuously evaluating its validity and testing the generalizability of its factor structure across different academic settings.  Given the many differences that exist between schools, it is highly unlikely that all alumni will use the same items to describe their attitudes and feelings of engagement. There is a core set of items that are applicable to most schools, but alumni from very different types of schools sometimes focus on behaviors and campus experiences that are different from the majority schools.  For instance, alumni from large public universities, small liberal arts colleges, single-gender schools, HBCUs, the Naval Academy or schools that are affiliated with a specific religious group are likely to have a subset of behaviors or experiences that are specific to their alumni community and affect alumni feelings of engagement.   As differences between schools become greater, it is very likely that there will be a few additional behaviors and experiences that will relate to feelings of engagement.  We are continuously evaluating the validity and comprehensiveness of the rating items we use.

Harford Survey Research has developed a large pool of Likert-type rating items that we use to evaluate alumni attitudes and engagement.  These items generally give us a very accurate picture of alumni attitudes and engagement levels at different schools. When schools are unique or very different from the institutions in which we have already collected data, we will conduct interviews or focus groups to develop new items that are specific to these academic institutions and alumni communities. We make every effort to insure that our Alumni Attitudes Survey evaluates the behaviors and experiences that relate to feelings of engagement at the different schools with whom we work.

Our Alumni Attitudes Survey uses approximately twenty-five Likert-type rating items to evaluate alumni attitudes and engagement. However, when we conduct focus groups at a school, there is always a chance that a few additional items will be  added to our measure of engagement. After collecting our survey data, we use factor analyzes to identify the variables that define alumni engagement for the alumni community we are surveying, as well as identify the rating items that relate to these individual variables. We compute Chronbach Alphas to evaluate the inter-item reliability of our scales, and we use simple multiple regression analyzes to evaluate the strength of the relationships between our measure of alumni engagement and alumni giving.  We obtain from schools the giving data on survey respondents for the last five years.  We compute the percentage of the past five years that an alum made a gift, and we use this participation rate as our outcome measure of alumni giving. The AES is an excellent predictor of alumni giving.

Harford Survey Research is continuously evaluating the validity of the AES.  Very often a three factor solution is obtained  with factor analyzes identifying three variables that affect alumni attitudes - the quality of the education they received, the extent to which the school was focused on students and the “sense of belonging” alumni feel.  Fifteen to twenty Likert-type rating items typically define these three variables. (See Table 3.)

We do not always obtain a three-factor solution on all alumni samples.  Schools can be different in the student experiences they offer alumni, as well as the amount of alumni outreach they are able to perform.  On a few alumni samples we have obtained a one-factor solution for our measure of alumni engagement.  This means that a single variable defines engagement.  However, when we have obtained a one-factor solution, many of the same rating items that were identified by the three-factor solution were also retained by the one-factor solution.  However, fewer items were retained.  Approximately ten items, as opposed to twenty items, are typically retained when we obtain a single-factor measure of alumni engagement.  Here again, HSR is continuously looking at how these factor structures vary across different academic settings.  This is the focus of our current Alumni Attitudes Study.

 

 

 

Table 3 (above) presents the results of the factor analyzes we performed on a set of items that were used to evaluate alumni attitudes at one University. A three-factor solution was obtained. The variables that were identified included quality of education; the extent to which the University was focused on student well-being and success; and the sense of belonging alumni feel. The first factor or variable was defined by six items and had a reliability of.92. (I have highlighted in bold the coefficients in the first column of coefficients that relate to this variable.) The first factor evaluated alumni perceptions of the quality of the education they received, as well as the extent to which they would say and do good things for their alma mater, such as, recommending the school to family and relatives. The second factor or variable, focus on students, was defined by four items and had a reliability of .80. It evaluated alumni perceptions of faculty accessibility, how well students were treated and if they received the guidance and attention they needed to succeed. The third factor evaluated the extent to which alumni feel a sense of belonging toward their alma mater – if they were actively involved in campus life, felt a part of the campus community and look forward to seeing classmates and friends. When alumni talk about their attitudes toward their alma mater, these three variables are the most important to alumni. They represent the lens through which alumni recall their academic and campus experiences. Table 3 provides a sample of the items we use to measure alumni engagement.  For this sample of university alumni (approximately 3,000 alumni from a single school)., these were variables and items that were important to alumni when they talked about their attitudes toward their alma mater. These were also the items we used to score the engagement levels of individual survey respondents.

Harford Survey Research has conducted numerous alumni attitude studies, and as already stated, the items and factor structures reported in Figure 3 are not always obtained. Sometimes different items are included on the survey because of the differences in the academic institutions with whom we are working, and this can result in different items being retained or different factor structures being obtained.  We would likely not obtain the same results, if these data were being obtained on alumni from a school that primarily uses distance learning or computer-aided instruction as its primary method of teaching. Similarly, there may be differences in how alumni from Ivy League schools, HBCUs, single-gender schools and religiously affiliated schools define engagement.  These are all questions being addressed by HSR's systematic program of alumni engagement research. However, although the individual items and factors we obtain may sometimes vary, the variables that are important to alumni have remained relatively constant across the majority of schools we have studied. 

The items listed in figure 3 offer a set of Likert-type rating items that schools can use to measure alumni engagement. We use a 5- or 7-point rating scale to evaluate the degree to which alumni agree or disagree with each statement or item. If you decide to use these items, we recommend that you conduct a few focus groups with alumni to insure that you are measuring all the behaviors and experiences that are important to your alumni. If you can, use the appropriate statistical procedures to identify the variables and items that define engagement for your alumni community.  We validate our scales for every sample of alumni on which we collect survey data. 

Finally, if you are considering conducting an alumni attitudes survey in the near future, you should contact Harford Survey Research.  We are preparing to conduct a national Alumni Attitudes Study directed at the validation of HSR’s Alumni Engagement Scale (the AES). To encourage schools to participate in our Alumni Attitudes Study we are offering extraordinary discounts off the cost of our Alumni Attitudes Survey. If you plan to conduct an alumni attitudes survey, this is an excellent opportunity to have this work done professionally and at a cost that all schools can afford. We have provided a brief description of our Alumni Attitudes Study for schools who would like to participate. Again, this is an extraordinary opportunity for schools to conduct an alumni attitudes survey at a cost all schools can afford.

I will talk more about our alumni engagement research in later blogs.

 

Comments
RagneemaAdese commented on 13-Sep-2012 12:37 PM
http://www.kdfokodlps.com - kdfokodlps ,
kdfokodlps , http://www.kdfokodlps.com
UNUNNYCENDY commented on 19-Sep-2012 02:28 PM
http://lsazxns.com - CRXGQ - gCvHdaAnFnvrAaS , http://mjhpeuho.com
paralegal schoolshttp://paralegaltrainingandjobs.com/how-to-become-a-paralegal/#educational requirements to be commented on 20-Sep-2012 10:07 AM
Hi, i think that i saw you visited my site so i came to “return the favor”. I am attempting to find things to enhance my web site!I suppose its ok to use a few of your ideas!!
sample thank you letters for pharmacy technicians commented on 22-Sep-2012 01:00 PM
Hi there, its pleasant article about media print, we all understand media is a fantastic source of data.
interview questions And answers for a pharmacy technician commented on 23-Sep-2012 01:06 PM
This paragraph will assist the internet viewers for creating new webpage or even a weblog from start to end.
lclqlzbph commented on 18-Oct-2012 06:54 AM
viagra achat en ligne
acheter viagra
generique viagra en ligne
viagra pas cher en ligne
Anonymous commented on 18-Oct-2012 10:58 AM
I truly appreciate this post. I have been looking all over for this! Thank goodness I found it on Bing. You've made my day! Thanks again!
free pokiesgames onlineaustralian online casino games
Anonymous commented on 18-Oct-2012 01:48 PM
You have brought up a very wonderful details , regards for the post. sump pump review
Anonymous commented on 19-Oct-2012 03:36 PM
Merely wanna comment that you have a very decent internet site , I enjoy the style it actually stands out.click thisCheck This Outcheck my siteVisit Your URLVisit Here
Anonymous commented on 21-Oct-2012 04:34 PM
Some genuinely wonderful content on this internet site , thankyou for contribution.halogen cooker
Anonymous commented on 22-Oct-2012 02:40 PM
I will immediately clutch your rss as I can not in finding your email subscription link or newsletter service. Do you've any? Please allow me recognize in order that I may subscribe. Thanks.suomi casino
Anonymous commented on 22-Oct-2012 05:19 PM
Good write-up, I am regular visitor of one's website, maintain up the excellent operate, and It is going to be a regular visitor for a lengthy time.Dehumidifiers reviews |coffee grinder | generator reviews |best Humidifiers |leaf blowers | paint sprayer
Anonymous commented on 23-Oct-2012 06:55 PM
F*ckin' amazing issues here. I'm very glad to peer your post. Thanks a lot and i am taking a look forward to touch you. Will you kindly drop me a mail?
online pokies
casino online
mobile casino
hchinmevec commented on 28-Oct-2012 02:06 AM
http://coachhandbagsoutletsale.us/ $86.15 cheap coach handbags $72.49 coach outlet online $78.64 cheap coach handbags $82.8 cheap coach handbags http://cheapcoachhandbagoutletonline.us/ $92.75 cheap coach $100.47 discount coach handbags $39.52 cheap coach handbags $97.97 cheap coach handbags http://coachbagsoutletsaleonlinestore.us/ $68.82 coach $48.99 coach bags outlet sale $86.89 coach bags outlet $73.39 coach bags outlet http://coachhandbagsoutletonlinestore.us/#official-coach-outlet-online $80.13coach $77.21coach bags outlet sale $39.44coach bags outlet $73.45coach bags outlet http://officialcoachoutletonlinestore.us/ $88.4discount coach handbags $88.44discount coach handbags $71.47cheap coach handbags $52.99cheap coach handbags
http://louisvuittonbagsoutletstore.us/#louis-vuitton-Monogram-Denim $57.52louis vuitton Monogram Empreinte $81.2louis vuitton Monogram Rayures $53.69louis vuitton Antheia Leather $71.25louis vuitton Monogram Multicolore Canvas http://louisvuittonhandbagoutletonline.us/#Speedy $76.55louis vuitton Monogram Vernis $99.62Suhali Leather $39.40louis vuitton Monogram Vernis $42.12Speedy http://louisvuittonbagsoutletsale.us/#louis-vuitton-outlet $64.3louis vuitton $86.45 louis vuitton store $52.8 louis vuitton $69.86 louis vuitton online store http://louisvuittonbagssaleonline.us/#louis-vuitton-outlet-store $83.43louis vuitton handbags $77.59louis vuitton $64.85louis vuitton handbags $93.4louis vuitton purses http://louisvuittonoutletonlinestore.us/ $42.50Monogram Canvas $80.19 Monogram Rayures $53.44 Antheia Leather $94.88 Monogram Multicolore Canvas
oGresstahg commented on 31-Oct-2012 11:42 PM
http://cheapdesignerhandbagsoutlets.us/ cheap handbags egzwys Wholesale Designer Handbags yzyrwtdiscount handbags fdnxfohandbags Wholesale mywrea discount designer handbags txcyxg
http://cheapdesignerbagsoutlet.us/cheap bags bnnhmdiscount designer handbags zdciacheap designer purses tnwkv wholesale designer Handbags ywcjgcheap handbags psdce
download firefox commented on 04-Nov-2012 10:42 AM
We absolutely love your blog and find a lot of your post's to be exactly what I'm looking
for. Would you offer guest writers to write content for you personally?
I wouldn't mind producing a post or elaborating on many of the subjects you write related to here. Again, awesome weblog!
Sainnacand commented on 08-Nov-2012 01:37 AM
buy clonazepam from india
BoashBow commented on 13-Nov-2012 03:45 AM
buy upjohn xanax online
buy xanax for cheap
buy alprazolam online cheap
where to buy xanax
buy xanax online legally cheap
buy xanax from australia
where to buy xanax online forum
best site buy xanax
buy gador xanax
order xanax no prescription overnight
buy cheap generic xanax online
how to order xanax bars online
xanax no prescription paypal
Gina commented on 14-Nov-2012 09:23 AM
Fantastic internet site you have right here.
Genevieve commented on 14-Nov-2012 10:01 PM
Good day I am so grateful I found your website, I really found
you by accident, while I was looking on Google for something else, Anyhow I am here now and would just like to say many thanks for a marvelous
post and a all round exciting blog (I also love the
theme/design), I don't have time to read through it all at the moment but I have bookmarked it and also included your RSS feeds, so when I have time I will be back to read much more, Please do keep up the great job.
vimax commented on 15-Nov-2012 09:50 AM
Hurrah! In the end I got a web site from where I be capable of in fact take
helpful information regarding my study and knowledge.
IVAN commented on 16-Nov-2012 04:13 AM
Official news: Holloway coach Crystal Palace
2012 Adidas f50
HuoLuoWei will take his assistant Keith miller got to his new post together.
49 Ian - HuoLuoWei left to Blackpool's hurst park, replace the dodge - Milton friedman legacy of crystal palace manager position.
Crystal palace has in the official push special account confirmed the news, saying it will work with its sign four and a half years of contract.
Adidas Predator LZ
mercurial 9

ergdgply684we
profollica commented on 16-Nov-2012 09:03 AM
Wow, awesome blog layout! How long have you been blogging for?
you make blogging look easy. The overall look of your website is
wonderful, let alone the content!. profollica
Yugcsb commented on 17-Nov-2012 05:50 AM
Petr cech: the referee boundary should also be open

Petr cech: the referee boundary should also be open
To this, petr cech think: "I do think the referee the party should also be more exchanges with the public, I think this is good. If you are a player, as long as you out admit: I'm sorry, I made a mistake, I should not diving, not shouldn't dry, then you will be free from admit his mistake and won trust and respect to the referee. It should also, they do the job also is very difficult, we also very understanding, but sometimes need further communication."
Last month, Chelsea complaints at referee carat teng in the field use of John obi mikel racial discriminating language, and the referee I in fa investigation tried to deny this, later on Monday, police also announced give up on the case continue to investigate trial.
Last month, Chelsea complaints at referee carat teng in the field use of John obi mikel racial discriminating language, and the referee I in fa investigation tried to deny this, later on Monday, police also announced give up on the case continue to investigateNike Mercurial Olympic trial.
But say back to drag the more than two weeks of case, petr cech or thought, "if have similar things happen, you'd better have what he said what, if don't do that, just hold up everyone's time stopped, and the football is no good. And, if now it put on record, notNike Mercurial Olympic Chelsea want to seek what, but this is to solve the event. I only hope it early early good."
As for this case, petr cech says it does not want to do too much evaluation: "I didn't come near, as a basic not involved, or did not hear players and referees specific something, I can't say too much."
As for this case, petr cech says it does not want to do too much evaluation: "I didn't come near, as a basic not involved, or did not hear players and referees specific something, I can't say too much."
To this, petr cech think: "I do think the referee the party should also be more exchanges with the public, I think this is good. If you are a player, as long as you out admit: I'm sorry, I made a mistake, I should not diving, not shouldn't dry, then you will be free from admit his mistake and won trust and respect to the referee. It should also, theyNike Mercurial CR7 do the job also is very difficult, we also very understanding, but sometimes need further communication."
But say back to drag the more than two weeks of case, petr cech or thought, "if have similar things happen, you'd better have what he said what, if don't do that, just hold up everyone's time stopped, and the football is no good. And, if now it put on record, not Chelsea want to seek what, but this is to solve the event. I only hope it early early good."

erhdfvd2wfwwr
Yugcsb commented on 17-Nov-2012 06:28 PM
Millwall will be banned, a 13-year-old fans again. Admission

Millwall official website confirmed, one as young as 13-year-old fans admit in on Zhou Yingguan game of the Bolton players Marvin - sauldre, shouted words like "slaves", will be issued an indefinite ban. This child at the same time were also asked to participate in an education classes organized by the club, and wrote to sauldre asking for forgiveness. Sauldre apology accepted, the England U21 international said in the game, several teammates were the racial discrimination of the word attack, including the "slaves".Millwall will be banned, a 13-year-oldNike Mercurial CR7 fans again. Admission
Millwall official website confirmed, one as young as 13-year-old fans admit in on Zhou Yingguan game of the Bolton players Marvin - sauldre, shouted words like "slaves", will be issued an indefinite ban. This child at the same time were also asked to participate in an education classes organized by the club, and wrote to sauldre asking forNike Mercurial Vapor VIII AG forgiveness. Sauldre apology accepted, the England U21 international said in theNike Mercurial Vapor VIII AG game, several teammates were the racial discrimination of the word attack, including the "slaves".

erhdfvd2wfwwr

Post a Comment




Captcha Image

Trackback Link
http://www.harfordsurveyresearch.com/BlogRetrieve.aspx?BlogID=7846&PostID=350866&A=Trackback
Trackbacks
Post has no trackbacks.

Follow Us!

RSSHSR Alumni Engagement Blog


Recent Posts


Tags


Archive

Need Help Getting Started?

Jim Flynn, Ph.D is available to speak at your school.

Alumni Boards, Advancement Professionals, Administrators, Faculty & Staff

 View More





SIGN UP FOR OUR NEWSLETTER





What Our Clients Say:
I highly recommend HSR’s alumni survey services. Dr. Flynn has extensive expertise in designing surveys, advising on strategies to encourage high participation rates and interpreting the data to advance the strategic objectives of the overall alumni program.

He also is able to clearly articulate the results to a variety of audiences. He has assisted staff in mining data to insure it is used effectively and accurately and in accordance with the program’s strategic objectives. He has also prepared and delivered presentations to the survey base so as to enhance engagement objectives and to university leaders to deepen their understanding of complex and difficult issues. This breadth of expertise has allowed us to not only conduct an accurate survey but also to immediately begin to utilize the data in a manner that has advanced our program.

Patricia McLaughlin, J.D.
Senior Vice President of Institutional Advancement
Johnson & Wales University
Providence, Rhode Island 02903