Measuring Twitter engagement: An example

The European Commission’s EMPL presence on Twitter in 2012: Engagement

Listening to the followers and engaging with them is a key action towards understanding what they say and want. The “engagement rate” concept has recently been introduced to explain and measure interaction of social media/networks users on different platforms. According to the literature, measuring engagement on social media platforms and on Twitter, in particular, raises a number of questions which have not been completely answered to date.

A number of commercial tool owners claim that their algorithms work very well, but their claims have not been yet validated by the research to date. For instance, Socialbakers[1] and Kaushik[2] created formulae that calculate the Tweet engagement rate, based on a number of metrics.

Figure 1: Socialbakers formula

Figure 1: Socialbakers formula

According to the Socialbakers formula (Figure 1) the engagement rate of the three account holders would be 12 for Social Europe, 12 for EURes and 43 for Commissioner Andor. The average rate of the three account holders would be 19 (Table 1).

Table 1: Engagement rate according to Socialbakers formula

Table 1: Engagement rate according to Socialbakers formula

Kaushik proposes another set of formulae, which provide some indicators about amplification, conversation and applause rates (Figure 2).

Figure 2: Kaushik's formulae

Figure 2: Kaushik’s formulae

Amplification indicates the rate a tweet gets retweeted. Conversation represents all the replies to a tweet while Applause designates all favourites (Table 2).

Table 2: Engagement rate according to Kaushik’s formula

Table 2: Engagement rate according to Kaushik’s formula

What both Socialbakers and Kaushik do not provide is a scale/benchmark to enable one comparing the results and therefore making judgements about the engagement rate.

Klout[3] provides a website and mobile application that enables users to get a score, a so called “Klout” score, which is based on a methodology that measures one’s activity on multiple social media and networks platforms. The score range is from 1 to 100. Klout has never disclosed the algorithm used to score people activity on the social media/networks platforms.

The Twitter engagement rate results when looking at four categories of metrics (replies, favourites, RTs and mentions) that are part of the formulae described earlier. The metrics reflect degree interaction that occurred between the account owners and their followers. The outcomes resulting by applying the two formulae cannot be compared to a scale since both Socialbakers and Kaushik do not provide a benchmark. Therefore, as the four metrics categories are part of this research I would suggest a solution that may provide a better picture of the three account holders’ engagement on Twitter. The solution helps answer one of my research sub-questions and it is not intended to be a benchmark, but a logical set of figures that could be discussed in relation to three areas: a) aggregate annual engagement breakdown, b) aggregate engagement country coverage and c) ratio: engagement, annual follower growth and tweet volume.

  1. a) Aggregate engagement breakdown in 2012
Figure 3: Annual aggregate engagement breakdown 2012

Figure 3: Annual aggregate engagement breakdown 2012

The aggregate engagement breakdown in 2012 resulted from summing-up the four metrics categories that indicate the engagement behaviour throughout the entire year 2012 (Figure 3).

The highest density of the engagement activities involving all three account holders together occurred in September 2012 with the highest number of replies, favourites and RTs. The highest number of mentions was recorded in December 2012 during the chat hosted by Commissioner Andor. The lowest density was recorded in July. In terms of the four engagement categories, RTs are in the leading position with the highest constant occurrence throughout 2012, while the replies were insignificant from January to August and they slightly increased from September to December 2012.

  1. b) Aggregate engagement country coverage

This indicates the interaction distribution by country and the intensity of the interaction in 2012, which varies from one EU country to another. Given the inexistence of a benchmark, I considered the full sum of metrics as a 100% engagement reference, to which I compared the percentage by individual country. If the 100%, in theory, would cover the 27 EU member countries (in 2012), I divided 100 to 27 and that makes a 3.7% average engagement by country. I then established a five level Likert[4] scale where I labelled each level and assigned a relevant range percentage to each of them, from 27.3% (the best, recorded by a country) to the poorest, which is 0.0% (Estonia, with 2 interactions).

Figure 4: Aggregate engagement country coverage

Figure 4: Aggregate engagement country coverage

I tried to balance the scale in such a way to make a reasonable categorisation and ranking based on the 3.7% average: outstanding (between 27.3% and 16.1%), good (between 4.6% and 2.1% and the average of 3.7% I consider closer to the level “good”), average (between 1.4% and 1.0%), unremarkable (between 0.8% and 0.2%) and poor (0.1% and 0.0%).

The aggregate engagement country coverage based on this scale is pictured in Figure 4, while the individual country results are available in Table 3.

Table 3: Aggregate engagement (Country coverage)

Table 3: Aggregate engagement (Country coverage)

In Table 3, engagement volume sums up the four engagement parameters: RTs, mentions, replies, and favourites. In terms of engagement country coverage the results are as follows:

1) Outstanding coverage: Belgium, Spain and the UK (so-called “old member countries”)

2) Good coverage: The Netherlands, Italy, France, Ireland, and Greece (so-called “old member countries”)

3) Average coverage: Sweden, Germany, Latvia, Hungary, Portugal, and Romania (combined old and new member countries)

4) Unremarkable coverage: Austria, Poland, Denmark, Luxembourg, Finland, Slovenia, Cyprus, and Bulgaria (combined old and new member countries)

5) Poor coverage: Slovakia, Czech Republic, Malta and Estonia (new member countries).

The statistics exclude countries outside the EU as well as suspended Twitter accounts. There was no coverage in one EU country: Lithuania.

  1. c) Ratio: Engagement, follower growth and tweet volume in 2012

Both ratios “Engaging followers vs. follower growth in 2012” and “Engagement volume vs. tweet volume” are also two relevant set of figures that I consider worthy of examination, when analysing the Twitter engagement of the three account holders (Table 4).

Table 4: Engagement, follower growth and tweet volume

Table 4: Engagement, follower growth and tweet volume

The highest and lowest ratios, which indicate the percentage of “engaging followers” from the total of the “follower growth”, is respectively 6% (1332/21097) in September and 1% (166/19025) in July 2012.

The ratios indicating “engagement volume” vs. “tweet volume” are 43% (184/426) highest in March and 23% (72/318) lowest in May 2012.

Figure 5: Ratio - Engagement, tweet volume & followers

Figure 5: Ratio – Engagement, tweet volume & followers

The engagement algorithms introduced previously are clearly the early stages of establishing standard engagement formulae, which should contribute to obtaining more relevant information leading to better and relevant judgements.

Previous articles on the same subject

Tweet me a URL and make your communication richer

Why mentions on Twitter help people communicate

The European Commission’s EMPL presence on Twitter in 2012: Content languages and hashtagging

Why mentions on Twitter help people communicate: The European Commission’s EMPL presence on Twitter in 2012

[1] http://www.socialbakers.com/

[2] http://www.kaushik.net/avinash/best-social-media-metrics-conversation-amplification-applause-economic-value/

[3] http://klout.com/

[4] http:// http://www.socialresearchmethods.net/kb/scallik.php

Leave a reply, don't be shy! :)

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s