If you can measure it, you can improve it!

    Who won the SEA Games?

    Now that the 2017 edition of the SEA Games is in the books it may be interesting to take a look at how the various teams really performed. We're going to do this in several ways comparing team results based on medals, points, population, and team size. We're also going to propose a different and better way to determine the winner of multi-sport events like the SEA Games so that final standings are more inline with how sport contests are actually scored.

    There is no 'official' way to determine the winner of a multi-sport competition like the SEA Games, the Olympics, the Commonwealth Games, or SUKMA. The accepted method, the ubiquitous medal count reported by the press, seems to have been adopted by governments, national sport bodies, and the public without giving it much thought. Additionally, when we talk about medal counts we really mean gold medal counts because silver and bronze medals have no real value in determining the final standings.

    The gold medal count

    Though traditional and widely accepted as the one and only way to rank teams in multi-sport contests, the gold medal count is a poor gauge of performance. It presents a limited view of overall performance because silver and bronze medals have no significance in the final standings other than to help rank teams that did not win any golds.

    Table 1 shows the official results of the SEA Games as reported by the KL2017 official website.

    If a team won any gold medals then that becomes the significant ranking factor. This can be seen by comparing the total medal count for Vietnam, Singapore, and Indonesia. Indonesia won more total medals but fewer gold medals than both Vietnam and Singapore so is ranked lower (5th) than both Vietnam (3rd) and Singapore (4th). This same situation occurred between Lao and Cambodia.

    Table 2 shows the final standings if total medal count is used to determine overall rankings.

    Obviously the total medal count is not really useful because all medals receive the same value; bronze medals are just as good as gold if we just count total medals. This would never be accepted so a proper proportional value must be established for each medal.

    A point system makes every medal count

    The purpose of a point system is to apply proportional value to each color medal. A simple 5-3-1 system awards five points for a gold medal, three points for a silver, and one point for a bronze. Table 3 shows what happens to the final standings of the 2017 SEA Games result if we score the event using a 5-3-1 scoring system.

    Malaysia is still the clear winner but notice that Singapore is now in 3rd place over Vietnam, and Lao and Cambodia have flipped their 'official' 8th and 9th positions. Vietnam won more golds but Singapore performed better overall with many more second and third place finishes. Likewise Lao had more third place finishes and thus managed to outscore Cambodia using the point system.

    A point system makes every medal matter to the overall result and is a much better indication of team performance than simply counting gold medals. It's a direct evaluation of performance and it's transparent and easily understood by sport associations, athletes, fans, and the media.

    Establishing any type of scoring system will affect how countries decide which sports to contest and how their teams are selected. Doing a point analysis after the fact, as we're doing here, is interesting but has no motivational effect on teams or athletes unless it is implemented before teams are selected. If countries attending the Games know that final standings will be established based on points then the makeup of the various teams and overall strategies might change.

    Scoring sports rather than events

    Multi-sport games are nothing more than a collection of separate sporting contests held at the same time and in the same place, and the overall medal count is an artificial way of keeping track of event winners; tracking the performances of athletes rather than teams.

    A good argument can be made that scoring sports rather than the events within those sports make more sense in large, multi-sport competitions. Sports with many events -- swimming or athletics, for example -- add disproportionate weight to overall results while team sports contest for only one medal.

    Scoring the various sport tournaments rather than the events within those tournaments would remove this disparity. Countries could contest for one medal in swimming by winning the swimming meet and for one medal in football by winning the football tournament thus giving both sports equal weight in the overall results. In multi-event sports, medals would still be awarded to the event winners just as they are now but the overall results of the sport competition would be reduced to the top three teams in each sport. This is a unique way of determining the results of a multi-sport event that has never been done to our knowledge.

    Other ways to evaluate team performance are population and team size

    Comparing point production based on population and team size uncovers some surprising insights but it's only significant in a sociological or developmental sense. It has no application to actual competition but should be part of any post-event analysis by sport associations.

    Table 4 compares point production relative to a country's population. This indicates the overall nature of sport participation and training within a country and neutralizes the direct competitive advantage that large countries usually have over smaller ones in sport competition.

    Fourth ranked Singapore now moves into first place based on their powerhouse performance of almost 92 points per million population. Likewise Brunei, with less than half a million in population, does well in this ranking moving from tenth place into second with 55 points per million population.

    This kind of comparison has to be considered in context though. Brunei's performance may not have raised any eyebrows during the Games but look at what they accomplished. Five silver medals and nine bronze which means they were in the battle at least in a few events beating out both Lao and Cambodia with 7 and 16 million people respectively in the silver medal count. This metric is also associated with economic strength with countries scoring low generally having weaker economies relative to those scoring higher.

    By comparing point production to team size we're able to calculate the average number of points contributed by each athlete to their team's final point tally (Table 5). This gives an indication of the overall strength and quality of each national contingent relative to other teams.

    Malaysia leads the way in this metric with each Malaysian athlete contributing 1.24 points on average to the team total. Vietnam and Singapore fill out the top 3 spots with 1.08 and 0.92 points per athlete respectively.

    Since the competition was at 'home' travel and housing costs were lower so it's reasonable to expect that Malaysia would pad their contingent with some younger, less experienced athletes just to get them exposure to international competition, and their total team size indicates that this is probably what they did. Younger athletes tend to be less productive than experienced performers so it's unusual for Malaysia to top this particular metric and represents a truly remarkable performance by the team overall.

    Other analyses that could be done include comparisons based on sex and sport. If the athlete list were broken down by sex then the same kind of comparisons as those above could be made based on male and female performance. Unfortunately this data was not available on the official 2017 SEA Games website.

    Also, as noted earlier, scoring sports rather than events is an alternative method for gauging success in multi-sport competitions. If hosts are able to provide the results of the various sports events in ways that lend themself to analysis it would be interesting to see how the final team standings might change.

    In summary, simple gold medal counts offer a shallow and incomplete evaluation of how well a team performs in multi-sport competition. A far better way to evaluate performance is by scoring these events using a point system that recognizes the value of each medal, not just the gold ones.

    Bill Price (price@sportkid.asia) is the owner and Chief Data Scientist at Sportkid Metrics.


    Don't miss any content
    Subscribe Now!

    What would you do differently if there were no such thing as talent? - 04 April 2021

    Athlete development measurements and the lingo that goes with them - 21 March 2021

    Retention and Training Age - 07 March 2021

    Fear of missing out is hurting youth sports - 23 October 2018

    Deliberate practice vs. late specialization - 24 September 2018

    Is talent identification even possible? - 17 September 2018

    Who won the Asian Games? - 10 September 2018

    Re-thinking the mission of Malaysia's sport associations - 03 September 2018

    Using maturity offsets to determine age at peak height velocity - 27 August 2018

    The youth sport talent illusion - 13 August 2018

    The tip of the iceberg - 30 July 2018

    7 things youth sport coaches should know - 25 June 2018

    Who is responsible for athlete performance - 18 June 2018

    Creating a culture of achievement in sport - 05 June 2018

    Sport development in the headlines (sort of) - 28 May 2018

    Who won the Commonwealth Games? - 23 April 2018

    Kaizen: Improving sport administration will improve performance - 02 April 2018

    What can Malaysia learn from Norway about sport development? - 05 March 2018

    Dealing with more than one email address and other communication ideas - 26 February 2018

    What can you do to work more efficiently? - 19 February 2018

    LTAD: Training to compete - 22 January 2018

    Sport clubs are the lifeblood of national sport development - 15 January 2018

    Take a chance! - 18 December 2017

    How we calculate age in youth sports can have benefits and consequences - 11 December 2017

    Can bio-banding help reduce the relative age effect in sport? - 04 December 2017

    Understanding the role that intrinsic and extrinsic motivation play in the athlete development process - 20 November 2017

    Great expectations: Expect more, get more! - 14 November 2017

    Why process is more important than outcome in a learning environment - 25 September 2017

    Sport associations are embracing physical literacy training - 18 September 2017

    Creating a true sport development system in Malaysia - 11 September 2017

    Who won the SEA Games? - 04 September 2017

    KL2017: Reporting individual sport results deserved better planning - 29 August 2017

    Can we please forget about ways to identify talent and just work on getting more athletes? - 07 August 2017

    Using the team selection process to boost motivation and increase athlete participation - 24 July 2017

    LTAD: The Train to Train stage - 10 July 2017

    LTAD: The Learn-to-Train stage - 26 June 2017

    Athletic training for youngsters - 12 June 2017

    Visualization and imagery in sports - 05 June 2017

    Young, single-sport athletes suffer more injuries and do not reach their full potential - 29 May 2017

    Transformational vs. transactional coaching - 23 May 2017

    Will they come back tomorrow? - 08 May 2017

    Advice to parents of young athletes - 01 May 2017

    Is VIP leadership of sport associations a good idea? - 22 March 2017

    What happens after an athlete's initial introduction to sport? - 27 February 2017

    "Where do athletes come from?" - 16 January 2017

    Understanding sport talent pathways - 09 January 2017

    Make 2017 the year of the growth mindset - 02 January 2017

    Teaching physical literacy skills in youth sport practices - 12 December 2016

    Developing sport from the ground up - 06 December 2016

    Pay for what you want - 21 November 2016

    The 10,000 hour rule: "Not for the faint of heart nor for the impatient" - 14 November 2016

    Parent involvement in their child's sport participation sometimes backfires - 07 November 2016

    How to do the measurements for determining peak height velocity (PHV) - 24 October 2016

    A foreign coach is not always the answer - 17 October 2016

    Tips on creating an effective coaching environment - 10 October 2016

    Peak height velocity and aerobic development - 26 September 2016

    Early sport specialization is still not a good idea - 19 September 2016

    What kind of data do we need to develop sports? - 13 September 2016

    The attrition and transformation models of sport development - 05 September 2016

    Solve for <x> - 29 August 2016

    Artificial elimination of athletes from training and competition hinders sport development in Malaysia - 15 August 2016

    Time is the most important factor in talent development - 01 August 2016

    What if opportunity never knocks? - 13 June 2016

    The long-term athlete development framework offers youngsters a chance at sport success and an active and healthy life - 06 June 2016

    Early sport specialization is not a good development strategy - 30 May 2016

    What does a declining population mean for sport? - 2 February 2016

    Coaching 'flow' - 11 November 2015

    The coach's role in creating a deliberate practice environment - 02 November 2015

    When should athletes specialize in a single sport? - 11 September 2015

    The Holy Grail of health, wellness, and sport development - 1 September 2015

    Revisiting the 10,000 hour rule - 10 August 2015

    The power of 'not yet' - 20 July 2015

    Let's stop trying to identify sport talent and start developing it - 22 June 2015