The objective of the world-level datamining completion KDD-2009 was a very common task inside the telecom industry and accurately represents the kind of tasks that are encountered in “real life” (in opposition to the purely “abstracts” tasks that are usually proposed in such academic competitions).
For the KDD2009, the final ranking is based on the average quality/accuracy (AUC) of 3 predictive models that are built using data coming from the “Orange” company (Orange is the number one of the French Telecom). The 3 predictive models to develop for the KDD2009 competition are one Churn model, one Upselling Model and one “Propensity-to-buy” (appetency) model.
The competition was a real “fight-to-the-death” because everybody wanted to demonstrate his superiority on “real world tasks”. At the KDD2009, there were:
- 1299 registered teams
- 7865 entries
- 46 countries:
Argentina Germany Malaysia South Korea Australia Greece Mexico Spain Austria Hong Kong Netherlands Sweden Belgium Hungary New Zealand Switzerland Brazil India Pakistan Taiwan Bulgaria Iran Portugal Turkey Canada Ireland Romania Uganda Chile Israel Russian Federation United Kingdom China Italy Singapore Uruguay Fiji Japan Slovak Republic United States Finland Jordan Slovenia France Latvia South Africa
As usual for such completion, your ranking is based on the (average) accuracy of the (3) predictive models (cross-selling, up-selling, churn) that you create, the higher accuracy, the better ranking. Here is a chart that illustrates the accuracy of the top 100 contestant: the RED dot is the accuracy obtained with TIMi:
I guess that the accuracy obtained with other “main stream” predictive analytics software is somewhere around 0.77 (AUC=77%). Although all major datamining re-sellers participated almost certainly to the completion, they did not reveal their final ranking… So we will never know for sure the accuracy of their tool… I wonder why….
These competition results place TIMi as the best “commercially available” predictive datamining tool in the world (the very few teams that managed to obtain a slightly better results than TIMi were all using software prototypes that are not available to the public). These results were obtained at a vendor-neutral world-level competition organized by qualified university researchers in the datamining field.
The added accuracy of the TIMi predictive models represents a tremendous difference in ROI for a telecom operator.