Wednesday, April 10, 2013

Lunchtime Sports Science: Introducing tanh5

As I mentioned in a previous article on ratings systems, the log5 estimate for participant 1 beating participant 2 given respective success probabilities \( p_1, p_2 \) is
p &= \frac{p_1 q_2}{p_1 q_2+q_1 p_2}\\
&= \frac{p_1/q_1}{p_1/q_1+p_2/q_2}\\
\frac{p}{q} &= \frac{p_1}{q_1} \cdot \frac{q_2}{p_2}
\end{align} where \( q_1=1-p_1, q_2=1-p_2, q=1-p \).

Where does this come from? Assume that both participants each played average opposition. In a Bradley-Terry setting, this means
p_1 &= \frac{R_1}{R_1 + 1}\\
p_2 &= \frac{R_2}{R_2 + 1},
\end{align} where \( R_1 \) and \( R_2 \) are the (latent) Bradley-Terry ratings; the \( 1 \) in the denominators is an estimate for the average rating of the participants they've played en route to achieving their respective success probabilities.

In a Bradley-Terry setting, it's true that the product of the ratings in the entire pool is taken to equal 1. But participants don't play themselves! Thus, if participant 1 played every participant but itself, the average opponent would have rating \( R \), where \( R_1 \cdot R^{n-1} = 1 \). Here \( n \) is the number of participants in the pool.

Our strength estimate for the average opponent faced is then
R &= {R_1}^{-\frac{1}{n-1}}.
There are two extreme cases. If \( n=2 \), then \( R = \frac{1}{R_1} \); as \( n \to +\infty \), \( R \to 1 \).

The limit extreme case is log5; the \(n=2\) extreme case I call tanh5. We compute
p_1 &= \frac{R_1}{R_1 + 1/R_1}\\
p_2 &= \frac{R_2}{R_2 + 1/R_2}\\
\textrm{tanh5} = p &= \frac{ \sqrt{p_1 q_2} }{ \sqrt{p_1 q_2} + \sqrt{q_1 p_2} }.
Why tanh5? We can think of log5 as derived from the logistic function by setting \( \log(R)=0 \) for the opponent's rating; analogously, tanh5 is derived from the hyperbolic tangent function by setting \( \log(R)=0 \) for the opponent's rating.

Note that we have a spectrum of estimates corresponding to each value for \(n\), but these are the two extremes. This also gives a new spectrum of activation functions for neural networks, but I'll explore this application later.

Wednesday, April 3, 2013

Lunchtime Sports Science: Fitting a Bradley-Terry Model

Power rankings are game rankings that also allow you to estimate the likely outcome if two opponents were to face each other. One of the simplest of these models is known as the Bradley-Terry-Luce model (or commonly, Bradley-Terry). The idea is that each player \( i \) is assumed to have an unknown rating \( R_i \). If players \( i \) and \( j \) compete, the probability that \( i \) wins under this model is expected to be about \[ \frac{R_i}{R_i + R_j}. \] This model is very popular for hockey and other games; one commonly seen version is called KRACH.

Let's fit a Bradley-Terry model to the current season of NCAA D1 men's hockey. The Frozen Four starts on Thursday, April 11, so you'll get to see how well your predictions do.

You'll need to have R installed. Once R is installed, install the "BradleyTerry2" package that's freely available for R (thanks to Heather Turner and David Firth). To do this, start R and run the following command; you'll have to pick a source.
Next, download two files from my hockey GitHub - R code that fits a basic Bradley-Terry model and a data file containing the NCAA D1 men's hockey game results going back to 1998.

Make sure both files are in the same directory and run the R code. That's it, you've built a power ranking using a Bradley-Terry model. You should get output that looks like this:

                            ability      s.e.
Quinnipiac              1.687042594 0.5678939
Massachusetts-Lowell    1.480098569 0.5872701
Minnesota               1.428503522 0.5638946
Yale                    1.115338226 0.5641414
Miami                   1.114307264 0.5479346
Notre Dame              1.109912670 0.5523657
Boston College          1.091836391 0.5815233
St. Cloud State         1.079314965 0.5573018

The order should be the same as USCHO's KRACH rankings.

How do we use these ability estimates to predict game outcomes? These values are the logarithms of the ratings I've mentioned above, so first apply the exponential to get the rating, then the estimated winning probability is the team's rating divided by the sum of the team and opponent ratings. For the teams in the Frozen Four we get a power rating of \( e^{1.687} = 5.40 \) for Quinnipiac and \( e^{1.079} = 2.94 \) for St. Cloud State, so we estimate the probability of Quinnipiac beating St. Cloud State to be about \[ \frac{5.40}{5.40+2.94} = 0.65. \] What's your estimate for Massachusetts-Lowell beating Yale?

Monday, April 1, 2013

Lunchtime Sports Science: Cracking a New Sport

This is the first and what will be a series of relatively short pieces on sports analytics. I'll be using a variety of sports for examples, including both team sports and single-player sports, and I'll also make my code available through my GitHub account.

Here are my recommended tools. If you're unfamiliar with some of these, don't worry. You'll pick them up as you go along, and they form a powerful suite that will keep you on the cutting edge even as a professional data scientist.
  1. Hardware - Ideally you want at least 4GB of RAM for larger data sets, but you'll be able to do high-level analysis with almost any modern computing hardware.
  2. Linux operating system - You can certainly do top-notch data analysis using any operating system, but Linux is an excellent (and free) working environment. There are a variety of ways to install and use Linux, but I'd recommend Ubuntu's Windows installer. This will allow you to easily install Ubuntu alongside Windows, and it also makes it easy to uninstall Ubuntu later (if you choose). Ubuntu is just one of the (many) Linux distributions available, but it's very popular and well supported.
  3. R programming language - R is a powerful statistical programming language, and it has thousands of available packages available. If you're using Ubuntu, installing R is simple - sudo apt-get install r-base. That's it!
  4. Python programming language -Python is a powerful and relatively easy to use programming language. One of the most common tasks for sports analytics is web scraping, and Python is an excellent choice thanks to libraries such as Mechanize, Beautiful Soup and lxml. It's also a great language for data cleansing. Installing Python under Ubuntu - sudo apt-get install python.
  5. PostgreSQL database server - There are many ways to store and analyze data sets, but a dedicated relational database server is necessary tool for high-level analytics. PostgreSQL is my personal recommendation, but there are other good choices (such as MySQL). PostgreSQL is free, fast, powerful and has a huge variety of procedural languages available (including R and Python). Installing PostgreSQL under Ubuntu - sudo apt-get install postgresql-9.2.
  6. GitHub account - Setting up a GitHub account is free and will allow you to automatically following any changes to various sports analytics GitHub projects (such as mine). Later, you can set up your own repositories if you'd like to share your own work with other people. Don't forget to install git under Ubuntu - sudo apt-get install git.
  7. We'll start with analyzing hockey. If you'd like to take a look at some of my hockey code and data that I've scraped, you can find them in my hockey GitHub repository. If you've set up Ubuntu and have installed git you can execute the command git clone to make a local copy of my repository.
Here's a basic outline for tackling a new sport.
  1. Understand how teams win - Build a model to project the likely outcome for a team when facing a particular opponent. Example - Krach for hockey (which is based on the Bradley-Terry model).
  2. Understand how teams score - Build a model to project how many goals/points/runs teams are likely to score or allow when facing a particular opponent. Exampe - Poisson distribution and hockey.
  3. Relate the two - Characterize the relationship between scoring and winning or losing. Example - Pythagorean win expectation.
  4. Understand how players score/prevent scoring - Determine which aspects of player performance impact team offense and defense and by how much.
  5. Understand player contribution to winning/losing - This is nearly automatic once you understand the relationship between team offense/defense and team winning/losing.
In my next article we'll build a basic power ranking model for hockey to predict likely game outcomes.