At the end of September 2012 Newcastle signed an eight-year contract with manager Alan Pardew. This followed a fantastic 2011/2012 season where Newcastle finished in fifth place, having long challenged for the coveted fourth spot, which turned out to be a bit of a dud. A few weeks later, Pardew said he fully expected Newcastle to challenge for Champions League qualification again this season.
That is an unrealistic aspiration for the Tynesiders. Newcastle is not among the six best teams in the Premier League, let alone the top four.
Don’t get me wrong, I am not trying to take anything away from Newcastle’s success last season. I hope they will bask in the glory of finishing above Chelsea, Everton and Liverpool (I know…) in a 38-game league. Their transfer market activity gets the highest mark after essentially trading Andy Carroll (£35m) for Cisse (£10m), Ba (free), Cabaye (£5m), Santon(£5.3m), Ben Arfa (£5.75m), Tiote (£3.5m) and Debuchy (£5.5m).
However, the Opta Sports dataset from the 2011/2012 Premier League season, kindly provided by MCFC Analytics, suggests it was momentum and “team spirit” (aka chance) rather than quality that dominated Newcastle’s last season.
As you can imagine, there is a strong correlation between goal difference and the total amount of points achieved in a season. After all, any individual point collection results from a neutral goal difference (one point) or a positive goal difference (three points). According to the data, a team’s goal difference can predict 96% of the variance in its points total. The graph below shows the distribution of teams according to goal difference and points for the 2011/2012 season.
There is only one statistically significant (95% individual confidence intervals) outlier – Newcastle United. Using only their goal difference as a predictor, Newcastle would have been expected to get approximately 55 points (1.46 points/game). That is 10 points less than the 65 they actually got (1.71 points/game). 55 points would have resulted in a seventh place finish last season, just behind Everton.
But was their goal difference an accurate reflection of their play on the field? Let’s look at the factors behind the goal difference; namely goals scored and goals conceded. It is conceivable that you score more goals the more big chances you create. Likewise, you will probably concede more goals the more big chances you allow against you. The dataset supports this set of hypothesis.
According to Opta Sports data, Premier League teams last year on average scored 0.78 goals per big chance they created. That does NOT mean the teams scored on 78% of their big chances – as anyone who watched Liverpool last season can attest – but you can also score goals without a big chance. Think of Cisse’s wonder goal away to Chelsea last season and you will understand what I mean. So in reality the conversion rate of big chances will be lower than the 78% quoted above. The data from last season suggests teams on average score approximately 15 goals resulting from other situations than big chances, and on 57% of their big chances.
Linear regression shows the amount of big chances produced explains 71.3% of the variance in goals scored. In other words the correlation is not as strong as that between points and goal difference, but big chances created is still a statistically significant variable in predicting the amount of goals scored by a team.
Unsurprisingly the top three teams created the most big chances with Man City and Arsenal creating 115 each and Man United creating 112. After that follows Spurs with 96 and Liverpool with 93, where after there is a marked drop to Chelsea with 68 big chances created for the season. Newcastle created 52 big chances. Only five teams did worse than Newcastle in terms of creation: Norwich (51), Stoke (48), QPR (42), Bolton (45) and Wolves (51). Two of those teams were relegated. Yet only five teams scored more goals than Newcastle.
In fact Newcastle beat the projections so much that they scored more than a goal for every big chance they created – 1.08 goals to be exact. That was the best conversion rate in the league. Liverpool were at the other end of the scale, scoring only 0.51 goals per big chance created. Liverpool needed to create almost twice as many chances as Newcastle to score the same amount of goals last season!
At this point, if you are a Newcastle fan, you may be rubbing your hands, feeling extra-smug about the sale of Andy Carroll to said Liverpool – but don’t get too excited just yet. Surprisingly, it appears the chance conversion has very little to do with the quality of your strikers. Observe the graph below. It shows the goals to big chance ratio for all teams in the PL last season, sorted by final league position.
Looking at the graph there is no clear pattern that the big teams score more goals per big chance they create. Last season’s top scorer, Robin van Persie, scored 30 goals but had 46 big chances (ratio of 65%). Statistically, a team’s ability to convert a big chance to a goal is insignificant (90% confidence intervals) in determining the amount of goals a team has scored. Contrary to popular narrative it appears the defining difference between top and bottom is not the ability to score on your chances, but the ability to create the chances in the first place.
Newcastle created 52 big chances in the 38 games last season. Based on linear regression they would have been expected to score a total of 44 goals from that amount of chances. They scored 56. It is important to consider that the sample sizes are relatively small for this so chance can play a significant part.
In the other end of the pitch, Newcastle allowed 78 big chances against them. Only four teams did worse; Norwich (101), Aston Villa (79), Bolton (101) and Wolves (94). Allowing 78 big chances against you should on average result in concession of 60 goals. Newcastle conceded 51. Only two teams in the league, Man City and Chelsea, had lower rates of goals per big chance conceded.
Again, the “ability” to avoid a goal from a big chance, does not look like an ability at all, judging from statistical analysis. It is an insignificant predictor (90% confidence intervals) of the amount of goals conceded. Perhaps a good goalkeeper will save more big chances, but many other factors come into play, including the size of the chance, the quality of the finish, luck and so on. All of these factors are more or less outside the control of the defending team.
So where does this lead us? Based on the skills of creating chances and avoiding chances against, Newcastle should have scored 44 goals and conceded 60, resulting in a goal difference of -16. Using multiple regression analysis we can create a new football metric called xPts; expected points total. The dependent variable is total points for the season and the independent variables are big chances created and big chances conceded. Based on the data from the 2011/2012 season that gives a model for xPts looking like this:
A rough interpretation of the model is teams pick up approximately 31 points as a basis and then an additional half point for every big chance they create over the course of the season. They lose approximately 0.2 points for every big chance they concede. The creation of chances is a better predictor of the total amount of points, than the concession of chances. This is in line with earlier findings that a goal scored is more valuable than a goal conceded is harmful. Likely this has something to do with the unbalanced nature of the points system. Although the concession of chances is statistically insignificant due to multicollinearity (that is, teams that are good at creating chances are generally also better at preventing them), I have chosen to include it in the model, as it is empirically an important part of a football game.
Inserting Newcastle’s numbers from last season in the newly derived xPts model we would expect a team with those stats to have picked up 43 points (30.696 + 0.502*52 – 0.183*78 = 42.526). The entire Premier League table for the 2011/2012 season based on xPts looks as follows:
The top four remains exactly the same, but at the bottom, Wolves would have survived if sun and wind was distributed evenly as would Blackburn (comfortably). Newcastle would have seen a less impressive 15th place finish, four points above relegation.
Is that Newcastle’s true talent level then? Probably not. Football is a complex game, and scoring goals alters the dynamic of a game. If Newcastle have been leading relatively more games than the statistics suggest they ought to, they will also have attacked (and tried to force a goal) relatively less than the model would expect. What it does suggest is Newcastle benefitted hugely from factors outside their influence. Before the season the smart money would have gone on Newcastle finishing outside the top 6, not on them qualifying for the Champions League. That is easy to say now, in hindsight, but the data from last year supports the conclusion.
If, as proposed in this post, converting chances to goals and avoiding goals on big chances are not primarily down to skill but to coincidence, it is prudent to expect those rates to regress back towards a more normal rate in the future. That is likely what has happened to Newcastle this season. Unfortunately I have no data from this season to back up this claim, but if someone does I would expect their numbers in terms of chances created and conceded, compared to goals scored and conceded to be more along the lines of the league average.
But Newcastle’s good season was not in vain. A season of over performance can lead to actual improvements. Prize money, European qualification and increased fan base (fans ARE vain – for further evidence see Simon Kuper and Stefan Szymanski’s fantastic book Soccernomics from 2009) all matter to the bottom line. Their recent success also means Newcastle will be a more attractive destination for good players, so they can use some of that extra money in a good way. If they take the opportunity to improve the squad and become a better team – they might end up earning that luck. If they maintain status quo and expect results similar to last season they will be bitterly disappointed.