WC 2014 Penalty Kicks and the Data Analysis Arms Race.
During the World Cup, my Twitter feed confirmed what everybody already knew: Goalkeepers are different. When all the “normal” people were dreading the prospect of extra-time and a shoot-out, most goalies were hoping for it, cheering for the keeper, no matter what team he was playing for.
So how did they do? In the last few years, penalty kick data analysis has exploded, giving both the shooter and the keeper additional tools and better insight in the battle for that little edge that will make the difference between a goal and a save.
I’m still not smart enough to use some of the data available from Opta or Infostrada so I had to do it the way our granddaddies did it: by watching grainy video on You-tube. The result is a somewhat correct overview of the penalties taken in the World Cup: 49 total, 13 in regular play, 36 in shoot-outs.
According to the data in Ben Lyttleton’s excellent book “Twelve Yards” the historical PK conversion rate in the World Cup is 80.3% in open play and 71.3% in shoot-outs. In the 2014 tournament, the conversion rate in shoot-outs is almost equal (0.9% difference) but the conversion rate from open play is 12.3% higher with only a single miss in 13 penalties. (Benzema missed a spotkick against Switzerland in the 31st minute, when the French were already up 2-0 by then and dominating the game).
Lyttleton also discusses the two basic penalty-taking approaches:, the “Goalkeeper-dependent” method of shooting which relies on the goalkeeper moving first (once the goalkeeper moves, the shooter sends the ball the other way, or through the center) and the “Independent method” where the shooter picks a spot and relies on his shooting skill to put the ball in a place the goalkeeper can’t get to. According to Littleton’s research, the latter is the more common approach while the former is currently the more successful.
How to rate goalkeeper performance against penalty kicks.
A few months ago, I wrote a blog suggesting that maybe it is time we took a more nuanced approach in analyzing a goalkeepers’ performance in regard to penalty kicks. I envisioned a calculation of “expected saves” vs. “actual saves” based on the placement of the shot in areas with high, medium and low save probability. (On Twitter Simon Gleave pointed out to me that Ken Bray came up with a similar graphic in his book “How to Score”. I haven’t read that book yet but I’ll be the first to admit that my outline is not based on any kind of scientific research. Doctor Bray’s graph is probably much more accurate than mine, but until I get a copy of his book, my unscientific graph will have to do.)
It should be pretty straight forward, take the area in which the shot was placed, take the probability that the keeper will make the save, do that for each penalty the keeper faced, calculate an “expected saves” amount and compare it to the actual saves made. The problem that arises (for me, anyway) is how to account for a keeper guessing the wrong way. In that case, the quality / placement of the ball becomes less relevant because the keeper never gave himself a chance to make the save.
With that in mind, I charted the placement of all the penalty kicks but differentiated between those where the goalkeeper guessed correctly, and those where he didn’t. This is what I came up with:
It shouldn’t come as a surprise that the majority of the shots in the medium/high zone have the goalkeeper either saving the shot, or moving the wrong way. There are only two kicks placed in the high/medium zones where the goalkeeper guessed correctly but didn’t make the save: Neymar’s penalty against Pitiekosa in the Brazil-Croatia game and Rodriguez’ spotkick against Cillessen, the winning spotkick in the Holland-Argentina shoot-out. Both goalkeepers made contact on those shots but deflected the balls into the goal. (We could get into a discussion about deflecting with open palm vs. deflecting with a fist but that’s not the topic here)
Interestingly enough both missed kicks (Willian against Bravo and Jara against Cesar) occurred even though the keepers moved the wrong way.
The analysis of the saves is more problematic. Did the goalkeeper force a shooter using the “goalkeeper-dependent” method into a bad shot by not moving or was it simply a case of bad execution by a player using the “Independent” method? Did the goalkeeper read the player who shot through the middle correctly or did he decide to react to the shot rather than guess?
Guessing correctly increase the chance of stopping a kick by about 30% . But even guessing correctly will mean nothing if the penalty is well taken. (Keepers saved only 14% of the kicks in open plays when they guessed correctly, but 44% in shoot-outs).
Goalkeeper Performance: Battle of the Back-ups.
The most heralded penalty save performances in the tournament were by Cesar (back-up at QPR), Romero (back-up at Monaco) and Krul (starting at Newcastle but third choice in the Holland squad). So how do their stats stack up against the other goalkeepers who faced at least 4 penalty kicks in the tournament. And do these stats tell the whole story?
With the average for penalty conversions being around 80%, it means that the average goalkeeper saves 1 in 5 penalty kicks or 20%. Guessing correctly on all five kicks roughly doubles that percentage. Using those numbers, the argument can be made that Navas’ performance was sub-par (11% save% while guessing correctly 56% of the time), Krul’s was as expected (40% save% while guessing correctly 100% of the time) as was Cesar’s; while Romero’s was excellent (50% save% while guessing correctly 50% of the time). However, when looking at the quality of the penalties faced, the story changes a little bit:
Of the 9 penalties Navas faced, 7 were well placed in the “low” area, 1 was in the “medium” area and 1 (his lone save) in the “high” area. Similarly, Krul faced five penalties only one of which was clearly in the “medium” area while the other four were in the “low” area, yet he saved two.
In contrast, four of the penalties Cesar faced where in the medium and low areas, while one of the three kicks placed in the high area hit the post. Similarly, three of the four penalties faced by Romero where in the high/medium areas and one barely in the “low” area.
Looking at these charts, one could conclude that Krul’s performance was superior to that of both Cesar and Romero while Navas was just extremely unlucky to face a series of very well-taken penalties at a particularly bad time for both him and the Ticos.
Costa Rica and Holland overview.
Navas’ (and Costa Rica’s) bad luck becomes even more evident if we take a closer look at the penalty placement of the Costa Ricans and the Dutch in their two shoot-outs.
It has been calculated that the team that takes the first kick in a shoot-out wins 60% of the time. However, both Costa Rica and Holland lost the shoot-outs in which they went first. In both cases, it was the team’s second consecutive shoot-out of the tournament, and no team has won two shootouts in a single World Cup finals tournament since Argentina in 1990, when they beat Yugoslavia in the quarter- and Italy in the semi-final.
The Dutch performance is like night and day. Against Costa Rica all four penalties were placed in the “’low” area with Navas guessing correctly twice, while against Argentina three out of four were in the high/medium area, with both Vlaar and Schneijder rushing their kicks, resulting in bad placement and relatively easy, but not less important, saves by Romero.
The Costa Ricans actually did much better than their Orange counterparts. Against Greece they managed to send Karnezis the wrong way on four out of five occasions while putting the ball in the “low” area the one time the Greek keeper guessed correctly. Against Holland, Krul guessed correctly all five times on the Costa Rican kicks, but every single one of these was placed very well, casting serious doubt on the theory that Krul’s pre-kick psychological warfare affected the shooter’s performance.
The Hoek Method: Anticipating vs. Reactive
The shooters didn’t lose the match for Costa Rica, Krul won it for the Dutch. And he didn’t get into the game until a shock substitution in the 118th minute. Which means that we should look into why the Dutch coaching staff made that change to begin with.
Several years ago, Holland goalkeeper coach Frans Hoek wrote an article (which you can read here) in which he categorized goalkeepers as “anticipating” vs. “reactive”. In reading the description, it would seem that starters like Navas, Cillesen and Neuer fit in the “anticipating” category while Krul, Cesar and Romero appear to be more “reactive” like. However, the “reactive” type keeper appears to be better suited for penalty shoot-outs. In addition, Krul is five centimeters (2”) taller than Cillesen, and looks somewhat more intimidating than the Ajax keeper who, at age 25, still can be mistaken for one of the ball boys.
Against Greece, the Costa Rican shooters appeared to favor the “goalkeeper-dependent” method. After the Holland-Costa Rica shoot-out, Tim Krul stated that in their game preparation, Hoek had instructed his keepers to wait as long as possible before committing and then explosively dive to the corner. This method takes away the visual cues a “Goalkeeper-dependent” type shooter looks for in his run-up. Not seeing the expected indications forces the shooter to change his plan just before he kicks the ball, increasing the chances of him making an error. Yet, as shown in the graph, the Ticos got off some excellent shots. But there might have been some kind of “tell” as Krul went the right way on all kicks, saving two of them in the far low corner. Of course, the more penalties a keeper faces in his career, the higher the chance becomes that he chooses correctly five –or more- times in a row, so this could have been one of those times. Only those inside the Dutch camp know for sure, and they aren’t talking.
Admittedly, 13 kicks is not exactly a large enough sample to draw any definite conclusions from but, as the data analysis arms race continues, it appears that the regular penalty takers might be gaining a slight edge for now.
In shoot-outs, the numbers remain relatively stable in comparison to previous tournaments, most likely indicating that near equal progress on both sides of the ball cancel each other out. With the “goal-keeper dependent” method of taking penalties gaining in popularity, the “Hoek-method” of saving penalty kicks might be something keepers and coaches will need to take a closer look at.
Whatever the next developments are in this area, we need better metrics to properly analyze goalkeeper performances in saving penalty kicks. In this article, I took a very amateuristic try at coming up with something. It is my hope that somebody much more qualified in this field than myself, can come up with such a method.