RealGM

After the 2009 season, former Baseball Prospectus author and quantitative consultant Tim Kniker developed a managerial reliever-use rating system, which he dubbed Bullpen Management Above Random. BMAR’s purpose is simple: It’s a way of assessing how closely each manager lined up his best relievers with his team’s highest-leverage relief opportunities. Essentially, it’s a stat that tells us how often a manager’s bullpen moves made fans lie awake and wonder, “Why him?”

BMAR rates managers based on their relievers’ Weighted On-Base Average allowed to righties and lefties over the previous 12 calendar months.2 BMAR’s baseline — the standard by which all managers are judged — is a team’s theoretical distribution of bullpen outings had the manager assigned relievers using eeny, meeny, miny, moe. To produce the ratings, Tim sums each bullpen’s actual totals of left- and right-handed batters faced and calculates how much more effectively the manager could have distributed the same workload, given the relievers on his roster at the time. You can think of each manager’s BMAR as an answer to this question: In light of the bullpen he had, how much better (in wOBA points allowed) were the relievers he did choose than the relievers he could’ve chosen at random?

A couple of caveats: First, while we know which pitchers were on the active roster on a given day, we don’t know which pitchers were actually unavailable (or less effective than usual) because of overuse, illness, nagging injury, or a stubborn latch on the bullpen-bathroom door. And we’re assessing only one aspect of bullpen management: which reliever the manager chose once he decided to make a move. If some managers are better than others at judging when to take out their starters, or at keeping relievers fresh by limiting how often they warm up or enter games — and some of them undoubtedly are — then this method will miss that extra value. And since we’re simply reassigning each team’s actual matchups against left-handed and right-handed batters to the most logically leveraged scenarios based on ability, we’re also overlooking the ability of some managers to gain the platoon advantage more often. But we can judge managers independent of bullpen quality: To adjust for the fact that some managers have better late-game relievers to work with, and can therefore make more of a difference by choosing their best available reliever at a crucial moment instead of their mop-up man, we’ll compare each manager to a personalized standard of how well he could have done with his own array of relievers.

According to Kniker’s wOBA-to-wins conversion, the difference between the best and worst manager by BMAR in a given season is on the order of four wins: not enough to turn a good team into a bad one (or vice versa)

Matheny and the Blue Jays’ John Gibbons may have been among the worst bullpen managers this season, but they didn’t prevent their teams from taking division titles.

Skin Blues wrote:He has had one of the best relief pitchers in baseball for three years. Last year that guy was second fiddle to Casey Janssen and Sergio Santos for most of the year (who were both terrible), and this year, to Roberto Osuna (who was good but not nearly as good as Cecil) and Miguel Castro (who was terrible). Castro was handed the job, and kept it for all of April, after one bad inning as closer from Cecil. It's no surprise that a stat that shows how innings are assigned to relievers gives him a poor rating.

Fairview4Life wrote:I am also not sure why you are so focused on the title of 'closer'. In the second half, when Cecil was insanely good, Gibbons used him in most of the Jays highest leverage spots. Saving him for the 9th inning is the kind of thing we shouldn't want Gibbons to do. Cecil was his man when it mattered for like 2 months of the year once he realized he could trust him, and he didn't need a day off or whatever (another thing those numbers admittedly don't account for).

vaff87 wrote:This thread is nothing until Randle chimes in.

Skin Blues wrote: and Sanchez and Lowe were right behind Cecil at 1.20 and 1.10.

Lateral Quicks wrote:Lowe was arguably our best reliever, and Gibby used him frequently for mop-up duty.

vaff87 wrote:This thread is nothing until Randle chimes in.

Randle McMurphy wrote:

vaff87 wrote:This thread is nothing until Randle chimes in.

Seems I've been requested here.

It largely comes down to Liam Hendriks and what Gibbons thought of him. Hendriks was, in actuality, the team's best statistical reliever all season (topped in both WAR/FIP). He especially annihilated RHBs (.219 wOBA). Despite this, Gibbons used him as a mop-up man to keep games from getting away. He did this because he did not trust the guy in the slightest based on how poorly he had done in high leverage situations earlier in the season (.433 wOBA in such spots). Whether you think that was a mistake or not (I do, I think Hendriks should be given more opportunity in 2016), he thought the guy couldn't handle the pressure and managed his bullpen accordingly.

This all meant, though, that Gibbons would have been docked in this particular stat virtually every time he used somebody else on the team over Hendriks (particularly against RHBs). Hell, Hendriks would have often even been the statistically optimum choice over Cecil or Osuna. It's no small wonder, then, that Gibbons' ranking on this stat is so poor this season and why his 2015 is significantly weighing down what would have been an otherwise above-average ranking in the previous two years. Of course, Cecil being an absolute untrustworthy mess (he couldn't even throw strikes to LHBs for a good stretch) for a few months there also would have hurt Gibbons as the stat would have simply looked at Cecil's overall 12 calendar months and not how poorly he was doing at that current point in time in the first half of the season. The possibility of a good-to-great reliever dropping off quickly like that (and the inability to actually adjust for it) would actually seem to be quite the flaw for BMAR.

I have other questions about the methodology here (such as whether they adjusted strictly for Sanchez's far better numbers as a RP or whether they just looked at Sanchez in the overall which would have hurt Gibbons appreciably in this stat), but if you're looking for answers here, Gibbons' odd but somewhat understandable relationship with Hendriks stands out prominently.