2010 MIT Sloan Sports Analytics Conference Review

On Saturday, March 7, I had the great privilege of attending the MIT Sloan Sports Analytics Conference. Hosted by the MIT Sloan School of Management’s EMS Club, the Conference featured lively discussion about many exciting aspects of the present and future of sport. It took place at the Boston Convention & Exhibition Center in downtown Boston. The conference featured four sets of breakout sessions with a lunch session and the main session in the middle.

Image courtesy sloansportsconference.com

First Session – Baseball Analytics

The first session I attended featured John Abbamondi – Assistant GM of the St. Louis Cardinals, John Dewan – Owner of Baseball Info Solutions, Dan Duquette – former GM of the Boston Red Sox and Owner of Duquette Sports Academy, Shiraz Rehman – Director of Baseball Operations for the Arizona Diamondbacks, and Tom Tippett – Director of Baseball Information Services for the Red Sox. Rob Neyer (ESPN) moderated the panel.

The discussion centered on key aspects of team management in baseball: the importance of being able to quantify defense, injuries particularly as related to pitching, and key parts of a quality baseball organization. I was extremely impressed with all of the panelists – they all came across as being very intelligent and very passionate about their job and the sport. I was particularly impressed with Dan Duquette. The Duke was extremely unpopular during his time with the Red Sox, both for his personality and for some of his acquisitions. After watching him on stage I can understand his lack of popularity – he lacks the sort of charisma that we often associate with quality individuals. Additionally, Duquette is clearly not afraid to make unpopular decisions and isn’t bothered by their reception. However, his lack of public rapport belies an excellent understanding of the intricacies of developing a championship caliber baseball team. One of the key questions that was discussed was why certain small-market teams (Diamondbacks, Twins, etc.) continually contend with large-market (or, more accurately, large-payroll) teams while others don’t (Royals, Pirates, etc.). Duquette focused on minor league player development, saying that teams that consistently contend have a solid decision making process regarding player development while teams that don’t contend don’t have this in place. He also said that the teams that are consistently at the top of the standings succeed because they have a top-quality individuals in every part of the organization – ownership, management, player development, marketing, and coaching – and because of this they consistently have quality players that aren’t overpaid. John Abbamondi talked about the importance within an organization of having sufficient quality information and being able to use it well. This was noted as a key difference between upper-tier and lower-tier teams.

The consensus regarding the development of statistics – this year’s darlings being Hit f/x and Field f/x – is not, as it might appear, a change in focus for baseball executives. Rather, it is a natural step in the progression of the relationship between technology and statistical analysis in baseball. Executives and coaches have been trying to figure out the impact of fielding, pitching, and hitting for years and have been trying to find ways to quantify player ability in these areas but have lacked the tools that are now becoming more widespread.  The goal of the stats is to be able to provide an absolute measure of the value of any player’s value and to properly project his future value. This isn’t a new goal – we simply have better ability to meet it than we did even 5 years ago.

Image courtesy blogs.southcoasttoday.com

One of the areas of growth in analysis that the panelists discussed is the ability to assess potential for injury in pitchers. Duquette pointed to pitcher management as the most important aspect of baseball management. One of the keys to the success of the 2004 Red Sox – according to Duquette – was that the Red Sox 5 starting pitchers took their turn every 5 days for the entire season. No minor league call-ups or middle-relief innings eaters were necessary at any point. The continuity of contribution from the players with the highest level of skill was an incredibly valuable piece of the team’s success. Duquette also discussed the development of technology in recognizing injury potential: Dr. James Andrews has developed a video system that looks at 14 aspects of a pitcher’s delivery. By looking at this analysis, baseball executives can better assess a given pitcher’s risk of injury. Duquette also discussed the importance of the development of mental training for pitchers. Shiraz Rehman and John Abbamondi talked about the value of the information created by Andrews’s information: on the one hand, changing a player’s delivery can significantly decrease the player’s injury risk. However, that risk needs to be balanced against the possibility that changing the pitcher’s motion might decrease his effectiveness. If a player loses the ability to pitch at a certain level, his relative ability to remain healthy loses much of its value. John Dewan talked about the magic 100-pitch number citing a study that his company conducted that showed (inconclusively) that limiting a player’s pitch count to 100 pitches early in the season can do a disservice to him. Some of this seems to be related to the mental aspect of the game. Duquette noted that increasing a player’s workload by more than 10% or significantly altering a player’s regimen are very risky for the player’s health. He also noted that some teams now see Tommy John surgery as a plus because it can strengthen parts of the player’s arm.

After the first session I became involved in a discussion with Tom Tippett, Rob Neyer, and Nate Silver (the creator the PECOTA projection system). Eventually, the discussion dwindled to Tom, several other conference attendees and I. We discussed the Red Sox trade philosophy and tactics and talked a bit about the Roy Halladay deal. Tippett said that, after Halladay was dealt to Philadelphia this past winter, Toronto GM J. P. Ricciardi told the Red Sox that they had made by far the best offer for Halladay’s services, but that the Blue Jays had made a decision not to trade him within their own division. Tippett questioned the wisdom of the decision while recognizing the validity of the perspective. As a result of this discussion, I missed most of the second session and wasn’t able to take any notes.

I caught a bit of the ‘Emerging Analytics’ session. The session featured Michael Forde – Performance Director, Chelsea Football Club, Simon Wilson – Head of Performance Analysis, Manchester City Football Club, Aaron Schatz – Editor-in-Chief, Football Outsiders, and Paraag Marathe, Executive VP of Football & Business Ops, San Francisco 49ers. Kate Fagan of the Philadelphia Inquirer moderated. What I caught of the session featured Schatz bemoaning the lack of access that the NFL gives to the media (with regard to statistics and film). He noted the anti-technology bent of the NFL: coaches are not even allowed to have calculators in the booth with them, let alone a laptop. Paraag Marathe talked about the inefficiencies of player analysis: he talked about the NFL Combine as analyzing players’ football ability by having them play another sport – track & field. He and Schatz also discussed the inherent inefficiencies in trying to create absolute statistics for player comparison. Michael Forde and Simon Wilson made interesting remarks regarding the financial decisions of European football clubs. Forde said that many fan perceptions about players often wildly unfounded.

Main Session – What Geeks Don’t Get: The Limits of Moneyball

This session was certainly the highlight of the day for most of the conference attendees, including myself. Michael Lewis, author of Moneyball, The Blind Side, Liar’s Poker, and several other books moderated a discussion of the limits of his 2003 work on the Oakland Athletics ability to contend in the American League with an extremely limited payroll. (If you haven’t read Lewis, you need to. He is one of my favorite writers: he is extremely insightful while maintaining very engaging prose. I highly recommend him.)

The panel included ESPN columnist Bill Simmons (The Sports Guy), Bill Polian – President, Indianapolis Colts, Daryl Morey – GM, Houston Rockets, Jonathan Kraft – Owner, New England Patriots, and Mark Cuban – Owner, Dallas Mavericks. The discussion was extremely humorous and gave some interesting insight into the complexities of sports decision-making.

Simmons opened by the panel by congratulating “…everyone in this room for the most dudes in one conference room record.” The panelists then discussed the limits of the stat revolution. Key limitations mentioned were inaccessibility to those without a background in stats, the limitations of context within a sport, and the differing ability to use information amongst individuals and teams.

Bill Polian, in particular lamented the limits of value of statistics to those lacking an academic background in mathematics and statistics. His line: “Speak English, please” highlighted the significant disconnect between ‘stat geeks’ and football executives. If the information isn’t comprehensible then it isn’t useful.

A good deal of the discussion centered on the limitations of statistics in football and basketball: Polian and Daryl Morey highlighted the importance of the intricate systems in their sports to individual player performance and the limiting effect this has on the value of statistics. Michael Lewis proposed the idea that baseball may have been a red herring: baseball is extremely easy to analyze relative to other sports.

Image courtesy blogs.jobdig.com

There were a number of things said on this subject that many of us in the audience found a bit uncertain: Bill Polian, in particular, made a number of remarks that implied the lack of existence of statistics of certain experts including Aaron Schatz and the crew at Football Outsiders. In general, he made the absolute assessment of players sound a bit more challenging and far off than it actually is.

The largest portion of the discussion centered around the disparity between top-level and organizations and the bottom feeders. LA Clippers season ticket holder Bill Simmons remarked, “Some teams are cheap. I have season tickets to one of them….Mark [Cuban] has a huge advantage over teams like that.” Cuban responded that certain teams use more stats in deciding player combinations and it is clear which teams fall into this category. When asked which teams he was referring to he said, “I don’t want to give examples….I don’t want to get fined.” Jonathan Kraft said that his family bought the Patriots because of the salary cap – they were in a position where success would not be determined by total amount spent on players but by how the money was spent. He discussed the inherent ability of different people to utilize information. He mentioned that under his family’s ownership the Patriots have had three different head coaches. “…Bill Belichick, Pete Carroll, and Bill Parcells…obviously there are very different intellectual capacities.” His comment drew raucous laughter – realizing what he had just said, he pleaded with the audience full of Blackberries, iPhones, and laptops, “Please don’t put that on Twitter!” to more laughter.

After this, the panel spent a significant amount of time evaluating the decision of the Patriots to go for it on a 4th and 2 in the Pats-Colts game in October 2009. The Pats were leading 34-28 and eschewed punting to attempt a 4th down conversion at their own 28-yard line with less than 2 minutes remaining in the game. Polian listed several factors (several injuries on the Pats defense, flow of the game) in defending the decision of his rival, saying that he would have done exactly the same thing. He went into significant analysis of the play called by New England discussing why it was chosen: the most successful play the Pats have run in the situation over the past several years is a QB Sneak. They lined up in the formation that they usually run it out of and then went to the second most successful play in the situation – a pass to Kevin Faulk. Kraft said that the play probably would have worked against 30 NFL teams noting that in addition to Brady the Pats had Faulk and Welker. Bill Simmons discussed the column that he wrote criticizing the system at the time saying that it seemed like a panic move. The silence that followed this proclamation was broken by Polian who said “I disagree.”

Image courtesy nba.fanhouse.com

Cuban then talked about a key difference between football and basketball. In football, the teams are trying to figure out which play the other will run. “In basketball, the coach yells out the play and the defense still doesn’t know what to do.” Morey agreed, saying that players are coached to defend solely against 3-point shots in end-game situations, but that when these situations occur, “It’s like a shiny object! They’re like ‘oh look there he goes – DAMNIT!”

The discussion then moved to what the panelists wished they knew about their sport. Cuban, predictably, responded with “Referees.” After a laughter-filled pause, he said, “Seriously…if a game that comes down to 3 points or less, 2 missed calls switch the outcome.” He also discussed that there was (at least at one time) an NBA referee who “refused to call 3 seconds. And we knew it.” Polian agreed, saying that NFL crews who call the most holding penalties in a season call 22-23 while the crews who call the least call 7-9, too large of a disparity.

The panelists agreed that they wished that they knew more about evaluating player psychology. Morey said that scouts are hard to take advantage of – how can a psychologist know who a 20-year old is when the player doesn’t even know who he is. Polian talked about the importance of the transformation of the individual from a 22-year old with no responsibility to a 28-year old with a house and a family to a 32-year old who is ready to retire and move on from football.

Cuban talked about his belief that, in 15-20 years, personalized medicine will be a key aspect of sports. He talked about the Spurs decision to draft Dejuan Blair after the rest of the league passed on him and the quality of play that he has rewarded them with balanced against the increased chance of serious injury given his health history.

The panel ended with Polian assessing the difference between top organizations and bottom organizations: he talked about the Red Sox and the quality of their marketing department in creating opportunities for the fun to reach out and ‘touch’ the product through tickets, jerseys, team yearbooks, and other ventures. He said that franchises that consistently fail either lack talent or have warring factions within the organization. Franchises that ‘get it’ are able to make themselves more than the sum of their parts.

Fourth Session – “Future of Attendance: Innovations at Arenas and Stadiums”

Shira Springer of the Boston Globe moderated this session. Panelists included David Holland – Senior VP and GM, Cisco Sports Business, Peter Moore – President, EA Sports, Tim Romani – CEO, ICON Venture Group, David Samson – President, Florida Marlins, and Alec Scheiner – Senior VP & General Counsel, Dallas Cowboys.

The contributions of Messrs. Holland, Moore, and Romani were very useful in their own right, but the highlight of the session

Image courtesy law.com

was the back-and-forth between Alec Scheiner and David Samson. Scheiner opened the discussion by talking about the $1.2 billion new stadium the Cowboys played in this past year. Samson, playing with his 2003 World Series ring said of his Marlins, “We are the exact opposite…for $1.2 billion I could get you three stadiums. And we’ve won the World Series.” There was a question at the end of the session about time spent by fans driving to and from the game and parking at the game. Scheiner talked about how his team created a web application to help fans choose the best route and best parking location. Samson chimed in saying, “We did the same thing that Dallas did and we did it for free.” Scheiner interjected, saying, “Wait – did you use our model?” Samson, grinned and proclaimed, “You’re damn RIGHT we did!” to great laughter. Scheiner shook his head in disbelief while Samson basked in his own rogue garishness. He then looked around and, realizing the situation, said, “Wait! Is this being taped?” The man behind the camera in the back of the room waved to him. Samson swore in disgust to even more laughter.

Image Courtesy cantstopthebleeding.com

The informative part of the session consisted of the executives talking about the business and technology of the stadium and the fan experience. Tim Romani talked about the strategy and budgetary limitations inherent in stadium building. He said that 80% of ticket revenue comes from the first 5 rows of the stadium while the last rows make up less than 1% of ticket revenue. He also discussed the importance of retractable roofs in decision-making related to NFL stadiums. He cited the Denver Broncos as a team that his company encouraged to consider the retractable roof when building a new stadium. However, Broncos owner Pat Bowlen said that this would conflict with his image and that of the general public of the Broncos as an ‘outdoor team.’ Other panelists chimed in about the inherent advantage of bad weather to teams in the north. Peter Moore said that, so long as bad weather is a possibility, there will not be a Super Bowl in New York. David Holland talked about architecturally designing stadiums so that they are able to evolve over time – something that the Cowboys did in building such a massive stadium. He also discussed the way in which technology relates to the stadium building equation: if your goal is to be technologically advanced, he said, you’re pursuing the wrong goal. Your goal is to provide the best fan experience possible and the components have to work toward the goal. If a technology doesn’t support the fan experience that you’re trying to create then you shouldn’t have it. Peter Moore talked about the way that the fan experience relates to the games created by EA Sports. He cited the Madden series as an ‘entry point’ for fans – a way for them to become acclimated to the game and to develop a desire to attend games and to participate in the fan experience of a given team. He also discussed the use of his company’s games in the broader culture: I was rather stunned to learn that in the final 48 hours prior to the 2008 US Presidential elections, Barack Obama’s team put ads on billboards within NBA Live in 10 battleground states for males aged 18-34.

As the session wore on, David Samson returned to his role as the ‘honest man’: he summarized the entire session by saying that the entirety of the discussion was “…talking about ways to get your money into our pockets.” He talked about several ways in which teams design ballparks to please the spectator with this goal in mind, including restrooms: “If you’re waiting in line to go to the bathroom you’re not spending money.”

The discussion as a whole was a bit stilting – it was quite clear that there were two discussions going on which happened to be occurring in the same time and place: the more theoretical discussion of the planned topic of the session and the interplay between Samson and the audience as he sought to give them a clearer idea of his vision of the issue. He is certainly a very intelligent individual but his dynamism is a bit off-putting. Still, his honesty has an endearing quality (to a degree). On the other end of the spectrum, Alec Scheiner has a very exciting charisma and yet a closer study reveals a shrewdness that can be even more demeaning than any readily observable quality of Samson’s. Two very interesting gentlemen all told – quite similar, yet quite different.

Fifth Session – Coaching Analytics

Ric Bucher moderated the final breakout session I attended. The panel consisted of Avery Johnson – former Mavericks’ head coach and current ESPN analyst, Buck Showalter – ESPN analyst and former Yankee manager, Kevin Kelley – the high school football coach who almost never punts and does many onside kicks, Nate Silver – creator of PECOTA and fivethirtyeight.com, and Brent Barry – former NBA player, current NBATV and TNT analyst, and ‘one of the tallest surfers in the world.’

The discussion floated between examples of coaches using (or not using) analytical methods, theoretical discussion of proposed situations, and Brent Barry making comedic remarks.

Avery Johnson gave several examples of situations where he used statistical information to inform his decisions as a coach and times when he was forced by players and other factors to abandon the stats. He discussed in detail the decisions he made during the infamous 2007 series where, following a 67-15 regular season, his team lost to the 8th-seeded Warriors in the first round. In the first game, his team lost by going small – which the numbers said they should do. However, the team missed almost 20 lay-ups. In the second game, they didn’t go small and won. The players felt that they shouldn’t go small anymore and Avery was forced to oblige. The team promptly lost the series.

Image courtesy umpbump.com

Buck Showalter talked about how the stats that are believed to be new and exciting are really old hat and how much of what is being done now with technology his staff did in the late 1980’s using colored pencils and graph paper. He also said that people often “…confuse change with a lack of respect for tradition.” He talked about the importance of embracing both what you see with your eyes and what the numbers tell you – you can’t become a slave to either. Kevin Kelley talked about the importance of making important decisions ahead of time so that you aren’t influenced by your emotions in key situations. The panelists also discussed coaches’ lack of experimentation. Nate Silver brought up the issue, asking the rhetorical question, “Why don’t coaches experiment more?” Showalter jumped in with “Spring training.” “Half of the guys are hungover,” retorted Silver. Showalter responded,  “They’re hungover? I’m hungover!”

Silver also talked about how, in many situations, the statistics in favor of a given decision are only 55-45 or 52-48 and how the failure of a riskier decision can be more detrimental to a coach then a less risky decision. He described baseball as being the most numbers friendly on the whole but the least numbers friendly on an individual decision. He talked about the importance of player psychology and health.

Avery Johnson brought up the situation of being ahead by 3 in a basketball game with 5 seconds remaining and the other team having the ball. He said, “When we didn’t foul we got burned, when we did foul it worked better for us,” but then talked about how certain players – Mario Elie and Dennis Rodman were his examples – refused to foul in the situation. He also talked about implementing different defensive schemes against key guys like Kobe and Steve Nash in clutch situations, citing the need to prevent Nash from “…[dribbling] 100 million times under the basket” off of a pick-and-roll.

Image courtesy zsakoljteiscsorike.blog.hu

Brent Barry’s contributions ranged from the insightful to the absurd. He talked about a remark he made as a rookie about his coach (who at the time had recently lost his thousandth game). At the time Barry had said that, ‘they say you learn something every time you lose. If that’s the case he should be Einstein by now.’ Barry noted that following this remark his playing time diminished. He later talked about the cyclical nature of the league’s coaches as a major problem: “If you’re a bad coach and you prove you can lose, someone will hire you to lose elsewhere.” He talked about the value of young coaches embracing the new statistics and the need for change: “I know Buck [Showalter, the senior member of the panel] doesn’t know how to type a text message but he’s learning.” (Buck wiggled his hands and nodded approvingly.) Buck then said, “I wondered why I was here, am I the dinosaur here?” Bucher piped in, “You did say Gale Sayers earlier.” Buck later discussed the contributions of stat geeks, saying of the short, skinny Nate Silver: “Nate may never have put on a jock in his life but if he has something to offer me then I want to hear what he has to say.” Barry asked, “What if he’s wearing one when he comes in?”

Kevin Kelley made some interesting remarks about the difference between his sport, football, and basketball and baseball: “We don’t have to be as bright as baseball and basketball,” he started, “In our sport, one guy makes most of the decisions.” Either the quarterback decides who to throw to or the running back decides where to run – the complex network of decisions has less impact on the outcome relative to basketball and baseball.

I greatly enjoyed this session: it was very informative and insightful while also being extremely entertaining and humorous. All of the panelists used comedy in their remarks and also contributed solid insights, particularly Avery and Kelly.

Errata

The two most significant areas for growth in providing absolute statistics were those pertaining to injuries and psychology. Various panelists discussed the lack of information that teams have regarding injuries: they don’t know enough about a player’s injury history and there isn’t enough information about things like recovery time, long-term effects, and other aspects of injury that pertain to future player value. On the psychological analysis front, many teams employ psychologists to analyze their players. However, there is significant room for growth in this area: several panelists discussed their desire for absolute valuation of psychological factors in various game situations.

I had the opportunity to meet many people at the conference, both from the student/young professional crowd as well as the celebrity crowd. I had the great pleasure of meeting Bill Simmons, which was a huge thrill for me. I also met several of the panelists and enjoyed talking to them in one-on-one or small group situations. The organizing committee did a fantastic job of selecting excellent panelists who were friendly, knowledgeable, and excited to be there. Kudos to the MIT group!

Advertisements
This entry was posted in Column and tagged , , , , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s