Thursday, February 04, 2010

Jeff Schultz Doesn't Bother to Divide

Out-of-touch liberal eggheads like your truly like spending our time reading books like Susan Jacoby's The Age of American Unreason so we can gripe in our heads about the lack of rationality and evidence-based argumentation in the modern media. (What Jacoby would think about me spending my free time writing about sports is a different matter entirely.) One of Jacoby's numerous complaints about the modern media is the prevalence of innumeracy, i.e. the inability to understand basic statistics and mathematical concepts. For instance, Brian Cook did a nice job recently in debunking Bruce Feldman's argument regarding the prevalence of lower-rated recruits among the Super Bowl starters. Because Feldman is a good writer and a rational guy, Cook was polite in his criticism.

I am under no such constraints when writing about Jeff Schultz's predictable blathering about Georgia Tech's haul on Signing Day. Schultz has filled Terence Moore's shoes as the AJC columnist whose arguments I read mainly because I want to mock them. With arguments like this, you can see why:

The NFL, generally recognized as the most successful sports league in the world, employs hundreds of scouts, spends millions of dollars, crosses the country several times over, breaks down every conceivable potential pro player from Big Campus U to Couldn’t Find Us Without GPS A&M and feeds it all into computers that spit out professional looking, color-coded spreadsheets, suitable for framing.

Know what? They still get it wrong more than half the time.

So what are we to think about national letter of intent day? College scouting is even less sophisticated than pro scouting. They’re dealing with athletes four years younger. Teenagers. More variables means more guessing, which means more darts are likely to hit the wall, three feet from the target.

NFL teams are almost uniform in the high value that they place on Draft picks. The major exception are the Washington Redskins, who regularly trade picks for established players. Ask a Redskins fan how that's working out for them. The Falcons acquired a Hall of Fame tight end last spring for a second round Draft pick and no one batted an eye. Bill Belichick, who has a reasonably solid reputation, traded Richard Seymour, his best defensive lineman and one of the pillars of the Patriots dynasty, for a first round pick and most people assumed that Al Davis was getting fleeced again.

So here's the question Jeff: are NFL teams misguided when they place such high value on the lottery tickets that they get to play each April? Have you intrepidly identified a market inefficiency? Or is it possible that you just don't understand the concept of probability at all? Assuming that you are right that NFL teams miss on over half of their picks, draft picks are still valuable because the more picks a team has and the higher those picks are in the draft order, the greater the likelihood that the team will find a good player.

And that brings us to recruiting. Schultz acts in a dismissive manner towards recruiting gurus, putting up sarcastic quotation marks around rankings, stars, and experts. Here is the meat of his argument:

Let me take this opportunity to give a value to these rankings on my own personal scale: one raspberry.

“The No. 64 guard in the country, you can’t tell me who that is,” Johnson cracked Wednesday. “I don’t even know who the No. 2 guard in the state is.”

The old expression of a team “looking good getting off the bus” applies here. They are recruits, nothing more, nothing less. Georgia Tech will be fine because the success of a football program is defined by coaching and direction, not by which recruits emerged from those inane hat-switching acts at press conferences.

The recruiting rankings tend to track the offers that a player receives. If Texas, Florida, and USC are all after the same player, then that player will get four or five stars. If the schools of the MAC are fighting for a player's signature, then he won't. If Schultz is right that the stars and rankings are meaningless, then he is adopting the position that the staffs at the elite programs do not know how to evaluate players. Care to defend the position that Urban Meyer doesn't know the difference between a player and a poseur, Jeff?

And then Schultz's crowing insult comes at the end, where he lists four players that Paul Johnson will have to replace this year: Derrick Morgan, Demaryius Thomas, Jonathan Dwyer and Morgan Burnett. Guess how many stars went next to Morgan's, Dwyer's, and Burnett's names when they were recruits? Cue Moses Malone: fo, fo, fo. Guess how many stars went next to Josh Nesbitt's name? The 2009 Georgia Tech team succeeded for a number of reasons. Good offensive coaching and good fortune come to mind immediately, as does Chan Gailey's recruiting. The assumption that Paul Johnson can continue to win at the same level without recruiting at the same level is just dumb.

The funny thing is that Tech's class is quite good if Schultz were able to understand how numbers actually work. He cites Tech's middling overall rankings, but doesn't account for the fact that the Jackets signed a small class of 18 players. By average star ranking, they do quite well: #26 by Rivals and #34 by Scout. Johnson brought in five four-star players, which is not far off the eight four-star players that Gailey signed in the 2007 class that formed the backbone of the 2009 ACC Champions. The class is defense-heavy, which is good for Johnson because he can cobble together an offense out of spare parts, but defense requires athletes and it isn't his field of expertise.

Yes, Jeff, there is no guarantee that these players will become stars on the field. The debris field of wasted talent at Florida State that paved the way for the Jackets to win the conference is a testament to the fact that talent alone does not win games. However, rather than reveling in the anti-expert, "no one knows anything about anything!" mindset that you encourage, Tech fans should be either reasonably happy because Johnson brought in a collection of good lottery tickets yesterday or mildly concerned because he didn't bring in more on the heels of two successful seasons.


Anonymous said...

There are three columns that drive by media recycle about recruiting classes every year:

1) If the class is indisputably poor and the writer likes the program, an article comes out about how the coach "added size and speed" with the class. This is a meaningless article, but it's true: almost all college scholarship athletes are big and fast. (examples: dozens of Steve Grinzel articles about Michigan State's crappy classes, current articles about Ohio U's and EMU's classes)

2) If the class is mediocre and the writer likes the program, write about how those eggheads with pointy heads "got 'em wrong" a few times in the past. Maybe Jeremy Crabtree isn't so smart after all, huh? (the column you cite seems to fit here)

3) If the writer hates the program, write about the recruits' poor academic credentials or "troubled pasts." This article works with a good class ("win at all costs!") or a poor class ("the program is declining so the coach is taking risks"). (example: FREEP)

Q-Was Feldman really arguing about the "invalidity" of recruiting rankings, or was he merely writing a filler piece based on a meaningless observation?

Jeff said...

After a work conversation about the extent to which highly rated recruits and highly rated draft picks pan out, the guy next door sent me this analysis of the players selected to this year's Pro Bowl (the first ones picked--not the injury replacements) and their draft position. Pretty startling, especially when you consider that nobody wastes a 1st rounder on several of positions:

QB - Manning - 1st Round
RB - Johnson - 1st Round
FB - McClain - 4th Round
WR - Andre Johnson - 1st Round
WR - Reggie Wayne - 1st Round
TE - Dalls Clark - 1st Round
OT - Jake Long - 1st Round
OT - Ryan Clady - 1st Round
OG - Logan Mankins - 1st Round
OG - Alan Faneca - 1st Round
C - Nick Mangold - 1st Round

DE - Freeney - 1st Round
DE - Mathis - 5th Round
DT - Ngata - 1st Round
DT - Wilfork - 1st Round
OLB - Dumervil - 4th Round
OLB - Harrison - Undrafted
ILB - Lewis - 1st Round
CB - Revis - 1st Round
CB - Asomugha - 1st Round
FS - Reed - 1st Round
SS - Dawkins - 2nd Round

22 Starters - 17 1st Round Picks, 1 2nd Round Pick, 2 4th Round Picks, 1 Fifth Round Pick, 1 Undrafted

QB - Brees - 2nd Round
RB - Peterson - 1st Round
FB - Weaver - Undrafted
WR - Fitzgerald - 1st Round
WR - Jackson - 2nd Round
TE - Davis - 1st Round
OT - Peters - Undrafted
OT - McKinnie - 1st Round
OG - Hutchinson - 1st Round
OG - Evans - 4th Round
C - Gurode - 2nd Round

DE - Allen - 4th Round
DE - Peppers - 1st Round
DT - Williams - 1st Round
DT - Dockett - 3rd Round
OLB - Ware - 1st Round
OLB - Briggs - 3rd Round
ILB - Willis - 1st Round
CB - Woodson - 1st Round
CB - Samuel - 4th Round
FS - Sharper - 2nd Round
SS - Wilson - 3rd Round

22 Starters - 10 1st Round Picks, 4 2nd Round Picks, 3 3rd Round Picks, 3 4th Round Picks, 2 Undrafted

Combined - 44 Starters - 27 1st Round Picks, 5 2nd Round Picks, 3 3rd Round Picks, 5 4th Round Picks, 1 5th Round Pick, 3 Undrafted

I'd love to see a similar analysis of the college All-America teams.

Stephen said...

Analysis of this would be pretty easy: It's an ordered probit model where the DV is the NFL draft round in which you're chosen and the relevant IV is your recruiting stars. Relevant controls might be your school's rankings while you played, your year, etc.

If someone knows where that data is easily accessible, I'll format it and run the regressions.

(On a random note: According to Wikipedia, the reason why Barca got Messi instead of River Plate... River Plate didn't have the money to pay for his growth hormones. Mas que club? Si. Mas que club y mucho denaro? No se.)

Stephen said...

Nevermind, we shouldn't use round drafted as the DV- that's throwing away some valuable variation, so we should use when you were drafted.

Following some blog cites, this Athlon article seems to be the best analysis relating rankings and player quality.

But this study is crappy for some pretty large reasons.

They select on the dependent variable: They only use the players who were selected in the first three rounds of the NFL draft. But what about high-star players who are drafted lower or aren't drafted? This is some pretty classic(ly bad) selection bias. They compound this by looking at only FBS players. We need non-FBS diamond-in-the-rough types to see if stars actually matter.

Also, doing this study correctly would be interesting because we could also compare the quality of different ranking systems.

Re: Age of Unreason-- I haven't read it. But my take on the blogosphere is that includes more numbers, but that rarely signifies an understanding of statistical analysis. That Athlon article is so methodologically flawed as to virtually say nothing. Fangraphs is guilty of this as well, but that's a rant for another day.

休息 said...