Wednesday, March 29, 2006

One of the Best Articles I've Read

I've become a big fan of the Baseball Prospectus over the past several years and one of their statistical tools that I've always wanted to see applied to the NFL is the way that BP attempts to normalize minor league stats to project them as major league numbers. They concede that they're work is only educated guesswork, but they do a good job of interpreting minor league numbers to evaluate prospects. They're also constantly re-evaluating their methodology. For instance, they used to treat walk rates as one of the most important minor league stats, but they've now tweaked that opinion based on an observation that hitters with great eyes, but limited power can draw walks in the minors because the pitchers tend to have suspect control, but they can't draw walks in the majors where the pitchers can hit their spots in the strike zone and where the pitchers won't be afraid to hit those spots when the hitter can't do anything more than hit a single.

Anyway, Football Outsiders has a terrific article up now where they've analyzed quarterbacks taken in the first two rounds of the Draft over the past couple decades, determined that completion percentage and games started are the stats that have the most predictive power for the success or failure of a prospect, and now speculate on Cutler, Young, and Leinart in the NFL based on those stats. Some thoughts:

1. I've always thought that completion percentage was a lousy way to evaluate a quarterback and that yards per attempt was the only important stat (with touchdowns and interceptions having some value, but only secondarily so). That may still be true in terms of a quarterback's value to their team, but in terms of evaluating a quarterback's talent going forward, it appears that completion percentage is quite relevant. This makes sense, since accuracy is the #1 physical tool that a quarterback needs and completion percentage is the best way to determine whether that QB hits their spots on a regular basis.

2. The implications for D.J. Shockley are not good. Shockley only started for one year at Georgia and the fact that he was never able to beat out David Greene, a decent but hardly overwhelming starter, ought to be a major strike against him. If he can't beat out a likely career NFL back-up (assuming that Mark Richt didn't keep him on the bench for factors other than ability), then how is he expected to be starting caliber in the NFL, or even back-up caliber? And the 55.8 completion percentage is nothing outstanding, especially since it is lower than any of David Greene's season completion percentages.

3. Excellent point on Vince Young: he never had to throw the ball that hard. That would explain why he never looked like his arm was that strong, and it's also possible given how open his receivers typically were.

4. It's good to hear someone else make this point:

When it comes to adjusting your system to your players, the NFL is nowhere near as flexible as college football. This is because in the NFL players are much more expendable. A key example of this would be a man to whom many misguided people compare Young, Michael Vick. Vick has two things that he is absolutely outstanding at, running and throwing the ball far. His major weaknesses are reading the field and accuracy. Yet for some reason he has been made to play in an offense geared to short accurate passing and quick reads, so go figure.


The problem with the first part of the argument is that Mike Vick is not really expendable. The Falcons are committing a huge chunk of their salary cap to him and his back-up has a completely different skill-set than he does. As a result, the team probably has him in a system that de-emphasizes running and long passing because they don't want him taking hits that will knock him out for an extended period of time. This is why I think the Falcons need to trade Matt Schaub for a high draft pick, axe Greg Knapp, find a Vick-like back-up (Brad Smith, anyone?), and install an offense tailored to Vick's talents.

5. The biggest problem with FO's theory is that it attempts to screen out system quarterbacks like Dorsey, Kingsbury, and Wuerrfel by limiting itself to quarterbacks taken in the first two rounds, but it does not take into account the effects of systems and surrounding talent as pertaining to QBs taken in the first two rounds. For instance, this argument regarding Matt Leinart is problematic:

The most statistically similar recent college quarterbacks are Ben Roethlisberger and Byron Leftwich, both of whom played in the MAC. Leinart had better talent around him, of course, but he also had to face much more difficult competition. Dip a little further into the past, and we find that Leinart?s college statistics are remarkably similar to those of another Pac-10 quarterback: John Elway.


Leinart did not merely have better talent around him than Elway, Leftwich, and Roethlisberger, he had better talent than any of the teams on USC's schedule. As a result of Pete Carroll's skills as a recruiter, USC's location in the middle of a talent-rich region, and the ineptitude of the rest of the Pac Ten at recruiting, USC has more talent than any other college program and certainly more talent than their opponents, who have players that, with very few exceptions, USC declined to offer. (Notre Dame will ultimately challenge USC on a talent basis, but they're going to suffer the after-effects of Ty Willingham's crappy recruiting in 2007 and 2008.) Leinart had the benefit of virtually flawless protection, the best running threat in the country, and a bevy of great receivers. Thus, his stats really aren't comparable to Elway's, since Elway competed with less talent that Stanford's opponents, or Leftwich and Roethlisberger, who competed with equivalent or slightly greater talent than their MAC opponents.

4 comments:

Fox said...

The problem in focusing on completion percentage can be summarized in two words: Tim Couch. Or anyone coached by Tedford. They were all highly picked at least in large part because they were in exceptionally quarterback friendly systems that allowed them to make short, low-risk passes. Don't you need to work in yards per completion to avoid that trap?

Mikey, I agree wholeheartedly with your points on Vick (since I've been saying the same thing for years) but had never thought of the like-skilled backup one. Considering that Vick is their franchise player it makes a ton of sense to gear the offense around him AND have a backup that plays his style, rather than place such a premium on a backup that plays a completely different style.

Michael said...

Couch is mentioned in the article as a product of Mike Leach's system, but he went #1 in the Draft, so he should have been good under the measurements set up by FO. That gets to my point about Leinart; he could be a product of the system and talent around him and comparing his stats to those of Leftwich and Elway misses that point.

The Falcons' problem on offense is that they have an offense designed for Matt Schaub, but Vick has completely different skills. The more I think about it, the more I want the team to take Brad Smith in the 5th round and mimic the Texas/WVU offense (with appropriate corrections for NFL defenses).

Fox said...

OK, I finally took the time to read the actual article, rather than just reply to your post. You're right--it's an excellent piece. But I think I'm most impressed by the flexibility and insight of his analysis of the three prospects, not his statistical methods.
It does seem though that his focus on completion percentage as the defining stat comes with an awfully big caveat: he's saying that completion percentage is the most important stat for the QBs with NFL-caliber talent but has not come up with a statistical way to identify those people. Instead, he trusts the scouts to correctly figure that part out, then uses completion percentage to determine which of the people they pick early will turn out to be good players. (I.e., he doesn't have any statistical way to sort out Ty Detmer from Carson Palmer--to take two Norm Chow coached QBs--he just says Palmer's better because he was rafted higher and moves on.) So basically, he's saying that of the players with NFL-caliber arms and heads, the ones who are the most accurate are going to have the best success. That doesn't seem like a big reach to me.

Also, I agree with you Mikey that he understates the talent Leinert had at USC--he was surrounded by top grade 1st round picks on that offense so he had a tremendous talent advantage over every team in the country, let alone the crappy Ds in the Pac-10.

I'm a Realist said...

I'm a Shockley apologist, but there was absolutely no way that Richt could bench Greene after he won the SEC title in 2002. It would have set up a Major Applewhite-Chris Simms situation where no matter what Shockley was able to do, there would be people that called for Greene to be the starter.

During the 2001 season, Shockley was scheduled to start against Houston...the game that was postponed due to 9/11. Greene was given the start against Tennessee, and won up in Knoxville in the "hob-nailed boot" game, which solidified his position as starter for the next two and a half years.

I don't necessarily think Shockley will be a great starter, but I think with adequate coaching, he could eventually be a contributor. Doug Johnson is still a third stringer, so Shockley should be able to find work somewhere.