Originally published February 2008
This is the time of year when all the prospect guru's start publishing their top ten prospects.
Baseball America (BBA) is probably the most well-respected of these lists. Its one of the few publications that really goes out and gets new, independent information about every team's players. Their writers talk to scouts and baseball developement staff to identify the best prospects. That said, the lists also reflect the judgments of the particular BBA staff.
John Sickels is another source of indpendent judgments. He does a lot of research to publish his book of prospects every year. Unlike BBA writers, who are constantly on deadline to produce articles for the magazine, Sickels actually has time to do "scouting trips" to watch prospects and make his own judgements. Of course, as a one guy show, he has doesn't have the resources to see everyone. And he is not a professional scout. So he may give too much weight to his own opinions and players he has seen personally and less weight to the opinions of those who get paid to identify the best players before its obvious to everyone.
Beyond those two, most of the other lists are rehashes of one another with a particular writer's biases about what is valuable. The SABR guys will add a stats overlay, people who are convinced pitching is important will emphasize pitching, people who think hitting is most important will grab people identified as the best hitters, etc. But most of these lists are based on reading BBA, Sickels and the blogsphere. They may or may not reflect the opinion of most scouts. In fact, in an effort to be interesting, they often have a bias toward having a semi-contrarian opinion.
Occasionally, they go beyond condensing others information and stray into outright plagiarism. One prominent Twin's blogger's list a few years ago was clearly just a rewrite of BBA's. He changed the order some and then rewrote the descriptions like a kid rewriting an encyclopedia entry for his school report.
There are also some local bloggers who take the time to talk to their local team's scouts and player development personell. They develop their own top ten lists for their favorite team. Those lists reflect a lot of input. While their judgments of prominent prospects may not be as well-informed as the national list makers, they often do a better job on players who are less prominent. If you are looking for hidden gems, these lists will often have them first.
Unfortunately, there are also lists that are based on little more than looking at the local leader boards for the minor leagues. Usually these stat-based lists are obvious, including players who are over-age for their league or harshly judging highly touted young prospects who have been less than successful while playing at a high level at an early age. These are the folks that projected Torii Hunter as a "slap hitter" while he was in the minor leagues because his power had yet to show itself.
The larger issue for every prospect list is that what makes a player a "top prospect" is often a moving target. Baseball America says its lists are based on a balance between a prospect's ceiling and how likely they are to reach it. So what you have is a subjective balance between two subjective evaluations. In short, these top ten lists are almost all art and no science. The result is that, while top ten lists are fun, they need to be taken with a very large grain of salt.
Last week I
looked back on Baseball America's top ten from ten years ago. The conclusion was that the order of the top ten was not very informative. Having followed BBA's lists for the last 20 years, that was not a surprise to me. While identifying that a player has major league potential is fairly easy, distinguishing how much of that potential each prospect will achieve isn't.
That doesn't mean there aren't players who stand out. There are. And most of those players go on to have great careers. But beyond a handful of truly special players with polished skills, the difference between number one and number 10 on most prospect lists is not very meaningful in the long run. It is likely that even a team's "top prospect" will not end up its best player.
The other thing to remember is that none of these list builders get paid to be right. If they are paid at all, its to be entertaining. And when they miss, they can quickly excuse their failure by pointing to the reality I just described that everyone misses. But major league scouts do get paid to be right. They don't have a job when they consistently under perform their peers. So when John Manuel at Baseball America makes Nick Blackburn the Twins number one prospect, and then admits he has him rated higher than the Twins, you have to wonder what he is thinking. If the team that has developed a player for five years isn't convinced, why should anyone else be?
That is one of the traps BBA falls into a lot. Over the years their lists are peppered with players who had one good season. When a player leaps from career minor leaguer to top ten prospect in one season, take it with a grain of salt. Player's tools don't usually develop that way. While they do have breakouts, it is not usually totally un-foreseen. They are players who have always had the potential and finally realize it.
An example from the BBA list ten years ago would be Chad Allen who is listed as the Twins 4th best prospect. Allen had a major league career, so you wouldn't call him a complete bust. But he was never in the same class as Torii Hunter, Jacque Jones, Corey Koskie or even Matt LeCroy, no matter where he appeared on the BBA top ten prospect list. My guess is Nick Blackburn ends up a similar case of overreaching.
On the other end of the spectrum is last year's draft choices. It is a rare player who changes his potential in the course of the summer after he has been drafted. With good reason, BBA usually lists the top draft choice from the previous draft in its top ten. It doesn't mean much about the player, but it tells you something about the strength of the rest of the list. If a player taken number 20 in the draft is immediately the team's best prospect, you know their farm system is not very strong. If the top ten includes a bunch of supplemental first round draft choices from the most recent draft, that is a clue that there wasn't a lot of competition for the list. Of course, it may be a player is better than his draft position. But that isn't usually something you find out over the course of their first summer.
So the basics of looking at prospect lists.
1) There really isn't much difference between number one and number ten on a list
2) Watch out for breakouts, don't pay much attention to recent drafts
3) Be very careful of being fooled by derivative lists and stat-based lists that appear on the internet. If a list doesn't reflect conversations with baseball development staff, it probably doesn't have much meaning.
Finally there is the recent trend of top 30 lists. To be blunt, these are little more than a laundry list of all a team's prospects. The rankings are pretty much meaningless and the difference between the bottom ten and the next 20 prospects won't mean much a year from now.
So as you hear about how the Twins didn't get the Mets "top prospect" for Santana, the appropriate response is really "so what". They got four of the Mets top ten and all four of them could turn out to be better than the guy at the top of the list.
John Sickels 2008-2009Seth Speaks 2008-2009Twinkie Town 2008-2009Baseball America's Twins Top Ten 2007-2008Josh's Top 50 Twins 2007-2008Baseball America 2006-2007Baseball America 2005-2006
Posts
