Here's how I decided to tackle the question...
First, I collected the data from the first 3 rounds from each entry draft starting in 1990 and ending in 2000. I figured that would be a generous sample size and it would yield players that are basically set on their career paths. Some guys from 2001 and upwards are still establishing themselves in the league and it would likely skew the results if I were to include those years.
I wanted to look at things from a pure, basic probabilistic model: like randomly choosing cards from a deck. I defined a "desired result" a little differently for every position...
For defencemen, I considered a "hit" to be a player that managed to play 160 games or more at the NHL level. That's it. I thought of including further parameters, such as production stats, but ran the risk of needlessly excluding useful players since the value of defencemen isn't tpyically captured in counting numbers like goals and assists.
For forwards, I used the same 160 GP line in the sand and added a minimum 0.30 career PPG pace parameter. Basically, I wanted to cull the Rico Fata-type players that manage to stick around long enough as journeymen (Fata played 230 games), but never really make any kind of impact to the unfortunate teams that get stuck with them.
For goalies, I dropped the "games played" requirement down to 100 and stuck with that. This seemed to seperate the actual players from the busts well enough .
Granted, the boundaries are somewhat arbitrary: I could have chosen 100 GP and a career pace of 0.50 PPG for forwards and gotten different results. However, I think the 160 GP is a fair minimum - it's about 2 seasons of work for a skater. If a guy couldn't stick around for at least 2 years of work in the league, it's fair to say he wasn't much of a player. Also, the 0.30 PPG rate managed to capture most of the effective forwards over the decade, even so-called "defensive specialists", while excluding the Rico Fata's and Tyler Wright's of the world. I think it's intuitive and reasonable to expect a forward to have some sort of offensive impact over a career. Fair to say, your Franchise hasn't gained much if it drafts Shawn Antoski (183GP, 3G, 5A, 8P).
Goaltending is a different matter altogether. Stats are very team dependent (and even "era" dependent) meaning I couldn't well rely on any kind of GAA or SV% line in the sand to cull the wheat from the chaffe. Since it's so singular and important a position, I figured any goalie that could play at least 100 games in the league could likely be considered an NHL caliber player.
I also added a subjective measure to the analysis by counting the number of players I personally recognized as "significant contributors" (ie stars and superstars). Sure there's probably an element of bias, but it gives at least some idea of the chances of hitting a "home-run".
The results:
(click to enlarge)From 1990 to 2000, there were 282 players chosen in the first round: 89 defencemen, 170 forwards and 22 goalies. Of the 282 players, 163 proved to be "contributors" at the NHL as defined by the criterion above. Sixty-four were defenders, 88 were forwards and 11 were goalies.
163/282 = 59% gross success rate across 11 first rounds. That's not too shabby.
Of the 163 contributors, 63 were "stars" by my personal estimation. That's a 22% success rate in terms of the total players chosen and a 39% chance you get a big-time difference maker as opposed to just Joe-Schmo NHLer.
Round 2 -
There were 302 players selected in the 2nd round during the decade in question. 104 , 175 and 24 of defencemen, forwards and goalies, respectively. As would be expected, the success rate declined significantly as compared to round 1: 74 players went on to become NHL players (only 25 of them stars).
74/302 = 24.5% chance of garnering a useful player from the second round. The probability of grabbing a star is a mere 8%.
Round 3 -
The difference between the second and third rounds wasn't as drastic. From 1990 to 2000, 288 players were chosen in round 3, 85 of which were defenders, 176 forwards and another 27 goaltenders. Eighteen blueliners went on to be players, as well as 32 forwards and just 5 goalies, for a total of 55 contributors.
55/288 = 19% chance of drafting an NHLer in the 3rd round. Twenty-four of the 55 were big time contributors by my count, making it an 8.3% chance of getting a front-liner (or top two defender, etc) when choosing in the 50's and 60's.
Admittedly, not the best of odds.
All Picks Aren't Created Equal
Of course, having the first choice overall isn't the same as having the 25th choice overall. Clearly, as the draft progresses the talent is whittled down. To generate more specific probabilities, I parsed the data into Top 5 picks, top 10 picks and top 15 picks.
Not surprisingly, Choosing in the top 5 meant a 98% chance of yielding an NHL player . Of the 55 players picked, only 3 failed to become NHL regulars (Alex Volchkov, Pavel Brendl and Jason Bosignore). Twenty-eight of the 52 went on to become stars (51%).
A top 10 choice was only slightly worse. 82 of the 110 players became contributors (75%) and 35 of them became stars (31%). The probability of getting an NHLer within the top 15 dropped down to 68% (112 of 165) with a 27% chance of grabbing a star (45/112).
Course, sample size becomes an issue here, but the general trend is hard to ignore: A top 5 pick seems to all but ensure a future NHL player.
The benefits of choosing near the top of the rotation dissipate beyond the first round, however. I gathered together the first 5 picks from the various 2nd rounds and discovered that only 17 of the 55 players in question became contributors (31%) and 5 became stars (9%), which pretty closely mirrors the general 2nd round probabilities. That suggests to me that future quality players are more or less randomly distributed amongst the turds after round 1, and organizational factors (quality of scouting, developmental system) are probably more important than draft position.
Issues
- Clearly this investigation is neither air-tight, nor exhaustive. I don't have the will or the mathematical chops to put this data through a more rigorous (and valid) analysis.
- Keep in mind that draft position changed as the league expanded. A 2nd round pick in 1991 was a first round pick in 1999.
- As mentioned, the "minimum requirements" for a player to be considered an NHLer, and therefore of future value (a "contributor") were pretty arbitrary. There's probably a more valid method of determining cut-off points within the data set itself, but I think the qualitatively intuitive parameters I chose work well enough for an initial inquiry like this one.
- In the future, a larger representative sample size, including equal round lengths (1-30, 31-61, etc) might smooth out some wrinkles.
- Treating the draft as a purely probabilistic event isn't very accurate. GM's don't just choose names out of hat (although the Flames Franchise probably would have done BETTER during the 90's had they done that). Quality of an organization's scouting staff and development system no doubt have an effect on a given team's success rate.
- Draft quality varies by year. Some summers yield numerous high quality NHLers (1997), while others are less fecund (1996). The depth of a given draft will no doubt effect success rates.
Conclusions
Assuming the limitations of my investigation don't render it completely worthless, there did seem to be some interesting observable trends. A first rounder looks to be slightly better than a coin flip overall, with a top 5 choice representing an almost sure-fire future player. The reduction in "talent density" in the 2nd and 3rd rounds as compared to the first is considerable, with the basic 3rd rounder representing roughly a 19% chance of choosing a contributor at any position.
Based on this info, the best strategy would seem to be gathering as many draft picks as possible in order to increase your chances of a "hit", or alternatively, banishing single picks beyond the first round for established NHLers. I think the percentages suggest that a lone 3rd (or even 2nd) rounder on it's own doesn't seem to have a great deal of utility.
EDIT - For anyone who wants to play with this stuff themselves, I've published the raw data at google spreadsheets here.
