Analytics guy says don’t believe everything you read about analytics

[dudacek]

I don’t think discussions about hockey analytics are as full of arrogant number-crunchers and Neanderthal “you-can’t-measure” heart types as they once were. We’ve done a better job adopting a middle ground.

Still, it was very interesting to read an analytics professional fresh off an 8-year stint with the Leafs offer the following:

You cannot take two graphs, one showing that one particular player is “good” and one other player is “bad” and let that be the focal point of your analysis. 

https://theathletic.com/3630588/2022/10/03/charron-what-i-learned-working-in-an-nhl-front-office/

The gist of the article is that the low-hanging fruit has been plucked and NHL minds have already adopted the obvious stuff. The public debate people invest a lot of faith in on message boards is based on far more limited data sets than what the pros have access to. And conversation should not be on who is “bad” and “good” but on isolating specific, meaningful ways players and teams can improve.

All this is to say that it’s not always as simple as one player having more blue on his bar chart and one player having more red. When making recommendations with millions of dollars of somebody else’s money on the line, an analyst needs to seek out the right tool for the job. Within our department, there was lots of disagreement on players that usually boiled down to how we weighted a player’s relative strengths or weaknesses. No opinion is totally objective, and some analysts will weigh certain metrics above others.

While that doesn’t bring us any closer to having a proper catchall measure similar to baseball’s WAR, I’d argue the collective understanding of hockey isn’t at a point where we can start to do that. 

Well worth the read, especially when you consider the investment the Sabres are making in this area.

[triumph_communes]

6 minutes ago, dudacek said:

I don’t think discussions about hockey analytics are as full of arrogant number-crunchers and Neanderthal “you-can’t-measure” heart types as they once were. We’ve done a better job adopting a middle ground.

Still, it was very interesting to read an analytics professional fresh off an 8-year stint with the Leafs offer the following:

You cannot take two graphs, one showing that one particular player is “good” and one other player is “bad” and let that be the focal point of your analysis. 

https://theathletic.com/3630588/2022/10/03/charron-what-i-learned-working-in-an-nhl-front-office/

The gist of the article is that the low-hanging fruit has been plucked and NHL minds have already adopted the obvious stuff. The public debate people invest a lot of faith in on message boards is based on far more limited data sets than what the pros have access to. And conversation should not be on who is “bad” and “good” but on isolating specific, meaningful ways players and teams can improve.

All this is to say that it’s not always as simple as one player having more blue on his bar chart and one player having more red. When making recommendations with millions of dollars of somebody else’s money on the line, an analyst needs to seek out the right tool for the job. Within our department, there was lots of disagreement on players that usually boiled down to how we weighted a player’s relative strengths or weaknesses. No opinion is totally objective, and some analysts will weigh certain metrics above others.

While that doesn’t bring us any closer to having a proper catchall measure similar to baseball’s WAR, I’d argue the collective understanding of hockey isn’t at a point where we can start to do that. 

Well worth the read, especially when you consider the investment the Sabres are making in this area.

Oh so can we ban people who slam post WAR charts instead of arguing?

[Thorner]

27 minutes ago, dudacek said:

I don’t think discussions about hockey analytics are as full of arrogant number-crunchers and Neanderthal “you-can’t-measure” heart types as they once were. We’ve done a better job adopting a middle ground.

Still, it was very interesting to read an analytics professional fresh off an 8-year stint with the Leafs offer the following:

You cannot take two graphs, one showing that one particular player is “good” and one other player is “bad” and let that be the focal point of your analysis. 

https://theathletic.com/3630588/2022/10/03/charron-what-i-learned-working-in-an-nhl-front-office/

The gist of the article is that the low-hanging fruit has been plucked and NHL minds have already adopted the obvious stuff. The public debate people invest a lot of faith in on message boards is based on far more limited data sets than what the pros have access to. And conversation should not be on who is “bad” and “good” but on isolating specific, meaningful ways players and teams can improve.

All this is to say that it’s not always as simple as one player having more blue on his bar chart and one player having more red. When making recommendations with millions of dollars of somebody else’s money on the line, an analyst needs to seek out the right tool for the job. Within our department, there was lots of disagreement on players that usually boiled down to how we weighted a player’s relative strengths or weaknesses. No opinion is totally objective, and some analysts will weigh certain metrics above others.

While that doesn’t bring us any closer to having a proper catchall measure similar to baseball’s WAR, I’d argue the collective understanding of hockey isn’t at a point where we can start to do that. 

Well worth the read, especially when you consider the investment the Sabres are making in this area.

I’d argue NHL could *never* come up with a stat like Baseball’s WAR: hockey is too relatively fluid.

Even WAR in baseball isn’t perfect 

[Zamboni]

“Eye test guy says don’t believe everything you see. Sight is deceiving sometimes”.

 

[Doohicksie]

4 hours ago, dudacek said:

The public debate people invest a lot of faith in on message boards is based on far more limited data sets than what the pros have access to.

Well, yeah.

Every team's analytics department is doing their own analysis, developing their own metrics; weighting parameters differently to see how to best predict outcomes.  I bet at this point the Sabres' have come up with a metric that quantifies why Tuch is better for the Sabres than Eichel, based on information that was available prior to their respective drafts, and they're applying it to current drafts to identify their targets in the most recent draft.  They knew Savioe, Östlund  & Kulich rated high on that scale which why they were thrilled when they were still available to the Sabres.  Other teams passed because their metrics guys valued them differently.

Are our analytics guys right?  We'll find out soon enough.

[Doohicksie]

4 hours ago, triumph_communes said:

Oh so can we ban people who slam post WAR charts instead of arguing?

What's the fun in that?

[PerreaultForever]

I think what @Doohickie said is right. Every team uses analytics to some degree now and there is a ton of information for everybody to wade through and each team decides which metrics or which aspects of certain things mean more or less to them and thus, as it was in the old days, there will be differing opinions of who is best. 

[pi2000]

What I used to love about hockey was the different roles that needed to be filled in order to ice a winning team. 

It used to be you needed some skill, some grit, some defensive specialists, some size, etc.

That's no longer the case.   

As a result of rules changes the modern game has evolved into a skills contest.   This makes it much easier to build a winner by simply drafting guys who post the best metrics as determine by analytics, hence the explosion of data scientists in front offices across the league. 

Is the game "better" than it was 20 years ago?  It's certainly more skilled.. but is that "better"?   Depends who you ask. 

Personally I miss the physical game.  Today, losing teams simply roll over and concede.    What made hockey unique was the toughness required to play the game.   Today's game is no different than basketball or baseball both massively successful sports, but nothing as entertaining as a hockey game used to be.

Think about it, what's the only play in hockey that gets fans of both teams out of their seats?  A fight.  You can't tell me that's not entertaining. 

Edited October 4, 2022 by pi2000

[matter2003]

13 hours ago, dudacek said:

I don’t think discussions about hockey analytics are as full of arrogant number-crunchers and Neanderthal “you-can’t-measure” heart types as they once were. We’ve done a better job adopting a middle ground.

Still, it was very interesting to read an analytics professional fresh off an 8-year stint with the Leafs offer the following:

You cannot take two graphs, one showing that one particular player is “good” and one other player is “bad” and let that be the focal point of your analysis. 

https://theathletic.com/3630588/2022/10/03/charron-what-i-learned-working-in-an-nhl-front-office/

The gist of the article is that the low-hanging fruit has been plucked and NHL minds have already adopted the obvious stuff. The public debate people invest a lot of faith in on message boards is based on far more limited data sets than what the pros have access to. And conversation should not be on who is “bad” and “good” but on isolating specific, meaningful ways players and teams can improve.

All this is to say that it’s not always as simple as one player having more blue on his bar chart and one player having more red. When making recommendations with millions of dollars of somebody else’s money on the line, an analyst needs to seek out the right tool for the job. Within our department, there was lots of disagreement on players that usually boiled down to how we weighted a player’s relative strengths or weaknesses. No opinion is totally objective, and some analysts will weigh certain metrics above others.

While that doesn’t bring us any closer to having a proper catchall measure similar to baseball’s WAR, I’d argue the collective understanding of hockey isn’t at a point where we can start to do that. 

Well worth the read, especially when you consider the investment the Sabres are making in this area.

The Sabres are interesting because I heard I believe from Paul Hamilton on WGR that they basically get all the people in the room together whwn they are discussing players for the draft or how they are developing, etc that include the scouts and analytics people and go over all the players and let everyone give their opinion on them, then if there are significant differences between what the scouts are saying versus what the analytic people are saying they dive into it to try and figure out why and get to the bottom of it.

He basically said they are very thorough with their evaluations, it's a total team effort where everyone's opinion matters and they don't rely on either side completely but take in all the information and are interested in learning about where there are differences so they can look to tweak their analytics to improve them. Also said there are no egos in there and nobody rolls their eyes when the analytics guys talk, they value what they have to say even if they don't agree with it all the time.

Edited October 4, 2022 by matter2003

[Pimlach]

6 hours ago, pi2000 said:

What I used to love about hockey was the different roles that needed to be filled in order to ice a winning team. 

It used to be you needed some skill, some grit, some defensive specialists, some size, etc.

That's no longer the case.   

As a result of rules changes the modern game has evolved into a skills contest.   This makes it much easier to build a winner by simply drafting guys who post the best metrics as determine by analytics, hence the explosion of data scientists in front offices across the league. 

Is the game "better" than it was 20 years ago?  It's certainly more skilled.. but is that "better"?   Depends who you ask. 

Personally I miss the physical game.  Today, losing teams simply roll over and concede.    What made hockey unique was the toughness required to play the game.   Today's game is no different than basketball or baseball both massively successful sports, but nothing as entertaining as a hockey game used to be.

Think about it, what's the only play in hockey that gets fans of both teams out of their seats?  A fight.  You can't tell me that's not entertaining. 

 

The old school talent checklist: 

Tier 1 (Fundamentals)  - skating skills, shooting, passing, hitting.   

Tier 2 (how does he play) - game speed, does he score, will he play hard, will he hit/fight?

Yes, I miss the physical game too.   I mean the pre- NHL Gretzky era.  Gretzky Era started the "bodyguard era" where each team had a person or two that would fight to protect their star players.  That led to post Gretzky era were teams would trot out their fighters to fight other teams fighters and actually suited up players that could not even play the game.  

In the 60's and 70's era the fighters were players too.  Even on the Flyers of the 70's, the fighters played a regular shift.  

I like the old game better, but some of that is nostalgia.  The speed and skill of today cannot be matched.  

Edited October 4, 2022 by Pimlach

[SwampD]

5 hours ago, pi2000 said:

What I used to love about hockey was the different roles that needed to be filled in order to ice a winning team. 

It used to be you needed some skill, some grit, some defensive specialists, some size, etc.

That's no longer the case.   

As a result of rules changes the modern game has evolved into a skills contest.   This makes it much easier to build a winner by simply drafting guys who post the best metrics as determine by analytics, hence the explosion of data scientists in front offices across the league. 

Is the game "better" than it was 20 years ago?  It's certainly more skilled.. but is that "better"?   Depends who you ask. 

Personally I miss the physical game.  Today, losing teams simply roll over and concede.    What made hockey unique was the toughness required to play the game.   Today's game is no different than basketball or baseball both massively successful sports, but nothing as entertaining as a hockey game used to be.

Think about it, what's the only play in hockey that gets fans of both teams out of their seats?  A fight.  You can't tell me that's not entertaining. 

Not sure what game you are watching. Maybe this is more true in the regular season, but watch the road the Avalanche took to winning their SC last year. They played some brutally physical series and it was certainly no “skills contest.”

[SwampD]

No player will ever get into the league on analytics alone.

[ddaryl]

Tools and data are only as good as the personell using them. You can translate data 10 different ways, so skill in the analyst department comes from within perspective of sed data and how it actually is appied to the whole team/picture. Got to have talent, and a Data Analyst talents is their ability to translate and apply to the situation at hand.

Just now, SwampD said:

No player will ever get into the league on analytics alone.

I think players who aren't flashy can do a lot of little things correctly and data can support that, but to never actually watch a player on film/in person wouldn't be prudent.

[That Aud Smell]

5 hours ago, pi2000 said:

What I used to love about hockey was ...

I enjoyed and respect this post, even though I don't agree much with it.

In bygone days and just by way of example, players reported to camp out of shape and Mogilny chain smoked filterless Camels at the bar of the old Macaroni Company. They were just such different times. It seems to me that the professionalization of the game, and especially of the players' bodies/health/fitness, is a function of market forces. As the amount of money at stake went up and up, the approach to getting that money responded in kind.

[Taro T]

7 hours ago, pi2000 said:

What I used to love about hockey was the different roles that needed to be filled in order to ice a winning team. 

It used to be you needed some skill, some grit, some defensive specialists, some size, etc.

That's no longer the case.   

As a result of rules changes the modern game has evolved into a skills contest.   This makes it much easier to build a winner by simply drafting guys who post the best metrics as determine by analytics, hence the explosion of data scientists in front offices across the league. 

Is the game "better" than it was 20 years ago?  It's certainly more skilled.. but is that "better"?   Depends who you ask. 

Personally I miss the physical game.  Today, losing teams simply roll over and concede.    What made hockey unique was the toughness required to play the game.   Today's game is no different than basketball or baseball both massively successful sports, but nothing as entertaining as a hockey game used to be.

Think about it, what's the only play in hockey that gets fans of both teams out of their seats?  A fight.  You can't tell me that's not entertaining. 

What play in baseball gets fans of BOTH teams out of their seats?  Same question for football & basketball.

NO other play in hockey gets fans of both teams out of their seats?

How about a penalty shot in a close game?  That gets everybody up.

The reason other plays don't get fans from both teams up out of their seats is pretty much every other play that is worthy of getting people out of their seats is momentary and has been decided 1 way or the other by the time the excitement has finished building.  Nobody gets out of their seat to praise McDavid or Crosby for making their goalie look foolish.  The penalty shot & the fight are high emotion events & the outcome is not known when the suspense begins.  Which is why everybody gets up (& not just because the idiots in front of them are standing & the people behind them have to stand to see).  If there were other in game play events like that, there would be additional reasons for everybody to stand.

 

[Thorner]

14 hours ago, pi2000 said:

What I used to love about hockey was the different roles that needed to be filled in order to ice a winning team. 

It used to be you needed some skill, some grit, some defensive specialists, some size, etc.

That's no longer the case.   

As a result of rules changes the modern game has evolved into a skills contest.   This makes it much easier to build a winner by simply drafting guys who post the best metrics as determine by analytics, hence the explosion of data scientists in front offices across the league. 

Is the game "better" than it was 20 years ago?  It's certainly more skilled.. but is that "better"?   Depends who you ask. 

Personally I miss the physical game.  Today, losing teams simply roll over and concede.    What made hockey unique was the toughness required to play the game.   Today's game is no different than basketball or baseball both massively successful sports, but nothing as entertaining as a hockey game used to be.

Think about it, what's the only play in hockey that gets fans of both teams out of their seats?  A fight.  You can't tell me that's not entertaining. 

This sorta encapsulates how I feel about the current NFL product. I get and support the safety reasons for some of the changes, but the game is essentially a track meet with grossly inflated passing stats at this point. Even more so than hockey the current NFL game bears little resemblance to the game of even the recent past.

[Thorner]

7 hours ago, Taro T said:

What play in baseball gets fans of BOTH teams out of their seats?  Same question for football & basketball.

NO other play in hockey gets fans of both teams out of their seats?

How about a penalty shot in a close game?  That gets everybody up.

The reason other plays don't get fans from both teams up out of their seats is pretty much every other play that is worthy of getting people out of their seats is momentary and has been decided 1 way or the other by the time the excitement has finished building.  Nobody gets out of their seat to praise McDavid or Crosby for making their goalie look foolish.  The penalty shot & the fight are high emotion events & the outcome is not known when the suspense begins.  Which is why everybody gets up (& not just because the idiots in front of them are standing & the people behind them have to stand to see).  If there were other in game play events like that, there would be additional reasons for everybody to stand.

 

3-2 count, bases loaded. Both fan bases stand. 

[inkman]

23 hours ago, triumph_communes said:

Oh so can we ban people who slam post WAR charts instead of arguing?

Slam post 

When a person on Social Media steals a pic/caption from another web-site, then re-post it on there page, multiple times per day, from different pages.

 

kinda diggin this unwanted interpreter role