Making a Better Football Fan
- Thread starter DropTopDoc
- Start date
DropTopDoc
20/20 Vision With my Buffs On
DropTopDoc
20/20 Vision With my Buffs On
Some viewers of Amazon Prime’s “Thursday Night Football” matchup between the Kansas City Chiefs and Denver Broncos got a special experience: seeing what was going to happen on a play before it actually happened.
Viewers who watched the game in Prime Vision with Next Gen Stats, one of Amazon’s three broadcast options, saw the unveiling of a feature called Defensive Alert that is powered by artificial intelligence to identify potential blitzes before the snap. The model highlights players it believes have a high probability of blitzing (crossing the line of scrimmage to rush the passer) with a red circle that appears under them.
As Sam Schwartzstein, one of the minds behind the model with Prime Vision and the Amazon Machine Learning team, eagerly watched his model work its magic on every snap, on one play, he became confused at why his program was highlighting a nickel corner who wasn’t giving any indication that he was blitzing. “Why are we highlighting this guy?” Schwartzstein yelled in fustration.
Was this a flaw? Did the machine get this prediction completely wrong? Schwartzstein, a former offensive lineman who played at Stanford with Andrew Luck, prided himself on being able to identify potential blitzes. His years of experience as a player and analyst told him the nickel wasn’t much of a threat.
Right before the ball was snapped, the inside linebacker dropped to the nickel’s side and the nickel finally moved toward the line of scrimmage. The program sniffed out this blitzer well before Schwartzstein, watching the game from a wide-angle camera shot. The weirdest part about this is no one really knows how Defensive Alert did it. It’s a self-learning program that has analyzed thousands of plays and movement patterns to understand how defenses move as a whole when certain players blitz.
The model is trained not to identify the usual four down linemen that typically rush the passer. It’s trained to identify unique players who rush the passer on 60 percent or less of snaps. It’s being fed tracking data from Next Gen Stats, which is derived from RFID chips in every player’s shoulder pads. The data includes the players’ acceleration, their orientation and where they are facing. From all that data, the machine starts to understand familiar movement patterns from the defense as a whole, which helps it predict which player is going to blitz.
“We’re highlighting things, starting at line set,” Schwartzstein said. “It’s happening in real time as information is coming in from the shoulder pads. And so you can see all this data coming in and (the model) gets more confident the closer we get to the snap because defenders have to more clearly define their roles the closer they are to the timing of the snap. One of the coolest features for me is we’re not just highlighting it at one time and sticking with it. It is on and off based on where players are moving throughout the play on both offense and defense.”
Defensive Alert, along with many other features on Prime Vision, has turned “Thursday Night Football” into a living, breathing broadcast that is helping the audience see all the nuances of the game as they are happening live.
The goal is to get viewers to see the game as the quarterback does. The quarterback isn’t certain who is going to blitz, especially early on when the defense is showing its initial disguise. But as the snap nears, players start to move around to get close to where they have to, to execute their assignments. The initial alignment and movement help the quarterback figure out who is blitzing or not. The best quarterbacks are coming to conclusions from their wealth of experience or film watching. The machine is processing information the same way, but it has an abundance of data that has been fed into it to pull from in an instant.
Viewers who watched the game in Prime Vision with Next Gen Stats, one of Amazon’s three broadcast options, saw the unveiling of a feature called Defensive Alert that is powered by artificial intelligence to identify potential blitzes before the snap. The model highlights players it believes have a high probability of blitzing (crossing the line of scrimmage to rush the passer) with a red circle that appears under them.
As Sam Schwartzstein, one of the minds behind the model with Prime Vision and the Amazon Machine Learning team, eagerly watched his model work its magic on every snap, on one play, he became confused at why his program was highlighting a nickel corner who wasn’t giving any indication that he was blitzing. “Why are we highlighting this guy?” Schwartzstein yelled in fustration.
Was this a flaw? Did the machine get this prediction completely wrong? Schwartzstein, a former offensive lineman who played at Stanford with Andrew Luck, prided himself on being able to identify potential blitzes. His years of experience as a player and analyst told him the nickel wasn’t much of a threat.
Right before the ball was snapped, the inside linebacker dropped to the nickel’s side and the nickel finally moved toward the line of scrimmage. The program sniffed out this blitzer well before Schwartzstein, watching the game from a wide-angle camera shot. The weirdest part about this is no one really knows how Defensive Alert did it. It’s a self-learning program that has analyzed thousands of plays and movement patterns to understand how defenses move as a whole when certain players blitz.
The model is trained not to identify the usual four down linemen that typically rush the passer. It’s trained to identify unique players who rush the passer on 60 percent or less of snaps. It’s being fed tracking data from Next Gen Stats, which is derived from RFID chips in every player’s shoulder pads. The data includes the players’ acceleration, their orientation and where they are facing. From all that data, the machine starts to understand familiar movement patterns from the defense as a whole, which helps it predict which player is going to blitz.
“We’re highlighting things, starting at line set,” Schwartzstein said. “It’s happening in real time as information is coming in from the shoulder pads. And so you can see all this data coming in and (the model) gets more confident the closer we get to the snap because defenders have to more clearly define their roles the closer they are to the timing of the snap. One of the coolest features for me is we’re not just highlighting it at one time and sticking with it. It is on and off based on where players are moving throughout the play on both offense and defense.”
Defensive Alert, along with many other features on Prime Vision, has turned “Thursday Night Football” into a living, breathing broadcast that is helping the audience see all the nuances of the game as they are happening live.
The goal is to get viewers to see the game as the quarterback does. The quarterback isn’t certain who is going to blitz, especially early on when the defense is showing its initial disguise. But as the snap nears, players start to move around to get close to where they have to, to execute their assignments. The initial alignment and movement help the quarterback figure out who is blitzing or not. The best quarterbacks are coming to conclusions from their wealth of experience or film watching. The machine is processing information the same way, but it has an abundance of data that has been fed into it to pull from in an instant.
Some skeptics believe Amazon is using a delay to see the blitzes coming and highlighting the player on the live feed. Though the processing required for Prime Vision to paint visuals does add some delay (usually three seconds or less), the model that powers Defensive Alert does not use that delay. The team has spent considerable effort to produce predictions as fast as possible — even installing dedicated hardware in Amazon’s state-of-the-art production trucks. There is no person or program trying to trick the audience about prediction capabilities.
One of Schwartzstein’s favorite moments in the game came when the Broncos brought a Cover 0 (man-to-man with no deep help) blitz, meaning they were bringing the house, late in the game.
The Broncos came out with a pressure front with both linebackers mugging the line of scrimmage and a safety showing blitz. The model highlighted all three players, as they were obvious blitz threats. But it also highlighted the nickel corner (top of the screen) even though he was lined up over the slot receiver. The model likely saw the safety lined up directly behind him and recognized this as a nickel blitz indicator.
At this point, you can hear Patrick Mahomes scream out, “Watch Cover 0! Watch Cover 0!” on the broadcast. Before he did, the audience could already feel and see the potential blitzes with all of the players highlighted.
After the ball was snapped, the right tackle blocked the wrong defender, allowing a free runner to get to Mahomes. One of the inside linebackers whom the model highlighted dropped, but the other three highlighted players blitzed.
Correctly identifying the linebacker who dropped from a mugged position would have been extremely difficult, but there might be a time when the model can do this as it keeps getting fed more data.
That play made it relatively easy to identify potential blitzers. Let’s look at the play in which the model saw a blitzer coming before Schwartzstein did.
(The names of the defensive players weren’t shown on the broadcast. I added them in to make it easier to follow along.)
Here, the defense had inside linebacker Nick Bolton mug the A-gap. Nickelback Trent McDuffie lined up outside leverage of the slot and was 4 yards away from the line of scrimmage. Right as the offense lined up, the model already identified McDuffie as a potential blitzer. Schwartzstein didn’t know why. He said before Bolton dropped, he would have kept the protection inside if he were on the Broncos offensive line rather than push out toward McDuffie.
(The arrows showing the defensive players’ paths weren’t shown on the broadcast.)
Right before the snap, Bolton dropped toward the three-receiver side and McDuffie advanced toward the line of scrimmage. At this point, it would have been too late to change the protection. There are protection rules that can ultimately lead the offensive line to pick it up, but it’s always advantageous to have a plan to pick up blitzers before the snap.
Both nickels blitzed, and the Broncos offense didn’t see it coming either because McDuffie blew right by the tackle, who was focused on an inside rusher.
After this play, Schwartzstein texted his producer, “I just lost to the model.”
Again, Schwartzstein doesn’t know exactly how the model is making some of these predictions. It’s learning on its own as it keeps getting data, but don’t worry, football purists. The model is also getting input from a panel of actual football people that includes former players and coaches like Andrew Luck, Geoff Schwartz, David Shaw, David DeCastro, Ryan Fitzpatrick, Andrew Whitworth, Nate Tice and Andrew Phillips.
The panel of experts reviews the film of the model making predictions and makes sure it’s identifying legitimate threats and not looking at players who could not be rushers to the well-trained eye. Some of their feedback, along with that of Schwartzstein, who provides feedback on every play, is fed back into the system.
Even when the model isn’t right on a blitzer, it’ll end up highlighting a player who is doing something out of the ordinary. There was a play Thursday when the Chiefs had two quarterback spies on Russell Wilson. They didn’t blitz, but the model highlighted them because the movement pattern told it they were doing something different.
Overall, the model is predicting blitzes at a very high percentage, but when it does miss one, the defense is likely doing something unique with a movement pattern that it hasn’t picked up before. The defense could be doing a very good job of disguising its intentions, but the model will record it so it doesn’t get fooled next time.
Schwartzstein said the idea for Defensive Alert stems from his time in college when he was already obsessed with the idea of machine learning and football. With his partnership with Amazon, he has the tools and backing to turn his ideas into reality.
“Betsy Riley, my boss, Alex Strand, Jared Stacey, Marie Donoghue and Jay Marine were challenging me to think bigger,” Schwartzstein said. “Because we do have a great machine learning team that’s international and you have access to them, what can you do? When we first started, we looked at making the predictions rules-based: Like what I would do if I was the center or QB? How many eligibles do we have versus how many defenders are on that side of the field? Field pressure is likely where we’re going to set the prediction. Where are we on the field? How close are the defenders? But then, as we got closer, we thought, let’s hand this over to the machine learning team and have them use an AI machine learning model to solve this problem.”
The international team wasn’t familiar with American football, so Schwartzstein had to educate them on the different positions and aspects of the game like motion and how it affects the defense.
“The thing is, these scientists are unbelievable — so this is cool for them because football is a very data-heavy sport with many different data sources you can plug in. They’re so willing and so hardworking at getting this to the next level. And now I’m having arguments with some of them about what teams are doing in certain scenarios. They’re talking about their fantasy teams because they’re wanting to get more invested into the sport.”
Just to ease the minds of concerned fans, teams cannot use this model to their advantage in games. Communication with the quarterback is cut off after 15 seconds of the play clock has expired and there’s no way to get information to the quarterback fast enough. Also, coaches in the booth don’t have access to the Amazon broadcast. Technology usage is extremely restricted for teams. They don’t have access to tracking data during games, and even when they are looking at their tablets, they are looking at stills, not video.
Not every viewer will choose the Prime Vision option. Some prefer to watch the game as they always have. At first, I loved the idea of Prime Vision because it allowed me to watch the game from a wide angle in which I could see all 22 players at once. How do you know what’s happening when you can’t see the secondary? Regular broadcast angles cut them off to zoom in on the QB. But with the added features that Schwartzstein and his team are including like Defensive Alert or Prime Target, which shows viewers which receivers are getting open with a green circle, viewers can watch the game unfold like never before. Viewers don’t have to rely on commentators to tell them the story of the game, they can see it for themselves.
One of Schwartzstein’s favorite moments in the game came when the Broncos brought a Cover 0 (man-to-man with no deep help) blitz, meaning they were bringing the house, late in the game.
The Broncos came out with a pressure front with both linebackers mugging the line of scrimmage and a safety showing blitz. The model highlighted all three players, as they were obvious blitz threats. But it also highlighted the nickel corner (top of the screen) even though he was lined up over the slot receiver. The model likely saw the safety lined up directly behind him and recognized this as a nickel blitz indicator.
At this point, you can hear Patrick Mahomes scream out, “Watch Cover 0! Watch Cover 0!” on the broadcast. Before he did, the audience could already feel and see the potential blitzes with all of the players highlighted.
After the ball was snapped, the right tackle blocked the wrong defender, allowing a free runner to get to Mahomes. One of the inside linebackers whom the model highlighted dropped, but the other three highlighted players blitzed.
Correctly identifying the linebacker who dropped from a mugged position would have been extremely difficult, but there might be a time when the model can do this as it keeps getting fed more data.
That play made it relatively easy to identify potential blitzers. Let’s look at the play in which the model saw a blitzer coming before Schwartzstein did.
(The names of the defensive players weren’t shown on the broadcast. I added them in to make it easier to follow along.)
Here, the defense had inside linebacker Nick Bolton mug the A-gap. Nickelback Trent McDuffie lined up outside leverage of the slot and was 4 yards away from the line of scrimmage. Right as the offense lined up, the model already identified McDuffie as a potential blitzer. Schwartzstein didn’t know why. He said before Bolton dropped, he would have kept the protection inside if he were on the Broncos offensive line rather than push out toward McDuffie.
(The arrows showing the defensive players’ paths weren’t shown on the broadcast.)
Right before the snap, Bolton dropped toward the three-receiver side and McDuffie advanced toward the line of scrimmage. At this point, it would have been too late to change the protection. There are protection rules that can ultimately lead the offensive line to pick it up, but it’s always advantageous to have a plan to pick up blitzers before the snap.
Both nickels blitzed, and the Broncos offense didn’t see it coming either because McDuffie blew right by the tackle, who was focused on an inside rusher.
After this play, Schwartzstein texted his producer, “I just lost to the model.”
Again, Schwartzstein doesn’t know exactly how the model is making some of these predictions. It’s learning on its own as it keeps getting data, but don’t worry, football purists. The model is also getting input from a panel of actual football people that includes former players and coaches like Andrew Luck, Geoff Schwartz, David Shaw, David DeCastro, Ryan Fitzpatrick, Andrew Whitworth, Nate Tice and Andrew Phillips.
The panel of experts reviews the film of the model making predictions and makes sure it’s identifying legitimate threats and not looking at players who could not be rushers to the well-trained eye. Some of their feedback, along with that of Schwartzstein, who provides feedback on every play, is fed back into the system.
Even when the model isn’t right on a blitzer, it’ll end up highlighting a player who is doing something out of the ordinary. There was a play Thursday when the Chiefs had two quarterback spies on Russell Wilson. They didn’t blitz, but the model highlighted them because the movement pattern told it they were doing something different.
Overall, the model is predicting blitzes at a very high percentage, but when it does miss one, the defense is likely doing something unique with a movement pattern that it hasn’t picked up before. The defense could be doing a very good job of disguising its intentions, but the model will record it so it doesn’t get fooled next time.
Schwartzstein said the idea for Defensive Alert stems from his time in college when he was already obsessed with the idea of machine learning and football. With his partnership with Amazon, he has the tools and backing to turn his ideas into reality.
“Betsy Riley, my boss, Alex Strand, Jared Stacey, Marie Donoghue and Jay Marine were challenging me to think bigger,” Schwartzstein said. “Because we do have a great machine learning team that’s international and you have access to them, what can you do? When we first started, we looked at making the predictions rules-based: Like what I would do if I was the center or QB? How many eligibles do we have versus how many defenders are on that side of the field? Field pressure is likely where we’re going to set the prediction. Where are we on the field? How close are the defenders? But then, as we got closer, we thought, let’s hand this over to the machine learning team and have them use an AI machine learning model to solve this problem.”
The international team wasn’t familiar with American football, so Schwartzstein had to educate them on the different positions and aspects of the game like motion and how it affects the defense.
“The thing is, these scientists are unbelievable — so this is cool for them because football is a very data-heavy sport with many different data sources you can plug in. They’re so willing and so hardworking at getting this to the next level. And now I’m having arguments with some of them about what teams are doing in certain scenarios. They’re talking about their fantasy teams because they’re wanting to get more invested into the sport.”
Just to ease the minds of concerned fans, teams cannot use this model to their advantage in games. Communication with the quarterback is cut off after 15 seconds of the play clock has expired and there’s no way to get information to the quarterback fast enough. Also, coaches in the booth don’t have access to the Amazon broadcast. Technology usage is extremely restricted for teams. They don’t have access to tracking data during games, and even when they are looking at their tablets, they are looking at stills, not video.
Not every viewer will choose the Prime Vision option. Some prefer to watch the game as they always have. At first, I loved the idea of Prime Vision because it allowed me to watch the game from a wide angle in which I could see all 22 players at once. How do you know what’s happening when you can’t see the secondary? Regular broadcast angles cut them off to zoom in on the QB. But with the added features that Schwartzstein and his team are including like Defensive Alert or Prime Target, which shows viewers which receivers are getting open with a green circle, viewers can watch the game unfold like never before. Viewers don’t have to rely on commentators to tell them the story of the game, they can see it for themselves.
DropTopDoc
20/20 Vision With my Buffs On
@Trav id love something like this, like a view like this and maybe coaches or coordinators explaining tendencies, I’d watch that over typical broadcasting, or on commercial breaks have a split screen with the commercials playing and them coaches breaking film down of the previous series
This was when i noticed how good Sumlin and Jimbo was along with some other coaches cuz they were breaking that nc down, people tend to shyt on coaches ( I’m guilty) but these guys know their shyt, they sometimes are just hard headed and don’t adapt or innovate
This was when i noticed how good Sumlin and Jimbo was along with some other coaches cuz they were breaking that nc down, people tend to shyt on coaches ( I’m guilty) but these guys know their shyt, they sometimes are just hard headed and don’t adapt or innovate
I tried the Prime Vision out during the Bears/WAS game. It was pretty drop. There were some pictures throughout the article I didn't get to add that shows how it works a little more in detail.
What I thought was dope was that it's not just AI but somebody who actually has experience (OLineman who played at Stanford with Luck) was one of the brain childs for the machine learning behind this. So, it's not just about feeding it #s and shyt but also, as you mentioned, having a mf who actually knows ball there to oversee it as well.
What's crazy though is the machine sniffed out a Nickel blitz before the Olineman dude did who thought it was a mistake. Shyt is spooky.
I also think shyt dope simply for the fact we don't get enough options to watch wide cam or 22 footage fr.
Wild shyt. I wonder how accurate it is
Seems like its already hitting at a high clip already from the article. It caught shyt before the Olineman/analyst did who helped create it. And the more games it reviews and feeds into it's system, it's only gonna become more accurate.
DropTopDoc
20/20 Vision With my Buffs On
Any ball you want to discuss fam ??
Some guys got more trained eyes, and experience, it shows you it’s levels to this shyt, that’s dope they got guys behind it, probably looks at tendencies too
DropTopDoc
20/20 Vision With my Buffs On
@skyrunner1 @mozichrome i know y’all like ball, anything y’all want to discuss or go over
@DropTopDoc
Why 2 Deep is giving these QBs hell lmao
And why it took long for every ream to do it. These QBs can't do shyt when there are two safeties deep instead of one I guess lol.
Are teams playing a lot of cover 2 man or cover 2 zone?
I know the 2 Deep look was popular in the early 2000s. I think slot receivers changed the game.
Why 2 Deep is giving these QBs hell lmao
And why it took long for every ream to do it. These QBs can't do shyt when there are two safeties deep instead of one I guess lol.
Are teams playing a lot of cover 2 man or cover 2 zone?
I know the 2 Deep look was popular in the early 2000s. I think slot receivers changed the game.