Problems with charting the Raptors defensive play, as a fan

When yours truly reads an article like this one …

Charting Toronto’s Defensive Play

it becomes easy to see just how problematic it is for the fanbase of this team to become sophisticated consumers of the basketball product which is put out by MLSE.

While this corner has considerable respect for an upstanding individual like Tyler Sherkin … especially, in comparison with some of the other regular “contributors” at a site that functions like RealGM.com … it does not mean that his observations in this article are correct and in any way present an accurate assessment of the actual defensive ability of the Raptors players, or the team in general, based upon the conclusions he’s drawn from the “stats” which he’s developed while working on this project. 

Here are some of the Main Problems which exist in this specific article:

#1. Data Invalidity - Type I: Evaluating the wrong event categories for Defense in the NBA

“Returning to the defensive stats, I was looking at a series of specific events. In particular, I tracked isolation defensive coverage, pick-and-rolls, help rotations/recoveries, contested defensive rebounds and transition defense. In the case of all but the latter, I tracked a positive and negative category. With transition defense, since the opportunities for positive events were so prolific, I only tracked negative events and there were very few, mostly because a player got fouled with no call and fell down (or flopped and fell down) and the Raps were short-handed as a result.”

Rebounding is a distinct phase within basketball and needs to be tracked as such. 

Other important defensive categories are not tracked at all, e.g. Off The Ball Positioning.

#2. Data Invalidity - Type II: Recording events in the correct categories

“The biggest problem, I found, was choosing whether to record pick-and-roll coverages by the help defender in the pick-and-roll category or the help rotation category. I unconsciously recorded several such events in different columns, so that is definitely something I have to fix the next time around by making a clear choice one way or the other.”

If the person recording these events does not have the proper Basketball Acumen and knowldege of a team’s actual strategies & tactics in use at the time, the data produced is not going to be accurate.

#3. Data Invalidity - Type III: Distinguishing accurately between a mis-applied strategy or tactic and an individual failure/breakdown

“The second biggest problem was determining how much was coaching strategy and how much was the players themselves.” 

If the person recording these events does not have the proper Basketball Acumen and knowldege of a team’s actual strategies & tactics in use at the time, the data produced is not going to be accurate.

———-

Here are some of the Unanswered Questions which you should ask while/when/after reading this article:

I. Where’s the actual data Mr. Sherkin?

II. When using descriptive and comparative terms like “poor”, “decent” and “good”,

i. How are these categories of performance distinguished from each other?

ii. How do the Raptors players measure in these categories relative to [A] one another and [B] the other players in the NBA who play those same positions?

———-

Here’s a list of specific conclusions which Mr. Sherkin has developed about certain Raptors, and the team in general, based on his observations from their recent games against the Pacers, the Thunder, and specifically the Bulls:

ONE - The basic assumption that Calderon is a poor individual defender was borne out.

TWO - The Raptors have but two players who are decent perimeter defenders against guys under 6’5 and those are Parker and Roko.

THREE - It is presumptuous to suggest that Anthony Parker’s individual defense is in declining. 

FOUR - The Raptors as a team did not look bad on many defensive possessions.

… while failing to account for the Main Problems [i.e. #1, #2, and #3] and the Unanswered Questions [i.e. 1 & II, i & ii, A & B] which are outlined above.

Caveat Emptor

Tags: , ,

3 Responses to “Problems with charting the Raptors defensive play, as a fan”

  1. The AltRaps Blog » Blog Archive » Toronto Raptors Linkage for April 1st through April 2nd Says:

    [...] khandor’s sports blog » Problems with charting the Raptors defensive play, as a fan - [...]

  2. Scott G Says:

    Nice critical piece. I think that most of the more general criticisms above could be leveled at most statistical analysis, particularly statistical analysis of something that is inherently extremely subjective (like Defense).

    In reality, I’m thinking that “objective” statistical analysis of defensive performances might be impossible — each can have their own opinion of the adequacy of any given defensive performance on any given possession, so I feel this makes any compilation of this information inherently suspect. As you said, caveat emptor…

  3. Linkage for Apr 1 12pm to Apr 3 5pm - Raptors Republic Says:

    [...] khandor’s sports blog » Problems with charting the Raptors defensive play, as a fan [...]

Leave a Reply