Problems with Statistics on Homeschooling

By Prarthana Jayaram

The ProPublica piece Small Group Goes to Great Lengths to Block Homeschooling Regulation examines critical questions about differences in homeschooling legislation between states and how the government may be lapsing in its role of protecting minors from abuse. The lapse is a result, more or less, of a lack of homeschooling regulation in most states; an organization called the Home School Legal Defense Association (HSLDA) has been responsible for blocking this kind of regulation for years (even where many non-member homeschooling families would be open to regulation).

Like most ProPublica stories, this one does a good job of intertwining compelling narrative from traditional “shoe leather” reporting with data analysis. I thought the way the story fact-checked statements made by each side of the fight – leaders at the HSLDA and researchers / policy-makers who believe there should be more legislation for homeschooling families – was telling.

Michael Farris (HSLDA founder) talks about the threat of government social workers to families who homeschool:

“Farris said his group gets 300 calls a year from dues-paying members worrying about ‘social workers at the door’. This number, however, represents just 0.35 percent of the HSLDA’s membership, assuming each call came from a different family.”

This makes it seem like it’s not a widespread concern. But then the article goes on to explain the other side’s counterargument, which I think (sort of) illustrates the concept behind Bayes’ Rule (which really should be Bayes’s Rule, but we can discuss that another time). Basically, an education researcher contends that Farris’s concern (that parents who are homeschooling their kids are constantly under scrutiny and generally afraid that social workers will get them in legal trouble because officials don’t understand homeschooling) is outdated. That kind of thing was a problem when homeschooling was first legalized, but it isn’t anymore. Rather, the issue now is the opposite:

“If social workers are particularly interested in homeschooling families, it’s not because they assume those parents are predisposed to be abusive, said Barbara Knox, a University of Wisconsin pediatrician who specializes in child abuse. It’s because parents who do have a pattern of abuse often pull their children from school under the guise of homeschooling in order to avoid scrutiny.”

The story mentions a small-scale study that Knox conducted in 2014 – in 38 cases of severe child abuse, nearly half the parents had removed the kids from school or else never enrolled them at all. So, very rudimentary calculation:

P(homeschool | abuse) = 0.5

Whereas P(abuse | homeschool) is unknown, but probably much smaller. It seems like this is the idea that Farris and the HSLDA are trying to push. There are obvious problems here, starting with the fact that Knox’s study does not have a large enough sample size to generalize to a national level. But at large, looking at this statistic both ways sheds a light on the issue. Yes, there’s a reason for the suspicion, but there is also a reason homeschooling families might balk at the mistrust.

Ultimately, though, I think the data in this story supports the anecdotal conclusions that the HSLDA is a little bit crazy.

It’s not an apples-to-apples comparison*, but if we pull in the numbers Farris cites about members who call to report having social workers check in on them, we have:

P(social worker visit | homeschool) = 0.0035

What’s nuts here is that the article states that 85% of homeschooling parents in the country are NOT members of the HSLDA (the author interviews a bunch of folks who support homeschooling and stricter government regulations for homeschoolers). So, that means that a) this group represents a very vocal minority, and b) the problem of meddling social workers is likely even smaller than the above number shows.

The story raises the problem of sample size and the significance of bias. There aren’t many independent studies on homeschooling. Both sides are sort of speculating on a lot of points here because it seems like there isn’t much fair and reportable data.

*Chiefly, the sample sizes are not the same for Knox’s study and the figure reported by Farris. Also, there are plenty of homeschooling families who are not members of the HSLDA, so that 0.0035 number derived from the 300 calls Farris gets per year, even if correct, is not a representative statistic.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s