Analogy, growing apples up to the speed of ridicule: A farmer wanted to sell more apples but was limited to a single tree. It was a lot of work for a modest profit. But one day a tourist came to the edge of his farm, pointed at a tomato and asked, “how much is that apple?” At a distance of 10 yards, the farmer learned, customers could not tell the difference between some tomatoes and most apples. So he put up a fence, separating the table of fruit from his customers by 10 yards. He sold 5 tomatoes as apples for every 10 real apples sold. If at 10 yards tomatoes could appear to be apples at a ratio of 1 for every 2, how many more “apples” would he sell if he separated the customer from the fruit by 20 yards? The difference between 10 yards and 20 yards, was the addition of 2 tomatoes for every 1 apple. He decided to set the distance at 30 yards, but told his customers that they were standing 20 yards away. Now according to his math, the new “fruit” added to the previous results would be 3 tomatoes for every apple. But at this point the customers threw up their hands, laughed and walked away. He apologized and said that it wasn’t that he misspoke; he simply measured incorrectly. He had accidentally measured 30 yards instead of the 20 yards he told them. He then matched up the measurement used in the actual evaluation with the message he gave his customers: 20 yards. Nonetheless, he still mixed this year’s tomato harvest with this year’s apple harvest and put up a sign that read: “We have more apples than previously estimated.” No one asked, why 20 yards and not 10 or 15? And then along came a prospective buyer.
- Prospective Buyer: So how many apples do you have there?
- Farmer: It all depends sir. How far can you step back without feeling ridiculous?
Review of actual case: The pharmaceutical industry wants to sell more of its drugs, but it is limited to those who inherited a specific LDL-R mutation: FH. It was previoulsy held among the scientific community that FH has a prevalence of 1:500. That would provide a modest profit. However, it is widely known that in clinical analysis mistakes are often made. Many of those who are determined by a clinical scoring system cannot actually be found to carry a mutation when subjected to molecular testing. The clinical scoring system only tells a doctor that those above a given detection point look like they might be carriers of the mutation. For example, Damgaard, et al, performed a study of those who scored in the top clinical category and found that 1 in 3 could not be found to have a mutation. As we move down the scoring system, loosening standards, the second lower category showed 2 in whom a mutation was not found for every mutation found. That’s 2 out of 3. Within this category, we have flipped the risk-benefit ratio on its head by lowering the diagnostic standard a single notch. The next lower category has a failure rate of 78%. This is what the authors did. They not only lumped the first two categories together, which then averaged 50%, but they also took a slice of this lower category with its 78% failure rate and blended it in, declaring a prevalence rate of 1:137 – extremely high. Forced, or unforced, the authors later issued an apology and correction. They had printed “6” as their DLCN cutoff point, but actually used 5 on the data underlying the printed result. It wasn’t that the text contained a typo; it’s that the greater part of their labor in crunching the data used the wrong number. Essentially, they put “cutoff 6 = prevalence of 1:137” on the table, while under the table they had arrived at 1:137 by using the cutoff point of 5. The “Corrigendum” essentially took out the slice from the lowest of the three categories, but still blended the other two. What were the reasons for the adjustments? Why one number and not the other? The clinical cutoff chosen appears to be arbitrary. Where was the concern or mention for false positives that are always a consideration with clinical screening? If one lowers the standard, wouldn’t one take it for granted that there will be more errors? … more false positives in the result? Where is the accounting for this?
- Patient, Doctor, and Investor: So what’s the appropriate clinical detection point to determine FH?
- Industry and Scientist: It all depends sir. How far can you lower the standard without feeling ridiculous?