Counting Chromosomes
A blog of random musings on genealogy, genetics, science, and history

People tend to hold overly favorable views of their abilities in many social and intellectual domains.... This overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it.
     —David Dunning and Justin Kruger,
         Journal of Personality and Social Psychology,
        1999 Dec;77(6):1121-34

Dunning-Kruger Effect
Image courtesy of YLMSportScience

This will be somewhat of a departure in content: I have a serious point to make, then I need some cathartic comedy time. Welcome to the ride.

I'm sure most of you are familiar with the Dunning-Kruger Effect. Since its proposal as a theory in 1999, it's been borne out in multiple studies—more than 100 to date—across varying environments. In a nutshell, what it tells us is that we're not very good at evaluating our own levels of knowledge and skill. Illusory Superiority: we judge ourselves as being better than others to degrees that violate the laws of mathematics. Those with the least ability are most likely to overrate their skills to the greatest extent, as the little infographic at right shows.

For example, 88% of American drivers rated themselves as having above-average driving skills. Software engineers were asked to rate their performance. In two different companies, 32% and 42% of the engineers, respectively, put themselves in the top 5%. College debate teams who were in the lower quartile—they were losing four out of every five rounds—thought they were winning over 60%.

People who, when tested, are measurably poor at everything from logical reasoning to grammar, math, emotional intelligence, and even chess all tend to rate their expertise almost as favorably, or in some instances even more favorably, as actual experts rate themselves. Poor performers lack the knowledge and expertise to recognize how badly they're actually performing. It isn't an ego thing...at least, not most of the time. As people gain a better understanding of the subject matter, they begin to recognize how poorly they previously performed and how much they have yet to learn.

Actual experts become more aware of just how knowledgeable they are and can more objectively rate their expertise, though they may also be more likely to slightly underrate themselves. But they often fall into another mistake: they assume that others have a grasp of the subject similar to their own. The second chart at right is another way of visualizing the progression.

Progression of Understanding
An annotated representation of the Dunning-Kruger Effect and the acquisition of subject matter expertise

Genetic Genealogist Accreditation

I have long been an advocate of a formal method of accreditation for genetic genealogists. I supplied a position to BCG this summer about their initial proposal for a new genetic genealogy standard, and much of my opinion centered around this. Without BCG taking on the role of formal accreditation of genetic genealogists—arriving at a tested "CGG" certification, for example, to accompany CG and CGL—I fear the final version will not be able to go far enough. And I know of no other certification body at this time who might properly undertake the task of managing accreditation.

In part, the for-comment draft of the genetic genealogy standard includes:

Genealogists consider all available relevant factors when they use DNA test results as a component of proving a relationship. Those factors include reported and typical amounts of shared DNA, sizes and locations of chromosomal segments, information about mutations, markers or regions that have been tested, number and genealogical expanse of people who were tested, and genetic groups, including meaningful triangulated groups.

Genealogists use valid tools and statistical algorithms from testing companies and third parties to interpret test results and establish conclusions about relationships or their absence. They cautiously form conclusions about the absence of relationships. Genealogists do not use DNA evidence to suggest genetic relationships beyond theoretically possible levels.

What the BCG is addressing here, and properly so, are the very core criteria of the Genealogical Proof Standard...from which no form of evidence, including DNA, is excluded:

  1. Reasonably exhaustive research in relevant records, using all appropriate methodology
  2. Thorough identification of each piece of evidence consulted
  3. Correlation and analysis of evidence, individually and collectively
  4. Resolution of any conflicting evidence
  5. A Written proof statement or argument that identifies the evidence and the reasoning

In the end, the ability to perform skilled analysis and correlation of DNA evidence will depend upon the genealogist's depth of understanding of genetics...a skillset separate and unique from traditional genealogy.

Driving the Car

One metaphor I've heard more than once is that you don't need to know how to build the car in order to drive it. The implication is that DNA testing and reporting companies provide us with tools that show us the information and all we need to do is apply it. If AncestryDNA says "possible range: 3rd – 4th cousins" or GEDmatch shows 3.8 generations to MRCA, we're golden. That's all we need to know.

Look again at that confidence spike on the Dunning-Kruger chart. No; that isn't all you need to know, not by a long shot.

Let's try a more genealogically—if over-long—metaphor. Let's say that you neither read nor speak French. You have discovered an image of a document written around the end of the 15th century you believe was penned by an ancestor. The handwriting is full of flourishes and flamboyance, and significant parts of some letters are abraded or faded and difficult to make out. Since you don't speak French, you certainly have no knowledge of ancien français or period or regional dialects like langues d'oïl in the north and langue d'oc in the south.

But you only need to drive the car, right? There's an app for that! You go to Google Translate and type in, as best you can tell, the letters you see...without the diacritics and orthographic ligatures, of course, because your keyboard doesn't have those. Et voila! Google Translate gives you a result! The English translation looks a bit weird and doesn't seem to make a lot of sense but, hey, you have a tool and the tool gave you the information. It must be accurate and correct, right? So let's just paste that translation into the genealogy...

If anything, the analogy is not extreme enough. Genetics is complicated. Trust me.

Now, mind you, I'm not saying everyone wanting to explore their family history by utilizing a DNA test needs to go get an MS or PhD in molecular biology or genetics. Far from it. It isn't difficult to learn enough via all the available online resources to become basically proficient...most especially in understanding where your own knowledge might break down and when to seek assistance or advice. See that "It's starting to make sense" recovery curve on the Dunning-Kruger chart.

The problem in our industry now and in the past couple of years is that there has been a massive increase in the number of people taking direct-to-consumer DNA tests (see "New Data Show Dramatic Growth in Consumer DNA Testing," from August 9). This means literally millions of people who have little to no knowledge of how to accurately apply DNA testing to genealogy. Over 10 million test kits have been sold since January 2017, and though the vast majority who bought them cared about nothing but the ethnicity report, a percentage wanted to dig deeper and find DNA cousins.

Frontier Barbers and Medical Care

The marketplace right now is very like the Wild West: any frontier barber can hang out a shingle proclaiming himself to also be a physician: a free shave with every bullet removal. There is no accreditation whatsoever required—or even available—for one to proclaim himself or herself to be a genetic genealogist...moreover, a professional genetic genealogist. The damage being done is not that many of the self-proclaimed genetic genealogists are at the "I know everything" spike on the Dunning-Kruger chart, but that they are actively representing themselves as experts, advising people improperly, and even teaching methods and procedures of applying DNA information to genealogy that are patently incorrect and unsubstantiated by any of the science as we currently know it. Out of ignorance, willful or otherwise, they are today's genealogical snake-oil salesmen.

Just as someone doesn't need a year of classes and training before taking a DNA test, genetic genealogists don't have to be among the real top 5% in their field. They don't need to be able to go into a genomics lab and start applying reagents for amplifying, fragmenting, and hybridizing samples. They don't need to ever have touched an Illumina iScan or HiScan system. They don't need to be able to walk up to a whiteboard and write the equations used by the Timber matching algorithm.

But they need to understand that Timber exists, what it does, and how the applied mathematics might affect reporting results at AncestryDNA. They need to understand what a centiMorgan is: not just read the output of a compiled report that says X.XXcM but understand what a centiMorgan actually means, what it represents, in general how the Kosambi linear equations arrive at the estimations, how the estimations differ from one genome map to the other, what the genome maps mean and represent and how they're developed and differ from one another. They don't need to be mathematicians but they need to understand why and generally how mathematical probabilities drive almost everything in genetic genealogy, from mutation rates to genotyping to imputation to calculating the aforementioned centiMorgans. They need to understand why and when Sanger sequencing is used; they need to understand what electrophoresis is; they need to understand how microarray chip and NextGen sequencing works; they need to have a firm understanding of the biological fundamentals of meiosis and mitosis, of chromosomes and karotypes and chromatids, of alleles and genes and inheritance, of the function and DNA structure of mitochondria, of the Y-chromosome and its unique characteristics and mutation rates, of the X-chromosome and its inheritance pattern and recombination volatility.

That, and much, much more. Not to engage in their own work, not any more than someone needs to be accredited a Certified Genealogist to work on family history. But to know enough to call themselves professional genetic genealogists, to presuppose that they can advise and educate others.

But Officer, the Run of Homozygosity Hit Me

Unfortunately, I've encountered too many self-proclaimed genetic genealogists that wouldn't recognize runs of homozygosity if they T-boned their cars at stoplights. It isn't very difficult discovering the ones who truly are experts in the field. I have well over 20 blogs bookmarked, subscribe to journals like PLOS and G3, and have Google Alerts set to inform me daily of news and publications I might otherwise miss. All it takes is getting to the "I'm never going to understand this" stage and you can start separating the wheat from the chaff, recognizing the people and publications you know are expert, identifying from whom you can learn.

But for the potential millions who never get to that point—or worse, are provided erroneous educations that steer them completely off the correct, scientifically valid path—the best solution is a structured, standardized accreditation of genetic genealogists. An objective body like the BCG that will provide credentialing necessary to show that, yes, this individual has a thorough enough grasp of genetics and its application to genealogy that he or she can be understood to truly be a genetic genealogist.

Comic Relief

The serious outweighed the comedic far more than I'd intended. A friend knows that I've been dealing with a situation in which some of those charged with defining and setting certain procedures ascribed to genetic genealogy are still languishing on the left-hand side of the Dunning-Kruger chart. This friend sent me links to a few YouTube videos. Nothing at all to do with genetics, but definitely germane to this topic.

From the, "But...it just doesn't work like that," department, short and well-produced videos from Constellations Creatives. Directed by Lauris Beinerts; stories by Lauris Beinerts and Orion Lee; based on characters by Alexey Berezin.

The Expert: Part One - The Kickoff Meeting

The Expert: Part Two - Wrong Angle

The Expert: Part Three - IT Support

The Expert: Part Four - Progress Meeting