MODIFIED Y-Utility: Y-DNA Comparison Utility, FTDNA 111 Mode BETA

Link to McGee's Y-Utility: Y-DNA Comparison Utility, FTDNA 111

What's new (Feb 01 2018)

• Implementation of Nordtvedt's variance method to compute the intralade aka coalescence age of a set of haplotypes.
• Added choice of Iain McDonald's mutation rates
• Added choice of the maximum-likelihood estimate of the TMRCA
• Multi-copy markers weighted to zero in FLUXUS
• Fixed bug sometimes affecting display of source code for Genetic Distance tables

What's new (Aug 06 2017)

• Allow for a link to be embedded in the Haplotype table
• Changed Hybrid GD calculations to match those now used by FTDNA
• Adopted FTDNA's term Tightly Related and modified genetic distance color codes to match
• Provide a checkbox to disable multivalued markers

What's Modified (Aug 25, 2015)
This is a version of McGee's Y-Utility modified by Colin Ferguson to incorporate some new features:

• Incorporation of Marko Heinila's 2012 mutation rates
• Added inverse mutation rate weighting option for constructed Fluxus ych data files
• TMRCA is reported as the range based on 5% and 95% cumulative probabilities
• Removed some features to facilitate modifications
• Modified instructions to match

What's New (May 1, 2011):
This version of the Y-DNA Comparison Utility has several new features:

• 111 Marker support: the new FTDNA results of 111 markers are now supported. These are supported through 102 cells, some of which have multi-copy markers
• Multi-copy markers are supported: multiple values in one cell, separated by a hyphen, "-", are supported.
• All multi-copy markers use the infinite allele method of calculating genetic distance regardless of the setup setting. All multi-copy markers use the same mutation rate within each multi-copy group
• All single-copy markers use either the stepwise or infinite allele method as specified in the setup
• A new CSV, comma separated value, output format is supported so you can paste the output of the comparison table to a .csv file for loading into a spreadsheet that can be loaded in older versions of Excel with acceptable handling of multi-copy markers. This is enabled by the new "Show Data in CSV Format" checkbox

This utility is free for your use. You are welcome to use this utility and it's output as you see fit.

Instructions

• Processing large datasets
  When using Y-Utility with large datasets, you may get the following message repeatedly in Internet Explorer:

  A script on this page is causing Internet Explorer to run slowly. If it continues to run, your computer may become unresponsive. Do you want to abort the script?

  If you frequently process large datasets, you may want to edit the Windows registry to increase the processing time allowed for scripts. By default Windows supports 5,000,000 script instruction executions before posting the warning. If you are experienced in performing registry edits, you may be interested in the Microsoft Knowledge Base article How To Set Timeout Period for Script. This procedure is not for novices. A mistake in performing registry edits can make your computer unusable. Should you decide to perform the edit, a value of 100,000,000 would give twenty times more script execution time than the default. There may be similar procedures for other browsers.

• Copying haplotype data
  This utility is designed for copy-and-paste input of haplotype data from web pages, especially from the new format FTDNA public project pages, or Excel tables. The haplotype data must be in the normal order provided by FTDNA.

  No non-numeric characters other than whitespace and hyphens ("-", for multi-copy markers) are allowed after the first column of allele data.

  Normally generated tables of data will copy correctly, but if you are copying from a manually created table where there may be extra whitespace between allele values this utility may not parse the data correctly. You can either edit the data so that no more than one blank space exists between allele values (or three blank spaces for a missing value), or you can copy the data first to an Excel spreadsheet then copy from Excel and paste into this utility. Data copied from an Excel spreadsheet is tab-delimited, and this utility should be able to parse it without problem.

• Y-DNA Allele setup, "Default" checkboxes
  These are nothing more than a display of the multi-valued markers ignored in a variance computation or fluxus diagram because they are incompatible with the assumptions there in. The one exception is DYS19 which rarely has a 19b value and can be disabled if need be.

• Y-DNA Allele setup, "Enabled" checkboxes
  You can enable or disable comparison of any allele. If the "Enabled" checkbox is not checked then the data for that allele will not be used in genetic distance or MRCA calculations.

• ID Column
  By default all characters before the first data column for each haplotype will be used as identification info. You can select a particular column to use for identification.

• 1^st Data Column
  By default this utility will try to identify the first column of allele data for each haplotype. It does this by looking for the first three consecutive numeric tokens that are within the limits for the first three alleles. Normally this works without issue, but you can construct a table of data where this technique fails, in which case you can specify the starting column of the allele data. Even this may fail in some cases, in which event, you should copy the data first to an Excel spreadsheet then paste from Excel into this utility.

• FTDNA haplotype comparison
  If this table is enabled, then a listing of all the haplotype data is generated in the FTDNA order. If the modal haplotype is enabled then a modal haplotype will be determined and listed as the first row. If difference highlighting is enabled, then difference between the reference haplotype and every other haplotype for each allele will be determined. The background color will be set according to the following:
  • 0 : none
  • 1 : green
  • 2 : yellow
  • 3+ : pink

• Fluxus Phylogenetic Analysis Software The .ych data is one of the formats supported by the Phylogenetic Network software available from Fluxus Technology from their Network Software page. You need to make sure that the haplotype data for all sets of results have the same number of markers. You may want to edit the ID field either in the source file or in the .ych data. The Network software supports fifteen character IDs. For duplicated haplotype data, the ID of the first set is used.
  • For an example of the output, please refer to: McGee Network Phylogenetic Diagram
  • There is a 2MB Flash movie of the phylogenetic network creation process: McGee Network Creation Demonstration
  • The .ych data is weighted by the inverse square root of the mutation rate for each marker. This builds a network with preferential treatment of the slower mutating markers which are more likely to characterize a cluster.

• Embed a relative URL for the root directory containing files named by ID:
  • When checked the ID is linked to an ID file containing more information. When the ID Column is set to All non-data the link is to files named as ID Column 1.

• Genetic Distance
  If this table is enabled then a table of genetic distances between every pair of haplotypes is created. The diagonal elements of the table indicate the number of allele data existing for that haplotype. The calculations can be of two types:
  1. Hybrid Mutation Model
     The target of this model is to match the method used by FTDNA. It uses the stepwise mutation model for all alleles except markers with hyphenated values such as DYS464 and YCA. The stepwise model says that each mutation is allowed to change the allele value by exactly one, so a difference of two means that two mutations occurred and a difference of three means that three mutations occurred. The infinite allele model says that the entire difference between allele values, no matter how large, is the result of one mutation.

     Please notify me if you find any inconsistencies between the genetic distances calculated here and those provided by FTDNA.

  2. Infinite Allele Mutation Model
     The infinite allele model says that the entire difference between allele values, no matter how large, is the result of one mutation.

     The close distances are highlighted for easier visualization of possible family relationships by the following:

     • Tightly Related : green
     • Related: yellow
     • Probably Related : pink
     • Possibly Related : light blue

• TMRCA - Time to the Most Recent Common Ancestor
  This is time from the birth of present generation to the birth of their most recent common ancestor. For example, for a man and his brother the TMRCA (pairwise) would be one generation. If they were both born about 1950 and a generation is assumed to be 31 years then the TMRCA is 31 years meaning their father was born about 1919. The TMRCA (group) between a man, his brother and first cousin would be 2 generations. In this instance the intraclad age would be 1.67 generations. If each of three men tested 111 markers it would not be unusual to find that they match 111 of 111 markers. The corresponding TMRCA (pairwise) in each case would be estimated to be between 0 and 5 generations. There would be no variance and the TMRCA (group) would be estimated as 0 generations.

• TMRCA (Pairwise)
  If this table is enabled then a table of generations or years to the most recent common ancestor for every pair of haplotypes is created. The numbers listed are the 5% to 95% probability that the TMRCA is no longer than the specified number of years or generations. The algorithm taken from Bruce Walsh paper, Estimating the Time to the Most Recent Common Ancestor for the Y chromosome or Mitochondrial DNA for a Pair of Individuals. There are other utilities related to MRCA calculations and can be found on the Y-DNA tools page of the ISOGG, International Society of Genetic Genealogy, website.

  The TMRCA calculations use the average mutation rate for all the markers in common between the pair of haplotypes being compared. There are three selections for how the mutation rate is determined.

  1. The first option for mutation rate calculation uses a constant mutation rate for all markers. This constant mutation rate can be changed in the box provided. The default mutation rate is 0.0024 mutations/allele/generation which represents the 60 total mutations during 24870 total allele meioses as given in Y-Chromosomal Microsatellite Mutation Rates: Differences in Mutation Rate Between and Within Loci by B.Myhre Dupuy, M.Stenerson, T.Egeland, and B.Olaisen; Human Mutation 23:117-124 (2004). Note that this rate does not include rates of some of the newer fast-mutating alleles that are currently being used.
  2. The second uses unpublished rates determined by Marko Heinila, Helsinki University of Technology, 2012. They are briefly discussed in this thread [GENEALOGY-DNA/2014-03].
  3. The third uses unpublished rates used by Iain McDonald, University of Manchester, in his spreadsheet TMRCA Calculator Version 4.1 Build 20170321. They are an ensemble average of three sets of rates: Heinila (2012), Burgarella et al. (2011) and Willems et al. (2016). The averaging is discussed in his file R-U106 – March 2016 overview report

  Years Per Generation - the table below was once published on the SMGF site prior to its shutdown:

  Mean generation interval:	32.6
  Median generation interval:	31
  Modal generation interval:	26
  Number of father/son pairs:	92,309

  The close distances are highlighted for easier visualization of possible family relationships by the following:

  • Cumulative TMRCA at 95%
    • 0-24 Generations: green [related]
    • 25-34 Generations: yellow [probably related]
    • 35-44 Generations: pink [possibly related]
  Note that if units of years are chosen, the number of generations are multiplied by the specified rate of years per generation.

• Intra Clade Age (Group)
  This is the method of Nordtvedt. The math is from a spreadsheet called Generations111T.xlsx that was on a website at knordtvedt.home.bresnan.net/, accessed 2 Mar 2014 (Note: that spreadsheet is no longer online, but as of 28 Jan 2018, it can be found in the Internet Archive: https://web.archive.org/web/20120616154221/http://knordtvedt.home.bresnan.net/).
  • Output Variables
    • Sum of Mutation Rates ± assumed uncertainty
    • Intraclade Age - this is the average TMRCA of the corresponding set of pairwise TMRCA.
    • Confidence Interval - the 5% to 95% confidence level of the Intraclade Age [see error analysis]
  • Marker Choices
    • Those excluded by default
    • Plus any other you disable
    • Note: you cannot enable any of the default exclusions
  • Special Handling - Blank and null values must be eliminated either by disabling the affected markers or deleting the affected haplotypes.
    • Blank Values - all haplotypes must have the same set of markers
    • Null Values - some markers have null values, e.g. DYS425.

• Modal Haplotype
  If this item is enabled then a modal haplotype will be created that is the haplotype consisting of the allele values with the largest number of occurrences in the haplotypes provided. In the event of a tie, the larger allele value is used.

• Show Legends
  This item causes a color legend to be associated with each chart for haplotype, genetic distance or generations to most recent common ancestor

• Show status
  This item causes a status box to be displayed at the top of the output window. Large numbers of haplotypes may take some time to process and this status box gives an indication of what items this utility is currently processing.

• Show ToolTip names
  This item enables the marker name to be displayed by hovering the mouse over the value in a results table.

• Show HTML Source
  This item causes the HTML source code for each table to be output to a text box for ease of cut-and-paste to project web pages.

• Highlight Reference
  If this item is enabled then the specified haplotype will be used as a reference for the haplotype comparison table and will appear with white background and blue fontcolor in all the tables.

• Disable Multivalued markers
  If this item is enabled then multi-valued markers are no longer used in the GD and pairwise TMRCA computations and appear greyed out in the haplotable.

• Clear Button
  Pressing this button will clear the input data textboxes but retain all other configuration information.

• Execute Button
  Pressing this button will cause a new browser window to open containing the specified tables of data.

• Example Button
  Pressing this button will initialize the haplotype box with 111 marker example haplotype data. This may be useful in understanding how the data should be formatted in the event that you have difficulty in using this utility.

• Debug box
  This textbox contains the processed data and includes line numbers. It may be useful to use this to identify where the input data is incompatible with the parsing algorithm used by this utility or to identify the row number of the desired haplotype for Highlight Reference selection.