Michael Fagan's blog

Continuing with my attempt to write about some things I’m doing at work, here are two related stories from a decade ago.

In these cases, they were projects involving plenty of other people. Although I played a significant role, I didn’t initiate either, and this post is from my own lens.

Pubic Accounts

The first one was Public Accounts, i.e. the financial records that the Ontario government publishes annually. Initially I was one of a few people given the task (summer of 2015) of creating a visualization for the release of the 2013-2014 fiscal year records. This was the first time I had even heard of public accounts.

What we were given were basically 3 datasets. One was a giant PDF. We wanted to publish this as machine-readable data, in addition to any visualization.

Obviously the best way would have been for the data to have been in a machine-readable format to begin with, but no such luck, nor was it practical with the time we had to make this happen. Effectively, each government ministry was given a Microsoft Word template, which they would modify, and the whole thing had been combined to create the PDF we were working with. Maybe we could have gotten some of those Word documents, but that wouldn’t have been much better.

Because of the complicated layouts and this being pre-AI, it wasn’t automatable. By this time we had joined up with what was then the Open Data team. After understanding the data, I created a basic schema (as in, spreadsheet columns), and the Open Data team collected themselves and any other co-ops and interns they had available, to split up the work of manually copying tons and tons of numbers from the PDF into spreadsheets. It was easy for each person to have their own spreadsheet and combine them later. I think it was perhaps 8 people in total, fortunately not including me 🙂

I think it was on the way home from work, possibly the day before the manual copy-pasting started, that I conceived a tool to check for copy-paste errors. The PDF essentially listed spending hierarchically, with both the individual amounts and the totals listed. There were more than 2 levels in the hierarchy, but in all cases the (sub)totals were given. So I got home and immediately started working again. By the time I went to sleep, it was done and one could copy the digitized spreadsheet into a textarea and have it sum up each part and compare it with the stated total, flagging any mismatches.

I showed all the “volunteers” how to read the PDF, to copy-paste into the right columns, and to use the QA tool. This tool became something to use regularly, not just at the end. It was very interesting (well, to me) that in addition to finding and fixing our own errors, it found errors in the original PDF. In many cases this was due to rounding, but at any rate we were able to get the data corrected.

Once the whole thing was completed and the numbers matched up, I also ran spell check (nothing fancy, just using Excel) and found more issues that got fixed.

We were able to release all the machine-readable data alongside a visualization as part of the release of public accounts generally, data which had never been published in machine-readable format before.

Aftermath

I wasn’t highly involved, but fortunately others were able to intervene in the process itself and get the data to begin life in a machine-readable format for the following year.

I helped out again, mostly just in rerunning another script I had made to convert from csv to the format used by the D3-based visualization, but I was a little sad to see that the new data format only included the detailed amounts, not the totals and subtotals. This meant that I had no additional numbers through which to validate the data, although presumably/hopefully something like this had been done upstream.

I was involved less and less every year, but one can see from the most recent release that the data I am referring to is now a datatable, using basically the same schema, with links below it to download the whole thing.

The Gazette

Around Christmas 2015, a coworker on an adjacent team asked if I had some time to help, as they had a big new workload. Some years’s worth of back-issues of the Ontario Gazette (not all the way back to 1868) were going to get converted to HTML to publish alongside current issues to meet accessibility legislation. The PDFs had already been converted into Microsoft Word.

The process being used was to manually copy from Word into the web-based WYSIWYG editor our content management used, then edit the source to fix any issues. There were a ton of issues, as Microsoft Word generates very messy HTML, and the conversion wasn’t aware of our CSS classes that would be used to match various Gazette formatting styles.

As much as I wanted to help, it was immediately clear that the two of us could work 24/7 on this and it would take much of our career. As with the previous project, this was very much before the LLM-era. I looked to see what I could automate and ended up with two practices.

If I was just looking at the HTML source in Sublime (the text editor I was then using), I could use it’s built in find-replace function with regular expressions to take care of a lot of things. I found a Sublime extension such that I could make a huge list of find-and-replace regular expression patterns that could run one after another from a single shortcut. This saved sooo much time. You still needed to manually review the HTML, but there was so much less to fix, and every time something got fixed, we would try to add it to the replacement patterns, so eventually this method was fixing almost everything.

The other automation didn’t save nearly as much time, but when you multiply it by how often it was needed, it was still a lot in total. In our Content Management System, to create a new page, you’d have to set a lot of metadata, but most of the fields were identical for every Gazette page. I created a bookmarklet which set that metadata. Bookmarklets are such a nice little bit of functionality that doesn’t get much attention these days.

Even with both of those practices in place, the number of steps required multiplied by the number of back issues, meant this was still a huge endeavour. As it happened, I had heard that my team had some remaining budget that could be used for temporary staff, which was unusual in that I generally know nothing about our budget. So I suggested we hire some people to do this work.

I was able to choose the people (based on their resumes), and we hired eventually 5 people working at the same time. More in total (some left, and I had to let one go), and I think it took perhaps 6 months of full-time work, but it did get done. I trained the first one to start and had them train the second, and so on. I reviewed their work initially until I was satisfied that they knew what they were doing, after which I only looked in when they had questions.

Aftermath

I think it’s pretty clear from the effort involved after my two automations why I had such a reaction to the workload initially.

The work got done, and at the time, it could only have been sped up further by hiring more people at once. As it was, we were able to keep some of them after the project for other work and I’m glad to have been a part of helping a few people with their careers, even in a small way. Between the hiring, firing, training, orientation, administration, and to some extent supervision, it is still the most I’ve ever been involved in HR. Fortunately for me the hires were done through a third-party agency for temporary positions, meaning I didn’t have to deal with the usual HR complexity of the public service.

Two projects I’m proud to have worked on.

I built a little game one weekend of March 2018, and tweaked it a bit over a few weeks. I always meant to polish it further and release it, and since it is 2026, I am doing just the releasing it part.

It’s only ever been played by friends, coworkers, etc., but never really by kids, which is who I had in mind when I made it.

The idea was to support neuroplasticity and also learn a bit. I won’t say much more, as the intention was (and is) that the instructionless game teaches you how to use it and reveals itself as you play it.

Without further ado, here you go: Langmath.

I have a pretty long list of planned improvements, and I may still make them at some point. If you enjoy it or have other thoughts, let me know.

This is the story of some work I did for an employer over a decade ago, although I think it is still very applicable today. I always meant to try to get some of the code released, but now much of it would be considered outdated, so I’m sharing the core of the algorithm.

Suppose you have your own search engine and you want to know if it is producing good results. How can you tell?

You can get the most popular search queries, manually judge what the best result is, and measure how well those best results rank for those queries. But in a broad search engine, this will be a negligible portion of overall queries and may not be representative. Not to mention that others may not make the same judgments as you did. Similarly, if you are a giant search company like Google, you can hire a lot of people to constantly judge search results.

But if you are smaller than that, what can you do? I won’t get deep into other methods, but I will share what I did. I think it worked very well.

I did find out later on that there had been an earlier effort to compare our results to Google’s results (limited to our website), with the assumption being that Google’s results were “correct” and the better ours matched Google’s, the better the relevance, but I wasn’t a fan of that idea. Nevermind that Google doesn’t want people scraping their results.

The algorithm

Basically, this method looks at what users do with search results. It is a backwards-looking metric. In other words, you can evaluate the relevance of the live search engine on your website, but it doesn’t really* enable you to predict how a proposed change will affect relevance. What you can do is launch the change, wait a few days (well, this will vary depending on search engine use, but for us it was a few days), and then see the results. If the results are good, keep the change, and if they are bad, roll it back. After-the-fact data is a lot better than no data.

I was aware of how the old Direct Hit search engine had pioneered the practice of tracking what results users click on and putting those pages higher the next time the same query was done, but my effort was focused more on measuring than affecting.

The formula evolved a bit over time but this is what it ended up like. It used three types of data for every query:

• how many search results were clicked on
• the position of those results (1 being the first result)
• how often another search was run immediately

var irrelevance = 1;
if (resultClicksTotal > 0) {
  var differenceFromSingleClick = 1 - ( resultClicksPerSearch <= 1 ? resultClicksPerSearch : (1/resultClicksPerSearch) );
  var positionFactor = Math.min(Math.log(averagePosition), 1);
  irrelevance =
    (differenceFromSingleClick * 0.46) +
    (Math.min(refinementsPerSearch, 1) * 0.3) +
    (positionFactor * 0.24);
}Code language: JavaScript (javascript)

Going through each part,

• irrelevance will be 0 (perfect relevance) to 1 (completely irrelevant)
  • when displaying this, the number will get converted to relevance, i.e. 1 - irrelevance
• resultClicksTotal is how many times a search result was clicked on for the given query
• if no results were clicked on, that counts as completely irrelevant, so we default to a value of 1 before checking
• if any results were clicked on, then irrelevance is based on three factors: difference from a single click, how often users did another search, and what position the clicked-on results were at
  • these are the weightings I found worked well for the three factors: 0.46, 0.3, and 0.24 (which sum to 1)
• difference from a single click is used, because clicking on exactly one result can be assumed to mean the user got what they wanted on the first try (very relevant), whereas clicking on a lot of results means they struggled to find it (irrelevant), as does clicking on very few results (less than one per search)
• the position factor uses a log function, to roughy account for how users actually click on results by position; I just looked this up now and you can see some example data here for comparison
• users doing another search is assumed to mean that they didn’t find what they wanted with their initial query (refinement = new search)

To summarize: if for a given search query, nobody ever clicks on any results, then it is completely irrelevant (1), whereas for another query that everyone clicked on exactly one result, and that result was in the first position, it has perfect relevance (0 irrelevance). Anything else is somewhere in between.

All of this is for a single search query. For each query, I also calculated an irrelevance value weighted to how important the query was, by multiplying it by what percentage of searches were for that query:

irrelevanceWeighted = irrelevance * viewsPercentageCode language: JavaScript (javascript)

Lastly, overall relevance (well, irrelevance) for the whole search engine is calculated by summing all the weighted irrelevance values.

Putting this into practice,

• a cron job every day to (1) download all the analytics data from the previous day of what searches were done and what results were clicked on and (2) calculate irrelevance for each query and overall
  • this included some normalization, like trimming the queries and converting everything to lowercase
  • the analytics data also included what page the searches were being done from, as that was used for other purposes
  • irrelelevance was calculated for the day as well as for the past 7 days (rolling)
• a second cron job run monthly

Using the data

I built a pretty robust dashboard for making use of all this data. The main page showed a sparkline-like bar graph of search volume, and a table with the top 200 queries, beginning with a summary row. The column showing relevance values had its background coloured to match the number, i.e. completely red = very irrelevant and completely green = very relevant (conditional formatting in Excel-speak), making it easy to quickly see how queries were performing.

The list could be resorted ascending or descending by relevance (weighted or not) or other values, and the whole list could be filtered by regex, which would update the sparkline as well.

Clicking on each query would add in details showing what pages the query was run from and what new searches were run instead.

In practice I found this extremely useful. One could detect if relevance or search volume was changing overall or for particular sets of queries, and the tool also helped to answer the question of “why” something was changing or had poor relevance. I ended up keeping a list of regular expressions to look at various types of queries and various topics.

Once a front-end encoding bug meant that anything you searched for yielded zero search results. This tool, which showed a drastic and sudden drop in relevance was how we found out… it’s not as if our users were going to be writing in to us about it.

By sorting by weighted relevance, you could see the queries most in need of improvement and target those. One thing that people were looking for often resulted in a page that had the correct information but it wasn’t explicit in the title and the content wasn’t at the top of the page. Some reworking of the content on that one page alone resulted in a measurable improvement to overall relevance. In another case, people were looking for a topic that we had content for, but not a single high-level page that linked to the specific pages, something that was easy to rectify.

Obviously fixing individual queries and groups of queries can only take you so far, but the tool also made it easy to see overall effects of changes to the ranking algorithm.

The tool also had a second view listing pages rather than queries. With this lens, pages with lots of queries could be assumed to need more content improvement. Just like with queries, a weighting factor would enable sorting to see the pages most in need of improvement. Anyone working on specific pages could easily use the data to see what people were looking for from that page as great evidence of what needed improving.

Final thoughts

* I noted earlier that this method can’t be used for prediction, but that’s not completely true. Although I never built any such functionality, one could use it to see whether the ranking of results better matched the frequency that those results were clicked. For example, if users clicked more on result #2, and less on result #1, and an algorithm change would cause the order of those two to swap, it’s a safe assumption that it is an improvement for that particular query. There are other possibilities as well… for queries that resulted in no result clicks at all, one could determine which of those was caused by there simply being no search results… and if an algorithm update meant more no-results queries got results, one could assume that would be an improvement.

Anyhow, that’s pretty much it. If you’ve done something similar or try this algorithm out, feel free to share. I’m haven’t worked on this in some time but you’re still welcome to share any improvements you make to it or note if different weightings for the 3 factors make more sense for your own use case.

The idea

In late 2012, I was running Waste Nothing, which listed options for reducing waste by type of item, and I wanted to do something that was more direct action that just listing others’ services.

I saw what was going on with Repair Cafés in the Netherlands. With my background in the software world, I was thinking agile, and minimum viable product. The MVP for this, for me, was to limit the scope to things that could be repaired with a needle and thread. While my early thinking was a single big event inspired by Soupstock, I settled on a different plan – although the “big event” thinking is when I came up with the name “repairathon.”

I would recruit volunteer clothing repairers, solicit material donations, and book our group at existing community events that already had lots of attendees. This would all keep the risk of failure pretty low: if nobody donated anything and nobody volunteered, and it was just me by myself with my own thread and sewing machine running the event, it could still be a success.

Aside from the goals of this new group (reducing waste, etc…. see the repairathon website), there were two things I wanted to achieve:

#1 – a money-free organization. Just once – I thought – we could have something that wasn’t an economic transaction. So we didn’t accept financial donations, we didn’t spend any money, we didn’t charge for events, we would not pay to participate, we wouldn’t attend events that were not free to attend.

I would say this worked quite well on the whole… there were some events that wanted us to pay to attend and when we told them of this policy, they would waive the fee. There was perhaps a single volunteer who wanted compensation for the transit fare to come volunteer, but that’s a longer story. And we did relent and accept donations in the form of food for volunteers to eat during the event if purchased from food vendors that were also attending the event.

#2 – a great experience for volunteers by expecting nothing of them other than showing up, and by giving them only actual useful work (i.e. clothing to repair). This grew out of my experience showing up to volunteer for things where the organizer didn’t have work for me to do yet, or wasn’t sure what to do, or had us doing not-especially-useful tasks.

This one mostly worked as well, although many volunteers did transition to taking on other roles. We were occasionally asked to attend an event that was more of an information-type event, where organizations had a table to tell other people about their work. I always told organizers that if we are attending an event, we are working. We will repair clothes, not just tell people we repair clothes. I think this worked well too.

Starting off

I posted online asking for volunteers, and later asking for material donations. It was very exciting to get both. Around the same time I reached out to to a few events that I thought we might join. We picked a date. We had an in-person volunteer meeting (just about the only one I ever held). I personally collected all the donated materials, by bike.

And, well… it worked. Our first event was in April, 2013. We planned a second event and we started to get some media publicity.

In keeping with my aim to have the volunteers do the rewarding part, I continued to recruit volunteers and donations, plan future events (after confirming with volunteer availability), manage social media, etc.

Around this time another group reached out to me as they too were interested in doing a Repair Café, an effort I supported from the beginning. We didn’t work directly together as much as I anticipated, but we promoted all their events and attended most of them (myself or another volunteer) with our materials.

What didn’t work, problems, and changes

I planned to collect a ton of data, and I had a laptop with a spreadsheet all ready to go, as you can see in the photos from the first event. I figured we would be able to say exactly how many of which types of items we repaired. This lasted a total of one event, as it wasn’t really practical to do all that data collection during the event… as I was also taking on the role of greeting everyone and introducing them to the concept. I didn’t end up doing much repair work myself.

We decided we were never going to use the large spools of thread used in serger machines that had been donated, so I donated them on to the Toronto Tool Library, a then-new organization that I also supported and did some volunteer work with starting when it was just an idea.

We were occasionally asked to do an event associated with a company, and I always said “no” to those. I didn’t want others to get credit for our work, for us to be seen as endorsing anything, or for us to attend events that didn’t feel “public.” I did encourage those organizations to run their own repairathons with their own volunteers/staff.

As those who have more experience managing volunteers than I do will attest, there can be problematic volunteers. People who complain a lot, ask for things that don’t align with the group’s goals, etc. We really only had one like this. Of course I was always nice to them, listened to their needs, explained in some cases why something wasn’t going to happen, encouraged them to pursue certain things on their own with any help we could provide, and directed them to more positive directions. In the end, they dropped out of participating, and that’s okay… you can’t please everyone.

That leads to the next issue… this person wanted to be compensated for the transit fare of getting to events to volunteer. This wasn’t a bad idea per-se but it conflicted with how I had hoped to match volunteers to events. What I wanted to do was have a handful of volunteers in every neighbourhood, and those people would do the events in that neighbourhood… as many or as few events as they’d like… not to have a few volunteers spread throughout the city and travelling greater distances to volunteer. We never really reached anything like this… in practice there evolved a core group of volunteers who did most of the events throughout the city. I did try to encourage and support other groups in running repairathons further from downtown and in adjacent cities, in some cases loaning all our materials out so that they only had to supply the volunteers.

The last issue for me was attendance. I always assumed that some volunteers would end up having to drop-out of an event last-minute, which I took into account when determining if enough people were available for an event to be scheduled. But then there was one event, where all the other volunteers didn’t show up, leaving me to do the greetings, repairing, photo-taking, and no bathroom or food breaks. It was doable but I was tired and decided it was time to actively hand off work.

The other volunteers more than took up some of the work, they completely took it over. I didn’t plan or attend a single event after that other than as another volunteer sewer. They chose, plan, and ran the events. It was great. All I was doing at that point was forwarding emails and keeping the materials (mostly a single suitcase) at my home. Events continued for years and I was barely award of them.

More outcomes and happenings

We really did so much great work. We fixed so many clothes and other textiles, across a lot of events throughout the city. And not just any clothes, clothes that people really loved. I remember a woman with tears of joy after we fixed her purse. And by “we,” I of course mean the actual volunteer that did the repair. Some random things too… like this couple that asked us to sew a waterproof cover on a dress as their infant daughter refused to wear a bib.

We heard from others outside of Toronto who wanted to do the same thing. A group in Edmonton comes to mind. We did what we could in terms of advice, encouragement, and promotion.

I was interviewed on CBC’s Metro Morning. While I often enjoy public speaking when I choose to do it, live radio with a huge audience was terrifying even though I had done it once before for Waste Nothing. I was on a panel discussion about clothing waste with people who were actually experts. I did a number of other interviews for newspaper, radio, etc., but at least they were not live.

Sometimes we got request for work that was not repairs, like making custom clothing or tailoring. We referred these requests to local businesses, or to any of our volunteers who wanted to take on paid work independently. People were volunteering their time and I was always happy for them to let people know if they also had their own businesses, as long as it was clear that the repairathon itself wasn’t promoting or endorsing it.

One thing I hadn’t planned on was the friendships. The repairathon coalesced around a small group of active volunteers who became life-long friends. That’s priceless.

Ending the group

The COVID-19 pandemic ended all in-person events of course, and the repairathon has not run an event since. And that’s okay. Neither I or the volunteers (which had long since not included me) felt the need to get it going again.

The Repair Café Toronto, which covers all types of items (not just textiles) is very much still running, and our former volunteers sometimes attend those. Earlier this year we gave all our remaining materials to the Repair Café.

Yesterday I finally updated the repairathon website to indicate all this.

The volunteers

This is my blog, so of course this is all written from my perspective and pertaining to me. I hope it’s clear that while I got things going initially, this was only ever possible because of so many others. Even those people who just volunteered at a single event and were never part of organizing things, their contributions were what made the repairathon a success.

For the friendships alone, this has been one of my greatest achievements.

Genealogy has been a hobby of mine for some time. One thing that has long frustrated me is that although there are good tools for creating a family tree—managing data, basically—there are not a lot of good ways for seeing and using it.

I suspect more people are interested in seeing their family than they are at doing research or data entry.

After I became a parent, I started working on creating a poster that could be displayed in our home, that showed our family. It’s finally ready for an alpha launch. Continue reading below or go see Family Circles Diagram.

Goals

The main issue for me is that most family diagrams show (a) a focus person and their ancestors, or (b) a focus person or couple and their descendants. I wanted to be able to show everyone. Overall, the goals were to create something:

• that shows all types of relatives
• beautiful by itself, not something ugly or plain that is just decorated (chartjunk)
• to be printed
• that focuses on the present; that is, family as it is experienced today
• readable from one position (not requiring rotation)
• in a format that enables further editing

To me at least, these have largely been achieved, although further improvements are still planned.

The basic design (and name)

A focus person (or people) are shown in the centre. Around them (arranged in a circle) are those one “step” away: parents, full siblings, children, and spouses. Around them are people one step away from them, and so on. This is illustrated better on the diagram website.

This is unlike any other family tree diagram I am aware of, as those will have only people of the same generation on each level or circle.

The name comes from the concept of social circles, with you in the centre, your close family around you, close friends around them, then acquaintances, etc.

What I have made is one (opinionated) implementation. The concept could be done in other ways as well, such as in an interactive digital form.

Some nice features

• It really can display all types of family. If someone has been adopted, both their biological and adoptive family are shown. In-laws and step-relations may not be “blood” relatives, but they are still your family.
• For the number of people displayed, it’s fairly compact.
• Some other diagrams may show siblings (and multiple partners) in the order they were born (or married), however I think this is clearer in this diagram, and it’s the only one I know of that clearly indicates twins/triplets/etc.

Caveats

You can go to the website now and create your diagram, but there are some caveats:

• Geni.com is used as the data source. Why? Because I use Geni. If your family tree is not on Geni, you can export it from your current software and import it into Geni. As Geni is a collaborative single family tree (not unlike Wikipedia) this can be confusing to newcomers. I do not have a date for when this tool will be available for any other website or for GEDCOM files. Any errors in the family tree data will of course be reflected in the diagram.
• As per the terms of the Geni API, there is a limit on how fast data can be requested. The more people are using this tool at once, the longer it will take to create your diagram.
• This tool is new and may not always work or work perfectly. If you experience a problem, please let me know. I may need to ask you to share your personal family data in order to figure out what the problem is and fix it. I also have very little time available, so I cannot make any promises about how long a fix will take.
• The tool provides you with a file in SVG format. This is great because it means you can edit it as you want (tips are on the website), however even if you do not want to make changes, you will probably need to use software (such as Inkscape) to save it as a PDF for printing. I do not yet know when I will be able to provide PDF files directly.
• Depending on who you choose to include, the diagram can be very large and is generally not suited to printing at home, or at least, not on a single sheet.
• The diagram is English and Western-oriented at this point. People’s names should display okay in any language (matching what you see on Geni.com), however texts including relationship names and the legend are only in English. The drawings used for people who have no photos may not match everyone’s culture. Place names are complicated and may or may not be in English.

Other interesting notes

• This really is a new way to look at both your family generally, as well as what information is on Geni. I think most people will see their diagram and realize they want to add more photos and other details before remaking their diagram to print.
• Complex relationships
  • In my limited experience, in most families you don’t have to look that far to find people who you are related to in more than one way. The diagram shows people only once, in whichever way is the fewest number of steps away. Other relationships with other people that are also in the diagram will be shown too, except in the case of siblings whose parents are not in the diagram (as siblings are drawn connected via their parents, not directly to each other).
• Data extrapolation
  • Because knowing someone’s birth date is so important to various aspects of how the diagram is drawn, estimates are calculated (where possible) for people with no birth year specified. These estimates are not directly shown, but are used in colour-coding as well as choosing images for people with no photo. Geni has it’s own birth date estimates, however they are not provided in the API, and at any rate, I believe my estimates are better, and I have some ideas of how to improve it further. Birth date estimates are also used to mark some people as deceased, even when the data from Geni has them as living.
  • In general, the tool tries its best to get or make use of the data available to fill in the desired data. For instance, the location listed for each person is meant to be their current location, but if that isn’t available, another location associated with that person will be used.
• Relationship names
  • Because relationships in a family circles diagram are not always as clear as in ancestor and descendant diagrams, in addition to showing it visually (by lines that connect partners, and arrows connecting parents to their children), relationship names are also provided. This uses English conventions, which can be ambiguous (for instance sibling-in-law could be either a sibling’s partner or a partner’s sibling). Other cultures have very different names and I’d love to make use of those one day. Relationship names may not match those on Geni for a couple of reasons, as Geni does not note half-relationships, nor use names like “step” and “in-law.”

The future

I have lots of improvements “planned”, but as mentioned, not much time to work on this. So if there’s a fix or improvement you want, or you just want to share your appreciation of the tool, send me a note.

Some time in the last few years, all of my Google Contacts contacts became “starred,” rendering the feature useless and annoying me constantly. I finally fixed the problem today.

The interface has no obvious method to bulk un-star contacts, and I didn’t find anyone else talking about having this problem. I wasn’t going to manually click to un-star several thousand contacts, especially when it takes a few seconds for the page to refresh upon un-starring, so I wrote a really simple script that can be pasted into the javascript console. I just had to let it run for half the day, but the problem is solved.

setInterval(function(){document.querySelectorAll("*[aria-label='Remove star']")[0].click()}, 5000)

Running this will (eventually) un-star all the contacts on your screen, at least until Google changes their interface.

For background, see Genetic Genealogy for Jewish Ancestry and Overcoming Endogamy in DNA (YouTube) by GeneaVlogger.

If you take a DNA test to find relatives, and you’re from an endogamous population (such as being Ashkenazi), you need to look at your matches with the longest shared stretch of DNA. That’s easy enough to do on MyHeritage or FamilyTreeDNA, but not possible in Ancestry or 23andMe as they do not display or let you sort by longest segment/block.

The next best thing is to approximate the longest segment by dividing the total amount of shared DNA by the number of shared segments. This can be a bit of a pain to do repeatedly, so I’ve created a bookmarklet to do this. I tried to include the bookmarklet here directly but WordPress is filtering the Javascript so instead you’ll have to add any link at all to your link/bookmarks bar, then replace the URL with the code in this gist.

It doesn’t actually sort your matches. What it does do is calculate the ratio (total/segments) for each match displayed on the current page, hide matches below a certain threshold, and add the ratio number to each match. On Ancestry, which uses infinite scroll, you’ll want to scroll the page to reveal lots of matches before you click the bookmarklet. On 23andMe, you’ll want to paginate and rerun the bookmarklet on each page.

The threshold it uses is a minimum ratio of 12 and minimum total centimorgans of 47, but that can be changed by editing the beginning part of the javascript

The calculation uses the total amount of shared DNA in centimorgans, however since 23andMe displays the total as a percentage, it firsts multiplies the percentage by 75 to get a fairly accurate centimorgan number. Some websites claim that the multiplier should be 68, but I believe this number was calculated for FamilyTreeDNA, not 23andMe.