Reading my grandfather's notebooks with a model that mostly fails

My grandfather kept notebooks from the mid-1970s until he stopped writing in 2019 — not a diary exactly, more a record of work, weather, prices, and occasional opinions. There are thirty-one of them in a box at my mother's house and almost nobody in the family can read his handwriting, including my mother, who can read about half. I started this after he died because I wanted to know what was in them, and because scanning them was something I could do in a week and reading them was not. I knew going in that handwriting recognition on one person's idiosyncratic script from photographs of ageing paper would not work well. It does not. It is still the difference between the notebooks being readable and not.

This is for my family, about fifteen people, and for nobody else. It produces a searchable transcript of each page next to the scan of that page, with per-line confidence shown honestly, so a reader can search for a name or a year and land on the actual image rather than trusting the text. The point is navigation rather than transcription: a half-correct transcript that gets you to the right page of the right notebook does the job, because a person then reads the photograph. Framing it that way is what stopped me from abandoning it when the accuracy turned out to be what it is.

All of it. I scanned all thirty-one notebooks, about four thousand pages, which took two months of weekends and is by far the largest part of the project. I built the page preprocessing — deskewing, removing the shadow in the gutter, and separating lines, which on paper this old is harder than it sounds. For recognition I fine-tuned an existing model on pages my mother and I transcribed by hand, around six hundred lines, which is a small training set and it shows. I also built the reader, which is a plain page with the scan on one side and the transcript on the other, and the search across all of it.

I decided to show confidence per line and to render low-confidence lines in a way that clearly says do not trust this, rather than presenting one clean block of text. A clean transcript of a bad recognition is a lie that ends up quoted in a family group chat as something my grandfather said. I also chose to let family members correct lines directly in the reader, with corrections stored separately from the model output, so the human text and the machine text never get confused — and about nine hundred lines have been corrected that way, mostly by my mother. I did not use anything hosted; the notebooks are private and that was not a hard call.

The genuine problem is that he wrote in two languages and switched mid-sentence, often with abbreviations he invented for his own work, and the model has no idea what to do with any of it. Accuracy on ordinary lines is around seventy per cent by character and drops under forty on the mixed ones, which is most of the interesting ones. I tried expanding the training set, which helped a little, and I tried a dictionary of his abbreviations that my mother and I built, which helped more. In the end I accepted that the tool is a finding aid and put the effort into search and into making the images pleasant to read, which was the correct place to spend the last three months.

Four thousand pages scanned and searchable, which means they exist somewhere other than a box in a house. Nine hundred lines corrected by hand, which will keep growing slowly. My mother found the entry for the week I was born, which she had been looking for on and off for years, in about a minute. Three cousins have read parts of them who never would have opened the box. The transcription is bad and the project succeeded, which is a distinction I did not understand before I built it.

My mother did the hand transcription that became the training data, built the abbreviation dictionary with me, and has corrected most of the nine hundred lines. My aunt identified people and places in the entries, which is a layer of the archive I could not have produced.