How I Photograph My Kid's Handwritten School Notebook, Have AI Read It, and Show It on a Display at Home

Good morning. It’s まさきん.

The contact notebook (renrakucho) my kid brings home from school is handwritten. The handwriting is also pretty unique, so sometimes it takes a while to figure out what it says.

I built a system that lets AI help with the “decipher it and pass it on to the family” part. In this post, I’ll walk through the actual steps, from preparing the image to refreshing the display.

It started because we kept checking it at the wrong time

You can’t even know the notebook exists unless your kid takes it out of their randoseru backpack. By the time you notice it, it’s often already evening.

I wanted the contents to be somewhere anyone in the family could see at any time. So I built a system that shows the notebook’s contents on the display we keep in the living room.

How the system works

It comes down to these 4 steps:

1. Take a photo of the notebook page with a smartphone
2. Correct the image orientation (photos of vertical Japanese text are not always upright)
3. Ask AI to transcribe it
4. Write the result into a data file and refresh the display

From photo to display update, the only step that needs a human is “take the photo.” Everything else is designed to run automatically.

Getting the photo into a readable format

Photos taken on an iPhone can sometimes make the AI misread the orientation if you use them as-is. HEIC files in particular may open fine in a standard image viewer, but some AI image-reading tools don’t support them.

So the first thing I do is convert to JPEG and fix the orientation. On a Mac, you can do all of this with the built-in sips command.

# Convert HEIC to JPEG
sips -s format jpeg IMG_xxxx.HEIC --out notice.jpg

# If the vertical text looks sideways, rotate 90 degrees clockwise
sips -r 90 notice.jpg --out notice_rot.jpg

Check the converted image once, and if the orientation is still wrong, add another 90 degrees or flip it 180. After a few tries you’ll know which conversion works, and then you just script the same conversion.

On top of that, the text is too small to read reliably in the full image, so I use PIL (Pillow) to crop the area where the text is concentrated before having it read.

from PIL import Image

im = Image.open("notice_rot.jpg")
# Crop the central text area (adjust to match the size of the actual sheet)
im.crop((1300, 500, 3100, 2800)).save("zoom_main.jpg", quality=95)

Passing only the zoomed-in text area, rather than the whole image, clearly improves the AI’s reading accuracy.

Dealing with handwriting

This is the part I’m most careful about. Kids’ handwriting is quirkier than adults’. Even with AI, there’s no guarantee it will read everything 100% correctly.

So I always include this instruction in my prompt:

This is a child's handwritten school contact notebook. Even if some parts are hard to read,
do not fill them in by guessing. Transcribe only the parts you can read with confidence,
and mark anything illegible as "illegible," making it clear where it appears.

If the AI fills gaps by guessing, there’s a risk of passing wrong information to the family. Having it honestly say “I don’t know” makes the system more trustworthy in the end. Anything it couldn’t read gets checked and filled in by a person at the end.

When you think about it, this whole system starts with a smartphone photo. And the display update also finishes with communication over the internet. Both the entry point and the exit depend on connectivity. It makes you want to take another look at your own plan.

How the display data is structured

The transcribed content is saved in a simple JSON format. The program on the display side reads this file and generates HTML.

{
  "updated": "2026-09-01T07:30:00+09:00",
  "valid_for": "2026-09-01",
  "schedule": ["Tuesday, September 1", "1 Japanese", "2 Math", "3 P.E.", "4 Music"],
  "homework": ["Kanji practice"],
  "items": ["Art toolbox"],
  "notice": []
}

The first element of the array is the title, and the rest are listed as “period subject.” On days with no notes, schedule becomes an empty array, and homework and items are handled the same way. On days I forget to take a photo, the whole structure is reset to null, and the display shows the message “Please check the contact notebook.”

I also record how many periods of the day’s timetable are filled in. That number turns out to be handy.

What the number of filled periods tells you

If every period in the timetable is filled in, you know it was a normal day. But some days, it’s only filled in partway.

On those days, there’s a chance it was an early dismissal or a special schedule. When fewer periods are filled than expected, the system doesn’t just push the update through. It flags it as “possible early dismissal or half-day schedule” and adds a check step.

The design picks up hints not only from the text in the notebook, but from “how much information there is.” This has nothing to do with transcription accuracy, but it’s a small trick that’s been quietly useful.

How the display update is automated

Once the JSON is updated, the display is refreshed in these 2 steps:

1. Run a script that reads the JSON and regenerates the HTML
2. Send a reload command to the browser on the display

The reload command doesn’t just refresh the page. It goes in this order: “clear the cache → clear stored data → reload the URL.” If the browser cache is still there, the screen can stay outdated even after rerunning the script.

I check whether the update worked like this:

• Every reload step returned a normal response (HTTP status 200)
• The “generated at” meta information embedded in the HTML shows the expected time

If either of these fails, I have a fallback: instead of reloading over the network, restart the display app itself. The screen goes dark for a moment, so I only plan to use it once I’ve confirmed it’s needed.

Tweaks on the display side

The display only shows a summary of that day’s notebook. If you show the raw text as-is, there’s too much information and nobody looks at it.

• What to do today (items to bring, prep for the next day)
• Whether there were any illegible parts
• A note if there might be an early dismissal or similar

By narrowing down the information, a quick glance while walking by is enough.

What I learned from doing this

The best part is that we no longer have to “wait for someone to notice.” As long as the photo gets taken, anyone in the family can see the same information.

That said, I don’t leave everything to AI. When “illegible” shows up, a person checks it in the end. Skip that, and wrong information goes straight to the family, so that’s the one rule I never break.

Who this is for

• People whose family tends to share school notebooks and handouts at different times
• People interested in having AI read handwriting
• People willing to write a simple script, like one that reads JSON and generates HTML

Summary

I built a system that has AI read my kid’s handwritten contact notebook and shows it on a display at home. The key points, I think, are not letting the AI guess, narrowing down what’s displayed, and reliably clearing the cache before reloading.

Technically, none of this is difficult. But I think piling up small systems like this makes sharing information as a family a little easier every day.

Snap it on a phone, deliver it over the internet. That’s the root of this system. Getting your connectivity in order seems to quietly pay off, too.

This article contains affiliate advertising. If you sign up for a product or service through a link on this site, we may receive compensation from the partner company. The site operator is also an employee of Rakuten Group and may receive compensation through an employee referral program. The content and opinions here are based on the operator’s own experience and research regardless of any advertising relationship, but please read with the above in mind. See the Disclaimer & Affiliate Disclosure for details. Disclaimer & Affiliate Disclosure

This article includes machine-translated content. Please check the official Rakuten Mobile multilingual page for exact terms.

If you have concerns about Rakuten Mobile coverage or signal quality, you can consult through the official signal improvement request form .