Jonathan LaCour

This post originally appeared in my Techish newsletter on LinkedIn.

You’re the Coolest AI of All

From a young age, I’ve always loved the magical experience of going to the movies. Torn ticket stubs, big booming sound, giant screens, and popcorn fill my heart with joy. There is nothing in the world quite like sharing in the community experience of enjoying a blockbuster in a movie theater. I suppose it's no surprise that I’ve loved living in LA so much!

Since 2020, theaters have struggled quite a bit, and for good and obvious reasons. I’ve truly missed taking my family to the theater. I’m a bit of a digital packrat, with a healthy collection of Blu-Rays stored on my home NAS. About a year ago, I finished renovating my home theater, which was a big step toward recapturing the magic of the movies with my family. But, no matter how great the picture and sound are, I still felt something was missing.

If you’ve been following me, you’ll be unsurprised to hear that I decided to scratch my itch with technology. What better way to explore the emerging and promising power of GenAI while honing my skill with HTML, CSS, JavaScript, and WebSockets?

Picture This

Part of visiting a movie theater is walking by a wall of big, colorful movie posters, which are, in my view, an underappreciated art form. A great movie poster creates a sense of excitement, anticipation, and wonder. A few months ago I stumbled on a project called Movie Posters Perfected, which was a huge inspiration for this project. While I could have gone down the path of just showing movie posters from this curated collection using a Fire TV stick, I wanted to create something even more magical. So, armed with inspiration, I set to the task of creating what I call my “Magic Poster.”

The hardware ingredients for the Magic Poster are simple:

• Samsung 43” Smart Monitor
• Raspberry Pi 5 and a case with a PCIe M.2 board
• HDMI Aware LED Backlight from AliExpress

Assembly was straightforward, thanks to double sided tape and zip ties. The one piece of hardware I wouldn’t use again is the Samsung monitor, which is a little too “smart” for it's own good. Getting it to be happy in portrait orientation was a nightmare! Still, I soldiered on and was able to mount the TV on the wall and drive the display with my Raspberry Pi 5, with the LED backlight matching the content on screen. Now, it was time to build some software.

Magic Poster’s Software Stack

While Raspberry Pis have become surprisingly powerful, they are still constrained computers with limited performance. I knew that I wanted to do more than just display static posters. I wanted to enhance the magic factor with animations, context awareness, and generative AI. When it comes to rendering performance, modern web browsers have decades of optimization, hardware acceleration, and features. I decided that my best bet was to use Chromium in “kiosk” mode, which displays websites in full screen, free of window decorations, toolbars, and tabs.

For the backend, I chose my programming language of choice, Python, along with libraries like Flask and SocketIO. Magic Poster is a JavaScript-powered frontend that communicates with a Python backend using WebSocket. I deployed modern CSS animations and transitions to create delightful views that can scroll, flip, and fade in and out.

After a few weeks of insomnia-fueled development, I had Magic Poster serving up beautiful posters from a hand-picked set of over 1,200 high-resolution movie posters, with four different styles: single poster, a “marquee” view with dozens of small animated posters, a “scroll” view that shows medium sized posters with a smooth scrolling animation, and a “flip” view that experiments with animation that really make the LED backlight sing.

Context Awareness

One of the benefits of powering Magic Poster with a network attached Raspberry Pi is that it can integrate with other systems on my home network. In my theater, I have an Apple TV 4K, a Zidoo Z9X and a gaming PC running ChimeraOS. Thanks to open APIs, I am able to detect when I am watching a movie or show, or when a game is being played. Using these APIs, I was able to have a small overlay appear on Magic Poster that displays information about the current activity.

Magic Poster was really taking shape, with a variety of different modes, context awareness, and even a simple REST API for controlling what is on screen at any given time. Still, I felt that there was something missing: interactivity. With the rapid and accelerating evolution of AI, there was an opportunity to take Magic Poster to the next level.

A Magic Sprinkle of GenAI and ML

Interactivity requires two-way communication. Attaching a keyboard or physical buttons to Magic Poster would be an easy path to providing input, but buttons are decidedly un-magical. I wanted Magic Poster to have a personality – to feel alive. Technology has an opportunity to become increasingly personal and interactive thanks to AI, and I concluded that the most magical way to interact with Magic Poster would be through conversation. Could I simply speak to Magic Poster, and more importantly, have Magic Poster respond?

Voice assistants have been “a thing” for quite some time. My experience with the three major players (Apple’s Siri, Google, and Amazon’s Alexa) have been mixed at best. All three assistants require “wake words” to begin an interaction, which feels forced. In the years since these assistants were developed, technology has evolved and advanced significantly. If possible, my goal was to have conversations with Magic Poster feel more natural.

As a longtime member of the Python community and a Fellow in the Python Software Foundation, I follow a number of prominent community members. Simon Willison is a Python developer that has become deeply interested in AI and LLMs, producing some great tools including Datasette, and more recently, Python LLM, which makes using both local and hosted LLMs from Python fun and easy. Through Simon, I discovered Vosk, which is an open source ML powered speech recognition toolkit with impressive accuracy and a very small footprint. I was quickly able to leverage Vosk to have Magic Poster passively listening to me in the background, ready to respond or take action based upon my feedback, including changing views, requesting specific posters, and more.

Now that Magic Poster could listen and take action, I wanted to give it the power to respond in its own voice. One of my favorite sites on the web is The Movie Database (TMDb), which is a free platform chock full of features, including an API with excellent Python client libraries. TMDb contains a lot of user-generated content, including a large number of reviews, with useful insights from a broad range of people. I decided to give Magic Poster the power to take hundreds of user reviews to generate a short summary of audience sentiment. When requested, Magic Poster reached out to The Movie Database, quickly pulls down all of the reviews for a movie, and then generates a summary using Ollama and the Llama 3 LLM. Finally, I feed the generated content to a text-to-speech system, and Magic Poster finally has a voice.

What’s Next?

It's been a lot of fun working on Magic Poster, and I am really happy with where it stands today. Of course, I have so many ideas about where to go from here, with additional voice features, display styles, and more. What do you think about Magic Poster? Do you have any suggestions for how to make it even more magical? Talk soon!

This post originally appeared in my Techish newsletter on LinkedIn.

This year, the LaCour family embarked upon a major home renovation, updating the main bedroom and creating an amazing home theater. While I was happy about the improvements to the bedroom, I'll be honest, the home theater was definitely the most exciting to me.

Fast forward to the end of our renovation, and we have an awesome place for the family to snuggle up and watch movies, or to catch a F1 race (Forza Ferrari!) with thundering sound on a big screen. Yet, the very first time my wife sat down to use the theater on her own, I got an annoyed text message: "Ugh, Jonathan, WHY do we have SO MANY remote controls?!"

Happy Spouse, Happy House

In my experience, there is no greater spark for innovation than an annoyed spouse. To her credit, my wife was absolutely right – we had way too many remote controls. Six, to be precise. Yes, I know, I have a problem.

My first instinct was to find a "universal remote," but the market is filled with ugly, difficult to use, under-featured options that wouldn't really make life any easier. So, I asked my wife: what is the best remote control you've ever used? She didn't hesitate: "our old TiVo remote control."

The Legendary TiVo Peanut Remote

When TiVo was first introduced nearly 25 years ago, it was a revolutionary concept. I remember the delightful experience of creating a "season pass" to my favorite show, LOST, and knowing that every week, I'd be able to sit down and enjoy an episode, zipping through the commercials.

While TiVo has faded into history with the rise of streaming services and pervasive "DVR" capabilities, its remote control, affectionately called "the peanut," is still a thing of legend. With its distinctive shape and smart button layout, it won awards for its ergonomic design.

Now, I was presented with a challenge. How could I replace six modern remotes and replicate my wife's experience with a non-programmable remote that has been out of production for ten years?

Thinking Like an Engineer

My first order of business was acquiring an old TiVo Peanut. Over the years, TiVo produced many different versions, but their "last hurrah" was the TiVo Slide Pro, released back in 2013. The Slide Pro slides open to reveal a full qwerty keyboard for text entry. More importantly, though, the TiVo Slide communicates to TiVo devices via RF (radio frequency), not IR (infrared).

RF is significantly more reliable than IR as it doesn't require a "line of sight" between the transmitter and the receiver. Late-generation TiVos featured built-in RF support, but older versions only supported IR. As a result, TiVo offered a dongle that plugged into the back of old TiVos via USB so they could use the Slide Pro.

Let's Go Down to Dongletown

I was able to score an unopened TiVo Slide Pro remote with a USB dongle for around $40. Once in hand, I made a discovery – in 2012, an enterprising developer added support for the TiVo RF dongle to the Linux kernel! So, I grabbed a spare Raspberry Pi 4, installed the latest version of Raspberry Pi OS, and plugged in the dongle.

After some tinkering, I was able to communicate with the dongle via the HID (Human Interface Device) API, capturing button presses, releases, and even the action of sliding the TiVo Slide Pro open and closed.

Six Devices, One Remote

Next up, I needed to figure out how to communicate with all six devices in the theater:

• Apple TV
• Zidoo Z9X Android Box
• Pioneer Elite Receiver
• AWOL Vision Projector
• Ceiling Fan
• Lutron Caseta Lighting

For the Apple TV, there is an outstanding Apple TV Python library that offers complete control over the network. The Zidoo Z9X, Pioneer receiver, and Lutron lighting are all able to be controlled over the network using Home Assistant. To control the projector and ceiling fan, I purchased a Broadlink RM4 Pro device, which can transmit arbitrary RF/IR signals, and trained it with their existing remote controls.

Introducing PNut

With all of the tools I needed in hand, I set to work writing some code to put it all together into a usable solution, which I call PNut. I made it general purpose so that anyone can use it, but I also included my own PNut configuration for reference.

I've been using PNut now for a few weeks, and it's pretty wonderful to be able to pick up a single remote and use it to dim the lights, turn on my ceiling fan, power up the projector and receiver, and control my Apple TV to watch the latest episode of The Righteous Gemstones.

Oh, and I'm pleased to report that my wife isn't annoyed anymore. Well, at least not about the remote controls...