jevans.bio

Table of Contents

  1. The Problem

  2. The Hardware

  3. Making It Smart

    1. His Blue Period

    2. IR Struggles, Part 1

    3. Brought To You By Node-RED

      1. The State Machine

      2. Implementation

    4. SSH, wlroots, and PipeWire

    5. The Dashboard

    6. IR Struggles, Part 2

  4. The End

  5. Addendum: Cable Management Sucks

  6. References

The Problem

I share a small apartment with my partner and I have a love for locally-controlled smart-home stuff. One of the biggest hurdles when trying to make our apartment “smart”, is that I want to make sure that my partner, and anyone who visits can interact with everything intuitively. This means that light switches have to work like normal switches, that a remote control does what you think it does, and, above all, that you never need to open an app to do anything you would do in any other home.

At the same, I try to avoid all-in-one devices as much as possible because they afford far less customization and control, and because they become a single point of failure that can end up being expensive and create a lot of e-waste. The biggest downside to this approach is that I often take a big hit in the “convenience” department.

It’s undeniable that integrating many parts of a system makes the user experience a lot easier. This is part of why internet service providers give customers an all-in-one modem, firewall, router, switch, and wireless access point. Managing individual devices for all of those functions for an normal user would be overkill and a nightmare to maintain.

At the same time, modem interfaces, CPU power, Ethernet controllers, and WiFi versions all evolve at different rates, and with an all-in-one device, you have to replace the whole thing any time you want a new feature in any one of those parts, or risk having to dive into the software/firmware of your device to make it work nicely with other parts and/or having the old device reduce the functionality of your new part.

The Hardware

Applying this philosophy to my home theater, I have it broken down into many components:

  • TV with most “smart” features disabled and with no access to the internet
  • NVIDIA Shield as the main TV user interface with as many default apps as possible disabled and replaced with open-source alternatives that don’t phone home
  • MiniDSP Flex acting as digital-to-analog converter (DAC), digital signal processor (DSP), and pre-amplifier
  • basic stereo speaker amplifier
  • stereo speakers
  • powered subwoofer
  • separate headphone amplifier
  • phono pre-amplifier
  • record player
  • my pc is also hooked up for couch gaming

As you probably noticed, there are still some devices that are performing multiple functions, namely the MiniDSP Flex and the powered subwoofer. Nothing is without compromise, and these components are far more difficult or expensive to separate.

With this set of components, there are three remote controls, a keyboard, a mouse, and a game controller needed to control everything and get the system into the state you need at a given time. Not only that, but each of the three remotes has volume controls that may or may not change the volume of a given state, and having multiple volume controls makes it very difficult to know how loud some content will be at a glance. This problem also makes it difficult to know what state the system is in at all.

For me (only because I was the one who set it up), this is a fairly middling annoyance, but for my partner and anyone else that might ever want to use the system, it’s a pretty insurmountable barrier without a detailed instruction manual next to the remotes. Since I have a Home Assistant server, I figured this would be a perfect time to put it to use!

Making It Smart

To start with, I wanted to figure out what my constraints are:

  • On the TV, I just want to be able to turn it on/off and switch the inputs.
  • On the MiniDSP Flex, I want to be able to switch inputs, change volume, mute, switch outputs, and control connected Bluetooth devices (play/pause, fast-forward, and rewind).
  • For the NVIDIA Shield, I want to be able to turn it on/off, use navigation buttons, and have a microphone in the remote for voice search.
  • For my computer I want to have a controller to play games and a way to navigate the desktop when needed.
  • I'd also like to be able to turn on/off every device to conserve power when parts of the system aren't being used.

Adding power control is as simple as adding a couple TP-Link HS-300 smart power strips. These can be set up to work entirely locally and allow for both switching on/off and monitoring power usage in Home Assistant[1].

It turns out that the biggest restriction here is the voice search functionality on the NVIDIA Shield. This is only possible on the official remote for the device, so to keep all of this functionality, I will at least need to have that remote.

For the computer, I certainly need a keyboard, but a nice feature of my favorite controller (the PS5 DualSense controller) is that it has a trackpad in the middle that works as a standard trackpad when connected to my computer, so I don’t need a separate mouse. One device gone!

The TV has wake-on-LAN functionality and, when it is on, can be entirely controlled by Home Assistant. Another down!

MiniDSP actually publishes the NEC IR codes to use for universal remotes, which is very helpful[2]. NVIDIA also added a nice feature to the Shield where users can choose to use the volume buttons on the remote to control a separate IR device instead of controlling volume on the Shield itself. Nice!

His Blue Period

Unfortunately, this is where things start to fall apart. That NVIDIA IR feature only lets you select from a pre-supplied set of IR codes for common AV Receiver manufacturers, and of course MiniDSP is not in that list. I tried all of the other options to see if any of them just happen to work. No dice.

I did find some chatter in forums about being able to program the MiniDSP Flex to use other IR codes, but after lots of digging, it turns out that they removed that feature when they changed the configuration software that they provide from a plugin for another piece of software to a standalone configuration application.

They are also very restrictive about what software they allow customers to download. To access any software at all, you need to provide the serial number that came with your product, and then they only give you access the most recent software for your specific device, so there is literally no way for me to use the old software and firmware to regain the feature. It's a terrible model and leaves an awful taste in my mouth. I will think twice before purchasing anything from them in the future.

IR Struggles, Part 1 (all the big movie studios are doing it these days)

I wasn't going to let this roadblock deter me, though. My first attempt to get around it was to use an IR blaster connected to Home Assistant and a button-remapping app on the Shield to call a shortcut that runs a webhook on Home Assistant when the volume up/down buttons are pressed. At the same time, I also remapped the Netflix button on the remote to run a webhook on Home Assistant to toggle the TV input between the NVIDIA Shield and the PC.

It was a bit of a chore to get the IR blaster to produce IR signals that worked for the MiniDSP Flex. I had to learn a lot about the structure of NEC IR commands and the raw IR commands that my IR blaster could produce. I ended up writing a Python script to convert the NEC codes to raw IR. These actually kind of worked!

Eventually, though, this process became a problem. There was a significant delay when pressing the remote buttons and the IR blaster sending commands. After lots of testing, I was able to determine that the vast majority of the delay was between the remote button press and the webhook call to Home Assistant. I also discovered that some Android apps don't like it when you run a shortcut while watching a video. The process causes the video app to lose focus for a split second, which can result in the screen flashing white, and the video pausing for that time.

As you can probably imagine, both of these problems were very annoying and caused both myself and my partner great frustration. To combat this I decided to try using the IR volume functionality on the Shield remote with an IR receiver from which I could tell Home Assistant to send commands through the IR blaster.

I whipped up a quick-and-dirty IR receiver using a TSOP38438 IR Receiver Module, and an ESP32 loaded with ESPHome firmware, set it up to work with codes that the Shield remote could emit, and it completely got rid of any perceptible lag in using the volume buttons. Success?

Well, it turns out that I should have thought about this a bit deeper. When you want to change the volume, you rarely only want to hit the button on the remote once, so when I was sending the second command to my IR receiver, the IR blaster was sending the first command to the MiniDSP Flex. This of course created a ton of interference and made the volume buttons do nothing. You would think that having a graduate degree in physics and a lot of experience with signal processing would have helped me foresee this pitfall, but nope 🙃.

I moved the IR receiver to the other side of the room, and covered most of the IR blaster in electrical tape to try and make the signals more directional. This helped, but I still needed to limit how rapidly I pressed the volume buttons and I certainly could not hold them down, which meant that changing the volume was slow.

This was a temporarily acceptable solution, and I was getting a bit burnt out with this aspect of the system, so I moved my focus elsewhere.

Brought To You By Node-RED

Node-RED is awesome. It's a Home Assistant add-on with a Home Assistant Community Store (HACS) integration that lets you use visual programming to create automations in Home Assistant, and it really makes building complex automations a breeze. Go get it.

I would have certainly given up on this project if I had to write the automations I needed in YAML or ... **shudders** ... in the Home Assistant GUI. The problem is that there are just so many inputs and so many states that all of the equipment can be in that it becomes unwieldy quite quickly. Even in Node-RED, building out the automations for this system was a bit of a chore. Thankfully, I was able to contain most of the craziness inside of a single state machine node.

The State Machine

If you're not familiar, a (finite) state machine is a set of states and allowed transitions between those states to define a dynamical system. This is a concept that is often talked about in the context of game design.

Super Mario World state machine example An visual example of a finite state machine from the instruction manual that came with Super Mario World[3].

For my system, there are 5 input states (off, PC, NVIDIA Shield, record player, and Bluetooth), 3 output states (off, speakers, and headphones), and 3 PC states (off, at my desk, and at the TV). A fully connected state machine would have \(5 \times 3 \times 3 = 45\) states and \(44^{45} = 9 \times 10^{73}\) transitions. Yikes!

Thankfully, the whole point of a state machine is that it is not fully connected, and in my case, the number of states is smaller, too. There are only 2 states when the home theater is off (PC off or PC at my desk). There are only 2 states when my PC is at the TV (headphone output or speaker output). For the rest of the inputs, there are 4 states each (speakers or headphones and PC off or PC at my desk). This makes the total number of states \(2 + 2 + 4 \times 4 = 20\).

The number of transitions is also much lower because transitions can only happen when one of either the input, the output, or the PC state changes. This means the number of transitions is \(4 + 2 + 2 = 8\) per state for a total of \(20 * 8 = 160\) transitions. This still sucks, but it's 71 orders of magnitude better, so I'll take it!

ℹ️

This is not the exact number of transitions because I have multiple ways that you can interact with the system that each trigger state transitions differently (the Shield remote, Home Assistant dashboard, turning on the headphone amp, etc.), but it is pretty close.

Implementation

After spending roughly an hour writing in states and transitions into a single node, I could finally make the automation that uses that state machine.

I first created nodes and connections for each of the various triggers for possible transitions, and feed them all into the state machine. For the output side of the state machine, I formatted the states like <input>, <output>, <pc state> and fed the result into a function node. This allowed me to split the state into its three components and use them to update a selection node for each one.

var states = String(msg.payload).split(", ");
var msg1 = RED.util.cloneMessage(msg);
var msg2 = RED.util.cloneMessage(msg);
var msg3 = RED.util.cloneMessage(msg);
msg1.payload = states[0];
msg2.payload = states[1];
msg3.payload = states[2];
return [msg1, msg2, msg3];

The selection nodes create entities in Home Assistant that you can watch for changes on, so for each one I created a flow that checks to see if the state has changed, and if so, performs a different action based on what the new state is.

The beauty of the state machine node is that it just ignores any trigger that doesn't cause a valid transition from the current state. This makes troubleshooting just so damn easy. If something weird happens, I know what state I was in, what state I changed to, and probably what triggered it, so it's as simple as checking the specific trigger node, one line in the state machine node, and one sub-flow after the state machine. Without a state machine, this automation would have to have connections that feed into nodes earlier in the system which could lead to undefined behavior because of the time it takes for signals to propagate through the flow.

SSH, wlroots, and PipeWire

I wanted to also make my Linux PC play nice with the automations, and automatically turn on/off my desk peripherals to save power. Home Assistant has a nice feature that allows you to add SSH commands as entities that you can include in your automations or dashboards.

I made sure that wake-on-LAN was enabled on my PC so that Home Assistant could turn it on. To turn it off, I added an SSH command to run systemctl suspend. Easy enough.

Switching the PC between my desk and the TV was a bit more complicated. I wanted to switch both audio and video outputs for each state.

For video, I run the Sway window manager which runs on top of the wlroots compositor, so I can use wl-randr to set the display configuration to be exactly what I need via the terminal (if you are running an X11-based window manager or desktop environment, xrandr has the same functionality).

For audio, I use PipeWire as my audio framework and WirePlumber as the session manager (this setup is pretty common these days). WirePlumber does have a tool to change audio outputs: wpctl set-default [ID]. The tricky thing is that the ID needed is not a static number that permanently matches a device; it changes all the time, and there is no other identifier that you can use to do this! Thankfully, there is a way to get the entire current PipeWire configuration in JSON: pw-dump. From here, it's the magical jq JSON query tool to the rescue.

After digging through the JSON output from pw-dump, I found that the ID is in .info.props."object-id", the device name is in .info.props."node.name", and the device type is in .info.props."media.class". I want the device to be an audio output aka sink, I want to then grab a specific device and get its ID. I can then pipe that ID into wpctl set-default and end up with this monstrosity for my TV:

wpctl set-default "$(pw-dump -N | jq '.[] | select(.info.props."media.class" == "Audio/Sink") | select(.info.props."node.name" == "alsa_output.pci-0000_0a_00.1.hdmi-stereo") | .info.props."object.id"')"

Now I can swap my PC outputs and sleep/wake my PC directly from my automations in Home Assistant. Woo hoo!

IR Struggles, Part 2 – The Re-struggling

At this point, there was nothing left to do but be frustrated with the IR setup, and I had enough. I decided that, if I'm going to use an IR blaster for the MiniDSP Flex, and not for anything else, why not attach it directly to the device and isolate that IR pair from the outside world. This would allow me to put my ESP32 IR receiver right next to the device and it would just work.

Hey, if I'm doing that, do I even need a separate IR blaster? Nope! ESPHome can do that, too. This way, I can also just have it act as an IR relay for volume commands from the Shield remote, and have them immediately trigger sending the commands to the MiniDSP Flex without ever having to go through Home Assistant. Lag free, baby! After a bunch of testing, I ended up with a short YAML file that just handles everything! I cannot stress this enough: !!!

I designed and 3D-printed a face plate for the Flex that I could mount an IR LED into. I coated the inside with electrical tape which blocks enough outside IR, while leaving a small hole for the LED. I soldered up a perf board with the IR receiver, a current-limiting resistor, and headers for the ESP32 and for the wires to the IR LED. And I made a simple enclosure for the ESP32.

The End

It fucking works.

It's done.

I now have one remote (and some buttons to swap to record player or Bluetooth mode) that handles everything. To use it, you point the remote where you would expect. To the user, the NVIDIA Shield might as well be built-in to the TV, and the speaker amplifier, headphone amplifier, and MiniDSP Flex might as well be an integrated receiver.

Or Is It? Addendum: Cable Management Sucks.

After finally getting everything working, the rats nest of cables behind my media console started to give me nightmares. I spent a long time trying to decide how to tackle it. I looked at all of the pre-made cable management systems, and honestly, they all are too expensive and too bad at what they do.

The solution I ended up with was cheap, easy to get from a local hardware store, and is flexible enough to work with a bunch of setups.

I got a 2'x4' sheet of pegboard and screwed it to the back of my media console with some spacers so I could get behind it. I picked up a 50' roll of ½” double-sided hook-and-loop and bundled all cables into a wiring harness based on where they run with hook-and-loop every 12” or so, keeping power and signal cables separate. I got the biggest bag of various zip ties I could and zip-tied my power strips, network switch, and whatever cables made sense to the pegboard, noting to keep power and signal cables far apart or have them cross at \(90^\circ\) angles.

It took me roughly 5 hours to deal with it all, but it's done. Sure, it's slightly more inconvenient than some popular expensive systems, but the whole thing cost less than $50, and I have tons of extra hook-and-loop and zip ties.

References

  1. https://python-kasa.readthedocs.io/en/stable/

  2. https://support.minidsp.com/support/solutions/articles/47001137495-ir-commands-for-minidsp-products

  3. https://www.mariowiki.com/Super_Mario_World#Power-ups

Find me on mastodon: @jevans@climatejustice.social

Follow this blog on mastodon! Search for https://jevans.bio/ in your mastodon instance.

RSS

Hello world.

This is where I will be posting my thoughts and pictures from now on.

Find me on mastodon: @jevans@climatejustice.social

Follow this blog on mastodon! Search for https://jevans.bio/ in your mastodon instance.

RSS

Enter your email to subscribe to updates.