Ensuring long term reliabiliy

General
1

Hi there.

I’ve been running mopidy and a suite of plugins for a few month now 24/7 on a RPi 4. It’s a normal raspbian image with mopidy running as a service.

Most important plugins are iris, spotify and my own for switching the external amplifier on/off.

The purpose of my contraption is to provide a unified, always available controller for music in a shared environment.

My issues are with the always available part. I have yet to find a suitable, automated solution that can replace me restarting the service (system is headless, so requires ssh-ing into the Pi) or power-cycling the Pi (also requires ssh-ing since I have not added any buttons to the Pi, and cutting the power hard might corrupt the micro-SD card).

It’s always just minor issues with the spotify plugin or short network outages somehow leaving mopidy or a plugin in a broken state even after network is restored).

My ideas so far:

  • Let systemd handle it
    -> The service keeps running, so a simple “restart after crash” not suitable. No idea if there are other measures I could try.

  • Script monitoring the log
    -> I get a wide spectrum of errors, seems hard to impossible to create something that catches every “right” error to restart the service and is also not too sensitive to harmless errors.

  • Using a custom plugin
    -> Nearly the same problem as log watching, but might be a little easier to implement “from the inside”.

  • Scheduled restarts
    -> Might leave service broken for a period of time or accidentally restart when in use.

Are there other avenues I haven’t thought of? Solutions I haven’t found yet? If not, what do you think might be the best approach?

2

Well I’d either go with

  • Script monitoring the log -> Could work well if you can find the most usual errors that affect you mopidy
  • Scheduled restarts -> Restarting at 4am would limit restart when in use

For a custom plugin (or another program), what you could try to do is the following, periodically:

  • if there is no playback going on
    • mute the volume
    • try to play something, and checks it is playing
    • restore previous state if everything is ok, else restart mopidy

But I’m really not sure it is worth it.