Library scan oddities

I have a strange problem. Using PiMusicBox 0.53. And while the library does find music on my NAS it displays them in an odd manner. For example. Some albums contain no tracks when you look at them from browse/albums. Yet these same albums do contain tracks if I view them through browse/files or browse/artists. Rescanning does not help. Unlikely to be problem in the file names or such, given that they are in the library. Only they are not displayed correctly? Any ideas how I could perhaps manually edit the entries, or other solutions?

EDIT: the problem appears to be that browse/artist/album displays some albums as empty, while those same albums are visible through browse/album.

You can try updating the Mopidy-Local-SQLite extension. This helped me with those kinds of problems.
Just execute the command that is mentioned in this post:

Thanks for the advice. I managed to update the extension, but for me it did not resolve the issues.

Ok. Listen to this. So, I updated the Mopidy-Local-SQlite extension. Then I removed the problematic albums from the library, did a rescan and then re-introduced the same albums (rescanned again). I was able to reproduce the same problems. And what’s more, using MPDroid from a phone displays the libraries completely correctly, it even retrieves album art for all albums. Yet, at the same time the browser GUI displays the information incorrectly. Most of these albums are standard angloamerican pop stuff, and the track names or artist names do not contain any strange non-English characters. doing “locale” outputs:

LANG=en_GB.utf8    LANGUAGE=
LC_CTYPE="en_GB.utf8"
LC_NUMERIC="en_GB.utf8"
LC_TIME="en_GB.utf8"
LC_COLLATE="en_GB.utf8"
LC_MONETARY="en_GB.utf8"
LC_MESSAGES="en_GB.utf8"
LC_PAPER="en_GB.utf8"
LC_NAME="en_GB.utf8"
LC_ADDRESS="en_GB.utf8"
LC_TELEPHONE="en_GB.utf8"
LC_MEASUREMENT="en_GB.utf8"
LC_IDENTIFICATION="en_GB.utf8"
LC_ALL=

So, could this be a problem of the GUI somehow?

The problems are:

  1. Some albums show up correctly under “albums” but the same albums looked at through “artists/albums” are empty.
    2)Some album art does not display

Album art is under control of the client. MPDroid uses a bunch of external web services to lookup the album art itself - nothing to do with Mopidy. Depending on the particular web client, some lookup via similar services, some try to use what Mopidy returns, some don’t bother at all.

Do I then understand correctly that this would explain 2) but not 1)? Because database giving the metadata (track names and so on… but not album art) is local?

Yes, it explains 2.

For 1) I would only say that the the regular album and artist listings you get through MPD work differently to the ‘browsing’ mode you’re using in musicbox-webclient. But I don’t know much about local-sqlite.

Okay thanks for the replies. This problem is not serious and I can live with it, just thought I’d mention. It’s not serious as it is always possible to locate the albums from some other path, just a minor inconvenience. BTW. I also checked the logs and there was nothing there.

After searching and analyzing file names and tags, I have nothing else except that the artists for which this seems to happen have multiple entries. Whenever there is only one entry there is no problem. I think what happens is that the sqlite extension skips the rest of the output when some error occurs. That is, there are always x number of entries that are ok, and then one which is not ok, after which nothing is. This, probably though, is something that the extension maintainer might be more interested in (than people reading this).

This is probably something for @tkem to comment on.

There are some serious issues with using MPD’s list commands with local libraries, which may result in only a subset of all artists/albums etc. being shown in MPD clients with the sqlite library. These are actively being worked on for Mopidy v0.20. In the mean time, you might

  • use the MusicBox Web client for browsing albums/artists/genres instead of an MPD client
  • try upgrading Mopidy-Local-SQLite to its latest version (v0.9.2), which contains a workaround for this
  • replace the sqlite library with the standard json library, as in Network folders are showing up empty in MusicBox 0.5.1rc)

If the problem persists, please share (via Dropbox, Google Drive, or whatever) your mopidy.log, preferable from doing a full rescan, and the Mopidy-Local-SQLite /var/lib/mopidy/local/sqlite/library.db metadata library, and give concrete examples (i.e. using album/artist names) of

  • what you actually do (i.e. MPDroid/Web client Browse -> Albums -> Artist Name)
  • what you expect to be shown
  • what the results are

I opened the db and noticed that many artists were not assigned a key. I then looked at the file tags and noticed that some had artist marked at “artist” and others at “albumartist”. So, I modified the tags of the entire library, so that they are now in “albumartist”, and now it works properly and it even resolved the issue with the scrobbler getting the album art.

All of the files have had their tags automatically created by different software along the years, and it seems that different SF use different tagging schemes. This causes the following things to happen in MusicBox Web client (PiMusicBox 0.53 sqlite extension v0.9.2): 1) if one artist “Beatles” is marked in some albums in "artist and in others as “albumartist”, your list of artists will display “Beatles” twice 2) if there is are several albums by the artist that are all marked with “artist” with nothing in “albumartist” you get a list of albums that are empty when you click them, the links are missing because of how the uris are assigned in the db.

Thanks a lot for doing all the analyzing! I thought I handled these artist vs. albumartist things well in recent versions of Mopidy-Local-SQLite, but I guess I have to re-check. Any chance you still have your initial .db file and are willing to share it?

Sorry, just tried to reproduce your #2, but it all seems to work fine for me. Here’s what I did:

  • cleared my local library
  • removed the id3v2 TPE2 tag from an album’s files (“Fever To Tell” by “Yeah Yeah Yeahs”), which should be the albumartist reported by gstreamer: id3v2 -r TPE2 *.mp3
  • imported the album; running mopidy -v local scan shows that only track.artists are present, but no track.album.artists:
    Adding Track(album=Album(images=[u'/images/1e96d7da3dae0964f27f59e17a4f635e.jpeg'], name=u'Fever To Tell'), artists=[Artist(name=u'Yeah Yeah Yeahs')], bitrate=273205L, comment=u'Amazon.com Song ID: 209222565', composers=[Artist(name=u'Nicholas Zinner')], date='2005-01-01', disc_no=1L, genre=u'Rock', last_modified=1425399458, length=240692L, name=u'Y Control (Album Version)', track_no=10L, uri='local:track:Yeah%20Yeah%20Yeahs/Fever%20To%20Tell/01-10-%20Y%20Control%20%28Album%20Version%29.mp3')
  • tracks are browsable via “Local media/Albums/Fever To Tell” and “Local media/Artists/Yeah Yeah Yeahs/Fever To Tell”, both using mpc and the Web client of my choice.

So, what’s the difference between my setup and yours?

Except that you’re using Musicbox, and I’m using Mopidy on a standard Debian system, I mean :wink:

So is it fair to say this is probably a bug in musicbox-webclient? I don’t have a local library setup here right now but I’ll try to reproduce it

Frankly, I don’t think it’s a Musicbox issue, just something I’m missing here. Can’t reproduce on MusicBox myself right now, since all my Pis are busy, but the new Pi2 is already on its way, so I’ll be able to set a spare one up for MusicBox testing, soon.

I would guess that it is not a webclient issue, since using other webclients (PiMusicBox comes with moped also, for example) doesn’t make a difference. It’s more likely the tags and how the database is created from them, but exactly how, I don’t know. I thought that it was the artist vs albumartist thing, but when I retagged the problematic files, I did so using MusicBrainzPicard, which unfortunately inserts many other tags (like artistsort, and all kinds of bar codes and id numbers). So, it may be that something other in the new tags is making the difference here. I can send some examples later.

tkem, notice that your experiment should not even reproduce the suspected 2) given that it only appears to occur when there are two or more albums listed by the same “artist” tag (that is, if my suspicions are right). This is one reason it was so hard to detect, because I had many artists that had only one album in the library, in which case they were fine, even though they had incomplete entries in the .db-file. But once you added a second album or more, the first album would still be normally listed, but all the rest of the albums would be empty.

Thanks for pointing out that it seems to occur only with two or more albums - that may indeed be the missing clue!
I noticed you stated so in your original post, but I completely missed that the first time, sorry!
Will check this out ASAP (which may be a few days).

Yes, the problem of listing the songs appears to only happen with multiple albums, however, the last.fm scrobbleproblem appears to happen even with only one album. when this happens there is something like “error. value not integer” in mopidy.log

I don’t really know much about the last.fm scrobbler, so the full error message (if you can reproduce it) would definitely be helpful.