RFC: Browse Local Files Extension

I would like to write an Extension, that allows browsing the local file system.
Comments on how to proceed are welcome.
The goal is to represent the local directory structure in a clients browse window.
This seems helpful for files that are not or badly tagged or where the desired browse structure should differ from the genre/artist/album/track metapher.

This Extension will be limited to browsing and lookups only.
Searches and listings will not be supported. This will remain to be done through the existing BE.
When a file is found it will be tested whether its a playlist or a track.
When a track is found it will be tested whether any tags are present.
Failing that the filename will be used as title and the parent directory as album.
Multiple media directories will be supported.
Not sure whether it is a good idea to allow browsing outside the media directory.
Any thoughts?

Doesn’t mopidy-local-sqlite already provide this?

AFAIK mopidy-local-sqlite scans a media directory, reads the tags and stores the result in a sqlite DB (hence the name).
On browsing it presents an artist/album/track tree that is abstracted from the filesystem structure.

The goal of mopidy-filebrowser would be similar to https://github.com/mopidy/mopidy/issues/1004

This sounds like #1004 to me, which we would love to get added in mopidy itself so we can move the bigger more complicated local backends out to external extensions.

Feel free to “steal” https://github.com/mopidy/mopidy/compare/develop...adamcik:feature/files-browsing as I’m not going to have time to prioritize #1004 for quite some time, or if nothing else just read over it for and then start from scratch. I also left some TODOs in there covering some extra nice to haves like image handling, but just getting the more minimal set of features needed to make this usable is more important of course :slight_smile:

FYI local-sqlite also lets you browse the directory structure (under Folders) without doing a scan. But from the sound of it I don’t think it behaves exactly as you want regarding parent folder as the album (arguably that’s not universally desirable). In which case #1004 looks like a great place to start.

Good. If thats ok i would start doing an external extension first and when the extension behaves well put it in mopidy itself later.
Or is that not a good aproach?

Either way should be fine, I was just thinking to get it in directly as it should be a fairly simple ext. and it unblocks the “kick local out plans”. Oh and the parent folder as album bit I’m a bit unsure about, but that’s mostly just personal preference and that I would like the browser to be really simple.

My reasoning for the external extension first approach was, that it might be easier to evaluate for those interested. Also i am more accustomed to writing external extensions.
Just uploaded a first shot at https://github.com/rawdlite/mopidy-filebrowser
While this works for me there is still a lot more checks to add.

I added a name to display for different root paths in in config via an optional :Name after the path.
Not sure if that is the best approach.

Also a whitelist for file extensions to show could come handy.

That happens to also be the reason for wanting to move local out. Though we feel we should ship with more than just streaming support so here we are wanting to get the local browser in :smile:

https://github.com/mopidy/mopidy/issues/947 comes to mind for the name mappings, but we haven’t actually added this yet.

ACK. I will fork and prepare a PR then.

Until https://github.com/mopidy/mopidy/issues/947 becomes a reality, i think path:name will do as the :name part is optional.
To make the ext just work with minimal configuration adding the home directory by default might be a good idea. A no_home configuration option would then disable that for the more advanced user.

Sounds good to me, except for using the directory name as album default; what about single tracks that are stored under an “artist” directory, or some completely unstructured collections? I think a simple “file browsing” extension (even if it makes it into core, it’s still an extension) should assume as little as possible about file/directory structure.
Anyway, I’d love to see this in Mopidy core so I could kick the “Folders”/“file browsing” directory out of Mopidy-Local-SQLite :wink:

Ok, that is a strong votum against directory name as album default was not the best idea.
I abandoned the idea a while ago already.

Just uploaded a first shot at https://github.com/rawdlite/mopidy/tree/file_browsing.

Its mostly working for me, except the playlist handling.
I am still struggling with that.

In my head adding a playlist handler for browsing doesn’t make any sense. You probably just want to make sure that playlists get exposed in the browse result. (Don’t remember if we want to expand them in browse)

As for detecting the playlists you want to be looking at the rest of the scanner results and checking for the mime type. Personally I would say just postpone playlists and get the basics right as we can do this in multiple phases.

Is there any reason the whitelist is needed with the current state of the scanner, or could it default to * perhaps?

Changing the util logging part is probably hidding some place where logging is being called incorrectly, and the code will still break when using other log handlers.

It would probably make sense to move this over to either the bug or to just go ahead and open a PR so we can comment on the code and work on getting this merged quickly.

Lastly please look at https://docs.mopidy.com/en/latest/contributing/?highlight=contributing#pull-request-guidelines - especially point 7 about commit messages and let us know if you need some git pointers to change the messages for the existing commits :slight_smile:

I was hoping it would be as easy as returning

models.Ref.playlist(name=name, uri='m3u:%s' % child))

But this is displayed as a directory in ncmpcpp.
Not what we want i guess.
I commented the code.

I still have to look deeper into the scanner results. I was trying to identify a playlist by its fileending being .m3u.
I postponed playlists for now.

Without the whitelist a lot of unplayable files will be shown in the client.
Setting

whitelist =

will disable the whitelisting.
The Settings in ext.conf should enable the extension to run without any configuration.

Ah, this was just me exploring.

just opened a PR.
Although i guess some more tests are needed.

I hope i did not miss anything.

Just looked into the scan code.
I think i should use the have_audio flag of the scan result and get rid of the whitelist.

Also i pushed some changes that implement the path.check_file_path_is_inside_base_dir check.

I made the mistake of branching from master instead of develop as specified in the guidelines.
If you can give me some time, i will setup a feature/file_browsing branch based on develop.

just did that.
Also now using the scanner result to check if file is playable.
This comes with a slight performance penalty for large directories, but getting rid of whitelist might be worth it.

I tested against some files that beets has not touched yet and
there is still an issue with a file i got from my nephew (don’t ask).

[quote] Paco_Osuna_§Mindshake_Plus8_Barcelona§Matteo_Spedicati_live§Mindshake_Italien§Jorge_Savoretti§Esperanza_Igloo__Rosario_Argentina§_-Mindshake_Night-Harry_Klein_Munchen-TVStream-_12…mp3
[/quote]

that ncmpcpp can’t parse. Not sure what the issue is. Mopidy-Mobile does not have that issue.