Auto Scan for New Content in the background - Constantly keeping library up-to-date?

[BigBellyBilly]

i know in files view you can manually re-scan and it will ask you if you want to just add new files;

but, as a suggestion; what about a new music settings option that would enable 'auto scan of new music files' (in a background thread) upon xbmc startup ?

i add music almost daily to my pc share, so this would save the repetative process of manually scanning.

[mitchell2345]

hi,

i use xbmc to play my music all the time. it works really well except i wish there was one more small feature.

builds over the months have gotten closer but its not there yet. i remember when you would scan your music to the database (db) and that’s all you could do. now xbmc will do it on the side and allow you to continue using the software. i’d like to take this to the next level and have it scan whenever xbmc is running. for example when xbmc is powered up, it checks for any updates to the db. then, depending on your settings in the music options, it would either update automatically, in the background, or prompt you to update.

i would like it to update completely invisible. that way, when i’m ready to listen to my music my new material is already in the db.

is this possible? how hard would it be? anyone else have and thoughts on this?

thanks,
mitchell

[HarshReality]

one process at a time.. cant do more than that so scan or dont scan but it doesnt multitask ;)

[mitchell2345]

cant you just start the scan process at startup. right now you can scan and navigate the gui, listen to music, or watch videos. it seems the functionality is already there.

mitchell

[spiff]

yes we can start another thread.

however, with all the threading issues that are bound to pop up this aint a trivial task. feel free to commit a patch. don't expect us to add it anytime soon.

[mou]

what about letting the ftp server (like auto name shortening) add the uploaded music files to the database on the fly??

[Livin]

what about having a scheduled task to scan at a user defined time?

this way the user can set it to scan at 4am when they are asleep and the xbox is doing nothing else?

[BigNoseJr]

Is there any current way to automate new content scans for each library? If not, instead of having to run a manual scan everytime I add new content to a folder, offering a background timer based cron job to run scans and/or cleaning libraries at user determined intervals would be great.

In addition to that, offering an easier way (other than the "watched" check mark feature) to display the newly imported video content much like the "newest music" feature would be great.

Are either of these ideas current features being worked on?

[spiff]

all them tools are avail to do this using python or whatever so no; its not being worked on and i wont add anything more.

you have your scheduler (alarmclock) and you have your builtin (updatelibrary)

[Livin]

I thought a background scan was in the works?

[spiff]

read my post. understand my post. realize its already possible, and for that reason i will not add it into the core.

personally i find it a rather unecessary feature. i know when i add stuff. hitting update library then isnt exactly much work.

[nickolasj80]

If I've scanned my music into library mode, will it update automatically if I add music? Or is the some quick way to update the library?

[HarshReality]

if you add to it just select scan for new content

[ShortySco]

Not automatically though.

Though, you can achieve this with a very simple python script, i have one (which someone posted) that auto updates my video library every time you switch on and every 10 hours thereafter.

I'm sure it'd be really easy to add music to it.
Let me know if you want it.

Shorty

[spiff]

just do updatelibrary(music) instead of updatelibrary(video). so yes very simple

[sickboy719]

Not automatically though.

Though, you can achieve this with a very simple python script, i have one (which someone posted) that auto updates my video library every time you switch on and every 10 hours thereafter.

I'm sure it'd be really easy to add music to it.
Let me know if you want it.

Shorty
I'd love that, if you wouldn't mind.

[gzusrawx]

Just for reference that post that includes the code for library auto update is here:
http://xbmc.org/forum/showthread.php?p=144695#post144695 (http://xbmc.org/forum/showthread.php?p=144695)

[ShortySco]

Yep, that's the very one i was thinking of. :)

Shorty

[Bad_Ad84]

Hi,

My setup is that I have seperate machine setup that automatically puts content into a network share, which is split up into sub folders based on show name and uses library mode as it gets the episode name/plot/etc and shows everything nicely.

However when ever new content is added I have to manually update the library everytime and it does a full scan which takes awhile..

Is there anyway when i browse a folder you can md5sum the combination of all the files names and compare this to the locally stored md5sum, if different just update that folder only (to increase speed and as its not often that all folders get new content on the same day)

[spiff]

this is EXACTLY what update library does. apart from using crc32 not md5

[Bad_Ad84]

well in that case, it doesnt seem to be working on my setup at all.....

it NEVER auto updates, any suggestions or things you want me to post if ive localed a bug?

[Bad_Ad84]

If im understanding right, when you press "update library" it scans all the folders and crc32's all the file names etc and updates where appropriate, which is similar to my request.

But what im asking is it checks the crc32 as you enter the folder of just the current folder and automatically update that folder if this doesnt match.

please correct me if im mistaken and this feature is already in, but just not working for me.

[spiff]

no, and i wont add any such nonsense

[Bad_Ad84]

Why is this nonsense? I really dont understand your stance....

With this the xbox acts like a PVR and doesnt require intervention everytime new content is added, this would just make the library more like non library mode and actually show all the content automatically.

When browsing the folder via samba you clearly get a list of filenames already, so could take less than a second to do a check vs the locally stored crc32? and save having to update manually everytime somethings added, especially in an automated setup like this?

[Bad_Ad84]

Infact, even windows apps like winamp etc do background scanning to update their librarys too.

[spiff]

winamp doesnt hit the net to do its lookups.

what shall we do when user gets impatient and browses to the next folder. cause things will take time - we're depending on the speed of the network and the servers involved.
what if you some files around you dont want added to your library? what if i just want to quickly launch a vid without it doing nasty things like looking up info for something i intend to wipe shortly afterwards?

i would absolutely LOATH code arsing up my library automatically. i wont add it. if others want to submit a patch that makes this an optional behaviour, i'm not stopping you

[Bad_Ad84]

well i was talking about only updating the new files, hence not screwing up anything existing.

compare crc32 of contents of folder, if different check name of files to see whats new/changed, then update that/those files in the background and as its only the current directory i dont see a great deal of strain or delay happening.

im not talking about updating the whole library everytime nor even every file in the current folder, only new items.

But you seem decided, so ill leave it at that.

[radio]

I wouldn't get bothered about those replies, any sort of usability requests get the same obnoxious replies from spiff if they don't fit in with his exact view of how XBMC should work. I reckon he'd do quite well on the GIMP's interface team though...

For something you might find useful though, there was an autoexec script that would run "Update Library" automatically as soon as XBMC boots. It does do the whole library, but might still be a bit slow for you?
I can't remember what thread the script was in, but it shouldn't be hard to find.

[Bad_Ad84]

Thanks for the helpful reply, unfortunately most of the xboxs in my setup are on permamently (i have 4 on the network in this setup) so this wouldnt resolve my issue.

If there was a way to do the whole library say every 30 mins/hour/what ever that would help as a "work around".

but thanks again.

[gzusrawx]

There's actually a script that someone wrote that you could set to launch at xbmc bootup and whatever interval you want after that that would update any folder you have set as a library folder. I haven't implimented it myself yet but I do have a link to it:

http://xbmc.org/forum/showpost.php?p=144597&postcount=17

ps. you have to create a seperate script for each library you want to update ex. Movies, Tv Shows, and Music.

btw what software are you using to auto sort your movie/tv show files?

[Bad_Ad84]

thanks for the reply, i will try that out. its abit more intensive than it needs to be, but looks like it will work.

[Bad_Ad84]

ok that works perfectly, but its a poor work around, shame the devs dont seem to think this is worth while.

guess ill just have to put up with the updating status bar and it checking every hour regardless of if it needs to or not.

but thanks alot

[Bad_Ad84]

This thread has been moved, yet clearly the dev doesnt have an automated setup, so you dont know when new content is added hence having to update everytime you come to the xbox.

[spiff]

okay,

radio, you wanted to provoke me, guess what. you've succeeded.
you're obviously not capable of using that puny excuse for a brain of yours for anything but being stupid. just because i dont agree with other ppl's opinions all the time doesnt mean i dont give usability consideration. i'm not even gonna start listing ideas i have
1) agreed with
2) IMPLEMENTED. you know. the thing that actually matters.

fyi. the video library has exactly 2 features which isnt my work. and even with it being 99% my brainchild, i have always stated PATCH WELCOME. i have NEVER rejected a patch (none of the whopping 3 i have received) so dont you come here saying i'm not open for other ppls opinions. but you cannot (well, you obviously can). let me rephrase that. any SANE human being would understand that you CANNOT expect me to implement something i dont agree with.

anyways, truth of it is that i've had it with this "community" consisting of 20 skilled ppl along with thousands of ungrateful arseholes like radio. i'm done.

[J_K_M_A_N]

spiff, don't let idiots like radio drive you away. I am sure you are well aware of the fact that there are thousands of people who LOVE your work. I ONLY use the library. It is by far the best part of XBMC for me (I don't play games and I hardly ever listen to music). I realize that not enough people come here and tell you guys what a great job you do but they all are VERY happy you do it!

I have to admit, at first I thought you were a crabby guy but after spending some time here reading some of the crap people put, I very QUICKLY understood. Actually, I love to read your comments. Especially if they ask a question that has been answered in the wiki. (I have done that myself :blush:)

All I am saying is, please don't let the dumb asses like radio get to you. If you want, we will start a who loves the library thread and I am sure there will be more than enough support. :grin:

J_K_M_A_N

[gzusrawx]

Seriously, all the attitude towards the devs is ridiculous. Did you pay for XBMC? no. Did you pay any of the devs anything, for them to hear your ideas out over their own? no. Like Spiff said and always says "Patch Welcome". What that basically means (in case you misunderstand) is that he doesn't think the feature is necessary enough for him to devote his personal time to implement. But if you can find someone who has the knowledge and the resources to do so, he's willing to accept the patch on his own work. Please for the sake of everyone who uses XBMC give them a break!!
thanks,
A happy user of something that's never given enough credit.

[Gamester17]

we will start a who loves the library thread and I am sure there will be more than enough support. :grin:http://xbmc.org/forum/showthread.php?t=29407

:rolleyes:

[Bad_Ad84]

Indeed, im very grateful for the library mode...

if you saw the way my setup works, you would see why the feature would be required (but its hard to explain without actually seeing it, but soon as you used it you would be like "that needs automating!")

Anyway, with some help from forum members im using a work around (script that updates in a set interval and later i will attempt to remove/move off screen the library update progress bar)

[ultrabrutal]

I agree. The library is by far the best feature. I also love streaming dvds with menus and all. Of course things can always get better, but we have to accept if a dev does not feel like it and don't hasstle them. If spiff do leave it will be a huge loss and I'll hold radio while others.... hmmm full metal jacket style?

[Kronenbourg]

I can certainly see the use of background scanning, during the autumn season I am currently downloading around 40 different TV shows so am constantly moving data over to my NAS, but pressing the 'update library' once a day really isnt a biggy.

I might sort out the script to run it on boot as I dont leave the box running all day.

[C-Quel]

Well looks like i need to Break The Ice as they say...

Well done 'radio' you have now officially been tagged with the name "PRICK!"

When you appear from under that rock you crawl from mind not to bump your head... the last thing we need is to lose a valuable/contributing user like yourself!

[djdafreund]

Oh Spiff, don't leave us please. The majority of us respect your views and your work, and love what xbmc is. Just ignore the "PRICKS!" out there. I personally love the library mode. It wasn't until i read from you guys "1st of all, if your not using library mode, WHY NOT!??!???" and sat down and took some time to set it up, and REALLY do see the big advantages to using it now. Which before, some things didn't match so i really didn't use it much except for movies mode. Now i use it for everything (well, about to start using it with music too.)

Hope you don't really plan on leaving though. We appreciate all your hard work into xbmc. And i know how people can be when they criticize work because a lot of the time they are either doing something wrong themselves, or they don't fully understand things. It's easier with xbox's, it always either setting up, or configs (IE-User error) since the hardware is the same, but just configuring some things on the user end isn't done right.

[havix]

As a former Meedio user I'm used to schedules for updating my libraries on my computer. I realize that XBMC can update on startup but it seems to me that most people running XBMC probably have out of date libraries on their system. Is it possible to have it check every hour or so for new media? I find myself using the file browser more often due to this issue.

[althekiller]

ATM there's no mechanism built into xbmc to schedule events. However, I believe there is a script to handle this. Try searching the script/plugin forums.