View Full Version : Update library without accessing scrapers - Option to only use NFO and local artwork?
steve1977
2009-07-04, 06:06
Have a feature request seeing how well the media managers such as EMM are developing!!! Would it be possible to allow the library to update and ONLY use existing nfo-files, artwork and posters/banners? Phrased differently, would love to stop XBMC access the scrapers.
Thanks a ton in advance!!!
smcnally75
2009-07-04, 06:14
I believe that is how it works now. If there is an nfo for the movie it will use it instead of scraping.
althekiller
2009-07-04, 06:20
Right, if we find a FULL xml nfo. We use that data and forgo the scrape. If we find a URL or MIXED nfo, the scrape will happen.
steve1977
2009-07-04, 06:23
Right, if we find a FULL xml nfo. We use that data and forgo the scrape. If we find a URL or MIXED nfo, the scrape will happen.
Sorry for being unprecise in my email. My issues are the following:
1) Add two new movies, but dont find the time to run EMM to create the nfo files. Still update the library at startup and it scrapes the ones. Later I do the nfo files, but the scrape already happened and I need to manually change it (which is painful given hundreds of movies, from which I need to remember, which is the new one to update).
2) Have some avi-files in the folders, which intentionaly dont have an nfo files (eg, sample files). These sometimes get picked up by XBMC and scraped leading to a movie, which in reality is only a sample file.
By disabling the scraping completely, but keep the library update running, I could work around. Any thoughts?
jmarshall
2009-07-04, 07:11
Adding nfo files should automatically queue an update, so update library should pick it up and take care of it. If it does not, it's a bug.
As for the sample files, what _exactly_ are the filenames? Are you adding these later and they queue an update, or what?
Sidenote: I wonder if the sample files are taken care of in generating the hash - someone mind checking?
Cheers,
Jonathan
steve1977
2009-07-04, 07:33
Adding nfo files should automatically queue an update, so update library should pick it up and take care of it. If it does not, it's a bug.
Hard to describe. If the nfo is in place and I run "update", it picks it up. If it is not, the scraper kicks in. If I later add a nfo file, it doesnt automotically change the content (as XBMC had successfully scraped it). I can select the individual movie and ask "update content", then it changes to the nfo file. But this is rather tiresome.
On alternative feature request could be: "update all movies" based on changes to the nfo files. this would help with my issue and also help if i change an nfo (eg, for more imdb ratings).This would be perfect!!!
As for the sample files, what _exactly_ are the filenames? Are you adding these later and they queue an update, or what?
Not consistent. Some have weird names, but all are rather close to the movie name. Because they end with avi but dont have an nfo file, scraper picks them up and adds another movie (which usually is duplicate given the real movie is already scraped based on the nfo file).
Sidenote: I wonder if the sample files are taken care of in generating the hash - someone mind checking?
Sorry. Dont really understand.
jmarshall
2009-07-04, 07:56
If the nfo file is added later, then all you need to do is do an "Update Library" and it should pick the changes up automatically. If it doesn't, it's a bug. Please file a bug report along with everything required to reproduce, including a sample filesystem that easily allows reproduction.
As for the sample files: There's a regexp we use to ignore them. You may have to tweak this, or rename your files: XBMC cannot ignore them unless you tell it to, or name your files suitably.
Cheers,
Jonathan
If the nfo file is added later, then all you need to do is do an "Update Library" and it should pick the changes up automatically. If it doesn't, it's a bug. Please file a bug report along with everything required to reproduce, including a sample filesystem that easily allows reproduction.
As for the sample files: There's a regexp we use to ignore them. You may have to tweak this, or rename your files: XBMC cannot ignore them unless you tell it to, or name your files suitably.
Cheers,
Jonathan
My understanding was that for the new information to be added/replace the existing information, you had to manually reload the file. This is the way it is when you add trailers or edit the nfo file.
steve1977
2009-07-04, 08:34
My understanding was that for the new information to be added/replace the existing information, you had to manually reload the file. This is the way it is when you add trailers or edit the nfo file.
Thats exactly in line with my understanding/experience, which I wouldnt call a bug, but the way it is meant to be. If there is any chance to change this behavious?
jmarshall
2009-07-04, 10:14
That should NOT be required. If it is, then IMO it's a bug.
Obviously an update is required, but a general "Update Library" should pick changes up.
As I say, if it's not, then it's a bug. Get it reproducible on a simple dummy filesystem, and post a trac ticket with full details.
Cheers,
Jonathan
steve1977
2009-07-04, 13:33
That should NOT be required. If it is, then IMO it's a bug.
Obviously an update is required, but a general "Update Library" should pick changes up.
As I say, if it's not, then it's a bug. Get it reproducible on a simple dummy filesystem, and post a trac ticket with full details.
Cheers,
Jonathan
Great, will do.
jmarshall
2009-07-05, 04:53
On consulting with my colleagues, it appears this is by design. Apparently we don't rescrape movies when the hash changes, all we do is add new movies.
This kinda makes sense, if you consider a folder full of 100 movies - we don't want to rescrape in this case unless we know they've changed. Given that we don't know which one has changed, we don't do it.
This is a very silly thing to not be able to pick up, and it's certainly something I'll address with the video library redesign. We can't do much about it until then.
Cheers,
Jonathan
steve1977
2009-07-05, 05:14
On consulting with my colleagues, it appears this is by design. Apparently we don't rescrape movies when the hash changes, all we do is add new movies.
This kinda makes sense, if you consider a folder full of 100 movies - we don't want to rescrape in this case unless we know they've changed. Given that we don't know which one has changed, we don't do it.
This is a very silly thing to not be able to pick up, and it's certainly something I'll address with the video library redesign. We can't do much about it until then.
Cheers,
Jonathan
Thanks. Then maybe treat it as a feature request. It would make a huge difference to me. As I am updating the IMDB ratings from time to time, basically the only way way I have to reflect these changes is to delete the library. As I have hundred of movies, it is not an option to update each individually.
Also deleting the library is hugely painful as my TV episodes (probably thousands) need to rescrape from the internet, which takes more than a might and then I need to manually go through them to change to the fanart I want.
Instead of having the reloading current movies as part of the update process, could it be implemented as a separate, manually triggered process? Something like a "Reload All" on the parent folder context menu. This way, if you haven't changed anything with the current movies and just want to have new movies added, you don't have to wait for it to rescan all current movies as well.
jmarshall
2009-07-05, 07:39
That's a workaround, not a fix. I'm not opposed to it, but won't waste my time implementing it when it doesn't actually address the problem.
The problem is trivial to fix: Instead of storing the hash just for the directory, we store a hash per item. Thus, not only do we know when a directory may need rescanning, but we also know which items in that directory need rescanning. Ofcourse, there's no point implementing this fix at this point - I shall simply ensure that this is part of the new filescanners.
Cheers,
Jonathan
That's a workaround, not a fix. I'm not opposed to it, but won't waste my time implementing it when it doesn't actually address the problem.
The problem is trivial to fix: Instead of storing the hash just for the directory, we store a hash per item. Thus, not only do we know when a directory may need rescanning, but we also know which items in that directory need rescanning. Ofcourse, there's no point implementing this fix at this point - I shall simply ensure that this is part of the new filescanners.
Cheers,
Jonathan
Gotcha... didn't know you guys created hashes for the directories, which is brilliant. Going to steal that idea. lol
steve1977
2009-07-05, 15:42
The problem is trivial to fix: Instead of storing the hash just for the directory, we store a hash per item. Thus, not only do we know when a directory may need rescanning, but we also know which items in that directory need rescanning. Ofcourse, there's no point implementing this fix at this point - I shall simply ensure that this is part of the new filescanners.
Sounds great. Waiting for the new release then. This change would make a huge difference for me.
I deleted yesterday my movie source to get the lib updated. For 1000 movies its not that hard but for an TV Show lib with 8k episodes xbmc need ~ 5h to rescan al.
jmarshall
2009-07-09, 01:22
Why did you need to rescan your shows - wasn't just the movies broken?
Removing the movie source should have cleared just them out.
Cheers,
Jonathan
I have 430 movies in movie named folders. Have used EMM to organise the collection so that each movie folder contains a NFO, a Cover/Poster .jpg, a Fanart.jpg and extrathumbs folder with 2 thumbs in each. All looks pretty and nice and neat. When I add that source to XBMC, will it import that data and not do its internal scrape? I was under the impression that it would. If it is kind of broken with the current development changes, is there a workaround?
Why did you need to rescan your shows - wasn't just the movies broken?
Removing the movie source should have cleared just them out.
Cheers,
Jonathan
I didnt say that i did rescan the shows :)
I did it exact the way you mentioning.
jmarshall
2009-07-09, 11:57
Phew - was just thinking of thetvdb's bandwidth :)