PDA

View Full Version : Documentaries scraper?


pko66
2008-08-19, 14:57
This should be another content type (along with movies, tv shows and music videos). There is a very nice compilation in wiki form at http://docuwiki.net/index.php?title=Special:Random (it is a spin off mvgroup, where there is a very rich source of information, although being in forum format is much more difficult to scrape, I think: http://forums.mvgroup.org/index.php?showtopic=10334

spiff
2008-08-19, 15:02
show me the scraper and i'll add the content type after the feature freeze

Gamester17
2008-08-19, 23:05
@pko66, checkout the XBMC Online Manual on how to write a scraper (the IMDb scraper is also a good example):
http://xbmc.org/wiki/?title=Category:Scraper
...and feel free to add anything to the manual that you think is missing (it is a wiki after all)

Also see: http://xbmc.org/forum/showthread.php?t=33710

pko66
2008-08-27, 02:44
Here goes another of my way-too-long messages, sorry, please be patient...

I am learning scraper creation; as practice, I am modifying the "culturalia.es" scraper to enhance it a little and I'm planning to incorporate thumbnail grabbing (and maybe other info) from IMDB; the functionality will be constrained by some limitations of the current implementation of scrapers: if I'm not mistaken, you cannot have the user to select a movie more than once, when you select a movie in culturalia, you use the translated to spanish title as usually you do not know the original title (sometimes they are VERY different, like "Sleepless in Seattle" that was called "Algo para recordar" here). So, the user selects the movie from culturalia and, among other data, you know now the original title which can in turn be used to search the IMDB, but in that second search you cannot have the user to choose the right movie since he already did with the culturalia search... as a workaround, I plan to simply select the first one with the same tittle and year, but it can be wrong one, of course. Is there a better way to do it?

As I learn, I'm writing a "manual", some kind of "scraper creation for dummies", right now is in spanish but I plan to translate it when it is finished. It could be a good addition to the wiki. If someone that knows spanish want to help me "foolproof" it, please tell me so

I suppose the search in IMDB and selection of the link can be implemented using "custom function" but there is very few documentation, and studying the imdb.xml scraper, there is some calls to a $INFO function that I do not know how they work (they seem related to the settings specific to the imdb scraper, that I suppose are stored in videodb)... where can I find information specific to that?

BTW, I think that the imdb scraper is NOT a good place to start learning scraper creation, filmaffinity.xml is way simpler and so much more appropriate to beginers like me...

spiff
2008-08-27, 15:02
correct on the limitation and how to best handle it currently (unless culturalia gives the imdbid).

$INFO[foo] is the value of the setting foo. you can use this to insert string values.
you can execute expressions conditionally using conditional="bar" where bar is a bool setting.