FilmAffinity (Spanish) scraper

[HectorziN]

Hello guys.

In last build, is included a new Spanish scraper by Jurrabi. This scraper uses www.culturalianet.com
I tried this, and I found this web page hasn’t enough information. I decided then making a new scraper using www.filmaffinity.com web page. This scrapper supports all of this:

- Title
- Plot
- Outline (when available for the movie)
- Year
- Director
- Country and original version title (used mpaa tagline for this)
- Credits
- Genre
- Nominations, Oscars, and other prices (used mpaa field for this)
- Rating
- Votes
- Runtime
- Actor name

I think that it could be useful adding it to next releases.
Where can I post the xml file and the gif file?


Best Regards.

HectorziN (From Spain)

[Nuka1195]

submit a patch on SF.

https://sourceforge.net/tracker/?group_id=87054&atid=581840

[HectorziN]

http://hectorzin.dynalias.com/FilmAffinity.zip
Regards

HectorziN

[HectorziN]

I don't know how submit a patch. I have posted here a link to download. Guys, try to test it and evaluate. I think that it is very good. I haven't found a movie that dont' work with this scraper.

The only problem I have found, and I don't know how to solve it, is that don't work when the search string contains ñ, á, é, í, ó and ú. you mast search using n, a, e, i, o and u.

If anybody knows how to fix, please, tell me.

I am waiting your comments.

[spiff]

sounds like encoding issues to me. make sure you properly specify utf vs not in your returned xml's.

[spiff]

seems to be working fine. i'll hold off a couple of days to see if you nail that issue, then i'll commit to svn.

cheers

spiff

[HectorziN]

the problem is that xbmc transforms ñ to ñ
In culturalianet.com, a search to ñ works, but filmaffinity.com needs to receive %F1.

The problem is that is xbmc who send ñ to the parer, then the parser cannot do anything with this.... This is what I think is happening.

Here is the log

INFO: Get URL: http://www.filmaffinity.com/es/search.php?stext=cuando+llama+un+extraño&stype=title

if you paste this url into internet explorer it won't find anything, but if you paste http://www.filmaffinity.com/es/search.php?stext=cuando+llama+un+extra%F1o&stype=title then it will work. How can this be fixed?

Thanks

[spiff]

we need some way for the scraper to say that it wants iso not utf chars. will have a crack at it later.

[HectorziN]

OK, thanks, I'll wait for the new scraper functionallity

[HectorziN]

Any progress???
is there a new definition of scraper languaje to explicit if dthe search using utf or iso?

Thanks a lot.

Also, anyone has test this scrapper? what do you think?

[HectorziN]

Hello spiff.

it will be abailable anything to select utf or iso when searching?
if not. I think that we could include the filmaffinity scrapper anyway. what do you thing?

Regards

[spiff]

oh my. this was an old one biting me:)

had forgotten all about this. i suggest we stick it in svn as is, then i'll look into a fix when i have the time

[HectorziN]

how can I stick it in svn?
thanks!

[spiff]

submit it as a patch on sf.

i will do the sticking:)

[HectorziN]

Sorry, I submited it, but assigned to nobody, I should mismatch this field...
I suppouse I should submitted it to you.

I have attach the scrapper itself

[spiff]

thats fine. will commit soonish (later tonite or if the beer gets me tomorrow)

[spiff]

done

[spiff]

the issues with the chars should also be fixed. your example movie name worked for me now atleast

[bugme]

Filmaffinity scraper dont work 1 week ago.
The scraper dont download thumbtails, genres, valoracion...
It seems something change at www.filmaffinity.com and know the scraper crashes.
Hectorzin or somebody could do something to patch the scraper?
Thanks a lot.
More info here (in spanish):
http://www.elotrolado.net/hilo_os-sigue-funcionando-el-scraper-de-filmaffinity_1042009
Bye byE
PS: With culturalia scraper are happenig similar something, it could be a change in the permissions to link the pages.

[HectorziN]

Filmaffinity scraper dont work 1 week ago.
The scraper dont download thumbtails, genres, valoracion...
It seems something change at www.filmaffinity.com and know the scraper crashes.
Hectorzin or somebody could do something to patch the scraper?
Thanks a lot.
More info here (in spanish):
http://www.elotrolado.net/hilo_os-sigue-funcionando-el-scraper-de-filmaffinity_1042009
Bye byE
PS: With culturalia scraper are happenig similar something, it could be a change in the permissions to link the pages.

I have updated it and it works again. I have submited it into sourceforged to be added in a next build of xbmc.
Please, say this to elotrolado.net, I am not registered in that forum.
If you are interested in this updated file, tell me were can i send you, also, it wold be a good idea posting it in elotrolado.net.

Thanks

[spiff]

new version is in svn (linuxport)

[bugme]

I have updated it and it works again. I have submited it into sourceforged to be added in a next build of xbmc.
Please, say this to elotrolado.net, I am not registered in that forum.
If you are interested in this updated file, tell me were can i send you, also, it wold be a good idea posting it in elotrolado.net.

Thanks
Thanks a lot friend, I will say this good new to elotrolado.net.
Im interested in the update file, can you upload to megaupload or rapidshare.
Thanks, thanks, thanks :grin:
Bye byE

[spiff]

as i said it's in svn - just grab it from viewvc

[bugme]

as i said it's in svn - just grab it from viewvc
Sory but i dont know viewc, explain me how to use it or ill wait to the new version of xbmc.

[spiff]

go to sf.net/projects/xbmc

go to code -> svn browse.

browse to the file in question. select download

for the lazy; http://xbmc.svn.sourceforge.net/viewvc/*checkout*/xbmc/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml

note that it won't work 100% on xbox as the searchstringencoding hasn't been merged yet

[HectorziN]

new version is in svn (linuxport)

Hi Spiff, I also included the new parameter for the encoding, but I think that this change is not implemented yet in the lastest t3ch build, don't you?

[HectorziN]

Hi Spiff, I also included the new parameter for the encoding, but I think that this change is not implemented yet in the lastest t3ch build, don't you?

Sorry, I didn't read your last message.

Then I report for everybody:

The xml fix the scraper and it work again like before.
I have implemented a bit change that allow searching using spanish characters, but this functionallity will work only with the next release of xbmc, not the current one.

I have in mind, trying to use the name of the actors to search in imdb for the picture, but I don't know if this is possible, because it needs to search again in a different xml... This is a "to do" for me.

[spiff]

it is certainly possible

[HectorziN]

it is certainly possible

I am trying to do this, and as an example I am looking at imdb.com scraper. The problem is that I cannot get cast thumbs using imdb scraper. It works in the last compilation released by t3ch (2008-06-30)?

Thanks

[HectorziN]

Hi!, I did it!
Filmaffinity scrapper looks for the cast photos in imdb.
It is slower, but it worth it!

Also include a setting menu to use the old methode of getting cast (faster but no photos)

Hi Spiff, I have tried to upload the file into sourceforge but it says I don't have any privileges to do this. It seems the security policies has changed.

How can I upload my file?

Another thing. With the last compilation I didn't get SearchStringEncoding to work :(

thanks!

[jelockwood]

The official/standard version of the FilmAffinity scraper produces Spanish language results. There's nothing wrong with that but the FilmAffinity website does offer an English language option and this maybe useful for some people.

As part of an effort to learn how to construct a Scraper, I have made a start at converting the Spanish version to access the English language version.

While the main difference is changing the country/language code in the URLs from es for Espagne to en for English the section that deals with actually scrapping the results also needs changing to look for different keywords. I have made a start on that as well but it still needs additional work.

As currently my knowledge of scrapers is limited I am hoping others here may assist in finishing it.

My initial version can be downloaded here http://homepage.mac.com/jelockwood/.Public/filmaffinityen.zip

[spiff]

if changing the language code is the only change needed, you should do it as a scraper setting. add the valid values to a spinner setting, you can see the tvdb scraper for how that is done (dont mind the extra complication of grabbing the valid line online). then you just replace the language code with $INFO[language] (if you called the setting language).

[jelockwood]

if changing the language code is the only change needed, you should do it as a scraper setting. add the valid values to a spinner setting, you can see the tvdb scraper for how that is done (dont mind the extra complication of grabbing the valid line online). then you just replace the language code with $INFO[language] (if you called the setting language).

While the main change is es vs en in URLs, the text labelling the fields also changes from Spanish to English meaning the regex code to match fields needs modifying as well. I have now managed to get all but two of the available working fields, there are some fields not listed on the English version one of which is the plot (so nothing can be done about that). The two fields not working, Writer and Duration, should work and I cannot see why.

Even the cast list, movie rating and movie thumbnail works.

I will upload my latest effort tonight to the same URL as per my previous message in this thread.

For your information, the English search result I have been testing against is at http://www.filmaffinity.com/en/film699914.html and the Spanish version is of course http://www.filmaffinity.com/es/film699914.html

[HectorziN]

Hi spiff, filmaffinity web, has a link to trailers of the movies, but this trailers are in youtube.

It is possibly to put as an url for the trailer a link to a youtube video?
I had try with same differents ways of youtube urls and xbmc don't show the videos.

Do you know how to put for example this trailer?

http://www.youtube.com/v/sQf8kP1dzPU

Thanks!

[spiff]

i do not know, i suggest you look at one of the youtube plugins/scripts

[HectorziN]

I did it.

The scraper now downloads actors thumbnails and also have a link for youtube trailers.

I have tried to upload the new xml scraper but Sourceforge has changed and I cannot find a way to do it. I don't know if I must to be a member in xbmc or something like that, but I cannot do it.

How can i submit this change?

thanks!

[spiff]

xbmc.org/trac is our new tracker

[HectorziN]

But can I submit a file without the use of any software?
I think it needs a software to upload files, right?

thanks

[spiff]

no software required. create a new ticket, tick 'i have files to attach to this ticket'

[HectorziN]

How can I create a ticket? I only see an option to view tickets, not create.

Should I be registered to do this? I cannot see a link to register, only one to login.

Thanks

[spiff]

you login using your forum credentials.

the there's a 'new ticket' in the upper right hand corner.

[HectorziN]

OK, I got it.

I submited the patch.
the only pendent issue the scraper has is that I don't know how to set the searchstringencoding

I included SearchStringEncoding="ISO 8859-1" but it don't work.
The page I am scraping is this one:

http://www.filmaffinity.com/es/search.php?stext=cariño

But the result is blank because the ñ
What is the format I shoudl use for the encoding?
I have tried a lot:
ISO8859-1
ISO-8859-1
etc.
but no one works.

Thanks

[fidoboy]

Hello HectorziN,

I'm trying to use your scraper but it has no config options in XBMC. I wan't to know how can i use it to manually search my movies, because sometimes it retrieves the wrong title. Also the posters quality is a bit low, can u enhace this scraper to get the covers from another source?

A bit offtopic; it's possible to use multiple sources (aka multiple scrapers) in XBMC?? For example if a movie is not found using one scraper, then use another, etc..

Many thanks in advance, kind regards,

[HectorziN]

Hello HectorziN,

I'm trying to use your scraper but it has no config options in XBMC. I wan't to know how can i use it to manually search my movies, because sometimes it retrieves the wrong title. Also the posters quality is a bit low, can u enhace this scraper to get the covers from another source?

A bit offtopic; it's possible to use multiple sources (aka multiple scrapers) in XBMC?? For example if a movie is not found using one scraper, then use another, etc..

Many thanks in advance, kind regards,

The config options are in the latest version, and it is not included yet in the last t3ch compilation.

The posters quality is low, but there is no way to use a poster from another address without a link in FilmAffinity web page.

It could be possibly to look the poster in imdb, but with the risk of getting a different movie, because the search in imdb could return more than one result.

when the search find a wrong result there is an option in the bottom of the window where you can set the name to search manually

I think is not possible having multiple sources. without changing set content option two times, of course.

[fidoboy]

Thanks for the answer HectorziN, where can i download the latest version of your scraper?? Also, i can't see that option at the window bottom, what theme are u using? i'm using MediaStream and can't see such option... :S

regards

[spiff]

updated scraper is now in svn, r15969

[spiff]

oh, and the search string encoding worked fine for me. i made a directory named cariño, set content, did the lookup. got the list your url pointed to.

[fidoboy]

And where is the SVN? can you provide a link to download the scraper or attach it here?

regards,

Fido

[w00dst0ck]

SVN: https://xbmc.svn.sourceforge.net/svnroot/xbmc/branches/linuxport/XBMC/system/scrapers/video/


@HectorziN:
It is possible to get the IMDB Link with a google search.
site:imdb.com +original title +year

I'm using a google wrapper to get the IMDB ID for fanart at my moviemaze scraper.

<!--URL to Google and Fanart-->
<RegExp conditional="fanart" input="$$8" output="&lt;url function=&quot;GoogleToIMDB&quot;&gt;http://www.google.com/search?q=site:imdb.com+moviemaze\1&lt;/url&gt;" dest="5+">
<RegExp input="$$1" output="\1" dest="7">
<expression>&lt;h2&gt;\((.*)\)&lt;</expression>
</RegExp>
<RegExp input="$$7" output="+\1" dest="8+">
<expression repeat="yes">([^ ,]+)</expression>
</RegExp>
<expression></expression>
</RegExp>

<!--GoogleToIMDB-->
<GoogleToIMDB dest="5">
<RegExp input="$$2" output="&lt;?xml version=&quot;1.0&quot; encoding=&quot;iso-8859-1&quot; standalone=&quot;yes&quot;&gt;&lt;details&gt;\1&lt;/details&gt;" dest="5">
<RegExp input="$$1" output="&lt;url function=&quot;GetFanart&quot;&gt;http://api.themoviedb.org/backdrop.php?imdb=\1&lt;/url&gt;" dest="2+">
<expression>/title/([t0-9]*)</expression>
</RegExp>
<expression noclean="1"/>
</RegExp>
</GoogleToIMDB>

<!-- Fanart -->
<GetFanart dest="5">
<RegExp input="$$2" output="&lt;details&gt;&lt;fanart url=&quot;http://themoviedb.org/image/backdrops&quot;&gt;\1&lt;/fanart&gt;&lt;/details&gt;" dest="5">
<RegExp input="$$1" output="&lt;thumb preview=&quot;/\1/\2_poster.jpg&quot;&gt;/\1/\2.jpg&lt;/thumb&gt;" dest="2">
<expression repeat="yes">/([0-9]*)/([t0-9-]*).jpg&lt;/URL</expression>
</RegExp>
<expression noclean="1">(.+)</expression>
</RegExp>
</GetFanart>

[HectorziN]

SVN: https://xbmc.svn.sourceforge.net/svnroot/xbmc/branches/linuxport/XBMC/system/scrapers/video/


@HectorziN:
It is possible to get the IMDB Link with a google search.
site:imdb.com +original title +year

I'm using a google wrapper to get the IMDB ID for fanart at my moviemaze scraper.

<!--URL to Google and Fanart-->
<RegExp conditional="fanart" input="$$8" output="&lt;url function=&quot;GoogleToIMDB&quot;&gt;http://www.google.com/search?q=site:imdb.com+moviemaze\1&lt;/url&gt;" dest="5+">
<RegExp input="$$1" output="\1" dest="7">
<expression>&lt;h2&gt;\((.*)\)&lt;</expression>
</RegExp>
<RegExp input="$$7" output="+\1" dest="8+">
<expression repeat="yes">([^ ,]+)</expression>
</RegExp>
<expression></expression>
</RegExp>

<!--GoogleToIMDB-->
<GoogleToIMDB dest="5">
<RegExp input="$$2" output="&lt;?xml version=&quot;1.0&quot; encoding=&quot;iso-8859-1&quot; standalone=&quot;yes&quot;&gt;&lt;details&gt;\1&lt;/details&gt;" dest="5">
<RegExp input="$$1" output="&lt;url function=&quot;GetFanart&quot;&gt;http://api.themoviedb.org/backdrop.php?imdb=\1&lt;/url&gt;" dest="2+">
<expression>/title/([t0-9]*)</expression>
</RegExp>
<expression noclean="1"/>
</RegExp>
</GoogleToIMDB>

<!-- Fanart -->
<GetFanart dest="5">
<RegExp input="$$2" output="&lt;details&gt;&lt;fanart url=&quot;http://themoviedb.org/image/backdrops&quot;&gt;\1&lt;/fanart&gt;&lt;/details&gt;" dest="5">
<RegExp input="$$1" output="&lt;thumb preview=&quot;/\1/\2_poster.jpg&quot;&gt;/\1/\2.jpg&lt;/thumb&gt;" dest="2">
<expression repeat="yes">/([0-9]*)/([t0-9-]*).jpg&lt;/URL</expression>
</RegExp>
<expression noclean="1">(.+)</expression>
</RegExp>
</GetFanart>

Thanks! it is a great idea but.... always returns the same movie? it could return a wrong one, right?

[HectorziN]

oh, and the search string encoding worked fine for me. i made a directory named cariño, set content, did the lookup. got the list your url pointed to.

Not a directory, the movie must be called cariño or another movie with a tittle containing ñ

If you search for a movie with the ñ character the scraper cannot find it because the encoding. Using the web browser in filmaffinity.com, it works.

Couls you test it, and... do yoy know the value for searchstringencoding that I need to use?

many thanks!

[spiff]

???

i repeat;
i made a directory named cariño, set content (including scan by dir name obviously), did the lookup. got the list your url pointed to.

[fidoboy]

Hi,

The encoding for ñ char is: %F1 but, anyway here you have the complete list (accents, etc):

http://www.jairoblanco.com/guia-rapida/html/html-url-encode-codificacion/

greets,

[w00dst0ck]

Thanks! it is a great idea but.... always returns the same movie? it could return a wrong one, right?

I've included moviemaze in my search string. If it's listed in the external review list of imdb.com [example (http://www.imdb.com/title/tt0852713/externalreviews)] I'll be sure that's the same movie.

[HectorziN]

???

i repeat;
i made a directory named cariño, set content (including scan by dir name obviously), did the lookup. got the list your url pointed to.

OK, but the problem I have is this one:
A folder called Movies
In this folder a lot of movies
one of them called "Cariño estoy hecho un perro"
I search information for this movie using the filmaffinity scrapper
and no results found, I change Cariño with Carino and it works.

The problem is that the search is not done with iso encoding, and I don't know the value to set in searchstringencoding

[fidoboy]

Hectorzin, have you readed my answer? You must encode your string, you should replace "cariño" with "cari%F1o" in your URL...

regards,

[spiff]

that will be done by the URL encoding applied prior to passing the argument to the scraper function...

[HectorziN]

My scraper is a lot complex. Is there any application to help debugger it?
I want to include impawards posters and I can't get it.

Thanks

[w00dst0ck]

I use xbmc for windows and watch the xbmc.log

There are also some online RegEx testers.

[HectorziN]

I use xbmc for windows and watch the xbmc.log

There are also some online RegEx testers.

Where the log file is stored in windows atlantis version?

thanks

[w00dst0ck]

Should be %HOMEPATH%\Application Data\XBMC\xbmc.log
or C:\Program Files\XBMC\xbmc.log

BTW: The german website http://www.regex-tester.de/regex.html translated to spanish http://tinyurl.com/5gfxx9 helps alot.

[HectorziN]

One question

I have some functions:

<GetMoviePosterDB clearbuffers="no" dest="12">
<RegExp input="$$1" output="&lt;thumb&gt;\1l_\2&lt;/thumb&gt;" dest="13+">
<expression clear="yes" repeat="yes" noclean="1,2">&quot;poster&quot;.*?src=&quot;(.*?)[a-z]_(.*?)&quot;</expression>
</RegExp>
</GetMoviePosterDB>


<GetIMDBPoster dest="5">
<RegExp input="$$16$$17$$13$$15$$18" output="&lt;details&gt;&lt;thumbs&gt;\1&lt;/thumbs&gt;&lt;/details&gt;" dest="5">
<RegExp input="$$6" output="&lt;thumb&gt;\1&lt;/thumb&gt;" dest="15">
<RegExp input="$$1" output="\1_SX$INFO[imdbscale]_SY$INFO[imdbscale]_\2" dest="6">
<expression noclean="1,2">&lt;a name=&quot;poster&quot;.*?src=&quot;(.*?)_S.*?(.jpg)&quot;.*?&lt;/a&gt;</expression>
</RegExp>
<expression clear="yes" noclean="1">(.*?_SX[0-9]+_SY[0-9]+_.jpg)</expression>
</RegExp>
<expression noclean="1"></expression>
</RegExp>
</GetIMDBPoster>



First function has a dest="13" inside
Second function has a dest="15" inside

As input of second function are $$13 and $$15

This works, and I have as a result the list of thumbs from both pages.

The problem I have is the next

if I insert this code in the <GetDetails> section:


<RegExp input="$$1" output="&lt;url function=&quot;GetFilmAffinityPoster&quot;&gt;http://www.filmaffinity.com/es/film214384.html&lt;/url&gt;" dest="16">
<expression noclean="1"></expression>
</RegExp>



you can see $$16 is also an input of the GetIMDBPoster function, but it don't work. Why?

Thanks

[spiff]

my suspicion is that you are bitten by the fact that order matters.

remember, the clearbuffers parameter say to not clear buffers after function calls.
so the point here is to
1) fill buffer 13
2) fill buffer 15
3) fill buffer 16
4) call GetIMDBPoster

not
1) fill buffer 13
2) fill buffer 15
3) call GetIMDBPoster
4) fill buffer 16

and i assume GetFilmAffinityPoster has clearbuffers="no"

[HectorziN]

Thanks, I got it. The problem was the order of functions.

I am testing with PC version and it works searching for cariño.
Is in xbox where don't work. It is possible that SearchStringEncoding is not implemented in xbox version?

thanks

[HectorziN]

I am searching for actor thumbs in imdb.
I have a problem with animation movies, in filmaffinity site de actor for this case is: "Animation"

Then my scraper searchs for animation in imdb and always finds this: Chuck Jones.

How can I avoid searching for actors when the actor is Animation? I should try to remove the Animation code from the buffer to avoid the scraper find it....

Thanks

[spiff]

use an expression that clears a buffer IF you find animation. let this buffer hold the function call. clear if expression matches. append the buffer. problem solved.

[HectorziN]

use an expression that clears a buffer IF you find animation. let this buffer hold the function call. clear if expression matches. append the buffer. problem solved.

I have tried it but....
If I write this:

<RegExp conditional="SearchCastThumb" input="$$1" output="&lt;url function=&quot;SearchCastThumb&quot;&gt;http://spanish.imdb.com/find?s=nm&amp;amp;q=\1&lt;/url&gt;" dest="5+">
<expression repeat="yes" noclean="1" trim="1">&lt;a href="search\.php.stype=cast.stext=([^&quot;]*)[^&gt;]*&gt;([^&lt;]*)</expression>
</RegExp>


it works, but if I change dest="5+" with dest="9+" then the scraper don't call the function. I know because if dest="5+" the log has this:

Get URL: http://spanish.imdb.com/find?s=nm&q=Mar%EDa+Jos%E9+Baus%E1

but with 9+ it isn't

why?

thanks

[spiff]

i assume buffer 9 is never transfered to the one containing the return value from the scraper function..

[HectorziN]

i assume buffer 9 is never transfered to the one containing the return value from the scraper function..

Yes, I use this:


<RegExp input="$$9" output="\1" dest="5+">
<expression></expression>
</RegExp>


I tried this code after the call to function and also with the function inside it:


<RegExp input="$$9" output="\1" dest="5+">
<RegExp conditional="SearchCastThumb" input="$$1" output="&lt;url function=&quot;SearchCastThumb&quot;&gt;http://spanish.imdb.com/find?s=nm&amp;amp;q=\1&lt;/url&gt;" dest="9+">
<expression repeat="yes" noclean="1" trim="1">&lt;a href="search\.php.stype=cast.stext=([^&quot;]*)[^&gt;]*&gt;([^&lt;]*)</expression>
</RegExp>
<expression></expression>
</RegExp>



I also tested it with buffer 20 because buffer 9 is used in the scraper and buffer 20 is never used. But don't work in any case.

Thanks

[spiff]

you need noclean on the outermost expression or all tags will be stripped off

[HectorziN]

you need noclean on the outermost expression or all tags will be stripped off

Thanks! Solved!

[HectorziN]

By the way, with Atlantis version searchstringencoding works!!!!

[fidoboy]

Is not working on current version (9.04 beta)

[dedaluz]

Any updates on this one?

[spiff]

yes, i commited a fix at r19978

[dedaluz]

Thanks!

[itombs]

Hi, today Im having some problems scrapping films.
I use the last version of filmaffinity scrapper downloaded from here:
https://xbmc.svn.sourceforge.net/svnroot/xbmc/branches/linuxport/XBMC/system/scrapers/video/

The problem is with the actors, now it doesnt work really fine, in some movies, the actor list is incomplete, for example:
The Doors
http://www.filmaffinity.com/es/film245613.html
Scrapper doesnt show Meg Ryan.

A Perfect Murder
http://www.filmaffinity.com/es/film871970.html
Scrapper doesnt show Michael Douglas

Death Race
http://www.filmaffinity.com/es/film127500.html
Scrapper doesnt show various actors

etc etc

I know that before, the scraper worked perfectly with actors but now doesnt work fine.

This problem is only mine or its necessary a modification for the scrapper??
Someone could try to scrap those movies and write the result here?

Thanks a lot.

Bye byE

[itombs]

Problem seems to be the multiplies results from IMDB when filmaffinity scrapper search an actor in IMDB.
When name only have a result (there is only an actor with this name), filmaffinity scrapper works well but, when IMDB return 2 or more results searching the name, filmaffinity scrapper are not prepared to select one and miss this actor.
A temporaly solution could be desactivate the option to download actors thumbnails from IMDB.
A trully solution could be search the title of the film in IMDB and download all actors information from IMDB (more complete than filmaffinity info).
More info (in spanish):
http://www.elotrolado.net/hilo_scrapper-de-filmaffinity-y-actores_1253389
Bye ByE.

[fidoboy]

Hi,

I'm using XBMC Live with SVN 21913, the filmaffinity scrapper doesn't seems to be working fine. It downloads the movie info, and the fan art but not the thumbnails. It works fine with IMDB scrapper, so i think that filmaffinity scrapper needs to be modified in some way.

Also i've noticed (before this bug, because some SVNs ago it was working fine) that for the movie "The Matrix Reloaded" it doesn't download the right thumbnail...

regards,

[vdrfan]

scraper needs some love as the thumb stuff was updated lately

[fidoboy]

Thanks for the info vdrfan, because i was going crazy trying to solve this issue. Can u give more info? May be i can try to repair the filmaffinity scrapper...

regards,

[spiff]

instead of wrapping the thumbs in a <thumbs> tag, return multiple <thumb> instead

[fidoboy]

How? can you provide a simple example?

[spiff]

sure - all the scrapers in svn that has been updated imdb tmdb tvdb, all the german ones.

[fidoboy]

Well, i've partially solved the problem that way (removing the <thumbs> tag) but there is also other problems retrieving the info for other sites. I'm not sure about the use of buffers (clearbuffer and dest) and i can find any documentation about it...

How can i see the scrapper generated output? May be that if i see the output i can understand better the problem.

regards,

[spiff]

debug log outputs all scraper results.

you do not need the clearbuffers trick any longer. just return <details><thumb>..</thumb>..</details> from the new functions. the clearbuffers thingy was needed when all thumbs needed to be wrapped in a <thumbs> tag, stuffing results in a buffer then finally adding them all at a final expression. you probably see one <RegExp> which has tons of input buffers at the end of the chain.

[fidoboy]

This is a total nightmare xD. I'm about to surrender... I don't understand the filmaffinityscrapper at all... i'm comparing it against imdb.xml scrapper but dest buffers are different and i don't know how to change them to fit into the filmaffinity scrapper functions...

[fidoboy]

What's the meaning of dest="4+"?? i don't understand what the + sign wants to mean...

regards...

[vdrfan]

this means: append to buffer 4

[vdrfan]

Updated scraper in SVN r21923. I think there's some room for improvements but for the moment this should (hopefully) work.

[fidoboy]

Many thanks, cause i'm going crazy trying to understand how the scrapper works. Can u post here the file?

thanks,

[vdrfan]

http://xbmc.org/trac/export/21923/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml

[fidoboy]

It works fine now, but trailers do not play...

[vdrfan]

They were working in the previous version?

[fidoboy]

I'm not sure...

[vdrfan]

Replace the scrapers and give it a try :)

[fidoboy]

I've tested it using the previous scraper version and it doesn't work. So it was not working before your changes...

[Filete]

nose ingles, pero si algun español de los k se pasan por aki me lo traducen, tengo un problema con los scrapers y es k solo me busca solo 1 miniatura.


not English, but if any of the Spanish overlooked k aki me translate it, I have a problem with Scrapers and k is just me looking for only 1 thumb.

[fidoboy]

Hola filete, lo que pone en los post anteriores al tuyo, es justo eso, la solucion es que descargues el nuevo scrapper aqui: http://xbmc.org/trac/export/21923/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml o bien que actualices tu XBMC usando la ultima version SVN, con eso esta solucionado.

Salu2,

[Filete]

Hola filete, lo que pone en los post anteriores al tuyo, es justo eso, la solucion es que descargues el nuevo scrapper aqui: http://xbmc.org/trac/export/21923/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml o bien que actualices tu XBMC usando la ultima version SVN, con eso esta solucionado.

Salu2,
Wenas ya me baje ese scraper y nada no me lo soluciona, y la 2 opcion no se lo k es.

[vdrfan]

English please, thanks.

[Filete]

English please, thanks.
Sorry pero porfavor dar soporte al idioma español

[vdrfan]

Uhm, you want me to use a online translation tool? Please respect our forum rules!

[itombs]

Hola filete, lo que pone en los post anteriores al tuyo, es justo eso, la solucion es que descargues el nuevo scrapper aqui: http://xbmc.org/trac/export/21923/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml o bien que actualices tu XBMC usando la ultima version SVN, con eso esta solucionado.

Salu2,

Hola, yo me he bajado esa version y sigue sin descargarme ams de una caratula.

Hi, i downloaded this version but only appers one thumb (imdb thumb).

Any solution?

Thanks.

[itombs]

And in the new version, problem with actors continues.

[Filete]

dowload the xbmc for here
http://www.sshcs.com/xbmc/

[mortisdj]

The last repo [21923] dont work. No download the covers, :(

[HectorziN]

hello again... i am a bit disconnected.

i will try to fix the cover problem. also the trailer problem, trailers are not working.

spiff, can i download the last version, modify it and upload the complete file? or should i use any tool to do it (like sourcesafe)

thanks!

[spiff]

just fetch the file, do your stuff and add it to trac. please mind the includes, i don't recall if i did this one or not. if i didn't, please make use of them

[HectorziN]

just fetch the file, do your stuff and add it to trac. please mind the includes, i don't recall if i did this one or not. if i didn't, please make use of them

this one was done by me, what do you mind by includes? indentation?

thanks!

[spiff]

heh, i know you did the scraper and in no way did i try to claim the credit!

what i meant was that i redid most video scrapers to use <include>..</include>. i checked svn and indeed i updated it so you'll see what i mean when you get around to working on it :) basically it's a mechanism to share code between scrapers.

[HectorziN]

heh, i know you did the scraper and in no way did i try to claim the credit!

what i meant was that i redid most video scrapers to use <include>..</include>. i checked svn and indeed i updated it so you'll see what i mean when you get around to working on it :) basically it's a mechanism to share code between scrapers.

WOW!! this is new for me! it is realy usefull, when I did copy/paste from imdb scrapper I though about what usefull a include sentence could be...

thankS! I will respect the includes! I will try to understand how it works.

[calibra3]

hola, hectorzin, se sabe ya por que no funciona bien filmaffinity?

Hello hectorzin , do you know because don't run filmaffinity scrapper?

[spiff]

he's the author - and he just said he'd look into it

[HectorziN]

heh, i know you did the scraper and in no way did i try to claim the credit!

what i meant was that i redid most video scrapers to use <include>..</include>. i checked svn and indeed i updated it so you'll see what i mean when you get around to working on it :) basically it's a mechanism to share code between scrapers.

I downloaded it from here: http://xbmc.org/trac/export/21923/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml

but i didn't see any include.

I found the problem, when there is more than one thumb, the scraper should return <thumbs><thumb>...</thumb><thumb>...</thumb><thumb>...</thumb>...</thumbs>
I am not including <thumbs> statement and it worked many time. I suppouse it is a requierement know.

I want to fix it but due to de include statements spiff said he has included i need to know where this version is to work arround with it.

[HectorziN]

hola, hectorzin, se sabe ya por que no funciona bien filmaffinity?

Hello hectorzin , do you know because don't run filmaffinity scrapper?

Hello hectorzin, do you know why filmaffinity scrapper it's not working? :)

why is for asking, because for answering
the scrapper runs, the problem is don't work ;)

what a bad english learning we have in Spain... me included, I like this kind of forums because I can improve my english.

I have answered your question in my previous message

[spiff]

try http://xbmc.org/trac/export/22635/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml

your link is to an old version

[HectorziN]

try http://xbmc.org/trac/export/22635/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml

your link is to an old version

thanks you are so fast.
i downloaded the file and it is including 4 files but movieposterdb.xml and impa.xml are not in the folder http://xbmc.org/trac/export/22635/branches/linuxport/XBMC/system/scrapers/video/

thanks

[spiff]

they are in the common/ subdir

[HectorziN]

they are in the common/ subdir

Ok, i think I need the last build because the one I have don't know what to do with include sentences.

where can i get windows binaries with a version that supports this new features?

thanks

[spiff]

uhm, search? ;)

http://xbmc.org/forum/showthread.php?t=35452&page=82

get a build from before 22529

[HectorziN]

uhm, search? ;)

http://xbmc.org/forum/showthread.php?t=35452&page=82

get a build from before 22529

I think links are not working.
I find this page with many builds http://danielpatton.com/user-accounts/XBMC-updates/ but all of them are older than 22529 :(

[spiff]

good. 22529 introduce major breakage, hence i said anything BEFORE :)

[HectorziN]

good. 22529 introduce major breakage, hence i said anything BEFORE :)

Ok, perfect.

I've downloaded last build and the scrapper works. I think people has downloaded last scrapper version but using an old XBMC version. Old XBMC version don't work with includes, and this is the problem.

People needs to download de last build and covers will work.

what I don't understand is why thumbs cast are not working.
this is the log file:


SearchCastThumb returned <details><actor><name>Kate Beckinsale</name><thumb>http://ia.media-imdb.com/images/M/MV5BMTY4Mzk2NjM2NV5BMl5BanBnXkFtZTYwODc2MDI2._V1._ SX100_SY137_.jpg</thumb></actor></details>


why only actors names are working?

[spiff]

no reason those shouldn't work. sure you have enabled actor thumbs in settings?

[HectorziN]

no reason those shouldn't work. sure you have enabled actor thumbs in settings?

sorry for my mistake. It seems that when installing lastest version I missed my config.

Everything is working except youtube videos.
I have found the problem, we should replace:

http://www.bajaryoutube.com/?youtube=http://www.youtube.com/watch?v=

with


http://www.yobajo.com/watch?v=


the web bajaryoutube.com it's not working for longer.

[jesusmabas]

posters and trailers don't work!! :(

[jesusmabas]

hectorzin, have you repaired it?? thx

[HectorziN]

a question:
I have this code to get cast name and thumb


<GetCastThumb dest="5">
<RegExp input="$$4" output="&lt;details&gt;&lt;actor&gt;\1&lt;/actor&gt;&lt;/details&gt;" dest="5">
<RegExp input="$$1" output="&lt;name&gt;\1&lt;/name&gt;" dest="4">
<expression noclean="1" clear="yes" trim="1">&lt;title&gt;([^&lt;|^(]*)</expression>
</RegExp>
<RegExp input="$$9" output="&lt;thumb&gt;\1&lt;/thumb&gt;" dest="4+">
<RegExp input="$$1" output="\1" dest="9">
<expression noclean="1">&lt;div class=&quot;photo&quot;&gt;(.*) alt=&quot;</expression>
</RegExp>
<expression repeat="yes">img border=&quot;0&quot; src=&quot;([^=]*.jpg)</expression>
</RegExp>
<expression noclean="1" clear="yes">(.*)</expression>
</RegExp>
</GetCastThumb>


how can i do to avoid including thumb when the thumb link is this?


http://i.media-imdb.com/images/nophoto.jpg


This image has no sense to include because wastes space and it is downloaded every time an actor with no thumb found.

thanks

[calibra3]

I have downloaded the las version 22616-gl and install in other computer, and I resolve it, the filmaffinity scraper, download again the info, fanart and tumbnails.

This weekend , I will update the XBMC of my HTPC, I hope that whe I update it, don't loose all the fanart and info of my 120 movies.

gracias hector!!

[spiff]

use negative lookahead

[calibra3]

use negative lookahead

hello .

this answer is for me or for hectorzin??

if it is for me, what is negative lookahead? where I can find it?


thanks

[spiff]

i only answer dev questions in the dev forums - extrapolate from there.

[JoshyFun]

Hello,

This is my first post at all :)

I had fixed some of the problems of "filmaffinity" problems (no covers, no trailers) using the "old" 9.04.1 version, tested on Windows and XBox.

I'm not the author, simply a "bug fixer" as the original author should be updating it to for upcoming releases (include tags).

Yesterday I know nothing about scrappers internal behavior but thanks to the scrappertester the fixes were possible (and some hours of work).

Now filmaffinity, ipwards and movieposterdb should work for covers, the file is at: http://www.zshare.net/download/65301318441ee46d/

Note: Moderator, if the link is not appropiate please move or delete it.

Thank you for a great software (XBMC and filmaffinity scrapper).

[spiff]

cool. please submit patches @ xbmc.org/trac

[HectorziN]

cool. please submit patches @ xbmc.org/trac

This patch cannot be submited because what he has done was remove includes and copy the include code inside the scraper.

The scraper is working, the problem is includes is a feature of latest versions of xbmc. The last scraper don't work with old xbmc versions.

I have last scraper and last xbmc version and it works.

I will submit a new patch today that fixes youtube videos and also covers from filmaffinity pages, including new poster page:
http://www.filmaffinity.com/es/filmimages.php?movie_id=449213

The patch from Joshyfush should only be used with older versions of xbmc, but should not include in xbmc trac

[vdrfan]

I've updated the filmaffinity scraper lately. Please check the current version of the scraper and submit a diff to that file in case something is wrong or missing.

[HectorziN]

Submited patch!
Now movie trailers are working and also all covers comming from filmaffinity web using large resolution!

[HectorziN]

I've updated the filmaffinity scraper lately. Please check the current version of the scraper and submit a diff to that file in case something is wrong or missing.

Sorry, i didn't notice this. i will download your file and check differences, i will submit a patch including your changes.

let me check it

[vdrfan]

Cool, thanks. (forget the ticket comment, dupe)

[HectorziN]

I've updated the filmaffinity scraper lately. Please check the current version of the scraper and submit a diff to that file in case something is wrong or missing.

can you send me a copy of your changes please?
i don't know how to get your version from xbmc.org
what did you change?
thanks.

[vdrfan]

http://xbmc.org/trac/export/22914/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml

[HectorziN]

http://xbmc.org/trac/export/22914/branches/linuxport/XBMC/system/scrapers/video/filmaffinity.xml

yes, I found it.

Don't worry, I see your changes. You used the new location of covers in filmaffinity using pics.filmaffinity.... and you changed youtube trailers using yobajo.com

This two changes still are in my version.

I also included more covers from the new cover page in filmaffinity ;)

[vdrfan]

Ok. Will add this once at home (or anyone else watching the ticket).

[HectorziN]

Ok. Will add this once at home (or anyone else watching the ticket).

I think you cannot download it yet. I think it must be aproved first. right?

[vdrfan]

Download? Approve? I've attached a diff to our current scraper to the ticket and will add it once at home.

[HectorziN]

Download? Approve? I've attached a diff to our current scraper to the ticket and will add it once at home.

sorry, I am a bit lost about how it works all of this.
I am a developer and usually I work with Team foundation server. This xbmc trac is a bit confusing for me ;)

[HectorziN]

I've updated the filmaffinity scraper lately. Please check the current version of the scraper and submit a diff to that file in case something is wrong or missing.

I have changed it again :S

I submited it before reading this post. sorry

[vdrfan]

Huh? In case you're modifying the scraper please ALWAYS use the latest SVN scraper to make our work easier. Also make sure to post updates to the given tickets instead in the forum, otherwise they might get lost.

If there's another change to the scraper, please create a new ticket and attach a diff to the CURRENT scraper ;)

[HectorziN]

Huh? In case you're modifying the scraper please ALWAYS use the latest SVN scraper to make our work easier. Also make sure to post updates to the given tickets instead in the forum, otherwise they might get lost.

If there's another change to the scraper, please create a new ticket and attach a diff to the CURRENT scraper ;)

sorry, but I don't know how to attach a diff
how can I do this?

[vdrfan]

There's a tool called diff. It is a tool that can be used to create a diff (aka patch) that contains differences between two or more files. The most common format is the unified format as its easier to read and work with.

If you can't supply a diff, please make sure you modify the current scraper sitting in SVN so we can easily replace the existing one.

[HectorziN]

There's a tool called diff. It is a tool that can be used to create a diff (aka patch) that contains differences between two or more files. The most common format is the unified format as its easier to read and work with.

If you can't supply a diff, please make sure you modify the current scraper sitting in SVN so we can easily replace the existing one.

I tried to find diff application but it is a common word to find it with google. can you give me a link to the application?

I will send aka patch once I install it.

Thanks!

[spiff]

uhm, windows? just grab tortoisesvn, it allows you to checkout stuff from svn (guarantee's latest version) and has a great diff tool (both for applying diffs and for generating them)

[HectorziN]

uhm, windows? just grab tortoisesvn, it allows you to checkout stuff from svn (guarantee's latest version) and has a great diff tool (both for applying diffs and for generating them)

I get this error trying to check out:

Checkout from http://xbmc.org/trac/export/22635/branches/linuxport/XBMC, revision HEAD, Fully recursive, Externals included
The OPTIONS response did not include the requested activity-collection-set
this often means that the URL is not WebDAV-enabled

what can be the problem?

thanks!

[HectorziN]

uhm, windows? just grab tortoisesvn, it allows you to checkout stuff from svn (guarantee's latest version) and has a great diff tool (both for applying diffs and for generating them)

I used Merge tool from tortoise and it shows me the differences but the option to generate diff file is deactivated.
I downloaded manually the file from xbmc/trac and did the comparison with both files from hard disk. I wasn't able to use checkou option

[spiff]

svn co https://xbmc.svn.sourceforge.net/svnroot/xbmc/branches/linuxport/XBMC

not the export url. also do your changes in the svn dir, THEN right click and select 'make patch'. that will give you a diff against the current file in svn

[HectorziN]

svn co https://xbmc.svn.sourceforge.net/svnroot/xbmc/branches/linuxport/XBMC

not the export url. also do your changes in the svn dir, THEN right click and select 'make patch'. that will give you a diff against the current file in svn

Thanks! it worked!

Is there a way to submit a patch directly using tortoise? or should I go to xbmc.org/trac and create a ticket? is there a way to checkout, change source and submit using only tortoise tools?

[spiff]

no, trac is not integrated, so you need to save it to a diff file, open a ticket and attach the file.

[JoshyFun]

This patch cannot be submited because what he has done was remove includes and copy the include code inside the scraper.
The scraper is working, the problem is includes is a feature of latest versions of xbmc. The last scraper don't work with old xbmc versions.
[...]

Hello,

I had just taken my 9.04.1 scrapper in XBox (that's my last usable version) and almost blindly modifiy it (without understanding sometimes what I was writing or why you write other things) :) That's the reason some unnecessary things were changed.

I had also noted that it is for revisions prior to include tags. Maybe scrapper fixes like this should be available to download separatly instead being "forced" to use the last SVN version :-?

I think you are also spanish so "Gracias" :)

[HectorziN]

no, trac is not integrated, so you need to save it to a diff file, open a ticket and attach the file.

I sent it but I think has not been included.

[vdrfan]

Added in SVN r23161. Cheers!

[AlbertQ]

After last changes in the scraper [SVN r23161], Fanart and posters distinct than Filmaffinity doesn't work :no: .



Despues de ponerme el ultimo scraper [SVN r23161], los Fanart y los posters de otros sitios que no sean Filmaffinity no se muestran.



:sad:

[peque]

Hello,

I had just taken my 9.04.1 scrapper in XBox (that's my last usable version) and almost blindly modifiy it (without understanding sometimes what I was writing or why you write other things) :) That's the reason some unnecessary things were changed.

I had also noted that it is for revisions prior to include tags. Maybe scrapper fixes like this should be available to download separatly instead being "forced" to use the last SVN version :-?

I think you are also spanish so "Gracias" :)
I'm glad to see you again JoshyFun. Thanks for this scrapper update.

Please contact me back.

bye!

pequesan_at_gmail.com

[HectorziN]

After last changes in the scraper [SVN r23161], Fanart and posters distinct than Filmaffinity doesn't work :no: .



Despues de ponerme el ultimo scraper [SVN r23161], los Fanart y los posters de otros sitios que no sean Filmaffinity no se muestran.



:sad:

Download latest version of xbmc. Last scraper only works with last version.

Regards

[HectorziN]

Hello,

I had just taken my 9.04.1 scrapper in XBox (that's my last usable version) and almost blindly modifiy it (without understanding sometimes what I was writing or why you write other things) :) That's the reason some unnecessary things were changed.

I had also noted that it is for revisions prior to include tags. Maybe scrapper fixes like this should be available to download separatly instead being "forced" to use the last SVN version :-?

I think you are also spanish so "Gracias" :)

You must understand that we must to maintain the last version, and not older version. ;)

[JoshyFun]

You must understand that we must to maintain the last version, and not older version. ;)

Hello,

Sure, of course, but there are some border cases, I think that "stable" releases should be maintained meanwhile development in SVN ones. In example my situation, my last usable version was 9.04.1 stable, and filmaffinity scraper does not work, Xbox SVN ones were not working properly, so my stable release does not work nor the SVN :(

A great solution could be adding scrapers to the new update facility (which runs for plugins, scripts and skins... maybe scrapers too?).

Anyway, this is not a discussion board, it is a development discussion one, and this post seems to be completly OT. If you wish to continue let me know by PM in which thread as this is the only section I read from time to time.

[nabz]

You must understand that we must to maintain the last version, and not older version. ;)
Hi HectorziN,
first of all, thanks for this scraper :) It's great!
Now I'm running the last SVN version (23756) with the last version of the scraper(Sept 25) downloaded from trac. I'm generating my library but it cannot downloads the fanart (only downloads the poster). Using the previous version of the scraper it can download the fanart but not the poster :sad:. I've tried to download them manually using the "Download fanart" function from the "Film info" screen, but when I choose the fanart (which I cannot see) it seems to be lock for a short time and after that the download window closes automatically without download the fanart.
Is it a bug? Or perhaps I'm doing something wrong... :sniffle:
Please let me know if can i do something else to give you more information (log file or something like that)

Thanks

NaBz


Hola HectorziN,
antes de nada, agradecerte este scraper... es la leche!
Te comento : estoy ejecutando la ultima version de SVN(23756) con la ultima version del scraper, del 25 de septiembre descargada de trac, y me pasa lo siguiente : estoy creando por primera vez mi biblioteca de videos pero cuando se pone a escanear unicamente me descarga las portadas de las peliculas, no descarga los fanart. Con la version anterior me hacia lo contrario, me decargaba los fanart pero no las portadas. He intentado hacerlo manualmente desde la ficha de informacion de la pelicula, usando el boton "DEscargar fanart". Me muestra los links de los fanart a elegir pero no me muestra la imagen. Aun asi, al elegir una de las que se muestran en la lista aparece la ventana de descarga del fanart, se queda un rato ahi parado y luego la ventana de descarga se para, sin haber bajado nada.
Puede que sea un fallo del scraper? O quizás estoy haciendo algo mal?:sniffle:
Por favor, dime si puedo ayudarte de alguna manera para que tengas mas informacion(algun log o similar)

Gracias por todo

NaBz

[vdrfan]

@nabz, see xbmc.org frontpage.

[nabz]

@nabz, see xbmc.org frontpage.
Hi vdrfan,
ok, I didn't see that.
I'll try it again later then TMDB works.
Thanks for all

[Baphomet08]

Hello.

I'm having some troubles with filmaffinity scrapper.

I've X-Men in my database but the XBMC can't get correct data from filmaffinity.

In search window, only shows 20 film, neither X-Men. When i go to filmaffinity web and search for X-Men, i get the data in 43 position.

How can i modify the scrapper to get the full search results instead 20?

Thanks.

[HectorziN]

Hi, I want to download latest filmaffinity scrapper but it seems http://xbmc.org/trac/browser/branches/linuxports has dissapeared.

the same for tortoise address. where is now the scrapers reppository?

thanks