PDA

View Full Version : filmstarts.de scraper development - help needed


floohh
2008-05-28, 21:30
Hi guys,

im currently developing around a http://filmstarts.de scraper. After hours I managed to get the XBMC to recognize the search results of filmstarts, but when i try to get the details page, XBMC requests a empty url. I think there is a failure in my RegExp Code

And here we go:

The Filmstarts-HTMl looks like:


<li><a href="/kritiken/35848-Der-Fluch-von-Darkness-Falls.html">
<img alt="" src="http://thumbs.filmstarts.de/nano/DerFluchVonDarknessFalls_poster_1.jpg">
<span class="r"> <img src="/designs/default/images/ratings/310er.gif" alt="Wertung: 3 / 10">

</span>
<span class="t">Der Fluch von Darkness Falls</span>
<span class="g">Teenie-Horror</span> </a></li>


<li><a href="/kritiken/36232-Fluch-der-Karibik.html">
<img alt="" src="http://thumbs.filmstarts.de/nano/fluchderkaribik-poster1.jpg">
<span class="r"> <img src="/designs/default/images/ratings/910er.gif" alt="Wertung: 9 / 10">
</span>
<span class="t">Fluch der Karibik</span>
<span class="g">Abenteuer</span> </a></li>


<li><a href="/kritiken/37419-Blueberry-und-der-Fluch-der-D%E4monen.html">

<img alt="Blueberry und der Fluch der Dämonen" src="/designs/default//images/no_film_small.gif" height="44" width="30">
<span class="r"> <img src="/designs/default/images/ratings/610er.gif" alt="Wertung: 6 / 10">
</span>
<span class="t">Blueberry und der Fluch der Dämonen</span>
<span class="g">Fantasy-Action</span> </a></li>


and my RegExp is:


<GetSearchResults dest="3">
<RegExp input="$$5" output="<?xml version="1.0" encoding="iso-8859-1" standalone="yes"?><results>\1</results>" dest="3">
<RegExp input="$$1" output="<entity><title>\2</title><url>http://www.filmstarts.de/\1</url><id>\1</id></entity>" dest="5">
<expression repeat="yes"><a href="/kritiken/([-.%\w]+)">[^<]|[\n]<span class="t">([-%. \w]+)</span></expression>
</RegExp>
<expression noclean="1"></expression>
</RegExp>
</GetSearchResults>






(I know the entities aren't converted, but I decoded them for better understanding)

The most important line is:


<expression repeat="yes"><a href="/kritiken/([-.%\w]+)">[^<]|[\n]<span class="t">([-%. \w]+)</span></expression>


to recognize


<a href="/kritiken/35848-Der-Fluch-von-Darkness-Falls.html">
<img alt="" src="http://thumbs.filmstarts.de/nano/DerFluchVonDarknessFalls_poster_1.jpg">
<span class="r"> <img src="/designs/default/images/ratings/310er.gif" alt="Wertung: 3 / 10">

</span>
<span class="t">Der Fluch von Darkness Falls</span>


The only thing XBMC does is to request "/"


Can somebody may help me?

floohh
2008-05-28, 22:59
okay i altered the term for skipping the unneccesary text, but now i only catch the first match, any idea how to solve?

<li><a href="/kritiken/([-.a-z0-9A-Z]+)">.*<span class="t">([0-9a-zA-Z .]+).*</li>

floohh
2008-05-29, 03:07
After hours of hard work, finally it worked

spiff
2008-05-29, 11:36
i assume your issue was that you didn't realize you are writing xml. so you need to escape special chars such as ", i.e. do &quot;

sorry i didnt see you inquery earlier. feel free to ask again i will try to be of help when i see it :)

floohh
2008-06-02, 16:14
Find solution here:
Link (http://sourceforge.net/tracker/index.php?func=detail&aid=1982083&group_id=87054&atid=581840)

spiff
2008-06-02, 16:30
great - the more the merrier. will add to svn :) cheers

tatoosh
2009-06-29, 09:46
Hey,

i cant download your filmstarts.de scraper. can u give me a link?
it would be nice to use this great website.

w00dst0ck
2009-06-29, 11:13
@Tatoosh: Mach doch mal ein Update deiner XBMC Version.
Alternativ kannst Du über http://xbmc.org/trac (http://xbmc.org/trac/browser/branches/linuxport/XBMC/system/scrapers/video/) die aktuelle Version aus dem SVN downloaden.