Does XBMC and XBMC Media Center end-users have permission to scrape imdb.com website?

ultrabrutal · 2009-01-12, 10:12

Just to take one, imdb:

IMDB Conditions

"Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below."

Do XBMC have this permission? Just curious...

**spiff** · 2009-01-12, 13:37

no, i was not aware of those terms (or atleast i could claim so).

not so much any longer. r17032 disables the scraper

ultrabrutal · 2009-01-12, 14:47

Shit! What a party pooper I turned out be

I'm afraid that most sites have conditions like this so most scrapers are in fact violating terms of the site

**theuni** · 2009-01-12, 15:59

Quote:

We do allow the limited use of robots and crawlers, such as those from certain search engines, with our express written consent. If you are interested in receiving our express written permission to use robots or crawlers on our site, please contact our Licensing Department.

I'd be happy to kick-off communications. I'd think this is the type of project that they would be open to.

Spiff: Let me know if this is something I should look into, or if it's up to the XBMC team to handle.

TheUni

**spiff** · 2009-01-12, 16:01

i think we will have to handle this ourself.

thanks for the offer though

ultrabrutal · 2009-01-12, 16:19

I'm afraid we eventually have to create our own movie database ala the mymovies database which we can all contribute to

**spiff** · 2009-01-12, 16:20

themoviedb.org is just that. i have already written the scraper a few months ago, but as the site had so little info i didn't bother with polishing it.

**pike** · 2009-01-12, 18:21

Let me quote Conan O Brien "Nicely Done"

Quote:

Originally Posted by ultrabrutal

Just to take one, imdb:

IMDB Conditions

"Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below."

Do XBMC have this permission? Just curious...

ultrabrutal · 2009-01-12, 18:32

Pike, when commitmonitor showed me the commits spiff had done I felt really bad. I use the scrapers also hehe. Of course XBMC cannot violate any licenses, so I totally get spiff.

spiff, I didn't know about themoviedb.org. Do you use an api or really scrape the html? I don't see an imdb link, so you use the movie name as a param?

**spiff** · 2009-01-12, 18:41

it's an xml based api.

see r17035

2009-01-12, 10:12	#1
ultrabrutal Fan Join Date: Feb 2005 Location: South of Heaven Posts: 954	Does XBMC and XBMC Media Center end-users have permission to scrape imdb.com website? Just to take one, imdb: IMDB Conditions "Robots and Screen Scraping: You may not use data mining, robots, screen scraping, or similar data gathering and extraction tools on this site, except with our express written consent as noted below." Do XBMC have this permission? Just curious...

2009-01-12, 13:37	#2
spiff Grumpy Bastard Developer Join Date: Nov 2003 Posts: 7,715	no, i was not aware of those terms (or atleast i could claim so). not so much any longer. r17032 disables the scraper __________________ Always read the XBMC online-manual, FAQ and search the forum before posting. Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules. For troubleshooting and bug reporting please make sure you read this first.

2009-01-12, 16:01	#5
spiff Grumpy Bastard Developer Join Date: Nov 2003 Posts: 7,715	i think we will have to handle this ourself. thanks for the offer though __________________ Always read the XBMC online-manual, FAQ and search the forum before posting. Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules. For troubleshooting and bug reporting please make sure you read this first.

2009-01-12, 16:20	#7
spiff Grumpy Bastard Developer Join Date: Nov 2003 Posts: 7,715	themoviedb.org is just that. i have already written the scraper a few months ago, but as the site had so little info i didn't bother with polishing it. __________________ Always read the XBMC online-manual, FAQ and search the forum before posting. Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules. For troubleshooting and bug reporting please make sure you read this first.

2009-01-12, 18:41	#10
spiff Grumpy Bastard Developer Join Date: Nov 2003 Posts: 7,715	it's an xml based api. see r17035 __________________ Always read the XBMC online-manual, FAQ and search the forum before posting. Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules. For troubleshooting and bug reporting please make sure you read this first.

2009-01-12, 14:47	#3
ultrabrutal Fan Join Date: Feb 2005 Location: South of Heaven Posts: 954	Shit! What a party pooper I turned out be I'm afraid that most sites have conditions like this so most scrapers are in fact violating terms of the site

2009-01-12, 16:19	#6
ultrabrutal Fan Join Date: Feb 2005 Location: South of Heaven Posts: 954	I'm afraid we eventually have to create our own movie database ala the mymovies database which we can all contribute to

2009-01-12, 18:32	#9
ultrabrutal Fan Join Date: Feb 2005 Location: South of Heaven Posts: 954	Pike, when commitmonitor showed me the commits spiff had done I felt really bad. I use the scrapers also hehe. Of course XBMC cannot violate any licenses, so I totally get spiff. spiff, I didn't know about themoviedb.org. Do you use an api or really scrape the html? I don't see an imdb link, so you use the movie name as a param?

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode