View Full Version : Best scraper for music?
Nazgulled
2009-04-07, 00:27
I don't know if I'm doing something or wrong or if the sites supported by the default scrappers are crap :/ (the sites, not the scrappers)
I tried all of them and it seems my music library is so incomplete. I'm talking about artists images and fanart. I already have all the covers for my albums in good quality. But I have no fanart nor good quality artist thumbs.
Which scrapper do you guys use for music? I suppose none of the sites supported by the default scrappers have proper support for fanart but they have for the artists thumb. However, most of the artists on my database don't have a picture (although the site has) and the ones that have it, are of very low and small resolution.
So, what do you guys use for music then?
Ninjamawwe
2009-04-07, 14:18
I use the allmusic scraper which has good and consistent artist/album info. As for the thumbnails I haven't found a good site, so I did all of them myself (folder.jpg in a /path/artist/album directory setup). Perhaps the owner of the home theater backdrops site would consider adding hi-res artist thumbs in addition to fanart. That site is already scrapable for fanart, so adding scraping of artist thumbs should be relatively straightforward (I think).
onesojourner
2009-04-07, 18:12
I have the same problem with my music. I never use the library mode in music because it never scrapes it right.
Nazgulled
2009-04-07, 18:38
@Ninjamawwe (http://xbmc.org/forum/member.php?u=25156)
I don't care about the albumart, that I prefer to do manually and I already have covers for all my albums. I'm only talking about artist thumbs and fanart, none of the scrappers seem to do a good job at it. I was just wondering if there was any other, or any other way for music...
It's a "SCRAPER", not a "SCRAPPER".
Sorry to correct you...
smcnally75
2009-04-07, 19:22
It's a "SCRAPER", not a "SCRAPPER".
Sorry to correct you...
LOL! I think it may be a location thing...I've noticed a lot of people calling them "Scrappers" lately.
Nazgulled
2009-04-07, 19:38
Not really, I'm Portuguese, I don't even know a word to translate that into :P I just thought it was "scrapper".
smcnally75
2009-04-07, 19:47
Not really, I'm Portuguese, I don't even know a word to translate that into :P I just thought it was "scrapper".
It means to scrape information off a website. (Like to scrape paint off of wood before re-painting it)
nuzecast
2009-04-07, 19:58
I use the allmusic scraper which has good and consistent artist/album info. As for the thumbnails I haven't found a good site, so I did all of them myself (folder.jpg in a /path/artist/album directory setup). Perhaps the owner of the home theater backdrops site would consider adding hi-res artist thumbs in addition to fanart. That site is already scrapable for fanart, so adding scraping of artist thumbs should be relatively straightforward (I think).
I've added a new category for square artist thumbs. 1080 x 1080 max.
I consolidated game backdrops and weather/photo backdrops under a Misc category as they are not used much.
Nothing will scrape this new category but since there aren't any images to scrape yet I guess that's OK :)
Yeah, if you put a "folder.jpg" in the root directory of an artist that will show up as the artist thumb/image. I found that a 600x600 image works fine. You could probably even get away with a 500x500.
Nazgulled
2009-04-07, 23:50
It means to scrape information off a website. (Like to scrape paint off of wood before re-painting it)
I know what it means, what I meant is that I don't know a Portuguese word that could translate to "scraper". The ones I could think of, don't make much sense in the context of "retrieving info off a website". But never mind about that...
@Arkon11
I wish the scraper could do that instead :P I don't really care for the artists thumbs to waste time with them but it would be nice to have everything cataloged. For now, I just disabled artists info...
I'm suprising a musicdb (like the moviedb and the tvdb) hasn't sprung up yet. I know I'd be more than willing to contribute if given the chance.
and i would gladly do the xbmc side
TeknoJnky
2009-04-08, 01:03
Like http://musicbrainz.org ?
musicbrainz does not hold much info, none at all about artist and just the basics about albums (which you get from tags anyways)
Ninjamawwe
2009-04-08, 09:57
@Ninjamawwe (http://xbmc.org/forum/member.php?u=25156)
I don't care about the albumart, that I prefer to do manually and I already have covers for all my albums. I'm only talking about artist thumbs and fanart, none of the scrappers seem to do a good job at it. I was just wondering if there was any other, or any other way for music...
Nazgulled, I am talking about artist thumbs, if you organize your music folders like I mentioned (/some/path/artist/album), and stick your artist image named as folder.jpg in the artist folder, xbmc will pick this as the artist thumb when scanning your library. Nuzecast was nice enough to add a artist thumb category to his site, now all we need is images and a friendly developer to make it scrapable.
Tariella
2009-04-08, 11:45
There is a database for video game soundtracks (http://vgmdb.net/). Something like that for all "other" music would be great.
Of course the cover art should be in a higher resolution. ;)
i just added htbackdrops.com artist thumbs to the allmusic scraper. for those who like ac/dc, alicia keys or some old guy i can't recall the name of; enjoy ;)
Nazgulled
2009-04-08, 16:10
Nazgulled, I am talking about artist thumbs, if you organize your music folders like I mentioned (/some/path/artist/album), and stick your artist image named as folder.jpg in the artist folder, xbmc will pick this as the artist thumb when scanning your library. Nuzecast was nice enough to add a artist thumb category to his site, now all we need is images and a friendly developer to make it scrapable.
But that's what I said... Only in a different way.
i just added htbackdrops.com artist thumbs to the allmusic scraper. for those who like ac/dc, alicia keys or some old guy i can't recall the name of; enjoy ;)
Thanks, that's a welcome addition. Too bad there's not enough artists thumbs though.
well duh, that is something you can HELP out with mate
@Nazgulled : Yeah I know what you mean, it's VERY tedious finding decent resolution images for artist thumbs. I have about 900 artists in my library, and I've been working all week to find decent images, rezizing them, and touching them up.
Anyone know where I could upload these, and maybe save other people some time?
Nazgulled
2009-04-08, 18:31
Yeah, I don't have that many artists, but still... I don't even have my music itself organized and I'm not going to look for artists thumbs when my own music is all messy on my hard drive. But I don't have time, at the moment, to organize my music, so...
Anyway, you can do that in the htbackdrops.com website, they have a music artist thumb category and as spiff said, the allmusic scraper now supports that :)
frumpy_uk
2009-04-08, 18:56
How about lastfm (www.lastfm.com) and/or discogs (www.discogs.com)? They have pretty wide selection of artist images.
I imagine you guys are aware of these sites (especially lastfm) and if they have already been discounted then please ignore.
Nazgulled
2009-04-08, 19:01
Last time I used the last.fm scraper, the artist images download were of very low quality/resolution. That's why I created this thread initially, cause none of the scrapers downloaded good quality pictures for the artists (at least 500x500 in my standards).
True, I agree that the last.fm scraper provide way to low rez images for my tastes. I'll start uploading those folder pics sometime this week.
Anyhow you think that 500x500 are satisfactory? I've found that it's pretty hard to find images where I can fit the entire music group into a 1x1 ratio image. It turns out quite a bit of the images for bands/artists out there are in like 2:1 ratio, which sucks and involves me doing alot of searching an eventual cropping.
the last.fm scraper and the discogs.com scraper wasn't hint enough that we have 'considered' it? ;P
Nazgulled
2009-04-08, 19:43
Anyhow you think that 500x500 are satisfactory? I've found that it's pretty hard to find images where I can fit the entire music group into a 1x1 ratio image. It turns out quite a bit of the images for bands/artists out there are in like 2:1 ratio, which sucks and involves me doing alot of searching an eventual cropping.
I based that value on my music covers... iTunes use 600x600 for music covers and I have some of them in that resolution but mostly, I have 500x500 and it looks good on a 1920x1080 screen on Aeon or PM3.HD (and probably all other skins).
The way Aeon or PM3.HD are made, it look good to have cover art or music artists in 1:1 aspect ratio. At least on Aeon it will be square anyway, even if it doesn't fit, Aeon will resize it (mantaining the aspect ratio). This doesn't happen on PM3.HD. I'm not explaining myself well but maybe you know what I mean...
That's why I recommended 1:1 oe 500x500, on Aeon, it would look good and be enough for 1920x1080. Maybe if the thumbs you got are higher, maybe you could upload them at 600x600. I don't think that more than that is need. This is artists thumbs we are talking about, not fanart.
But this is just my opinion.
Yeah I have, and always have planned on using 1x1 ratio images for artists. I primarily use Aeon (especially since Stark has been released) and I've noticed that a 500x500 icon looks decent as long as the source image is clear. If I find any that are great quality I'll put them up as 600x600, but as far as I'm concerned theres no need to upload huge 1000x1000 images. That'll only increase the file size, loading time, and would just be overkill.
(oh yeah, and I knew we weren't talking about fanart :cool: )
Nazgulled
2009-04-08, 20:13
If I find any that are great quality I'll put them up as 600x600, but as far as I'm concerned theres no need to upload huge 1000x1000 images. That'll only increase the file size, loading time, and would just be overkill.
Exactly :nod:.
(oh yeah, and I knew we weren't talking about fanart :cool: )
It was just to make sure everyone was on the same page :rofl:
i just added htbackdrops.com artist thumbs to the allmusic scraper. for those who like ac/dc, alicia keys or some old guy i can't recall the name of; enjoy ;)
spiff, if you are dealing with allmusic scraper. Could you please have a look at this please?
http://xbmc.org/trac/ticket/6144
Thanks!
i'm already cc'd on the ticket. that means 'whenever i/we have the time'
i'm already cc'd on the ticket. that means 'whenever i/we have the time'
Yeah, okay.
I hope you don't resent me for asking this...
not you personally no. but in general i don't like such nags ;)
not you personally no. but in general i don't like such nags ;)
Yes, I understand that. But you always say, that we(users) should "catch" you on the forum and try to convince and explain about new features/ bugs and get some attention to tickets.
It was a soft try. :cool:
indeed. which is why i didnt go by my nickname ;P
Nazgulled
2009-04-11, 20:53
Yeah, if you put a "folder.jpg" in the root directory of an artist that will show up as the artist thumb/image. I found that a 600x600 image works fine. You could probably even get away with a 500x500.
I tried this but it doesn't work for me... The allmusic scrapper does not seem to look into that file, can you take a look at that spiff, please? When I tried, it downloaded an artist thumb from htbackdrops.com without checking for a local "folder.jpg". (I have my folder structure correctly setup as "/Music/Artist/Album")
Also, it didn't work in file view the same way it works for album covers. When if file mode, the "folder.jpg" for an album is automatically picked up and displayed for that album. As per the quation above, I though this would work for artists too, but it seems it doesn't...
Or am I missing something here because it seems to be working for Arkon11?
Alright, lets see if I can help.
1.First of all make sure you're running a fairly recent build of XBMC.
Ok, to try this out, I went and deleted my userdata and my cache folder (under users/yourname/app data/roaming/xbmc.
2.So I'm starting from a blank slate. What I did was go and add my music folder in file mode.
3. On your music folder, hit "c" and select "scan to library"
4. Now go into library mode, and select "artists" and press "c".
5. Now choose "change scraper"
6. Make sure you select "discogs" and make sure you turn on "automated scan".
It'll now scan your libary and download artist thumbs, but if you have a "folder.jpg" in your artist directory
(for example: I have a folder called MGMT, and inside I have a folder called "Oracular Spectacular" and a jpg called "folder.jpg")
When it scans your library and downloads stuff from discogs, the thumbnail in your folder should take priority over anything that it's downloaded. The same should go for a fanart image "fanart.jpg"
hope that helped man.
Nazgulled
2009-04-13, 05:04
I don't like discogs, I prefer allmusic or last.fm... Allmusic is my preference though, it scraps more things as far as I've seen (on my library at least).
Maybe that's the problem? I can't test it right now to see if the problem is the scraper or not...
I doubt it's the scrapper thats the issue. XBMC (as far as I know, should always choose local thumbs/fanart over downloading new ones.)
If you don't mind the trouble, try deleting your cache, and userdata folders.
Nazgulled
2009-04-14, 00:21
OMG, I'm so stupid... Nevermind, stupid, stupid me... It is working...
I keep my music in sync (with the same folder structure) with my laptop and the XBMC machine and I was doing the changes (adding a folder.jpg to a artist folder) in my laptop and not on the HTPC. I mixed this because I use my laptop to do all the necessary changes on the HTPC through network drives.
I can't believe I mixed this...
Sorry!
hahah hilarious. Don't worry about it, but I did get a decent laugh :-p.
Hope you've got everything figured out.
Nazgulled
2009-04-14, 02:18
Yeah, everything's working now... I just need time to download proper thumbs for all my music artists :P
I'll get around to uploading some. I've just been crazy busy lately, and i'm going down to Texas for some school stuff for a week :(
Have u guys thought about the method of "Freevo"?
"Unless you want to bother signing up for an Amazon.com associate developer account (or have one already), comment out this line to stop freevo moaning at you on the console:
plugin.activate(’audio.coversearch’)"
I used it back in the days with Freevo, and got all the covers nicely....
How can i add a new scraper for music?
Thanks.
michael1t
2009-10-30, 01:52
Hello spiff,
I wanted to learn about Your music scraper I am a programmer
and would like to review the code ...
If this possible and if so could you post a link so I can go Learn more
so I might be able to help in the future ?
thanks in advance ...
Michael T.