View Full Version : [Request] new gen of AQTbrowser
not sure everyone knows, but i got it from a trustworthy source that there's a way to make aqt more bulletproof. the solution is quite genious.
here's the story: itunes has apple trailer support. it does this by an .xml file somewhere on their site. if we could sniff that xml out and parse it, we would be there. :saint:
i'm not a skilled python coder, in fact, i never done anything in python before. nevertheless, i'm gonna see if i can figure it out, but no promises whatsoever.
any takers? (skilled coders?)
from guybrush:
http://www.apple.com/moviesxml/h/index.xml
or
http://www.apple.com/moviesxml/h/view1_all.xml
(set your user agent to "itunes/4.5")
sneaky eh?, xml is perfect for these kind of things ;)
itunes uses these urls for trailers, so the format wont change or they will break their own application.
firefox plugin to change user agents:
https://update.mozilla.org/extensions/moreinfo.php?id=59&vid=617
great stuff... now is there a way to specify user agents in python?
if so this change would be relatively simple. im still a python newb myself.
yes there is, pretty ez actually
http://diveintopython.org/http_web_services/user_agent.html
nheinemans
2005-02-02, 14:23
:bump:
anyone interested in looking into this?
solexalex
2005-02-03, 21:12
yes
have a look in python script launchbrowser for example (from flash)
it uses http-header
sorry to be ot, but is there a way of increasing the amount of cache used by aqt ? currently i get about 10 secs of good fluid motion with large trailers before they jam up. medium quality is crap in comparison, so i'm sure a "x secs content" cache would work so much better... is this even a part of aqt tho, or does xbmc take over this bit and i can adjust this in the settings menu ?
sgtstadanko
2005-02-09, 22:29
i dont think its the cache...mine is hard locking the xbox. i increased the cache to 16mb and it played for lunger but still locked up.
sorry to be ot, but is there a way of increasing the amount of cache used by aqt ? currently i get about 10 secs of good fluid motion with large trailers before they jam up. medium quality is crap in comparison, so i'm sure a "x secs content" cache would work so much better... is this even a part of aqt tho, or does xbmc take over this bit and i can adjust this in the settings menu ?
hmm.. i think mine downloads the entire trailer before playing.. it take a while but the quality is awesome...
yeah i think flash modded it to have a download-and-play option available by editting the file.
as long as the current version is working we should keep aqt as is but if it breaks again than switching to this xml can be done then.
here's the xpaths to use to find the relevant information in the xml files from apple.com. i tested most of them using java/jdom to make sure i had the right values.
:kickass:
i'll probably change aqtbrowser to use those if i can find a usable xml/xpath lib for python.
index file: http://www.apple.com/moviesxml/h/index.xml
$base = /document/scrollview/matrixview/vboxview[1]/matrixview
exclusive: $base/vboxview[3]/vboxview[2]/matrixview/view/vboxview/matrixview/vboxview/hboxview/gotourl
url = ./@url
title = trim(./textview[1])
opening this week: $base/vboxview[3]/vboxview[@viewname='opening this week']/vboxview[2]/matrixview/view/vboxview/gotourl
url = ./@url
title = trim(./textview[1]/b)
studio = trim(./textview[2])
categories: $base/vboxview[1]/vboxview[4]/matrixview/view/vboxview/hboxview/textview/gotourl
url = ./@url
name = trim(./value())
movie studios: $base/vboxview[1]/vboxview[6]/matrixview/view/vboxview/hboxview/textview/gotourl
url = ./@url
name = trim(./value())
new movie trailers url: $base/vboxview[2]/view/matrixview/textview[2]/gotourl/@url
new movie trailers: http://www.apple.com/moviesxml/h/view1_all.xml
note: the following xpaths also apply to all trailers index files downloaded from the urls found in the xml files.
"x-y of z": trim(/document/scrollview/matrixview/matrixview/vboxview/hboxview[1]/textview/b)
note: use this to know if there is more page: if (z > y)
movies: /document/scrollview/matrixview/matrixview/vboxview/matrixview/hboxview/vboxview/matrixview
url= ./gotourl/@url
thumb= ./gotourl/pictureview/@url
width= ./gotourl/pictureview/@width
height= ./gotourl/pictureview/@height
title= trim(./vboxview/hboxview/textview/gotourl/b)
studio = trim(./vboxview/textview[1]/gotourl)
actors = trim(./vboxview/textview[2])
category = trim(./vboxview/textview[3]/gotourl)
rating = trim(./vboxview/textview[4])
movie trailer page: $url from trailers index xml
smallurl = /document/scrollview/matrixview/view/vboxview/hboxview/matrixview/hboxview/vboxview/hboxview/gotourl[1]/@url
mediumurl = /document/scrollview/matrixview/view/vboxview/hboxview/matrixview/hboxview/vboxview/hboxview/gotourl[2]/@url
largeurl = /document/scrollview/matrixview/view/vboxview/hboxview/matrixview/hboxview/vboxview/hboxview/gotourl[3]/@url
note: those are relative urls... need to resolve them to absolute urls using the current url path.
movie page: $smallurl or $mediumurl or $largeurl
movieurl = /document/scrollview/matrixview/view/vboxview/hboxview[2]/matrixview/movieview/@url
or //movieview/@url
just did an initial text with pyxml:
loading http://www.apple.com/moviesxml/h/index.xml alone takes 9 seconds. that includes only getting the file from the website and parsing it into an xml document.
if i get this down to an acceptable value, i'll continue the script. otherwise, we'll have to keep aqtbrowser the way it is.
jsvensso
2005-03-18, 16:54
is it 9 sec during startup of the script? i could live with that if it would secure the script from being outdated....
i did some basic parsing. i can now extract the exclusive movie trailers (6 of them on the index xml page) and find their respective url. some have one 3 urls (for small, medium and large), others have as many as 9 (3 formats x 2 teasers + 1 tv version). i'll try to show all the available movies, but apple don't include any intelligible text with those links, so i'll probably show the end of the urls (small.xml, fullscreen.xml, teaser/small.xml, etc.)
by using threads, and parsing the information in the right order, i think i will be able to get this script pretty usable.
should have a test version avail. in a couple of days.
*edit: now waiting for an answer to http://www.xboxmediaplayer.de/cgi-bin....t=11430 (http://www.xboxmediaplayer.de/cgi-bin/forums/ikonboard.pl?s=;act=st;f=21;t=11430) before this can work...
the xml processing is done and working.
i tried to migrate the original aqtbrowser.py to use it, and it's not working very well.
http://pub.abuserz.com/xmlqt.rar
copy the file + directory from lib/* to your xbmc/python/lib directory.
copy aqtbrowser directory and .py file to your xbmc/scripts directory.
it doesn't work very well (or even at all).
if you quit the script without doing anything, xbmc hangs.
if you try to play any trailer, it might kinda play, and will probably hang during play or if you try to play a second trailer.
i already took 4 days of my time on this. i probably won't invest any more time...
i don't know what the problem(s) is, so if anyone can fix it to make it usable, then simply post a working version when you're done.
maybe darkie, who i think implemented python into xbmc, could debug and see what's happening.
there is no thumbnails in this version, even if they are cached.
- mouton
message deleted.
ohh.i read your side post now.--never mind
oh c'mon... don't give up when you're so close :p
i might try again later when the xbmc python implementation has been changed; it freeze the whole xbox without giving reasons, the debug window (console) doesn't show the complete stdout, xmbc.player().play() behaves hereticaly, etc.
the python script i have works without problem on my mac, but it doesn't work on xbmc.
anyone who want to use my xbmcxmlqt.py 'library' is free to do so. i personally won't work anymore on this. wasted enough time as it is.
DarkVamp
2005-03-26, 11:55
hi,
i will cross fingers you will have the chance to complete your script... :kickass:
i tried another approach...
i thought i might be able to preload everything, save it in a data file, and have aqtbrowser load the movies from there.
my mac can load the whole 1026 trailers information in 50 secs. xbmc hangs at the 3rd or 4th category. i can't go see if anything appeared in the python console in xbmc, since xbmc itself is frozen. if the python console was logged to file as xbmc.log is, that would help.
xbmc also seems to have a problem when i try to load all the categories in parallel (using threading - enable that by using the flag at the top of the script). maybe it's the same problem and it hits the problem faster that way... who knows.
if an xbmc developer can debug my script and tell me why it makes xbmc hangs, or if someone can add python stdout output to a file on disk, like xbmc.log (and xmbc.old.log), then i'll continue working on this.
until then, i'll continue using aqtbrowser.py
test script for anyone who wants to try to help me:
http://pub.abuserz.com/xmlqt_preload_text_1.rar
the "xbmc" module has two functions
xbmc.log(string) --outputs to xbmc.log at log level 6
xbmc.output(string) --outputs to xbmc.log at log level 1
the log level is set in xboxmediacenter.xml...
-ast
i'm already using output for all my print()... but it doesn't help that much when xbmc just hangs.... appart from inserting an output() between each line of code, i can't really know what line is the problem... and even then, it doesn't tell me what the problem is...
no other python developer willing to lend a hand or two to mouton :o
i did use the 'insert-a-debug-line-between-each-line-of-code' technique and was finally able to make this work!
memoryerror was the problem apparently. i don't know how much memory python scripts can use on xbmc, but mine was apparently using too much. i was able to lower that by unlink()ing my dom documents when i'm done with them, and by limiting the number of threads running at the same time.
now that i'm able to download the actual data (which takes around 3-4 minutes to complete), i'll try to save it to a file and allow the user to trigger an update whenever he feels like it.
i though it would be useful to be able to mark movies as 'i-wanna-see'. dunno yet if python on xbmc has a way to easily select an item in a list, but at worst, i'll make a button 'mark last played trailer'. that way, i'll be able to watch trailers, mark the movies i wanna see, and refer to that list in the future.
bam! :thumbsup:
finally!!
todo:
- dialog visual (it's black now)
- ask whoever did the pm2 background3.png to remove the apple logo, since there's buttons there now. if you know who it is, please ask him for me and report back. :) thx!
- add the 'i-wanna-see' feature
- add update button
you can get it from http://www.xbmcscripts.com
enjoy!
really nice stuff!
a couple of comments
on the screen where you sometimes have multiple trailers for 1 movie, is it possible to get names for the diff trailers?
for fullscreen trailers, possible to add an (optional) download before start playing ? (flash's aqt has this)
also, this is offtopic, but now that you know how to do the xml parsing ;) there's also a musicvideo section on that site :saint:
on the screen where you sometimes have multiple trailers for 1 movie, is it possible to get names for the diff trailers?
sadly enough, not really. they're not anywhere in the xml data. itunes uses graphics and link the graphic to the trailer.
for fullscreen trailers, possible to add an (optional) download before start playing ? (flash's aqt has this)
i'll probably allow downloading of any trailer, probably by using a 'switch to download mode' button. selecting any trailer would then download it instead of playing it.
also, this is offtopic, but now that you know how to do the xml parsing ;) there's also a musicvideo section on that site :saint:
yeah, i know. should be pretty easy to do once i'm done with this one.
never thought the day would come, but this thread now belongs in development subforum
thx mouton.