New script "flexibrowser"

[Bernd]

hi everybody,
i was creating a script for team xbox game trailers/movies and thought:

"why do i have to clone the aqtbrowser over and over again.
wouldn't it be nice to seperate the gui part from the webpage parsing part?
so anybody can quickly create new browsers for new webpages?"

:idea:

so i decided to create a script with some kind of plug-ins in which you can browse the scripts in a filemanager-like fashion.

the script comes with some ready to go plug-ins including:
- team xbox gametrailers / movies (they have tons of xbox movies)
- channel9 (microsoft behind the scenes)
- gamespot (cloned from gamespot.py)
- alpha centauri (cloned from acbrowser.py)

these plug-ins also serve as samples on how to create your own plug-ins, which is quite easy since you basically only need to do the regex parsing.
see "flexibrowser.py" on instructions how to create your own plug-ins.

it has download support (press x to toggle)

comments and contributions (plug-ins and also on the gui side) are very welcome.

you can grab it at
http://www.xbmcscripts.com


please make sure you extract the zip including its folder structure! you should have a subfolder flexibrowser in scripts-folder.

have fun
bernd

[thor918]

why not send it to xbmcscripts.com
where you can upload it yourself.

[solexalex]

you can send it to me, i will upload it on my (french) website :
http://xbmc-scripts.gueux.be/

[Bernd]

dotspf has put it on his site!

flexibrowser.zip (http://pythonscripts.dnsalias.com/files/flexibrowser.zip)

thanks :bowdown:

@thor918:
i've registred there but the site doesn't allow me to upload anything. after loggin in, clicking on "submit content" (just as told in the faq) the webserver (not the mambo scripts) asks me for a login. tried my username and pw, but it doesn't work.
i'll try again tomorrow.

@solexalex:
your mail address isn't listed in your profile. tell me where to send my script to and i will do it.

[Asteron]

i just canceled that dialog and was able to proceed relatively fine... though the text edit box doesnt work in firefox...

also do not cancel after you upload the file.

[solexalex]

it is ok bernd !
i got it here. i will test it as soon as possible !
i'm on my script right now : mycalendar, it is about to be finished for a public version :)

[Freddo]

why not submit this to scriptrunner, that way people can more easily get hold of it.

[Asteron]

the scriptrunner guy (threezone) hasnt updated the database in a while and has been missing from the forums since late february :-/

it would be great if we could combine scriptrunner and xbmcscripts.com.

i really like the idea behind flexibrowser though.... are there any ideas for other sites which there should be flexibrowser plugins for?

[Bernd]

thanks asteron.

the next plug-ins on my personal wishlist are plug-in versions of gamespotplus, aqt and ifilm.
these are the scripts i personally use the most. all these are also very flexibrowser-suitable.

when these plug-ins will be available is just a question of spare time.

i would really apreciate it if some other pythonists would create plus-ins.
maybe i should create a howto thread in the python developers forum.
edit*: howto thread can be found here (http://www.xboxmediaplayer.de/cgi-bin/forums/ikonboard.pl?act=st&f=21&t=11927)

the basic idea behind flexibrowser was to create an open source like script:
everyone can contribute. all will benefit. :d

[Livin]

it would be great if the greatest scripting minds on the board could come up with a standard framework (as the idea of flexibrowser suggests) and then the entire community can try to "enforce" (through peer pressure) script writers to use that framework.

this 'should' create a standard that will make things easier to develop to as well as use for end-users.

thoughts?

[Bernd]

i've created a new plug-in for ebaumsworld

current plug-ins:
-ebaumsworld (new! lots of hillarious movies)
-channel9
-team xbox
-alphacentauri
-gamespot

bernd

[snowatom]

i can't seem to figure out how to cache insteas of downloading the whole file first.

i have tried change the 0 to 1 in the line below:

self.downloadandplay = 0 # play direct or download before play

but that didn't do me any good, is it possible ??

snowatom

[Bernd]

press the x button on the controller.
this will toggle stream/download.

if flexibrowser is in download-mode if you will see an '*' at the start of the path on the top of the screen.

it should default to stream when the script starts.

only the teamxbox plug-in will always download, since when i created that plug-in xbmc had problem playing the videos directly from the teamxbox site.

bernd

[snowatom]

thanx, that did the trick ;o)

snowatom

[madlobster]

i'd really like to see a script for archive.org. the site uses xml pretty heavily, so it should be a rather simple task compared to many other sites.

[Bernd]

*the site uses xml pretty heavily, so it should be a rather simple task compared to many other sites.
they have a rss feed. but it only contains the newest additions, like many other feeds.
or is there a place to get all movies of one archive in xml? haven't found one.

i'd really like to see a script for archive.org.
nice site though, lots of intresting movies

[madlobster]

the site uses xml pretty heavily, so it should be a rather simple task compared to many other sites.
they have a rss feed. but it only contains the newest additions, like many other feeds.
or is there a place to get all movies of one archive in xml? haven't found one.
unfortunately, i haven't been able to find one either.

perhaps the best solution would be to have the script an first run search ftp://moviesxx.archive.org (with xx being 02 to 14) for all *_meta.xml files and build a local database of the movies. then, on future uses, it would probably be best to update the database using the various rss feeds.


there are 13,878 movies here...

[RocafellaSqualie]

just grabbed up the new version of flexibrowser and for some reason it's not working on my xbox

i put all the files in a "flexibrowser" directory like the description stated, and even created a "flexitemp" & "temp" directory.

all i get is "running" for half a second then it dissappears :(

[Bernd]

i put all the files in a "flexibrowser" directory like the description stated, and even created a "flexitemp" & "temp" directory.
the directory structure should be as following:

{flexibrowser}
+--flexibrowsergui.py
+--cachedhttp.py
+--clientcookie.zip
+--{plugins}
** * *+--plug_teamxbox.py
* * * +--plug_gamefaqs.py
* * * +--plug_(other plugins).py

just as it is stored in the zip file.

all i get is "running" for half a second then it dissappears *:(
if you still experience probs please have a look at the xbmc logfile. it may help to pin down the problem. (remember to set the loglevel to 1 or 0 in xboxmediacenter.xml if you haven't already done so)

bernd

[morgoth123]

it doesn't work here either...i have the exact dir structure but i get some attribute error.
file "q:\scripts\flexibrowser\flexibrowsergui.py", line 96, in ? class flexibrowserhttp(cachedhttp.cachedhttp):
attributeerror:'module'object has no attribute 'cachedhttp'

[Bernd]

it seems the script cannot find the cachedhttp.py file.
is it in the same folder like the flexibrowsergui.py ?

[morgoth123]

yes it is so i really dont understand it..i take it case (in)sensitive has got nothing to do with it?

[Bernd]

yes it is so i really dont understand it..i take it case (in)sensitive has got nothing to do with it?
the case doesn't matter.

l would suggest the you do a fresh install:
-download the latest flexibrowser package from xbmcscripts.com
-extract the contents of the zipfile to a directory of your choice. make sure you get the right directory structure (see above)
-ftp the contents of that directory to the flexibrowser subdir in your scripts directory.

hope that helps
bernd

[joaomgcd]

hi! :)

what a great app, thanks for creating it! :d

i've created a plugin to stream videos of a portuguese games site, and it works really well, but i have a couple of technical questions...

- are you considering implementing image viewing? because the site in question has both images and videos for download...

- how do you handle videos with linebreaks in the url? for instance, when a the pages source code is like this
<a href="http://123.com/video.wmv
">video</a>
, i can't parse it correctly, and no video is found. i tried using the
data = data.replace('\n', '')
function but it just locks up the whole system, for no apparent reason, because i use it on another part of the code and it works fine.

- when you write

p = re.compile('<a\s+href="([^/w].*?)">(.*?)</a>', re.ignorecase or re.dotall)

for mov in movies:
m = gogameelement(mov[1], strbaseurl + mov[0])
how do i know what part of the regex is mov[0], mov[1] etc?... i know it works this way, but is it the "()" that define them?.. i hope i'm making myself clear... :p

well, thanks again for your nice little app :)

[Bernd]

hi! :)
what a great app, thanks for creating it! *:d

thanks i appreciate that. i'm happy you like it.


- are you considering implementing image viewing? because the site in question has both images and videos for download...

i'm currently working on displaying images so we can see all the screenshots of upcoming games.


- how do you handle videos with linebreaks in the url? for instance, when a the pages source code is like this
<a href="http://123.com/video.wmv
">video</a>
, i can't parse it correctly, and no video is found. i tried using the
data = data.replace('\n', '')
function but it just locks up the whole system, for no apparent reason, because i use it on another part of the code and it works fine.

normally the re.dotall should work, but i had also problems with it, so i tried the replace('\n', '') approach.
what do mean by "lock up"? does the script just stop responding, but ftp still works?
i was just trying to hunt down this kind of bug.
i will check this out on the weekend and post here if i find out anything.


- when you write

p = re.compile('<a\s+href="([^/w].*?)">(.*?)</a>', re.ignorecase or re.dotall)

for mov in movies:
m = gogameelement(mov[1], strbaseurl + mov[0])
how do i know what part of the regex is mov[0], mov[1] etc?... i know it works this way, but is it the "()" that define them?..

i use () to tag parts of the regexp. the mov[n] statement means the n-th tagged expression.
in the regexp '<a\s+href="([^/w].*?)">(.*?)</a>'
mov[0] is the first part ([^/w].*?) and
mov[1] is the second part (.*?).

you can use named tags (http://www.amk.ca/python/howto/regex/regex.html#section000740000000000000000) if you don't like the simple indexed version. you can check out that link for good examples on python regexps

if you have any other questions feel free to post them in flexi browser dev thread (http://www.xboxmediaplayer.de/cgi-bin/forums/ikonboard.pl?act=st&f=21&t=11927).

bernd

[joaomgcd]

hi again! thank you very much for the prompt response. :)

i'm currently working on displaying images so we can see all the screenshots of upcoming games.

very nice! :d that will be great. audio support would be nice too ;)

normally the re.dotall should work, but i had also problems with it, so i tried the replace('\n', '') approach.
what do mean by "lock up"? does the script just stop responding, but ftp still works?
i was just trying to hunt down this kind of bug.
i will check this out on the weekend and post here if i find out anything.

by "lock up", i mean the system really just stops responding to anything, and even the ftp server doesn't respond anymore. it's not like when your script has invalid code, it's really everything freezing. strange thing is, it never happened to me before using that expression. only if i include that line of code does it happen. oh, and sadly the re.dotall doesn't work either in those cases i mentioned...
let me know if you can find a solution. i'm only just learning python with this plugin so i'm really not familiar with it... :shifty:

i use () to tag parts of the regexp. the mov[n] statement means the n-th tagged expression.
in the regexp '<a\s+href="([^/w].*?)">(.*?)</a>'
mov[0] is the first part ([^/w].*?) and
mov[1] is the second part (.*?).
great, that is what i thought, thanks :)

if you have any other questions feel free to post them in flexi browser dev thread.
i'll post this in that thread too.

thanks again for your support!

edit: by the way, do you use any way to debug your code, so you can know where the problem is instead of having to guess what it is everytime?

edit2: if you want added internationalization you can add the lines

decoded = string.replace(decoded, 'ã', 'a')
decoded = string.replace(decoded, 'ç', 'c')

to your decodehtml function in the flexibrowser.py file, since xbox doesn't support ã or ç. :p

[BigMitch]

i have a bunch of csi movies on a computer downstairs and it runs mandrake with apache web server. i would like to watch them from my xbox with out having to ftp them up. here is what it looks like when you browse it for firefox. http://192.168.2.9:8002/mmedia/tv/csi-ny/ then that is the directory with the avi's.

can you guys help make a script do do this?

[Bernd]

i have a bunch of csi movies on a computer downstairs and it runs mandrake with apache web server. i would like to watch them from my xbox with out having to ftp them up.
why not use a ftp server. afaik recent builds of xbmc can access files on ftp servers directly.

or use samba on your server which will work with older builds of xbmc.

another simple option is to create a m3u playlist with direct links to your movies or your web server. check the manual on how to do this.

[Bernd]

hi,
i have a problem with the alpha centauri, its not showing me the video
i have noticed also that newer episodes don't play.
have you tried to download older episodes?
i haven't checked it, but it may be possible that new episodes use a newer version of real media, which cannot be played by xbmc.


its also not downloading any temp files to z:\ :(
first there must be an asterix (*) in the path when flexibrowser (press x to toggle). otherwise the flexibrowser will pass the url of the movie to xbmc, which will stream it directly
second the z: drive in xbmc-python is actually the x drive (at least in avalaunch the files show up in on the x: drive)

it may be possible that bronline changed their page layout so the url cannot be found. you can turn loglevel in xboxmediacenter.xml to 1 or lower and check your xbmc.log. you should see the url xbmc is trying to stream. check what the log says. is it a script error or xbmc error.

bernd

[Der Hesse]

hi,

no video from alpha centauri is working :(
also not all ebaum videos are running, when i pressing a its just making the click sound and nothing happens :(

i have the newest flexibrowser

[javadog]

hi,

no video from alpha centauri is working :(
also not all ebaum videos are running, when i pressing a its just making the click sound and nothing happens :(

i have the newest flexibrowser
same here, except with the ch9 plugin. seems that one a video is watched or started, it is difficult to rewatch it. i choose any of the videos that i've watched before, and all i get is the "click" sound. it doesn't attempt to play or re-download.

is there a cache somewhere that i can clear out?

i also get the total "hang" that joaomgcd mentioned. it will totally freeze the system to where even a 4 finger salute won't work. i have to manually restart (push the on/off button).

great app otherwise. great potential. keep up the good work.

[Bernd]

the alphacentauri issue seems to be some kind of xbmc playback problem.
when you select a movie you'll see the xbmc *messages scrolling through the progress dialog.
you can check the xbmc.log and see the error messages.

i don't think there is much i can do about it.
:verysad:

this issue should be posted in the "bug discussion" forum.

[javadog]

bernd,

sorry if i'm posting in the wrong section, but this section says its for scripts support. the "bug" secion seems to be for xbmc bugs and not scripts... am i wrong. if so, can you provide a link to where i am supposed to post?

the "bug" i am referring to does not even trigger the progress dialog to display. seems the only way i am able to download or stream is after i reboot (if this is not the first time i've attempted to download/stream a particular video). so, i go into the ch 9 group and select video x to play. it either downloads or stream. after that, i can choose another video, but if i select x again, all i'll get is a "click". no dialog. i can then navigate up and down the list, but going back out will the the plug-in selection screen (where you choose channel 9 or one of the other plug-ins) is not possible. freezes the entire system. if you want, i can write down exact steps to reproduce this.

since you are just a script running under xbmc, it very well could be an issue with that, but i wouldn't know since it is your script i am running. i saw you mention that you have a "gui" that is something somewhat of a different thread than the download plug-ins. in my armchair programmers assesment, it is almost as if the plug-in(s) get into a state where they no longer respond. hence, your gui allows you to select streams, but if the plug-in is busy or hung in the background, all it does is "click" and return... and, if a plugin is hung, going back a level to list the available plugins hangs your gui.

like i said, if i'm posting in the wrong place, i'm not sure where the "right" place would be for support on your excellent script. (i consider it and the flickr plugin to be showcases of the xbmc scriptin ability...really!) i wish i knew a little more about the scripting so that i could dig in and figure this out to help more. i would love to have persistence of the (x) download feature so that i don't have to remember to trigger it each time (perhaps that does work if i were ever successful at cleanly exiting the script?)

one other question (not a bug). when you choose to download, where are the videos downloaded to? can you directly access them with xmbc w/o going into fb? how long do they "stick around", and can they be deleted once watched?

(feel free to move this to a "correct" forum thread)

[Bernd]

-alpha centauri is a xbmc issue
can't help on that.

-the channel9 plug-in should work.
i tested it yesterday and it worked.

-ebaumsworld needs a small change due to page layout changes.
i already fixed that. for those of you wha cant't wait should change the reg-expr in plug_ebaum.py in ebaumvideoelement.geturl():
m = re.search('<a href="(http://media.ebaumsworld.com/.*?)">', data, re.ignorecase or re.dotall)
if (m != none):
*return m.group(1)


-some teamxbox trailers may not work.
they have added an age verification for m rated games.
i already fixed that.

all fixable changes will be released with the upcoming flexibrowser1.4 release.

bernd

[Bernd]

one other question (not a bug). when you choose to download, where are the videos downloaded to? can you directly access them with xmbc w/o going into fb? how long do they "stick around", and can they be deleted once watched?the downloaded videos are put in the cachedhttp cache. this usually resides on the x: drive in the folder ~httpcache.
x: is a temporaray drive. any xbox software started may delete all contents on x.
you cannot access this drive with xbmc directly. if you use avalaunch for example you should see this folder.

the flexibrowsergui script tries to delete all cached data if you exit via the back button. to be able to get to the downloaded movies you must perform a reset instead of pressing the back button. or you can comment out the following line in flexibrowsergui.py

self.http.cleancache()
by putting an # at the beginning of the line.

but you should be aware that watching/downloading many movies may then fill up your x: drive.

bernd

[zhugeliang]

something strange seems to be happening when i attempt to watch videos from teamxbox. i can get into where they display each movie for each game but when i try to select one, it doesn't do anything. then when i try to back out of that category then the entire box freezes. i have the latest version of the script from xbmc scripts. can anyone help me out with this?

[Bernd]

something strange seems to be happening when i attempt to watch videos from teamxbox. i can get into where they display each movie for each game but when i try to select one, it doesn't do anything. then when i try to back out of that category then the entire box freezes. i have the latest version of the script from xbmc scripts. can anyone help me out with this?
teamxbox slightly changed their site and thats why it doesn't work atm. see my post above :rolleyes:

i'm currently testing the my fixes before releasing it.
btw. in the next version you will be ablöe to watch teamxbox's screenshots :cool:

bernd

[thor918]

great job on spiked
:d
it crashed on one,
but i expect that is the usual mplayer crashing.
i hope they add some errorhandling in mplayer soon.
it's much more enjoyable to get an error message insted of the xbox freeze.

[Livin]

bernd,
a thought... what about making the modules directly accessible without needing to go through the flexibrowser menu to get to them.

i'm not a huge fan of menu systems and like direct access, this is why i put all my most used items on my main xbmc screen, almost never use the submenu.

since i would only use a few of the modules in flexibrowser, and many others may as well, how about making it so we can launch them from a bookmark?

[Bernd]

great job on spiked
:d
it crashed on one,
but i expect that is the usual mplayer crashing.
i hope they add some errorhandling in mplayer soon.
it's much more enjoyable to get an error message insted of the xbox freeze.
you can distinguish between flexibrowser and xbmc/mplayer crashed by the progress dialog.
if it hangs and the progress dialog has flexibrowser in the title bar it was my fault :lookaround:
if it has something different like open file, etc it is mplayer. :(
unfortenatly there is nothing i can do about mplayer bugs.

bernd

[Bernd]

bernd,
a thought... what about making the modules directly accessible without needing to go through the flexibrowser menu to get to them.

i'm not a huge fan of menu systems and like direct access, this is why i put all my most used items on my main xbmc screen, almost never use the submenu.

since i would only use a few of the modules in flexibrowser, and many others may as well, how about making it so we can launch them from a bookmark?
what do you mean by directly accessible?
do you want to flatten the hierachy completely?
this isn't such a good idea since many of the entries are generated dynamically. that means a webpage is retrieved, parsed and elements are generated.
if you flatten the hierachy it would mean that the script has fetch a lot of pages before it can display something. and this would take a long time before it could display anything.

bookmarks would be technically possible. but maybe i could just add a function to add a specific playable item to a pls playlist.

bernd

[thor918]

yep. it was like this.
selected one movie, no movie got played and tried another movie, then it froze.
so it was a mplayer crash.

[Livin]

bernd,
a thought... what about making the modules directly accessible without needing to go through the flexibrowser menu to get to them.

i'm not a huge fan of menu systems and like direct access, this is why i put all my most used items on my main xbmc screen, almost never use the submenu.

since i would only use a few of the modules in flexibrowser, and many others may as well, how about making it so we can launch them from a bookmark?
what do you mean by directly accessible?
do you want to flatten the hierachy completely?
this isn't such a good idea since many of the entries are generated dynamically. that means a webpage is retrieved, parsed and elements are generated.
if you flatten the hierachy it would mean that the script has fetch a lot of pages before it can display something. and this would take a long time before it could display anything.

bookmarks would be technically possible. but maybe i could just add a function to add a specific playable item to a pls playlist.

bernd
sorry, i reread my post and i'm not sure it was clear...

i mean being able to launch something directly like this...

<execute>q:\scripts\flexibrowser\flexibrowsergui.py comics</execute>

... basically calling the "function" within flexi directly

[thor918]

well that isn't possible to pass execute parameters as the api is now. so i will suggest shortcutscripts to be used from skin calls

<execute>q:\scripts\flexibrowser\shortcuts\comics.py</execute>

i posted a suggestion about python execute parameters to the sourceforge a long time ago, but non of the developers have taken up the idea.
https://sourceforge.net/tracker....=581841 (https://sourceforge.net/tracker/index.php?func=detail&aid=1157794&group_id=87054&atid=581841)
(i took the passed parameters a little further in that request.)

[pexians]

great script!

i'd like to see script for www.wwitv.com (http://www.wwitv.com) there is over 1000 tv-channels so xbox would become digibox.

[thor918]

great script!

i'd like to see script for www.wwitv.com (http://www.wwitv.com) there is over 1000 tv-channels so xbox would become digibox.
as i see it that site only list other sites that have strems.
that makes it impossible to make a script for that.

[pexians]

as i see it that site only list other sites that have strems.
that makes it impossible to make a script for that.

yes it lists streams, but also give links to streams. look sources or statusbar when over green "200k" image.

there is nasty javascript protection, but it can be passed turning javascript off when page is loaded.

sample page uk (contains all three formats):
http://wwitv.com/television/217.htm

country list:
http://wwitv.com/television/tvbar.htm

i think this can be done...

[Bernd]

i will check that out since the stuf looks promising.

but it will take a while since i'm soon going on vacation and will be back in the end of september.

so you gotta be patient. :cool:

[zhugeliang]

am i the only one here that is not able to play streams from teamxbox?

[Bernd]

am i the only one here that is not able to play streams from teamxbox?
tried it today and it works fine :d

bernd