Any plans for reformating the Scraper XML format any time soon?

[Nicezia]

Only reason Why I ask is because i'm developing a scraper editor/tester for the current format of the XML...

Sneak Peek:
So far its awfully simple but foran amateur programmer behind the scenes its pretty complex.

I've already implemented all of the options in the class (minus the custom functions which I'm having a difficult time wrapping my headd around just how i want to go about implementing those, and so far nesting RegExp's is not exactly as complex as the actual XML allows)


But if I follow my todo list it should be able to create the simpler movie scrapers by Thursday or Friday...

So again my question is, do you foresee me having to relearn the structure of your scraper XML anytime in the next few months?

[Schenk2302]

Hi,

it seems you could read my mind. I'm really looking forward to your editor

Thanks in advance

Schenk

[vdrfan]

Cool. Nope, scrapers won't change except of regexp stuff, but the layout (backend) willl most likely stay the same. Any plans of making it for other platforms as well? Maybe MONO is the way to go.

[Nicezia]

Yeah, actually I've just made a copy of the class written in c++ considering my C++ isn't really up to par, the c++ version may be a little longer in the making...
but doing it in visual basic for now is giving me insight eough to be able to visualize how to port it to c++ (sans .NET). the only real problem i see so far in making it cross platform is
A) breaking my reliance on linq.... (but linq makes XML so easy....)
B) I have no experience in building a gui without visual studio designer (i am looking into information on x-windows programming though)



I'll look into cross platforming & particularly Mono as soon as i can get a working version out, first things first.

[rwparris2]

Looks cool!
I started writing a tester in python, but haven't gotten very far with it.

If you know any python maybe we could work together? I don't have any experience with c#/c++ so I wouldn't be much help there.

Also, IMO a GUI for writing scrapers isn't really necessary. It is nice, but being crossplatform is nicer Editing the scraper's XML directly isn't hard if you have a simple way to test it.

[spiff]

great stuff!

the scraper xml format will not change, however i will change the default to matching regular expressions case insensitive and add a tag to indicate sensitive matching (see #6262). also i plan some minor changes on the returned xml format, but that doesn't really matter for you.

the key to the nesting stuff and other functions is recursive code (see CIMDB::InternalGetDetails).

[Nicezia]

Originally Posted by rwparris2

Looks cool!
I started writing a tester in python, but haven't gotten very far with it.

If you know any python maybe we could work together? I don't have any experience with c#/c++ so I wouldn't be much help there.

Also, IMO a GUI for writing scrapers isn't really necessary. It is nice, but being crossplatform is nicer Editing the scraper's XML directly isn't hard if you have a simple way to test it.

Sure it isn't neccessary... but I started this because i got tired of making the tiny stupid mistakes, like leaving a semicolon off the end of an entity. First i wrote a little program to replace entities in a string of text. Then my thinking turned into it would be nice to be able to see what's going into what buffer as i'm building my sections, and then i figured why not take it all the way....

As for python, that's my next area of study, (thinking about making a python plugin for XBMC for libpurple so instant messaging can be integrated into XBMC)

Originally Posted by spiff

great stuff!

the scraper xml format will not change, however i will change the default to matching regular expressions case insensitive and add a tag to indicate sensitive matching (see #6262). also i plan some minor changes on the returned xml format, but that doesn't really matter for you.

the key to the nesting stuff and other functions is recursive code (see CIMDB::InternalGetDetails).

Cool I've already allowed for expanding to support new attributes (Assuming you plan to use the case insensitivity as a conditional attribute)

[rwparris2]

What are your thoughts on this thing working crossplatform? Will you be able to get away from linq? Can you make the gui and backend seperate so that it can be run from a command line?

I need to know whether to continue with my python implementation or not... linux & osx users could use a similar tool as well

[Nicezia]

Originally Posted by rwparris2

What are your thoughts on this thing working crossplatform? Will you be able to get away from linq? Can you make the gui and backend seperate so that it can be run from a command line?

I need to know whether to continue with my python implementation or not... linux & osx users could use a similar tool as well

Definately on the cross platforming, plan on making a console implementation of it as well (my initial reason for rewriting the class in C)

When it gets to that point however i might need some help as far as setting it up so that it can compile in different enviornments (makefiles and all that since i have never worked with compiler options before .. my only programming experience is with Visual Studio flavors... ) I'm going to switch the entire project to Mono as suggested as soon as i finish coding it in Visual Basic... seeing as it has Linux & Mac implementations that should solve the cross platforming problem effectively.

It would be shortsighted of me not to make it cross platform, since i actually use XBMC through my Linux box.

[Nicezia]

Originally Posted by spiff

the key to the nesting stuff and other functions is recursive code (see CIMDB::InternalGetDetails).

Thanks a million for that reference!!!! I should be able to implement nesting fully now