SingStar mode (Performous / UltraStar ) Karaoke Pitch Match feature built-into XBMC? [Archive]

El Piranna

2007-06-08, 02:16

Reading this topic (http://xbmc.org/forum/showthread.php?t=26764) it has come to my head a little idea: XBMC can be able to play karaoke CDs, but what about add a system similar to the one that UltraStar (http://en.wikipedia.org/wiki/Ultrastar) implements using Mp3s and text files? We could borrow the code from UltraStar proyect or from it C++ cousing UltraStar-NG so it would't be very dificult to get it. Yes, i know there is very few people that has Karaoke Revolution microphone or Live! headsets, and having Karaoke Revolution you would't want anymore, but i think it would be easy to get and very versalite. And, of course, later add the speech recognition and on-screen display systems of UltraStar-NG... The most dificult piece of port UltraStar is work with audio hardware on XBox, and this work is done with XBMC, so why not to add that feature on it? :)

spiff

2007-06-08, 09:24

patch welcome

Gamester17

2007-06-08, 09:59

FYI; XBMC do not support reading Karaoke CD+G subchannel, it does 'only' support MP3 plus CDG files (which can be ripped off Karaoke CD+G CDs using custum tools on your computer). This is because no one has implemented it in XBMC yet, (and it is limited to Xbox Samsung DVD-ROM or PC DVD-ROM drives hardware as only those can read subchannels, but with XBMC being ported to Linux and... :;):

Hopefully someone could implement it in a way so it does not bloat XBMC nor increase the memory footprint of the XBMC application, so IMHO, please do not try to implement this into XBMC if you can not do it without sacrificing too much RAM memory usage when not in activated.

I submitted suggestion to add true CD+G sub-channel support to the libcdio library once but no programmer was interested:
http://savannah.gnu.org/support/?104390

PS! Also see these other XBMC related karaoke suggestions:
http://xbmc.org/forum/showthread.php?t=181

Gamester17

2008-08-31, 21:01

I would like to request that UltraStar NG (USNG) UltraStar Next Generation be implemented directly into XBMC.

UltraStar NG (USNG) is an open source clone of SingStar (http://en.wikipedia.org/wiki/SingStar), the famous karaoke music game. UltraStar lets one or several players score points by singing along to a song or music video and match the pitch of the original song well. UltraStar displays lyrics as well as the correct notes similar to a piano roll. On top of the correct notes UltraStar displays the pitch recorded from the players. UltraStar allows several people to play simultaneously by connecting several microphones possibly to several sound cards. To add a song to UltraStar, a file with notes and lyrics is required, together with an audio file. Optionally a cover image, a backdrop image and a video (even music video) may be added to each song. A feature unique to USNG is drawing the singing pitch as a line on the screen, in addition to the notes. This allows the player to easily correct his singing pitch instead of having to guess which way to correct.

UltraStar NG is written in C++ with the graphics rendering based on OpenGL for high performance even on slower machines, with full effects. Background video playback is supported using FFmpeg, (same open source codec suit that Boxee uses). USNG currently is currently available for Linux and Mac OS X but the code uses SDL (Simple DirectMedia Layer) (http://www.libsdl.org), the same cross-platform multimedia library that XBMC uses, so hopefully porting to Microsoft Windows XP and Vista should not be too much trouble.

USNG features better pitch detection with fast fourier transform than the competing projects open source projects "Ultrastar" and "Ultrastar Deluxe" which use autocorrelation. Multiple simultaneous tones can be detected and separated by the FFT post-processing algorithm, but the game currently only uses this information in the practice screen, not in the singing screen.

http://ultrastar-ng.sourceforge.net/
http://ultrastar-ng.sourceforge.net/develop.html
http://sourceforge.net/projects/ultrastar-ng/
http://en.wikipedia.org/wiki/UltraStar

Any volunteer(s) to code this for XBMC? :grin:

El Piranna

2008-08-31, 22:33

Hahaha! Well, now i have the XBox very abandoned but i'm working in a Guitar Hero clone in C+` for it (i haven't tried it but at least it works in MacOS X, Windows XP and Linux!!! :D ) and i was thinking to add UltraStar support to it (voice + guitar = ... :) ), so maybe one day if i have time i could choose to work with this in first time since it was my idea... :D

One point: the most stable and complete code is from UltraStar Deluxe, i have tried the other clones and they sucks... :(

Gamester17

2008-08-31, 23:24

i'm working in a Guitar Hero clone in C+` for it (i haven't tried it but at least it works in MacOS X, Windows XP and Linux!!! :D ) and i was thinking to add UltraStar support to it (voice + guitar = ... :) ), so maybe one day if i have time i could choose to work with this in first time since it was my ideaAdd StepMania to that and you have yourself and all-in-one hit musical game, hehe :laugh:

One point: the most stable and complete code is from UltraStar Deluxe, i have tried the other clones and they sucks... :(UltraStar NG was the only C/C++ port of UltraStar that I could find ???

ultrabrutal

2008-09-01, 00:26

El Piranna, you are doing a Guitar Hero clone which could run on the old Xbox using packs like GH for Frets on Fire?
Wonder how well the Guitar works with an adapter. Guess I could fine mine and try it out

El Piranna

2008-09-01, 00:47

StepMania is the next logic step since all the three games are the same only with diferent input types :D I thinked about it but the problem is that maybe you could get a heart illnes if you play it... X-D Yes, i'm triying to make it run on the XBox using SDL and the Guitar Hero packs for Frets on Fire. In fact, i decided to make it since the port of Frets on Fire using the XBMC python engine was simple impossible. Now i'm adding new features and i have some several memory leaks (it doesn't use SDL directly, it use a custom interface to be able to use SDL, OpenGL, DirectX... on the fly) but some months ago when i did check the memory use i could be able to download it to 60 MB, so maybe it could be posible to run it finally on the XBox... As i told you, it runs on MacOS X, Windows (with some problems, gettimeofdate() doesn't exists but i have found a workaround that i have to try) and Linux. Ah, and i'm making a custom guitar with a XBox controller, too :)

And yes, UntraStar-NG is the only one C/C++ SingStar clon, but it has very little few features, i only say UltraStar Deluxe is the best and most complete i have found.

Now i am with exams at university and at free time i'm with a little personal project, but maybe i can be able at latest September to try something with this point... somebody else want to work in it?

El Piranna

2008-09-01, 22:24

El Piranna

2008-09-09, 23:57

Good news... UltraStar-NG is not the only one SingStar clone made on C/C++ :-P

http://sinatra.nu
http://www.canta-game.org

They seems to be very bizarre, but not very resource hungry too. At least now UltraStar is not the only one option to look about :-)

I was thinking about the problem of were to integrate it, and i think the best option is as an actual karaoke (CD+G) feature sub-option. There wouldn't be so much competition or song list but at least it could work. Now, i need to know two things: where in the source code is the karaoke config and the karaoke option itself. Any help?

Gamester17

2008-09-10, 09:28

I have a little question: this is something similar to the LinkBoks fork or the webserver or the ftp server, it's said, "binary plugins" (to call it something) using DLLs, but how it can be implemented? And how it can be done to not load ultil requested?

And more important that this... where should it be implemented, inside music (music plugin? is there something like this apart visualizations?) or as an independent button in home window?I am not a programmer myself but I think that it would probably be best if it was implemented using the visualization/screeensaver API (meaning as a DLL renamed to .vis) but for that to happen you would probably have to extend the API to include two-way communication, see:
http://xbmc.org/forum/showthread.php?t=137
and
http://xbmc.org/forum/showthread.php?t=7552
Also look at the code of other visualizations. Start a new thread in the development forums if you have any coding questions.

Good news... UltraStar-NG is not the only one SingStar clone made on C/C++ :-P

http://sinatra.nu
http://www.canta-game.org

They seems to be very bizarre, but not very resource hungry too. At least now UltraStar is not the only one option to look aboutSinatra looks to be written in the C programming-language so could be an alternative to UltraStar NG, but Canta is written in Python and is in as a while 3D GUI game so I suspect that would be harder to integrate into XBMC even though XBMC features an embedded python-engine.

El Piranna

2008-09-10, 12:49

I am not a programmer myself but I think that it would probably be best if it was implemented using the visualization/screeensaver API (meaning as a DLL renamed to .vis) but for that to happen you would probably have to extend the API to include two-way communication, see:
http://xbmc.org/forum/showthread.php?t=137
and
http://xbmc.org/forum/showthread.php?t=7552
Also look at the code of other visualizations. Start a new thread in the development forums if you have any coding questions.

Good links, thanks!!! :grin: It sounds really simple to implement, so as in about 3 hours i have my last exam i'll try to see it (at least, make a simple vis to now how it works and how to compile on linux :-P ). The only one thing is what i told you about it would be too much competitive (i have try with CD+G that it's implemented as a vis too and you must to launch it after start playing music) but at least the code would be implemented and later it could be put in another place more util (xbmc/bins/*.xapp as happened with screensavers? :rolleyes:)

Sinatra looks to be written in the C programming-language so could be an alternative to UltraStar NG, but Canta is written in Python and is in as a while 3D GUI game so I suspect that would be harder to integrate into XBMC even though XBMC features an embedded python-engine.

Sinatra seems to be very little but i don't know what format it uses, and it's not intended to be compatible with UltraStar clones, too. At least is an option is UltrStar-NG gets bloated. In any case, UltraStar-NG blog say it has new features on SVN so they are good news too :-D Canta i thougth it was made in C since the 3D engine. Python is a good option, but it seems to be more simple that Sinatra, if posible. I think I (we) should use UltraStar since it's the most known and maybe it's more updated than others.

succo

2009-01-10, 02:42

as far as i can see, sinatra txt files are just like ultrastar ones, so it should work with ultrastar songs too
i'm not under linux right now so can't say for sure if it plays ultrastar/performous files, but files have just the same syntax, so, again, it should

now that mp3+g code is in linuxbranch this would be a nice addendum, i think

AaronG85

2009-01-10, 06:23

i want this feature so bad as ive been using ultra star for ages

eilz

2009-01-30, 18:36

Could you run this using a unix build, say DSL then use WINE to run the application. Then use the XBOX microphones. Most people running Ultrastar said it run ok on Wine, but not tried XBMC via Unix build yet

Just a thought, but Ideally would be much better on XBMC itself, as its ab absolutely brill program, simple and easy, and anyone can make there own track words etc on it

oldnemesis

2009-02-10, 08:34

i want this feature so bad as ive been using ultra star for ages

Aaron, could you please advise how many of those songs are available to general public? So far I only found around ~20 songs total; if that's all we could find then it doesn't make any sense to implement such feature. If we're talking about several thousand of quality music, it's a different case.

frostwork

2009-02-10, 09:34

There are several places where you can download thousands
of song text files for ultrastar.
Licence wise it seems to be unclear if this is allowed, even if only the
text files are provided.
The by far most adavanced cross-platform karaoke game is
http://ultrastardx.sourceforge.net/
You can find on their homepage also several free to use music tracks
with ultrastar songtext files.
The game is written in Pascal
Build from svn for linux binary.

eilz

2009-02-10, 15:08

oldnemesis

2009-02-24, 06:50

Do a search for ultrastar DJPO

Got it. So far we're pretty close to implementing playback for those songs. The main thing we need now is input support.

tetsuo55

2009-02-24, 16:10

I don't know how i missed this before.

But this idea is awesome!

Some points:
-Full support for CD+G and CD-TEXT
-Full support for MP3+G, BIN, MIDI Karaoke (MID and KAR), KMA.

Similarly to the subject but not exactly the same
-Frets on Fire (Open Source Guitar Hero)
-StepMania (Open Source Dance Dance Revolution)

If all 3 could be added to XBMC in the suggested method it would be great.
Although none of the 3 would be as feature rich as the standalone application it does have some benefits:

You could choose either 1 of the 3 or a mix of the 3 or all 3 at the same time!

All we would need is the 3 add-on files on top of the MP3. (Singstar file, guitar hero file and DDR file)

oldnemesis

2009-02-24, 20:22

-Full support for CD+G and CD-TEXT

I'm not going to do it. There are several reasons for that, but to start let's say I do not have a single CD+G :)

-Full support for MP3+G, BIN, MIDI Karaoke (MID and KAR), KMA.

XBMC already supports MP3+G and MIDI. No reason to support BIN, and I have no idea how popular KMA is.

Similarly to the subject but not exactly the same
-Frets on Fire (Open Source Guitar Hero)
-StepMania (Open Source Dance Dance Revolution)

This functionality is unrelated to karaoke, and it's hard for me to see a reason to start integrating all existing games into XBMC. Why just not run Frets on Fire separately?

I'm not going to integrate the whole ultrastar game into XBMC either - just to enable karaoke playback. High scores, user profiles and competition modes belong to games and should be in games, not in the media center as it's hard to fit it into current concept.

oldnemesis

2009-07-06, 06:00

Added support for UltraStar karaoke text files. Video background is also supported. No pitch detection yet, but may be added in future. As stated before, scoring/profiles is not even planned.

El Piranna

2009-07-06, 10:35

Added support for UltraStar karaoke text files. Video background is also supported. No pitch detection yet, but may be added in future. As stated before, scoring/profiles is not even planned.

This are good news, i wanted the UltraStar text files support (i have a lot of songs :-P ). A little question: pitch detection is about to get the singer tone? It would be available for the XBox trunk too? How much difficult it would be to add scoring when pitch detection will be available? Maybe it would be an interesting project for me to start with XBMC code in a serious way...:rolleyes:

Gamester17

2009-07-06, 13:39

Voice masks on non-Xbox platforms (Linux/Mac/Windows) would also be very cool, maybe that could be possible via filters after phi2039 have finished with the initial Master Audio framework. You see the voice mask feature can not be fully ported from the Xbox version of XBMC as today the Xbox versions of XBMC uses the API built-into the XDK (Microsoft Xbox SDK) for the voice masks and microphone input, that is closed source code libraries.

Alternatively implement an other open source solution for it, (like for example LV2 API and plugins (http://lv2plug.in)). LV2 is the successor of LADSPA (Linux Audio Developer's Simple Plugin API) (http://www.ladspa.org).

A little question: pitch detection is about to get the singer tone? It would be available for the XBox trunk too? How much difficult it would be to add scoring when pitch detection will be available? Maybe it would be an interesting project for me to start with XBMC code in a serious way...:rolleyes:I guess for that you will need to integrate the full UltraStar NG (UltraStar Next Generation) or at least parts of it:

http://en.wikipedia.org/wiki/UltraStar#Performous_.2F_UltraStar_NGPerformous / UltraStar NG

UltraStar Next Generation (USNG; since version 0.3.0, Performous) is a similar game written in C++. USNG is currently available for Linux and Mac OS X. USNG features better pitch detection with fast fourier transform than UltraStar and UltraStar Deluxe which use autocorrelation. Multiple simultaneous tones can be detected and separated by the FFT post-processing algorithm, but the game currently only uses this information in the practice screen, not in the singing screen.

A feature unique to USNG is drawing the singing pitch as a line on the screen, in addition to the notes. This allows the player to easily correct their singing pitch instead of having to guess which way to correct.

USNG uses SDL (Simple DirectMedia Layer) cross-platform multimedia framework, and the graphics rendering is based on OpenGL for high performance even on slower machines, with full effects. Various background video formats are supported using ffmpeg.

http://performous.org
:oo:

El Piranna

2009-07-06, 13:51

I guess for that you will need to integrate the full UltraStar NG (UltraStar Next Generation) or at least parts of it:

http://en.wikipedia.org/wiki/UltraStar#Performous_.2F_UltraStar_NG
:oo:

Uh-oh... And i was thinking about a simple python script to show the song lines and the scores when the pitch would be available...:sniffle:

In any case, if UltraStar files are parsed, is should be easy to get an obscure-not-so-known API to parse the tone from that files so they could be shown one day in the future by a script and have a reference of the tone to sing it, without scores, isn't it?:oo:

steve1977

2009-07-06, 18:20

Added support for UltraStar karaoke text files. Video background is also supported. No pitch detection yet, but may be added in future. As stated before, scoring/profiles is not even planned.

What does this mean? Is anything Ultrastar related working as of now? This would really be amazing. Dont care about all the scoring, but to sing karaoke based on the Ultrastar-raw songs (including photo, text and audio) woudl be perfect!!!

oldnemesis

2009-07-15, 03:29

Voice masks on non-Xbox platforms (Linux/Mac/Windows) would also be very cool, maybe that could be possible via filters after phi2039 have finished with the initial Master Audio framework.

I agree, but I cannot do it. My knowledge of cross-platform input sound processing is pretty much minimal.

Once input processing is added (would also be useful for those with HD audio and no karaoke mixers), adding pitch detection should be fairly easy.

I guess for that you will need to integrate the full UltraStar NG (UltraStar Next Generation) or at least parts of it:

It is not cross-platform - looks like Linux only, so little use for us.

Also I do not see the point of integrating the whole game into XBMC, as it requires much different framework (score table, easy selectable user profiles and so on). For those who want to have game functionality it's much easier to launch the original game instead from XBMC.

oldnemesis

2009-07-15, 03:31

This are good news, i wanted the UltraStar text files support (i have a lot of songs :-P ). A little question: pitch detection is about to get the singer tone?

Not yet. Maybe in future.

It would be available for the XBox trunk too?

There is no platform-specific code, but I suggest asking XBox maintainer.

How much difficult it would be to add scoring when pitch detection will be available?

It is pretty difficult to make it looking 'nice'. At this moment karaoke functionality is just piggy-backing the audio player, so once the song is done, the control basically goes away from the karaoke module. It could be changed, but I'm not sure it worth it.

oldnemesis

2009-07-15, 03:32

What does this mean? Is anything Ultrastar related working as of now?

Only text and video. So far I wasn't successful trying to show the photo over the video.

El Piranna

2009-07-15, 09:27

It is not cross-platform - looks like Linux only, so little use for us.

Also I do not see the point of integrating the whole game into XBMC, as it requires much different framework (score table, easy selectable user profiles and so on). For those who want to have game functionality it's much easier to launch the original game instead from XBMC.

Well, integrate the full game is overbloated, i'm the first one to say this, but allow the interfaces so you can make a python script that "emulates" the game can be a good idea... I have been the other day thinking about the required C/C++ interfaces and it's only neccesary to get the song notes infro (so the parse it's not done twice, one in C/C++ and another in python) and the FFT so it can be quickier than in python (obviusly :p). For the other things, the XBMC python support is capable of the video overlaying, show the notes, manage the scores and more :-)

El Piranna

2009-07-15, 09:32

There is no platform-specific code, but I suggest asking XBox maintainer.

Who is the XBox maintainer? T3ch? I'm asking this because i use primarily the XBMC in my XBox and since i knew about the talks about UltraStar support i was waiting just to be able to play on the console... :p

I love python, so i could be able to make the game if i have the necesary interfaces (and the time :-P ). I love C++ too, but i was confused as a cow in a garage when i started to read the code (too much C, DirectX, DLL and Windows style code for a MacOS X and Linux game programmer guy... :p)

blittan

2009-07-15, 10:33

The one maintaining the xbox code is Arnova, T3CH is someone who releases builds of the code from svn.. He's not affilated with xbmc at all.

El Piranna

2009-07-15, 10:34

The one maintaining the xbox code is Arnova, T3CH is someone who releases builds of the code from svn.. He's not affilated with xbmc at all.

Ok, i'll send him a message in a break. Thanks for all!!! :-)

oldnemesis

2009-07-15, 12:02

Well, integrate the full game is overbloated, i'm the first one to say this, but allow the interfaces so you can make a python script that "emulates" the game can be a good idea...

Providing the Ultrastar-specific game interface (like real-time access to the input buffer and FFT routines - how many other scripts honestly would need it?) will be even more burdensome than integrating the game itself.

El Piranna

2009-07-15, 12:11

Providing the Ultrastar-specific game interface (like real-time access to the input buffer and FFT routines - how many other scripts honestly would need it?) will be even more burdensome than integrating the game itself.

LOL! It's true, the FFT routines i don't think they will be used for any script, i was talking about to have it in C for speed improvement (in a DLL maybe? just kidding... :p), but the rest of functions can be done in python. Look at Frets on Fire, it's written in python...

(P.D.: by the way, i tried to port Frets on Fire to XBMC but the code was a really mess, so i started a new one in C++. I'll have to see FoF Reloaded to see if the code is better...)

bidossessi

2009-07-15, 12:13

I believe the work being done on the games library is supposed to allow running "native" games. Ultrastar qualifies as a game imho, so why try to integrate it, and not rather put effort on making the games library, so as to be able to run ultrastar AND other native games (like wesnoth or nexuiz) in a clean and "xbox/ps3/whatever-way"?
That way, games become much more like plugins (albeit OS-dependant), and xbmc remains lean. If you want to run a game through xbmc, install it as you normally would, and "scrape it" with xbmc. No need to rewrite the game nor bloat XBMC. Of course that depends on the games library being made functional. But i honestly think it would be a waste of dev resources to go ahead with this idea.

@El Piranna: me disculpes, i'm not trying to bash your idea. but it seems to me that your C++ skill could be more efficiently used in contribution to XBMC by helping make a powerful games library (capable of running native/emulated games on any platform in a seamless manner) come to life quicker. This would scratch both yours and Leo2's itches, and make XBMC much more versatile and complete as a living room multimedia experiences.

El Piranna

2009-07-15, 12:17