Proposed changes to Watched status (video library)

[jmarshall]

Hi All,

I'm planning to look at enhancing and cleaning up the Watched / Unwatched status stuff in the video library.

The idea is basically to remove the "Watched / Unwatched / Show All" button completely, and simply show watched and unwatched via the watched overlay (or via UI conditional).

The watched/unwatched status is only useful (IMO) at the Titles and Episodes levels primarily, so having code in place to try and filter the higher level views (actors/genres etc.) complicates the code and doesn't really make all that much sense.

In the case of Movies and Music videos, you'd only get watched/unwatched status at the title level, and the title level would still display every movie or music video, but ofcourse you'll be able to see which ones you've watched as it'll be indicated via the watched/unwatched condition (overlay/text/however the skinner wishes to demonstrate it)

TV shows is a little more complicated as having the number of episodes that you still have to watch is useful to know at the show and season level (this is currently not available). So the proposal is to allow these levels to show both the episode count and the unwatched episode count.

In addition, it is proposed that instead of just storing a "true/false" field, we instead store a "lastplayed" field, and in addition perhaps a "playcount" as well. This way smartplaylists (suitably extended) could be used to grab all
unwatched episodes or movies, or all movies you haven't watched for 2 years and so on.

This is unlikely to be fully implemented for a while yet, so here is your opportunity to comment on the pros and cons of both the current system, and the proposed improvements. Any and all comments will be taken on board.

Cheers,
Jonathan

[agathorn]

I know you and I talked about this briefly last night, though we hadn't talked about removing the button and filter :p

I do agree though. I think in the end what matters most is the ability to be able to tell, easily, what is what.

The ability for that to link into Smart Playlists makes it even better and I definitely think this change would be for the good overall.

[skunkm0nkee]

I have to say I'm a fan of the watched/unwatched filter and make use of it all the time. I tend to keep stuff in the library after I've watched it but rely on it being hidden by default by having the filter only show unwatched stuff.

I know that it would be ideal to have the watched/unwatched filter available throughout (to hide TV Shows where there are no unwatched episodes for example) but if it's a choice between having no filtering or having filtering at the lowest level only (the current situation) then I'm happy with the current filtering situation. It certainly cuts down the amount of stuff I'd have to wade through looking for an unwatched overlay.

Off the top of my head I think the watched/unwatched flag is a field on the episode table (I'm thinking about this from the perspective of tv shows) in the video database, is this right? When navigating the library, the data that is displayed in the containers is retrieved via sql and at the moment because the watched/unwatched only comes in at the episodes level then it is ignored at the other levels? If this is the case then would it be possible to change the sql that retrieves the tv shows and the seasons so that it references the episode table as well? (If this is possible but you have problems with the sql then I'd be happy to write the sql code)

[agathorn]

I have to say I'm a fan of the watched/unwatched filter and make use of it all the time. I tend to keep stuff in the library after I've watched it but rely on it being hidden by default by having the filter only show unwatched stuff.

I know that it would be ideal to have the watched/unwatched filter available throughout (to hide TV Shows where there are no unwatched episodes for example) but if it's a choice between having no filtering or having filtering at the lowest level only (the current situation) then I'm happy with the current filtering situation. It certainly cuts down the amount of stuff I'd have to wade through looking for an unwatched overlay.

Off the top of my head I think the watched/unwatched flag is a field on the episode table (I'm thinking about this from the perspective of tv shows) in the video database, is this right? When navigating the library, the data that is displayed in the containers is retrieved via sql and at the moment because the watched/unwatched only comes in at the episodes level then it is ignored at the other levels? If this is the case then would it be possible to change the sql that retrieves the tv shows and the seasons so that it references the episode table as well? (If this is possible but you have problems with the sql then I'd be happy to write the sql code)

The change you propose is a possibility and we did discussed it.

With that said however, this latest proposal I think is more robust. First off bear in mind that with the status of an item available to the skin, it would be possible for the skin itself to show or not show the episodes. In a sense you would have the same functionality if you so desired it, with a lot more benefit.

This is what i'm suggesting:

An InfoBool, perhaps ListItem.Watched() which returns whether or not the item is watched/unwatched OR if a parent item it would return based on its contents. This could be used as a visibility flag by the skin and be much more flexbile than the current ListItem.Overlay Infolabel.
An Infolabel that returns the number of unwtached items under a parent
An Infolabel that returns the number of watched items under a parent

[skunkm0nkee]

Cool, if the functionality will still be there but transferred to the skin level then that sounds good to me :grin:

[Jeroen]

I agree too. Possibilities are gained and none are lost if I understand correctly.
So, would the last played and playcount be available in the music library as well? These would be a very welcome addition especially for my music.

Also on a related note, I think it would be a nice addition to have an "unwatched episodes" entry in the library, something like the "recently added episodes" entry we already have now.

[sho]

I agree this needs an overhaul.
I currently only use the checkmark* to indicate what has been watched and could not really care about the toggle options.
One of the reasons being that different members of the family have watched different files and not all remember to toggle the status.

*That is, before I lost my Library .... sniff .

Oh and playcount for music would be very nice :)

[timdog82001]

So, would the last played and playcount be available in the music library as well? These would be a very welcome addition especially for my music.


That would be awesome if so. I personally think it would be most beneficial for those with large TV collections, especially if you could reference them on a per show basis. I also had an idea a while back about having a sort of "calendar" section of all your most commonly watched shows, and clicking on any particular show would automatically launch either the most recent partially viewed episode, or if it was finished, then the next episode in the series. It would be great if this were able to help out in this regard, though I think it would still need other cbmc changes which probably won't come for some time. I'm probably dreaming anyway.

Anyway, to be honest, I may be one of the few, but I never really use the watched/unwatched filter so it makes no real difference to me in the end, but the proposed idea sounds good to me.

[jmarshall]

Just to clear some things up:

1. Playcount and lastplayed are ofcourse already available to music (probably not to the skin, but you can ofcourse use smartplaylists etc. with them).

2. The skin will NOT have control over what is visible and what isn't in the library views. ListItem.Watched (or whatever it is) can be used to change the state (colour/text/imagery etc), but cannot be used to hide the item completely, as all lists will be present in their entirety.

3. With that said, as smartplaylists can use it, then you can easily list episodes and movies that you haven't watched by setting up a smartplaylist to do it. Ideally the smartplaylist could return either at the show level or the episode level (season level IMO makes little sense).

4. The main reason I dislike the watched/unwatched stuff is the following user comment:

"I just played a movie in XBMC and then it deleted it!!!!"

(User had "show only unwatched" on but didn't know about it - this is quite hard to offer support on.)

Cheers,
Jonathan

[bripeace]

Just to clear some things up:

1. Playcount and lastplayed are ofcourse already available to music (probably not to the skin, but you can ofcourse use smartplaylists etc. with them).

2. The skin will NOT have control over what is visible and what isn't in the library views. ListItem.Watched (or whatever it is) can be used to change the state (colour/text/imagery etc), but cannot be used to hide the item completely, as all lists will be present in their entirety.

3. With that said, as smartplaylists can use it, then you can easily list episodes and movies that you haven't watched by setting up a smartplaylist to do it. Ideally the smartplaylist could return either at the show level or the episode level (season level IMO makes little sense).

4. The main reason I dislike the watched/unwatched stuff is the following user comment:

"I just played a movie in XBMC and then it deleted it!!!!"

(User had "show only unwatched" on but didn't know about it - this is quite hard to offer support on.)

Cheers,
Jonathan

Are you saying that all hidden stuff would no longer be hidden. And in addition to that we would have to specifically create playlists (and i sasume many of them if we don't want one global list) just to see whats Unwatched?

I personally always have unwatched on as it saves the trouble of going through episode by episode looking for where the checkmarks stop just to see what episode I'm on.

[jmarshall]

Correct, the hidden/unhidden will no longer be available. Simple reason is that this is a bad UI design, the result of which causes nasty stuff to be done code-wise.

Ofcourse, the functionality of that will be available with a single smartplaylist for TVshows and a single smartplaylist for movies. (eg "Titles, hide all unwatched" will be doable) so you won't be missing out on that particular feature.

Cheers,
Jonathan

[bripeace]

Correct, the hidden/unhidden will no longer be available. Simple reason is that this is a bad UI design, the result of which causes nasty stuff to be done code-wise.

Ofcourse, the functionality of that will be available with a single smartplaylist for TVshows and a single smartplaylist for movies. (eg "Titles, hide all unwatched" will be doable) so you won't be missing out on that particular feature.

Cheers,
Jonathan

So a single 'smartplaylist' which I dont' even know what that is will allow me to say browse to a show and hide all watched shows - displaying all the unwatched ones.

What i definitely do not want is just a global list of all unwatched shows or movies - considering I may have entires shows and season of unwatched junk and don't really need to scroll through all that just to get to the next episode of Deadwood I haven't seen for instance.

[jmarshall]

That's what I said, yes. A single smartplaylist could be used to return exactly what the "Titles" node of the tvshow library does as the moment, with the added bonus that it'll only return shows for which you have unwatched episodes.

Useful smartplaylists (such as this one) will be published, or possibly even included by default. They're very easy to setup, however - check out the music library for instance.

You can have a global list as well if you like, but as you say, it'd be fairly useless in most circumstances.

Cheers,
Jonathan

[bripeace]

That's what I said, yes. A single smartplaylist could be used to return exactly what the "Titles" node of the tvshow library does as the moment, with the added bonus that it'll only return shows for which you have unwatched episodes.

Useful smartplaylists (such as this one) will be published, or possibly even included by default. They're very easy to setup, however - check out the music library for instance.

You can have a global list as well if you like, but as you say, it'd be fairly useless in most circumstances.

Cheers,
Jonathan

Sounds awesome, sorry about my confusion!

[SandmanCL]

Reading through this thread has rougly covered some of the thoughts I had for a feature request. Since my idea isn't all that original I'll piggyback on the ideas here.

If xbmc provided a library toggle value for hiding watched items, the skinner could implement the feature with a button or menu through the UI and/or mapped to a key. This would allow the user to browse through TV shows and quickly hide or show any watched show/season/episode.

[jmarshall]

Did you read the thread?

I'm REMOVING said ability as it already exists and is IMO a horrible user interface.

[SandmanCL]

Sorry, I guess I've been too trigger happy on the 'Post Quick Reply' button today. I browsed this thread and saw some mention of what I wanted ("An InfoBool, perhaps ListItem.Watched() ") so I missed the real meaning of the first port.

I've been working on setting up the TV Library and got all gung ho about getting it to work right :grin:

[monti]

Would it be possible to maintain watched/unwatched status for videos that aren't scanned into the database?

I usually just use the file system to access my movies and tv series, and ~never use the library mode.

[J_K_M_A_N]

No. That has been covered before. They need the library to keep track of those kinds of things.

One question though...why the heck wouldn't you use the library? You say you want to keep track of watched and unwatched so it sounds like you aren't deleting the shows after you watch them. The library is one of the greatest things about XBMC. I would guess if you try it you would be hooked. (Just my personal opinion)

J_K_M_A_N

[monti]

No. That has been covered before. They need the library to keep track of those kinds of things.

That's a shame :(

One question though...why the heck wouldn't you use the library? You say you want to keep track of watched and unwatched so it sounds like you aren't deleting the shows after you watch them. The library is one of the greatest things about XBMC. I would guess if you try it you would be hooked. (Just my personal opinion)

Well, the times I've tried using it, I've been disappointed:
- The scanning process has mistaken the first double episode of a serie as a single episode -- and visa versa on another serie.
- The process is really too slow for me.

I'm sure the library is great, but I still prefer plain file system. It's fast and it's hassle free. Except for the watched/unwatched issue, that is. :P

[agathorn]

That's a shame :(



Well, the times I've tried using it, I've been disappointed:
- The scanning process has mistaken the first double episode of a serie as a single episode -- and visa versa on another serie.

Probably because thats the way the episode(s) actually are as far as production went. This doesn't affect your files in any way though.
[/quote]

- The process is really too slow for me.


spiff made a lot of enhancements very recently that sped up library scraping about 300%

[monti]

Probably because thats the way the episode(s) actually are as far as production went. This doesn't affect your files in any way though.
Well, in both cases the whole season of episodes ended up with the wrong episode numbers and details. Fairly annoying, especially since the episodes had clean and easily readable/parseable file names and numbers to begin with.
spiff made a lot of enhancements very recently that sped up library scraping about 300%
That's good to hear!

[ForgotMyUsername]

Referring back to the original post, although I've read the thread:

This is unlikely to be fully implemented for a while yet, so here is your opportunity to comment on the pros and cons of both the current system, and the proposed improvements. Any and all comments will be taken on board.

For me, completely removing the watched/unwatched button and the functionality to hide watched items at all levels would be a major setback and make the library useless. To show you why, I'm going to explain how I use the library at the moment and why the watched/unwatched button is essential for me.

My library currently holds 1881 movies. This said you can imagine that some parts of the library are unusable here. Title view e.g. takes 2:40 Min until it is loaded. If you then press Info for one of the movies in the list, and go back to the list, you have to wait 2:40 Min again. Scrolling the list to see a movie starting with the letter "T" will take some minutes, too. So the titleview is hardly usable (as is actors, this takes ages to load, last time I tried it, I turned the box off after 8 minutes waiting).

However, I can use Year or Genre as these are much smaller lists. Year isn't a problem at all, Genre is more difficult, BUT lists get a lot smaller as soon as the watched/unwatched button comes into play. Many of my Genre-Lists get reduced from 180 to 34 objects when I switch to unwatched only which speeds up things a lot.

When I am alone, I usually have the button set to "unwatched" when I'm deciding what to watch that evening. I then go to the Genres and pick a Genre I feel like watching. I also often use the year as I'm collecting movies but often don't want to watch old b/w stuff or silent movies on a Saturday evening (or I might want to watch exactly that).

When I have guests, I switch to "watched", ask my guests what kind of movies they like and switch to Genres accordingly. Even if I could use the title-view, this is a lot better because it reduces the number of possible movies. When "watched" is set, I can comment on any movies my guests might want more information about. All other movies will not come up in any list and I never have to tell them I don't know anything about them.

I noticed that most of the people who commented before referred to TV Shows. I don't use the database for the few TV Shows I have, so this might be useful for that (although I don't really understand why you need watched/unwatched for this, I usually start with episode 1 and after that I watch episode 2...).

If I understood correctly what you intend to to, I would have to create tons of smartplaylists, staring with "Action/Watched", "Action/Unwatched, "Adventure/Watched" .... then going on to "1922/Watched", "1922/Unwatched" until I finally come to "2007/Unwatched".

Hope you can understand why for people with large collections the proposed change would make the whole library virtually unusable. Isn't there any possibility to preserve the current functionality and add new possibilities anyway? Also, a setting for the "Watched/Unwatched-Button" could be added to AdvancedSettings and switching it off by default to prevent new users from thinking they deleted an entry by watching it.

ForgotMyUsername

[monti]

I noticed that most of the people who commented before referred to TV Shows. I don't use the database for the few TV Shows I have, so this might be useful for that (although I don't really understand why you need watched/unwatched for this, I usually start with episode 1 and after that I watch episode 2...).
If you watch an episode or two per week, and the tv serie season has 22 episodes, it's fairly difficult to keep track of what you watched the last time. Was it episode 7? or 9? The watched/unwatched status is great for keeping track of this.

[jmarshall]

Some comments:

1. First off, thanks for your comments. I'm after beta testers for this very feature, and you sound like an ideal candidate - can you run XBMC_PC on a win32 box?

2. If titles is taking 2:40 then IMO that is too damn long. I shall see if I can speed it up a bit - music songs with over 4k items only takes a few seconds, albeit the amount of information it's retrieving is less. Do you happen to be using profiles, as that slows things down a fair bit? A debuglog will tell you the actual times taken for the various portions (query, retrieval, formatting).

3. The plan is to eventually allow the entire structure of the library to be XML-ized. I've already done smartplaylists for episodes, shows, musicvids and movies, so that takes care of the "title" level views. I have not yet decided how to do the other entries.

4. Further on 3, it may well be possible to have the genre et. al. options filterable by watched status (as it is at the moment) - they would be separate categories though "Watched Genres / Unwatched Genres". I'm not sure whether the watched status (overlay) will be available at these levels or not - it does complicate the queries somewhat.

5. It may be smarter to offer a more general filtering mechanism that is applied after folder retrieval. This gives essentially the watched/unwatched button but generalizes it to allow you to filter out whatever you like (eg filter by rating as well as watched status for instance). I'm not sure how well it would work from a UI perspective though, and I suspect it may overly complicate things. Perhaps the allowed filters could be XML-ized and available via a dropdown list to allow them to be edited for advanced users.

Please consider the beta testing. I have had no responses to it as of yet.

Cheers,
Jonathan

[ForgotMyUsername]

Great, it really sounds as if there is a chance to improve things. Really was afraid that functionality might be lost after so many years of steady improvement.

1. I have already read your post concerning beta testers. The problem is that I cannot compile, and I've never tried XBMC_PC (except for some minor changes to the skin). So I think I wouldn't really be of any use. I've been using XBMP/XBMC since 2002, but only use it on my local boxes in the living-room, bedroom and guests' room. The only thing I can do is to test the latest t3ch-builds, but I'm afraid that has nothing to do with helping you, really. Anyhow, as soon as I might find some spare time, I'll try to dig into compiling/XBMC_PC deeper. Perhaps I then might be able to contribute.

2. First of all, I don't use any profiles, never even tried it. The problem with loading title-view is not that it takes so long, but that xbmc has to reformat the whole list after you return from the info-window. That is what slows things down so much and makes it unusable. There seems to be no sort of "cached" view, that is accessed but xbmc seems to rebuild the list from scratch, every time it is accessed. This can also be noted if I switch from watched to unwatched etc. It just takes ages. I'm going to do a debuglog for titles/actors/studios etc. tomorrow or tonight (I just returned from watching a movie with a friend and the projector needs to cool down before I switch it on again).

4. This sounds like a good idea to preserve current functionality. With Genres this should be easy to implement. It might also be a good idea to have smartplaylists with playlists like "Movies ABC" "Movies DEF" "Movies GHI" etc. I've already structured my files on File-Level according to this scheme as it reduces scrolling times and entries in the listview to an acceptable degree. You've seen my db lately, so I hope you know what I'm talking about because it's a bit difficult to explain the principle. For me this was the easiest way to organize the amount of data collected over the years. As the general tendency seems to be that there are more and more collections like mine, I think it's worth thinking about how to handle large amounts of db-content in acceptable time.
Preserving the possibility to browse watched/unwatched movies by year seems to be harder to implement (and I really love to "watch a movie from 2007 tonight"). Have you already found a solution for that already? Of course you could do playslists for each year starting from 1900. For my collection this would be nice as there will be entries from 1922-2007, but I think there are many users out there who appreciate that only years appear in the list which you actually have movies from. So the smartplaylists with zero entries should not be shown. Is this possible?

5. This sounds like a fantastic future for the database, but I think this really is something that might be interesting in the remote future. The Actors-view also is a great idea, but I think nobody uses it except for testing-purposes as it's just too slow. If it's possible to implement this with acceptable speed I would be glad to filter all movies with a rating below 7.0 on some evenings :-)

If there is anything else than contributing ideas and thoughts, please let me know.

ForgotMyUsername

[ForgotMyUsername]

Here's the log (boot, go to library, get title list, get info for a movie, return to title list, get info for another one, get directors list, get actors [crashed]).. things seem to have sped up a bit since I last tried it, but it's still a pain...

http://pastebin.com/m4313b2d3

[jmarshall]

Thanks for the log - it helps pinpoint some timings:

1.4 seconds to do titles query
28 seconds (this is the first bit to focus on reducing) to retrieve information from the db -> fileitem.

Interestingly, there's also as least 10 seconds to retreive the cached list, which I'm certain can be reduced significantly.

Actors failed due to out of memory - I need to redo this to fetch in chunks, as this is one list that's gonna be very large.

BTW: For beta testing this new stuff, you don't necessarily need to be able to compile. All you need is time to test using XBMC_PC.

Cheers,
Jonathan

[ForgotMyUsername]

Fantastic, sounds like there might really be hope for a noticeable speedup.

Tried XBMC_PC in the meantime and after some problems I got it to work. I can access my library and everything seems to be fine, but I cannot get access to any smb: shares (no matter if I use name or ip or set a default user in the network settings). Although it's not really enlightening, here's a log http://pastebin.com/m5ea90d09

Perhaps you could tell me what exactly XBMC_PC must be able to do so that I can help you (I might be a bit stubborn but I don't really understand what sense it makes as of course on this machine, sorting/accessing lists ist really fast, but I hope you're going to tell me).

ForgotMyUsername

[Clumsy]

ForgotMyUsername: Do you have the xbmc_pc build from the t3ch package, or from the sourceforge page here (being the sdl version) ? I have the sdl version working with smb shares here, so maybe that is the better option. If so, try these steps:

1. extract the t3ch build
2. install http://sourceforge.net/project/showfiles.php?group_id=52340 - the runtime setup.exe
3. Download: https://sourceforge.net/project/showfiles.php?group_id=67586 - the win32 zip file, and extract the glew32.dll from the bin folder in the zip to the xbmc dir.
4. Get the latest win32 sdl build: http://downloads.sourceforge.net/xbmc/XBMC_win32_12563.zip?modtime=1207603787&big_mirror=0
and extract the .exe to your xbmc dir.

Folder structure looks like this:



20.04.2008 01:08 <DIR> .
20.04.2008 01:08 <DIR> ..
20.04.2008 01:08 0 bla.txt
20.04.2008 00:07 <DIR> cache
20.04.2008 00:02 <DIR> credits
06.04.2008 15:35 7.761.920 default.xbe
27.12.2007 22:10 212.992 glew32.dll
20.04.2008 00:02 <DIR> language
20.04.2008 00:02 <DIR> media
20.04.2008 00:07 <DIR> plugins
20.04.2008 00:02 <DIR> screensavers
20.04.2008 00:02 <DIR> scripts
20.04.2008 00:02 <DIR> skin
20.04.2008 00:03 <DIR> sounds
20.04.2008 01:03 <DIR> system
20.04.2008 00:12 <DIR> UserData
20.04.2008 00:04 <DIR> visualisations
20.04.2008 00:04 <DIR> web
07.04.2008 22:58 9.003.008 XBMC.exe
20.04.2008 01:01 0 xbmc.log
20.04.2008 00:29 339.602 xbmc.old.log


Then add a network share, the browsing of the workgroup doesn't work for me, but what works is to add the smb share manually by just entering the ip address.

[ForgotMyUsername]

hotzenpl0tz: Thanks for your help. I found a how-to in the forums and have both, the build from the t3ch package and the sdl version. I already installed the sdl-runtimes and glew32.dll, but it's the same with both exes, I can't access the shares, no matter wether I use name or ip, as stated above. I manually created the cache directory as I saw this in the log posted above. But this seems to be unimportant.

Just to make this clear, I'm not interested in using xbmc on my pc, I just want to be able to contribute something and help jmarshall with speeding up the library. So it's not important that I can play movies etc. the only problem at the moment is that I don't really know what I need to be able to help. Perhaps it's enough to be able to use the library. However, it might be helpful if I could scan my shares. I'll just wait and see.

jmarshall: Just tried Actors in XBMC_PC, it also crashes here, so there seems to be a real problem. All other queries work like a charm....

ForgotMyUsername

[jmarshall]

You need to use the DirectX version - I will provide it for you when it's ready to be tested in detail (I still need to do a little bit of stuff over the next couple of days).

SMB shares don't work, but that's not really much of an issue as you can just mount your shares directly on win32 (or they're already there).

The speed will obviously be far increased, but what I'm really after is indications as to what's missing UI wise for the smartplaylists + watched/unwatched tagging and so on. Unfortunately I can't test everything myself - I inevitably will miss things.

I'll PM you the link to a build when it's ready to be tested.

Cheers,
Jonathan