I would like to present an idea I have been mulling over for video manager. See what others think, especially Jbernardis since it is HIS program :-)

It's a great idea, but unfortunately I don't have the bandwidth to tackle that right now. Maybe for vidmgr 2.0

Of course the downside of a flat file system and relying on metadata is that the PyTivo NPL representation would lose its structure. I'm like you - I have my films all divided into directories by genre, but I also still use the PyTivo pull interface too and I like to see them consistent with one another.

I no longer use the pytivo pull interface at all since all my video is stored as mp4 and would have to be transcoded to transfer.

My biggest problem with filing by genre is that so many movies belong in more than one category and when I go looking for one....I always am looking in the wrong one. Perhaps using a browse interface for searching is the wrong approach.

Have you seen any easy way to gather the movie meta data into a single file? Getting all of it to a simple spreadsheet would at least allow me an index to keep handy. Suppose I could relearn visual basic and do it in excel.

__________________
Current : Roamio Base with 2TB drive and 2 Premieres and a mini. OTA. kmttg, pyTivo, running with a 78TB Synology 1511 NAS....serving up the world.

Setup help for pytivo under windows: To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Anyway, I am back to running all this under windows. The files are all on the NAS. If you think you can give me the scripts and a bit of direction, I'll give them a go under windows. What language are they in?

Bash. I know there are at least a couple of bash interpreters for Windows out there. I have no idea how well they work or whether they support any of the newer bash forms. A better solution might be Take Command.

My biggest problem with filing by genre is that so many movies belong in more than one category and when I go looking for one....I always am looking in the wrong one.

You may have lost me, there.

Quote:

Originally Posted by jcthorne

Perhaps using a browse interface for searching is the wrong approach.

If you are going to relegate the function to a PC, rather than the TiVo, then what I would do is write CGI scripts that relegate the functions to a web page. You can use Perl, PHP, or whatever to write the script, and arrange the database any way you like. Indeed, you could create the data base using whatever database manager you like. I'm doing that very thing here.

You will have to either run a web server on one of your machines, or else purchase a web page from one of the myriad web hosting services.

Quote:

Originally Posted by jcthorne

Have you seen any easy way to gather the movie meta data into a single file? Getting all of it to a simple spreadsheet would at least allow me an index to keep handy. Suppose I could relearn visual basic and do it in excel.

Collecting the relevant data into a single file is not difficult. 'Simply parse the metafiles and extract the fields into a database file. A tab-delimited file would work, or perhaps semicolon. Extracting a page for display filtering by genre is also not difficult.

Thanks for your usual very detailed response. I often learn quite a bit from them even if they only lead me in directions I had not considered before. That can be a good thing

Quote:

Originally Posted by lrhorer

You may have lost me, there.

By example, From Russia With Love belongs in each of the following Genre

Classics
Action Adventure
Thrillers

In my case its located in the Action & Adventure directory but I would just as likely go looking for it in the Thrillers or Classics sections. Symlinks could help with this but I still am not sure I can make this work effectively under windows. The basic problem being that windows does not keep track of the links. When the base file is moved, renamed or deleted, the symlinks do not get updated and now point to the wrong place.

__________________
Current : Roamio Base with 2TB drive and 2 Premieres and a mini. OTA. kmttg, pyTivo, running with a 78TB Synology 1511 NAS....serving up the world.

Setup help for pytivo under windows: To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

If you are going to relegate the function to a PC, rather than the TiVo, then what I would do is write CGI scripts that relegate the functions to a web page. You can use Perl, PHP, or whatever to write the script, and arrange the database any way you like. Indeed, you could create the data base using whatever database manager you like. I'm doing that very thing here.

You will have to either run a web server on one of your machines, or else purchase a web page from one of the myriad web hosting services.

Collecting the relevant data into a single file is not difficult. 'Simply parse the metafiles and extract the fields into a database file. A tab-delimited file would work, or perhaps semicolon. Extracting a page for display filtering by genre is also not difficult.

I would like the info presented to the user on the tivo, not on a pc. Away from the tivo, a simple spreadsheet database is sufficient. dlfl provided a simple utility to do just that:

By example, From Russia With Love belongs in each of the following Genre

Classics
Action Adventure
Thrillers

In my case its located in the Action & Adventure directory but I would just as likely go looking for it in the Thrillers or Classics sections.

OK, I see what you are saying. You only have it extant under one category. It was this very dilemma that prompted me to start down this path.

Quote:

Originally Posted by jcthorne

Symlinks could help with this but I still am not sure I can make this work effectively under windows.

It will definitely be more difficult than in Linux. Once again, Take Command may help. Cmd.exe is really awful.

Quote:

Originally Posted by jcthorne

The basic problem being that windows does not keep track of the links. When the base file is moved, renamed or deleted, the symlinks do not get updated and now point to the wrong place.

Linux does not, either. Hypothetically it could be possible for an OS to keep track of symlinks, but it's probably not practical. That means the app has to do it, or else the user. In the files I posted, the `Verify` routine is run manually to add a title to the list of verified movies (which means basically it has been edited to suit my tastes and has been pushed to a TiVo to make sure it transfers without error.) In addition to adding it to the verified list of movies, it also asks the user to classify the video by genre. All of this could fairly easily be automated to pull the info from the metafile, should one choose to do so, and it could be run once an hour, say. The Windows scheduler is quite pathetic, but it is sufficient for this task.

I have set up the `Verify` routine so the user does not have to enter the entire video name. Any part of it is sufficient to bring up a list of all videos whose names contain the string. Thus, for the video "Raiders of the Lost Ark (Recoded Sun Jan 07, 2004, HBOHD)" one need only enter the string "raid" or "ark". Of course, using as common a string as "ark" may bring up more than one video name. On my server, it brings up 14 different names. The user simply enters the number of the video in question from the list. Note coding this was very easy using the Linux command `find`. It will be more difficult under cmd.exe. Take Command may have a suitable function, I don't know.

Once the user has selected the video, the `Verify` routine cleans up all the temporary files that went into creating the video, and then it checks to see if the video is already listed as being verified. If not, it renames the video to eliminate leading articles ( "The" or "A" ) by calling the routine `Rename`.

Once this is all done, the `Verify` script calls the routine `Vlinks`, which creates all the links based upon the users selections. Again, a list of 25 genres is displayed for the user, and the user selects which ones apply. As I mentioned, this could be easily automated to pull the info from the metafile, requiring no user input at all. My biggest issue with this - and the reason I do not do it - is the genre data may be missing or IMO incorrect in the metafile. That, and the genre info in the metafiles encompasses more genre specifications than I care to consider.

At the very end of the `Verify` script, I also run the `GroupAlpha` routine, which creates a symlink to every video from a directory whose name name is equal to the first letter of the name of video (or "0-9" if the first character is a number). This share allows me to find a video by name in the NPL or vidmgr very quickly, despite having literally thousands of videos on the server. I also run the `GroupAlpha` routine under cron twice an hour.

If the user moves the file, the `Verify` routine will need to be run again. (Or if the user is not concerned about the verification aspect, one could simply run `Vlinks` directly on the fully qualified file name.) With little trouble, this could also be easily automated to run every hour or so if the `Vlinks` routine were modified to pull the info from the metadata.

Obviously, none of this takes care of any broken links caused by moving or deleting the file. To handle this, I have the routine `AuditDatabase`, which also runs twice an hour. It first checks for broken links, removing any it finds. Then it updates the database of movies with any links it finds that do not have any corresponding genre entry in the database (this functionality could be removed if the system obtains the genres from the metafiles). Finally, it creates any links that are missing according to the database.

I also manage the video system using the script `VideoScribe`. This typically runs automatically twice an hour, although I frequently run it manually. Among other things, it creates links for all videos that have not yet been verified in a share labeled "Unverified". That way I can easily tell which videos have not been verified, and when I select them to push to one of the TiVos, they are all collected in a single folder named "Unverified". It also calls the script `DateGroup`, which creates links to every video based upon its recording date (not the file create or modify time) in reverse order - newest first. This share allows me to quickly choose from among the videos which have been most recently recorded.

Personally, I would choose a much more portable method than this. It's fairly trivial to implement, even using cmd.exe. 'Just parse the metadata using the cmd.exe "find" command and spit it out to a delimited database file. Excel can read tab or semicolon delimited files.

Quote:

Originally Posted by jcthorne

My Synology box has a built in simple web page server so could to that but its not for the pc, I need it on the tivo.

Short of writing some HME or HMO code, links are the only way I know to handle this. They can be easily implemented directly on the Synology system, even with your TiVo servers running on a Windows PC, or they can be implemented atop SMB on the Windows PC. They don't haveto be symlinks, BTW. If they are on the NAS, then they can be hard links, which might make things easier for you. I chose symlinks because I needed the links to span physical volumes, which hard links cannot do.

Quote:

Originally Posted by jcthorne

Thanks for the scripts. I am studying them for ideas but at the moment don't see how to use them or similar under windows in an effective manner.

The algorythms for the most part will work under any script language, including cmd.exe. The issue is Windows does not provide the vast library of external functions like `find` and `select` that *nix does, nor does cmd.exe provide them internally. It's also vastly less powerful than any of the *nix shells, although it is possibly sufficient to the task at hand, which is not very complex.

It's probably simpler to implement them on the NAS, but certainly it is not impossible in Windows under a Windows based interpreter - even cmd.exe.

This seems a fundamental error, but I cannot get around it, please, anybody?

I can't say what's wrong here - this is the third executable line in the file. What version of python are you using? I assume it's not 3.x; vidmgr (and I believe pyhme) requires 2.7 or thereabouts - but hasn't been tested with 3.

I noticed that I didn't have an import statement for time - not sure if it's required - but you could try that - add an "import time" statement above the print statement. I can only assume that if it's necessary, then I was getting the definition of time from the "from hme import *" statement, and you and I are running different versions of pyhme (I'm running 0.18 BTW).

As far as the other issue you are having, vidmgr does not do the actual pushing - it makes an HTTP request to PyTivo to do this. Are you certain that you have your PyTivo information correct in your config file? The message you quoted "Exception ..." occurs when the URL Open request fails. This usually indicates a bad IP address or port for PyTivo, an incorrect TSN or an incorrect share name.

That's no guarantee that the IP addresses are correct. Vidmgr does not use pytivo to browse the files - it just goes to the directories themselves. The only time the ip addresses are used is to make the push request.

NOTE: This is the IP address of pytivo that you need to specify - not the IP address of the tivos themselves.

I spent some time this weekend getting an education on windows NTFS hardlinks and junction points. Also learning that when they are applied to entries on a NAS server emulating an NTFS file system, things can be a bit different.

I found a useful shell extension for windows file manager that greatly assists with the creation and management of hardlinks. It also shows which files have more than one link by modifying the icon in file manager. Works pretty slick. Moving the links or files does not destroy the links as long as they remain on the one volume.

Using the tools, I was able to move all my movies to an alpha storage system and then create links for a genere storage set.

The end result is that vidmgr now presents the user both alpha and genre menus. Was not as diffecult as I had thought once I determined to learn how they work.

Thanks to lrhorer and orangeboy for the pointers and info. I still do not have it automated but now that I understand the process, that seems almost possible.

__________________
Current : Roamio Base with 2TB drive and 2 Premieres and a mini. OTA. kmttg, pyTivo, running with a 78TB Synology 1511 NAS....serving up the world.

Setup help for pytivo under windows: To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

I found a useful shell extension for windows file manager that greatly assists with the creation and management of hardlinks. It also shows which files have more than one link by modifying the icon in file manager. Works pretty slick.

That's no guarantee that the IP addresses are correct. Vidmgr does not use pytivo to browse the files - it just goes to the directories themselves. The only time the ip addresses are used is to make the push request.

NOTE: This is the IP address of pytivo that you need to specify - not the IP address of the tivos themselves.

Regarding the time issue, I learned when I got home that I definitely need the import statement. Not sure why it was working for me (and others) but not for you, but the next release of the code will have the import statement. I hope to be releasing it in the next day or so - just doing some testing right now.

Anyone else see a problem where the wrong
Episode of a show transfers? It pushes,
but not the episode I select

Anyone else ever see that?

I never saw that happen, and nobody has reported it either. Do you have specifics that would help debug?

vidmgr does not know that it is an episode of a show - it's just a video file that exists in a directory with a bunch of others. The behavior of a collection of movies in a directory would be no different, so there is plenty of opportunity for this to have shown up - but nobody else has reported it.

Anyone else see a problem where the wrong
Episode of a show transfers? It pushes,
but not the episode I select

Maybe you have a disconnect between the filename of your metadata file and the actual content? Like, the file is named for Episode 2 but the text inside it is for Episode 3? So you see the Episode 3 text in vidmgr but the corresponding video file is actually Episode 2?

(not sure I explained that clearly, but hopefully you get the gist ...)

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

I have posted version 0.7c of the app to git. This release has a few changes that I've been working on with lrhorer over the past couple of weeks. Two major changes:
1) better logic for scanning titles in dvdvideo directories. I used to stop at gaps in the numbering sequence - not anymore - also titles are sorted more intelligently.
2) some changes in the way metadata is merged from multiple files. If this is not an issue to you, the default behavior will have no affect on you. However, there are a couple of new options for the config file that control how this is to be done. More information can be found in the changelog and readme files.

One other minor change - I added an import statement for the time module which apparently I was missing. Not sure how this hasn't surfaced before, but hopefully it's fixed now.

Don't forget the title numbering in the metafile starts with 0, but the first video on the disk is Title 1. Title 0 is the longest piece of contiguous video on the disk, regardless of the order on the disk. In some case, this content could be the same as #2, or #5, etc. Also, the "cover" video may not come before some other video. For example, I have a Robert Donat DVD whose cover video is The 39 Steps, that being the more famous of the two enclosed videos. There are two movies on the DVD, however, the other being The Ghost Goes West. As it happens, The Ghost Goes West is a few minutes longer than The 39 Steps, and the first videos is just a short clip, so Title 0 is The Ghost Goes West, Title 1 is nothing, Title 2 is The Ghost Goes West, and Title 3 is The 39 Steps. I have pyTivo and vidmgr ignore Titles 0 and 1.

Note also the order the videos are listed on the cover are not necessarily the order on the disk itself. I have several multi-episode DVDs whose Titles do not fall in the same order they are listed on the disk.

Maybe you have a disconnect between the filename of your metadata file and the actual content? Like, the file is named for Episode 2 but the text inside it is for Episode 3? So you see the Episode 3 text in vidmgr but the corresponding video file is actually Episode 2?

(not sure I explained that clearly, but hopefully you get the gist ...)

i'll retry this problem with a few more, and the one that failed to see if it was intermittent. The txt meta file matches the file that I selected and if I use pytivo to do the transfer, it picked up the correct episode.

Don't worry about message B - that is normal on the first startup - there is no cache yet.
...........

I just installed vidmgr -- very slick! It seems to be running OK but in addition to the message thomb mentions above, I also get this message:

"error opening thumbnail cache file for write"

Is this also normal, i.e., because there is no cache yet? The thumbnails seem to be working OK although I need to add them for most of my videos.

__________________
"It must be swell to be so perfect and odor-free" -- Del Griffith

VideoReDo users: Try To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. and To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

No that's not normal. The thumbnail cache is written into the vidmgr subdirectory from the current directory. Normally the current directory is the same directory as the pyhme python code, and it has a subdirectory named vidmgr (there the vidmgr code resides). This means that under normal situations, the cache is written into the same directory as the vidmgr python code. Perhaps you are either working in a different directory structure, or you have the directory write-protected?

If it can't write the cache, it will create a new one every time you restart. Not the end of the world, but you might lose a little response time until the cache is built up again

Thanks, I found the problem immediately -- I once again made the mistake of installing a program in the Program Files folder tree on Win7, where programs generally do not have write access to data files residing there. And where if you try to edit or replace a file (e.g., config.ini), it will most likely put the replacement file in a virtual folder with no clue to the user until the file still behaves like the old file.

I've moved my entire pyhme and vidmgr folder tree under a new folder I created call "programs" and I no longer get the error message and I can see the cache file that has been created in the vidmgr subfolder.

There are solutions such as placing data files in the %APPDATA% folder that avoid these issues but a simple solution is just don't put such programs in the Program Files folders. Maybe it would be good to put a warning about this in the readme (?).

__________________
"It must be swell to be so perfect and odor-free" -- Del Griffith

VideoReDo users: Try To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. and To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.