rodercot

While I scratch my head on this one, I'm going to throw in an experience. When first setting up NFS, performance was slow and unreliable. Turned out to be a switch. So the lesson learned is to try different network gear/config when things are flaky. May or may not help, but you might try eliminating or replacing any switches and/or cables between your core and NAS.

Roy,

You know I would think of something like that as well. If I had not changed the board in that server this past weekend to allow for more drives at a later date and I also changed the switch from a 5 to 8 port linksys which I know works great. I also have and can stream all the files from the server no problems with any other setup.

I did run this command after doing some research.

mysqlcheck -u root -p pluto_media --auto-repair and it returned all OK's for the whole list. I also ran that command on pluto_main and it returned all ok's as well. this was run from root@dcerouter in a ssh window (putty).

I take it this still does not eliminate the table or one table could be corrupted causing a problem I guess. I am not sure a) if CHECK TABLE works with this db properly or not. b) I am not sure of the syntax for proper use of that command either.

I am wondering if just deleting the proper db or tables and having it rebuild itself would be the wise thing to try here. but I guess that with a clean install we pretty much covered that.

I have a fresh MD sitting on the table ready to install as well which I am very anxious to do, but would like to get this issue fixed first. On that note. Do we still need to run the setup diskless md script from Webadmin prior to booting up the MD with 710RC2. I read somewhere that you did not have to with this version and I want to make sure I do it properly. Sorry did not want to get off topic.

Dave

« Last Edit: July 08, 2008, 12:46:16 am by rodercot »

Logged

rodercot

I finally ran top while running a drive scan (resync) from webadmin and then a ssh window running in the background and the most cpu usage I saw was 22% once it hit 34% for a very split second again I am not sure if this is the proper way to run Top.

I did run a drive resync on my 250gb Music disc which are all flac files and the resync took almost 1.5 hours to complete but it did not freeze anywhere it just kept trucking along at a very snail's pace.

Now one thing I notice is there are a lot of Windows Media Player files in those music directories that are not in the database as I guess LMCE does not recognize them they are files like album art large.jpg and album art small.jpg there are no spaces in the titles I am just typing them that way but in some directories there maybe five of these orphans. Just guessing at anything at this point for with this not fixed it taked me three times as long to update a movie. I have to go into FAMS (Files and Media Sync) get the cover art - which FAMS does ok, then I delete some attributes it deletes them right away and then in windows explorer i just get the busy world icon in the tool bar like it's seraching for a web-page. Then I have to close that session via closing explorer and open a new one log in again and then set the media sub-type hit update and the same thing happens it updates. ( I do not see this happen) then again close and open a new session to work on a new file.

Can I delete a drive at a time from the server to see which drive maybe causing the issue. If so, is that just a matter of deleting a particular child device under the server or can I just mark it ignore and then reload the router and try a resync that way.

Sorry if this is getting long, I really want to get this fixed so I can move forward with the install.

The 250GB of music isn't really relevant, only the number of music files. If you are talking about a couple thousand music files, then 1.5 hours is reasonable.

Correct - in a music folder, UpdateMedia is only looking for audio files. There is no reason to assume it would recognise another media players files (eg cover art). If you don't need it anymore then just delete it. If you do, just ignore it - it doesn't make any difference to LMCE.

As for the web admin issue - can I suggest that you use the Firefox browser actually from KDE desktop to eliminate any strange compatibility or network issues.

Logged

rodercot

Yes I have tried updating directly on the hybrid via kde in firefox with the same results. Do you have any idea if we can run a CHECK TABLE command on the db somehow or are there actual db queries for lmce as I cannot find them. I have read some not all of the mysql troubleshooting and faqs.

I am still wondering if I can just check ignore under a child device and then rescan to elminate a drive from the nas and try a rescan just to see if it makes a difference.

One other Question. Should this NAS be a low IP address device as it is set at 80.2 which I think I forced it to. I think I tried to change it but the system set it back to 80.2 when it found it the second time after fixing the mac address problem.

rgds,

Dave

« Last Edit: July 09, 2008, 01:52:13 pm by rodercot »

Logged

rodercot

I added a new movie today and decided to let it run out after amazon added the cover art I chose media sub-type Movies and file DVD , then I clicked Update and it took exactly 2.5 hours to refresh the media sync window to the point I could choose another file to update or another option in the Webadmin same session. This is from kde - firefox - internal lan.

rodercot

Quick thought. Can the core connect to the internet? Just guessing an excess of connection timeouts might be the culprit.

HTH,Roy

Roy,

Yes the Hybrid is what I am using can connect no problem to the internet. Everything is fine that way. How can I just delete the pluto_media db and or rebuild and does the ignore button under each child device move the shares to unknown devices. where I am lost is that this was all working 100% my refresh time was like 10-20 seconds on a whole disc resync which contains between 100-150 files per disk then the power outage of a 2 months ago. Then upon a reinstall I have the same resync slow down problem. Which totally makes me think there is a corrupt file in the db id3??? BUT Why is Disk10 working fine, IT was not in the server before the power outage.

Is there no way to rebuild the database from within LMCE. Any help from some mysql guru's out there. There are so many versions out there and none seem to match up with the one we have in this system and command line options seem to be different spanning different revs.

The only other thing. If I wanted to remove the server and reinstall it disk by disk. where is the dhcp table (folder - file) to edit so the server does not default to 80.2 this time as I said I forced it the last time.

I've been working on the Media Browser and Media Files Sync all week, so I am quite familiar with the code. Can you better explain the problem? Are you getting these hangs only if you try to change the Media Type from Media Files Sync? Please tell me all actions that are causing your system to hang.

A couple of other quick notes.. File paths are passed as UTF-8 encoding. Make sure your naming convention conforms to this.Also, when ever you change something from Media Files Sync (such as the media type you mentioned), the binary file UpdateMedia (located in /usr/pluto/bin) is run, then the page refreshed. It sounds to me like the problem is in UpdateMedia.. Maybe you should ssh to the core and try running it from the commandline to see if you get some output that could be useful for debugging...

rodercot

I've been working on the Media Browser and Media Files Sync all week, so I am quite familiar with the code. Can you better explain the problem? Are you getting these hangs only if you try to change the Media Type from Media Files Sync? Please tell me all actions that are causing your system to hang.

A couple of other quick notes.. File paths are passed as UTF-8 encoding. Make sure your naming convention conforms to this.Also, when ever you change something from Media Files Sync (such as the media type you mentioned), the binary file UpdateMedia (located in /usr/pluto/bin) is run, then the page refreshed. It sounds to me like the problem is in UpdateMedia.. Maybe you should ssh to the core and try running it from the commandline to see if you get some output that could be useful for debugging...

Well Yes! The problem only seems to happen when trying to update a media file within FAMS. (files and media sync) ok so I rip a movie to disk 6 lets say. open up media file sync and of course the file is not available yet as that disc has not yet been updated so I resync that whole disk (takes about 1hr to do this) now I have access to that new file. I click on the new file add the amazon cover art (works ok) now I have the file with the attributes and the cover art. I click on a few of the attributes I do not keep and hit delete selected and I get the problem here (no refresh but it does update the upon closing and opening a new session) I am now in a new session again I goto the new file and select it and choose the media type and file type and then select update. (same problem it does update the file but take 2.5hrs on average to refresh)

So really the problems are one very long resync on any disc or file in the db or being added to the db on disk 1 - 9 which were also on the old build off 710rc2 before the power outage. disk 10 works fine which was added to the same server while I was not running LMCE over the months or so. So I reinstalled LMCE a clean install then let it find my server and set it up with sh script. Let it run media update after 4-5 hrs it only scanned in Disk 1 (half of the files that were on it) and none of the other drives had been updated so I resync'd them all myself from the webadmin to speed up the Update_Media process.

I hope this explains it. as i side note. I have installed phpadmin and I just ran a check table on all tables in the Plut_Media db and I get all OK. I take it that elminates ANY corruption and the files names would be fine. I do have some dvd's stored on the server that do have spaces in the folder names. IE. The 40 year old virgin instead of The_40_year_old_virgin. Please do not tell me I have to change all these. It worked fine with those names before the power outage.

The other issue - IS it possible to somehow have a count of the files the system finds. IE. please forgive me here. In XBMC when you set up multipath shares in folder view at the bottom of the page it tell you the number of files in that whole share. so Ie I had 400 dvd folders in 4 drives shares it would say 400 files found at the bottom the page. So Now when I set those file up in Library mode i know if I am missing any movies as the same files in library number appears at the bottom of that page. Just wondering. It helps and the reason I ask. Is that I have my movies set to Movies-DVD, Movies-Music Videos-Movies-HD1080. But when I select Movies-ALL in the gui and count them up I know I am missing about 17-20 movies from that option. It took me a week to set them up as they currently are to go through files and media sync and find the 17-20 titles that either do not have the media sub types set or double cover arts installed would be a big deal and to scroll through all files in the gui and count them is not an option as you know all media is mixed in there.

I will try the update media as you suggest and post finding. Can you supply an example of an actual path to file is looking for please. Are you suggesting like Windows-Share disk 1/Movies1/Movie_Title_Folder/Video_TS

for a path, try /home/public/data/videos/..... all the way to your destination device.. for example, if I were to do it for my software raid device, my path would be:"/home/public/data/videos/Software Raid 5 [64]"

For those that don't know, the pluto_media database, like the other pluto_* databases, takes the HABTM (Has and Belongs To Many) database design pattern to whole new heights. There are multiple join tables, which depend on each other, creating a nested join situation, a mis-match in any of the keys in the join table, will create a rippling effect that causes subsequent entries into tables like Attributes, etc. to become more increasingly corrupt, which is why it is a good idea TO NOT FUCK WITH THE DATABASE MANUALLY.

While sqlCVS -v can fix some of the problems in the database, it only can fix problems that are easily identifiable such as null foreign keys, or references to foreign keys that no longer exist.

Thom, you just beat me to pointing out that the db check only verifies the db infrastructure and not the contents. I've played a little with a brute force content checker but it currently runs really slow. Oh, and on the HABTM design pattern, I wish it were, but unfortunately the join tables are not pure (they have fields in excess of two foreign keys), so it's more like a HABTM implemented with the Has Many design pattern.

Going back a few messages, if I understand correctly, UpdateMedia ran fine from the command line, the problem is just when updating from FAMS. If so, then the place to look is /var/log/apache2/error.log.