I haven't used Newsbin for a while, but decided to try again and upgraded to v6.80b9.I selected a lot of headers and added to download list. The files are added to the list and PAR2 files starts to download. After a minute or so, Newsbin crashes. I have tried to restart Newsbin but it crashes again as it is trying to download files.

DownloadsV2 - Download, wish failed lists. Downloaded.db3 - Files list. DownloadMarker.db3 - notes the hash of what files you download (it's what marks files as "Downloaded"). Might only delete this as a last resort or if it's really big.

Quade wrote:Me either. How much are you dumping to the download list at a time? Are they search or header based files?

About 7000 header based files.

I downgraded from 6.80b9 to 6.73 and continued with the current download. Newsbin doesn't crash now. Thus, there is something in 6.80b9 that causes Newsbin to crash on my system.

EDIT: Should I try to upgrade again (maybe to another beta? which?) and delete DownloadsV2.db3, Downloaded.db3 and maybe DownloadMarker.db3 ?

Alternately, and maybe better, would be to use 6.80b9 with a "clean" data folder. If I copy and modify paths in my current *.nbi file to an empty directory, would the other needed files be created when running Newsbin?

Alternately, and maybe better, would be to use 6.80b9 with a "clean" data folder. If I copy and modify paths in my current *.nbi file to an empty directory, would the other needed files be created when running Newsbin?

None of that should be necessary. Are you adding large files or small?

Basically I'm trying to characterize what you're doing so I can try to reproduce it. There's not really enough to go on at this point.

I know it's 7000 of some kind of files and I know it's headers. I might "Post Storage/Delete Stored Posts" on the group and re-download headers if you try B9 again.

Alternately, and maybe better, would be to use 6.80b9 with a "clean" data folder. If I copy and modify paths in my current *.nbi file to an empty directory, would the other needed files be created when running Newsbin?

None of that should be necessary. Are you adding large files or small?

Basically I'm trying to characterize what you're doing so I can try to reproduce it. There's not really enough to go on at this point.

I know it's 7000 of some kind of files and I know it's headers. I might "Post Storage/Delete Stored Posts" on the group and re-download headers if you try B9 again.

I am adding a mix of small and large (RAR) files.I have 6.73 and 6.80b9 as "portable" installations so that I can run either one.

I deleted the stored posts for the group and re-downloaded the headers (which took many, many hours...). I then selected the same ~7000 files to add to the download list (and bypassed filters, e.g. duplicate detection, since the files had already been downloaded). Crashed again with 6.80b9 (I had no problems with 6.73).

EDIT: After the crash, I restarted Newsbin, cleared the download list and added the files to the download list again. I also disabled AutoPAR to see if that somehow caused the crash. Now nothing is happening. Status is running, but nothing is being downloaded!? I enabled AutoPAR again, but still nothing is being downloaded.

Something must have gotten corrupted in the crash.

Last edited by ppan on Wed Oct 04, 2017 6:52 pm, edited 1 time in total.

Noticed that you are also using newshosting, and I am not getting any issues with b9. My previous issue with sticking gz files has not appeared, but this version sure takes a lot longer to process these files compared to previous builds before I get to the 200/200/0 sweet spot.

So I did one more test; deleted DownloadsV2.db3 and Downloaded.db3 again and also disabled Autopar. Now Newsbin didn't close down but it stopped downloading after a few minutes and the error log still showed crashes:

I restarted Newsbin after the latest crash. At this point, all the headers had already been added to the download list. The download chugged along for maybe 30 min, and then I got the errors again (AutoPAR disabled).

p.s. I have 75,434 files in my chunks folder. I thought v6.80b9 was supposed to reduce the number of chunks?

I added 20 TB to the download list, 20,000 items and notice Newsbin is using 13 gigs of RAM. I have enough RAM so, it's not much of a strain but if you're running a 32 bit version, you're probably running out of RAM. That would explain the mystery crashes too.

It's like demand loading is firing pre-maturely so every download list item has all the information is needs to download.

I doubt very much that the crashes are related to RAM. I am running 64 bits windows (and presumably 64 bit Newsbin). I have 8 GB of RAM. When I am running Newsbin, it may use up to 200,000 K but usually less. My physical memeory is then at 65-80%.Furthermore, I have no crashes with v.6.73 only with 8.80b9.

I have been running several test and I got it to work fine intermittently for a while. However, there is something else very strange going on with header downloads. I will post that separately.

I doubt very much that the crashes are related to RAM. I am running 64 bits windows (and presumably 64 bit Newsbin). I have 8 GB of RAM. When I am running Newsbin, it may use up to 200,000 K but usually less. My physical memeory is then at 65-80%.Furthermore, I have no crashes with v.6.73 only with 8.80b9.

I added 20,000 files and Newsbin was consuming 13 gigs of RAM when they finished being added. Well over the amount of physical ram you actually have. Since I have 64 gigs of RAM. My windows didn't much care about how much RAM I was using so, I got no crashing or any bad behavior whatsoever.

There's a good chance 6.73 doesn't load up the ram the same way 6.80 is. So, right now I'm assuming RAM till something else pops up. Your very strange symptoms suggest some major fault like running out of RAM. That's why everything starts crashing internally after you add a bunch of files. On restart, it won't use that much ram because of how it demand-loads the files.

I doubt very much that the crashes are related to RAM. I am running 64 bits windows (and presumably 64 bit Newsbin). I have 8 GB of RAM. When I am running Newsbin, it may use up to 200,000 K but usually less. My physical memeory is then at 65-80%.Furthermore, I have no crashes with v.6.73 only with 8.80b9.

I added 20,000 files and Newsbin was consuming 13 gigs of RAM when they finished being added. Well over the amount of physical ram you actually have. Since I have 64 gigs of RAM. My windows didn't much care about how much RAM I was using so, I got no crashing or any bad behavior whatsoever.

There's a good chance 6.73 doesn't load up the ram the same way 6.80 is. So, right now I'm assuming RAM till something else pops up. Your very strange symptoms suggest some major fault like running out of RAM. That's why everything starts crashing internally after you add a bunch of files. On restart, it won't use that much ram because of how it demand-loads the files.

I am sorry, but I don't understand. In my case, with 7,000 files, Newsbin was always using less than 200,000 K (and physical memory only at 65-80%). In this case, why would it be a RAM issue for me?

I am sorry, but I don't understand. In my case, with 7,000 files, Newsbin was always using less than 200,000 K (and physical memory only at 65-80%). In this case, why would it be a RAM issue for me?

As I said, on restart it won't use as much ram. It's when adding large numbers of files that the RAM use increases. Newsbin assumes that if you're adding files right now, it doesn't have to go into RAM savings mode so, it keeps the information about how to download the file in memory. On restart, this information remains in the DB until the download for the particular file starts.

It's not necessary that you understand this process. I just like to explain for the people who are interested.

I interpret your original comments as "If I add 7000 files in one go, it crashes during the process of adding". Not as "I have 7000 files in the download list and it's crashing".

I deleted DownloadsV2.db3 & Downloaded.db3I started Newsbin and added 7000 files and Neswsin started downloadingAfter several minutes Newsbin crashed.

I monitored Newsbin RAM during these steps, and it never exceeded 200,000 K.

p.s. In another test:I deleted DownloadsV2.db3 & Downloaded.db3I started Newsbin and added 7000 files while in pause mode. I then started downloading the files.One time it worked w/o a crash. Another time I think it crashed. I am still repeating these tests.

Sometimes Newsbin continues after the errors and sometimes it closes down and I need to restart. I wish that there a was some way to troubleshoot this (it is definitely not a memory issue; using < 70% physical RAM when crashing)

What exactly do these messages mean (what is Newsbin doing in each case?):

If it's 32 bits, anything over 2-3 gigs of ram use will run Newsbin out of RAM. 32 bits can't use more than 2 or 3 gigs. 64 bits doesn't have this limitation. Running a 32 bit version in 64 bit windows also has a hard RAM limit which will be less than total ram.

There's some reason only a couple of you seem to have this issue. B12 is coming close so we can only see if this continues.