Since SETI has been around for many years, nobody would of guessed that so many people wanted to be a part of it.
Maybe the number of users should be capped at a certain amount; and only allow new users as others uses drop out.
That way a maximum user number could be establised and the system could be tuned to that max user number.
It is like selling tickets to a concert, once the number of seats have been sold, no one else can get in unless someone drops out.
Just my 2 cents worth. Happy Crunching.....

The current limits of 100 for the cpu and 100 per gpu were introduced to protect the working database (BB2 in Richard's description) from overloading, which it is now suffering (or at least appears to be suffering from). Increasing these limits at this time would not help the situation, indeed might have undesirable effect of making things even worse. Thus there may be an argument for reducing these limits, and strictly enforcing them.Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Has anyone considered making the WU's longer in order to reduce the average number of transactions on the database? Figuratively, for example, making new WU's that contain 3x the data to reduce on the client than is normal today. A significant increase would be needed, I suspect.

And, is it possible to send such large WU's only to clients running GPU's? It seems absurd that GPU's should be treated the same as CPU's when sizing WU's or specific client compute tasks. Other projects do this, I think, e.g. PrimeGrid.

Has anyone considered making the WU's longer in order to reduce the average number of transactions on the database? Figuratively, for example, making new WU's that contain 3x the data to reduce on the client than is normal today. A significant increase would be needed, I suspect.

And, is it possible to send such large WU's only to clients running GPU's? It seems absurd that GPU's should be treated the same as CPU's when sizing WU's or specific client compute tasks. Other projects do this, I think, e.g. PrimeGrid.

This fixed MW in the past. 5 tasks were bundled together since they only took a few seconds on a 79xx AMD card. Now it has other DB issues.

...The tasks which make the database 'messy' will be the ones which hang around, and their wingmates - the errors, the failed validations, the ghosts, the users who walk away without clearing their caches...

Speaking of ghosts: Is anyone working on removing the 71 ghost MB 7.0 WU's? That might clear up some space, as not only the space for those WU's could be re-allocated, but once they are gone, the entire set of MB 7.0 executables could be deleted... These ghosts have been bothering me ever since the number of "in the field" 7.0 WU's went to zero - and I'm on one of the ghosts! (as in: I processed the WU, and a wingman also came in, but the WU never got deleted!...)

And as far as "Walking away without clearing the cache" goes: sometimes you have no choice about it - sometimes a computer will fail on you without warning, and you have to either buy or put together a new one, and let those WU's that were on the failed computer get picked up when they exceed deadline.... Hello, from Bangkok, Thailand!...

But on server-grade kit like this, they probably have access to multi-channel RAID controllers, and lots of drive bays, so my 'two database containers' suggestion doesn't mean two physical servers or even disk access contention.

Yep.
But the more storage pools there are, the greater the work of the server to manage them, and the more tweaking required to balance available RAM (for caching) between the pools (I've no idea what automatic resource allocation is available for such systems, nor what it's capabilities are if it exists).
The big advantage of multiple storage pools on a given server is that existing HDDs can be replaced with Flash storage so that at the very least indexes & logs can be placed on them, or if there is enough space in the server for the capacity necessary, one or more of the tables with the greatest I/O load could be moved to the Flash storage pool and save the HDDs for the not so busy tables.

Has Seti gone back to Hardware RAID controllers?
I recall they did have some, but when one died they didn't have any spares of the same model & version & so were unable to rebuild that particular array, hence Matt went for Software RAID. Cheaper and easier to deal with hardware failures, but certainly not the performance of dedicated hardware controllers.Grant
Darwin NT

Hello all of you.
I think everyone see the problems y differents ways. All of uou, experimented tester knwns much ore than me.
My only idea is only to stop producing work. Arecibo is out. So no new tasks at this time.
Stopping producing work will for sure be not apreiated to all of us. But.
A least, it will give time to server to absorbe all WU in queue.
Servers will be more able to give more power to all other application.
So I really thik, after a few days, all very late works will be resorbed, the bottlneck reduced.
Then staff can work more easily on DB.
DB try to work, but receive more and more WU ! It is never ending.
Why not to FULLY stop the project for a few days and so give fresh air to server to breaths (also to staff)
I think everyone become tired about all the problems.
We, normal cruncher as me, probably, we will lost CR and AVG due to longer outage. And then ????
We will not die !
It is my suggestion : stopping producing work. And let the time , the time to solve by self, with help from staff.
Sorry, I stay with my idea : stopping producing work, Disable up/down loas server. I know, some WU will be out of time.
Or perhaps, no any work sending to us, upload server active to let finish WU out in field, And then closing up/down
after one week. Then let the staff working. It need one week ? two weeks ? 3 weeks ? no problem.
As long staff give information.
Best regards from Belgium

I cannot understand that with so many millions of donations in the last few years technically hardly anything has happened. In any case, not really positively noticeable. It was purchased and that (what you can find and read on the internet about the purchases at SETI) and sometimes the software improves from 7.8.x to 7.9.x but with SETI only a “handful of people work” ... - where is all the money there? Already consumed everything?

And if there are so many problems with hardware and software - what do they do all day? Look at the monitors and wait for the big WOW-two signal to come? The database problem and hardware difficulties have existed for months. It looks like Eric has to do it all by himself.

Maybe it would make sense to make a comprehensive improvement plan and then shut down SETI for 2-3 weeks. Clearing old stuff, clearing up statistics and "corpses", processing all outstanding data (WUs), then reworking the entire hardware and software and then going back online clean. ET will certainly not send a signal in the 3 weeks. But when it comes - and it will come - and all the hardware and software (as it is now) does not work properly, the whole years were really free for each and every one of us.

Much more likely it is for me that the signal is already there and is not recognized ...
But I'm not giving up yet and continue to search with you - if I get WUs.

So many good people write here who are familiar with databases and hardware but are not from SETI. Why are not these experiences, ideas and helps flowing into the project? Do the ideas evaporate here in the chat?

Science means to create knowledge and that is what many do here!
It is clear to me that in scientific projects, which are also subject to a certain level of security, not everyone can and should be allowed to work and have insight. But there is always talk of so many donations (money). Why are the experiences, ideas and knowledge donations not accepted?

Certainly, very creative and intelligent people work at SETI. But even these are limited with their knowledge and their performance. Maybe SETI will start a donation call - Mining of knowledge - for thoughts and ideas of professionals? I am sure that we can find a solution so quickly.

Well, for the most successful and implemented ideas there are no credits but instead of a donation star, graphics card symbol or disk icon, a bronze, silver or golden, pulsating brain as an achievement 😉

As reminder:
- (Info: 2011) Every year SETI needs about 2.5 million US dollars to operate.

I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me.

I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me.

They recently (about 6 months ago, IIRC...) doubled the size of the WU, first on Beta, then on production.. Hello, from Bangkok, Thailand!...

I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me.

They recently (about 6 months ago, IIRC...) doubled the size of the WU, first on Beta, then on production.

I remember that too. The other issue we have is this project wants to support crunching on smart phones. They are snails compared to high end GPU's. W/U size has to stay reasonable for them as do deadlines considering their flop counts. Now if some of you programmer types want to change BOINC so it can issue dynamic deadlines based on reported flop count, the code is on github ... Making W/U sizes dynamic is much harder as that is the science code.

The "doubling" of the task size was triggered not by increasing the number of data points but by increasing the resolution. This change has had very little impact on the execution time of a task.Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?