Just happened to me on an Asus EEE 901. I was copying large files from a pendrive, and after few minutes my desktop was almost frozen, unusable. The system responded for a mouse action in 30-60 seconds. The hard drive led (SSD in fact) was continuously on. I managed to switch to vt1 and log in (it took a minute), and ran top. It showed that update-apt-xapi was using 100% CPU. After about 8-10 minutes of struggling, the process finished it's mysterious work, and the system was back to normal again. But rendering the box unusable for minutes can be frustrating. At least can somebody tell me why we need this background process at all? Can I uninstall it, without breaking functionality?

The program is described in apt-cache show as supporting the index of
packages that you see when you use "add/remove" from the applications
menu in the gnome panel. I uninstalled it and nothing bad happened (I
use synaptic or apt to install/uninstall applications).

I think it trawls through the titles and descriptions of your apt database of packages making it possible for you to search for packages not just for their title but also by their description. However I think in the past I have been able to search by package descriptions long before this process was crippling my PC.

Anyway my solution is a little less dramatic than uninstalling the thing (if its existence is necessary I don't know, I doubt it but have kept it in case). I just went into the directory /etc/cron.weekly and moved the file apt-xapian-index to the folder /etc/cron.monthly. It still rears its ugly head occasionally but at just six times between releases I can live with it.

Confirmed problem on Ubuntu 9.04 x86_64 Jaunty. I am running in Intel Core 2 Quad Q9650 CPU, and the update-apt-xapian process takes 100% of one core. As of this writing, it has run for 468:50:30 according to top with no end in sight. This is absurd.

I tried purging the package and killing the process, but then the Quick Search window in Synaptic no longer works. I tried making the /etc/cron.weekly/apt-xapian-index script non-executable, but it runs anyway.

Other than the Quick Search in Synaptic, I see no use for this process or its huge database. It either need to be fixed so that it doesn't eat 100% of a CPU core, or eliminated as a requirement for Synaptic's Quick Search so that it can be removed from the distribution.

I just killed the process I wrote about this afternoon. In checking the database, I noticed that it was last written over 2.5 hours ago. Yet, the update-apt-xapian process was still taking 100% of one CPU core until I just killed it. I took David's advice and moved the executing script from /etc/cron.weekly to /etc/cron.monthly. This needs to be fixed.

I have an old 866Mhz machine with 256MB RAM.
Top only shows this process consuming around 40% of the CPU.
However, this process brings my machine to its knees.
The mouse doesn't move well. Menus take forever to pop-up.
Programs won't close without the warning "Application is not responding".
I think perhaps I am out of RAM and thus thrashing the cache.

I don't know how the kernel handles a background task wanting all available memory, thus leaving none for the application in focus.

That sounds like a band-aid solution. This must be broken either in code or concept if it consumes so much resources. Why doesn't each package installation add itself to a database, instead of having an indexing monster run regularly even if no new packages are installed?

Any changes I made to the status of this bug where accidental. Currently update-apt-xapian-index is running and was making my system extremely sluggish with lots of IOWait, and my mouse clicked the Fix Released button on accident.

Considering the Add/Remove programs doesn't have much of a "live search" feel to it (when I search for packages it doesn't update for a few seconds and takes a lot of CPU) I don't see what this index is accomplishing.

As someone already mentioned, why can't this just be a done at package install/removal time? The prelink package does this. People expect the system to be under heavy usage while doing package management, but a daemon either needs to be unobtrusive and quick.

Also from a scalability point of view I imagine that this daemon redundantly touches a lot of packages that haven't changed, since it's execution time seems to (understandably) grow with the amount of packages I have installed. A post-install solution would cut down on the overall time wasted creating this index.

Mirza: Please read the other posts before adding comments. #21 is clearly still affected by this issue and Yes it takes waaaay too much system resources to update its index. On my AMD 3700+ it uses 100% CPU for at least 20s. (I'll monitor it the next time it runs in Karmic to get more exact time.)

Is it only Synaptic that uses this index? Does anyone know? Does Software Center use it as well? I mean, a non-geek person uses Software Center and the ultimate geek uses apt-get. This massive indexing affects all users but how many need or use this feature if it's only Synaptics Quicksearch that takes benefit from it.

The general search function in Synaptic does its job. The user types in a word, clicks enter and expects its CPU to work in order to find the target. Isn't the purpose of an indexing service to speed up the searching? It does, BUT, right now it's almost like its just MOVING the time when the CPU must work, FROM that time when the user expects it to work (when the user wants to search) TO that time when the user maybe watches a movie or playing a game? At least update-apt-xapian-index can in Karmic still bounce up while surfing the net, making the computer quite sluggish.

Not trying to cause a "me too" storm or anything, but since I promised to mention more exactly for how long it uses my CPU I'll add yet another comment. On AMD 3700+ it uses maximum resources for 40-60s. The situation is better now than in Jaunty where it would consume resources on almost every boot while in Karmic it runs more seldom.

Still bad though when I was running Gimp, Word 2007 through Wine and Firefox that this suddenly bounces up and makes everything sluggish :S If it has to run, run when computer is not in use.

I have just switched to Kubuntu 9.10 from the Ubuntu Gnome distros, only to find it is all locked up until I kill this xapi process. It's pretty serious - completely locking up my computer for ages (until I kill it!).
I am amazed reading these comments it has been around so long and still hasn't been dealt with. I'm a bit annoyed because i finally persuaded my Dad (a fervent Windows man) to give Linux a go and gave him the Kubuntu disc I'd just written. D'oh.

In Jaunty, I found an interesting piece of code in /etc/cron.daily/mlocate. It checks if the laptop is on AC power, and if not, it exits without updating the locate database. I know this is not closely related to this bug, but apt-xapian-index doesn't seem to have this check. Maybe someone who has a laptop should run some tests to see if it worth putting into the script and could file a bug report if necessary. (I've never had a laptop, so I can't experiment with it).

running 9.10 on a 500mHz P3 notebook, this application pretty much shuts it down for a couple of hours on saturday morning. My experience is same as others I read here. I figured out what it was, killed it and saw the disk-banging stop and got my cpu back.

If all this does is give Synaptic Package Manager quick-search a current index - at least on saturday efternoon, it has to be the silliest resident ap I've ever seen.

I'm taking it out of cron.weekly, making it available to be run manually, while I'm on vacation and seeing if that lets it do whatever it seems to be doing.

If the wise folks who watch the gates on what can get into a Linux installation would keep an eye out for stuff like this, some of use with older machines would be better served.

To think that the Linux believers think that Windows is the only system with bloatware. This is a prime example.

boozehead, if it was running several times, maybe you experienced bug #594820 ?

For those wondering why this bug report is marked Fixed, and who did not read the whole thread because it has become unreadable, see for instance comment 77. See also bug #655831 which is still open. As I explained there, 100% CPU usage in itself is not a problem.

In my opinion, this is not solved. With nice and ionice the system performance may be less affected, but the power drain by heavy CPU usage will still remain.

The question are: Does the update need to be that costly, and does the user need this updates altogether. As far as I see, the index provide two functions: The quick search mode in synaptic pagage manager, which is not installed by default, and the package recommenadtions in the terminal, rarely useful to normal users either, and expendable by power users if it comes at this large cost.
That's it, or have I forgotten something? If not, I would conclude that apt-xapian-index just should not be installed by default, and has to prevail for users that intentionally install it. It must be made sure however, that no one misses the features without knowledge how to get them back.

Agreed - this is not fixed. Bug took one of my 12.04 servers out of commission. update-apt-xapi ate up 123MB of memory (the server only has 256MB) and then proceeded to top off the swap until the machine hung hard, forcing us to hard reboot.

A weekly fatal unable to fork error message from /etc/cron.weekly/apt-xapian-index on a server with 512MB memory with a web server and database server running leads to this bug. That cron job calls /usr/sbin/update-apt-xapian-index, which is a Python script that imports axi and axi.indexer.

Sorting is a classic computer science problem with both time and space complexity. For time think CPU. For space think memory.

The solution is to change the sorting algorithm in the python axi and axi.indexer modules. The first priority is to switch to an algorithm that consumes a whole lot less memory (i.e., each step of the algorithm keeps less objects in memory), and it will stop crashing and stop thrashing (memory swapping to disk).

The second priority is of lesser importance (because renice can solve a lot of the effect), which is to switch to an algorithm that takes a lot less time to run (i.e., takes fewer steps to complete), and it will stop consuming so much CPU for so long.

On an idle system with 512M memory and 768M swap, while /etc/cron.weekly/apt-xapian-index was running, I used free to see memory and swap used. It rose to then peaked at this:

total used free
Mem: 503528 497736 5792
Swap: 786428 231196 555232

Total used memory is 728M.

After ending, it immediately dropped back to this:

total used free
Mem: 503528 259160 244368
Swap: 786428 224484 561944

Total used memory is 483M.

I conclude that apt-xapian-index consumes the difference, which is 245M.

Running "apt-cache stats" I see at the end "Total space accounted for: 26.0 M".

Therefore, it takes 245M to sort and index 26M of information. This seems conclusive that the algorithms, containers, and functions chosen are very inefficient. It should definitely not require 10 times the memory space of what is being indexed.

I just encountered the problem with LUbuntu 13.10 32-bit, after installing "Ubuntu Software Center". I then replicated on Ubuntu 13.10 32-bit.

System stats once "update-apt-xapian-index" begins: 99% CPU usage on FOUR CORES with 257MB of memory used by the process which renders the system completely unusable.

I'm seriously tempted to just re-write the entire program and push it all upstream for the sake of the community. This is RIDICULOUS! FIVE YEARS after the bug was noted and there is NO fix outside of simply *not using it*.

My 12.04.5 LTS occasionally grinds to a halt with login nearly impossible. Finally I caught this remote embedded 256Mbyte system in the act and found update-apt-xapi had driven the machine well into swap. Hopefully apt-get remove apt-xapian-index will prevent this happening again until this long-standing defect is fixed. Perhaps mmap its big RAM data to /tmp/something and put in a substantial sleep in its main loop.

This happened to me today on XUbuntu 15.04 final with updates. It seems to stay on hogging as much as CPU as it can get its hands on (nice 10 apparently) and using about as much memory as stated earlier here. It just keeps on hogging, making me wonder if something's gone wrong.

I probably can figure out how to disable it somewhere..an average user would not.