A NOTE has been added to this issue.
======================================================================
http://bugs.bacula.org/view.php?id=1871
======================================================================
Reported By: luca@...
Assigned To: kern
======================================================================
Project: bacula
Issue ID: 1871
Category: configure/build process
Reproducibility: always
Severity: minor
Priority: normal
Status: closed
Resolution: won't fix
Fixed in Version:
======================================================================
Date Submitted: 2012-05-19 13:34 BST
Last Modified: 2013-05-23 15:50 BST
======================================================================
Summary: readline support not available without TERM_LIB
Description:
This bug has been reported in Debian by Sven Joachim [1].
[1] <http://bugs.debian.org/646730&gt;
It is impossible to build the bconsole readline support in Bacula_5.0.2+ without
the TERM_LIB. I know what the Bacula manual says about readline:
- System Requirements [2]
* If you want to enable command line editing and history, you will need to
have /usr/include/termcap.h and either the termcap or the ncurses library loaded
(libtermcap-devel or ncurses-devel).
[2] <http://www.bacula.org/5.2.x-manuals/en/main/main/System_Requirements.html&gt;
- Building Bacula from Source [3]
The --enable-conio or --enable-readline options are useful because they
provide a command line history and editing capability for the Console program.
If you have included either option in the build, either the termcap or the
ncurses package will be needed to link.
[...]
readline is no longer supported after version 1.34. The code within Bacula
remains, so it should be usable, and if users submit patches for it, we will be
happy to apply them. However, due to the fact that each version of readline
seems to be incompatible with previous versions, and that there are significant
differences between systems, we can no longer afford to support it.
[3] <http://www.bacula.org/manuals/en/install/install/Installing_Bacula.html&gt;
However, the readline support is never available if you do not have TERM_LIB
because of [4], which basically put conio and readline at the same level in
terms of dependencies. I was not able to find any reason for this change in the
ChangeLog neither in the BTS and similar bugs were either too old
(http://bugs.bacula.org/view.php?id=743 [5]) or not related
(http://bugs.bacula.org/view.php?id=1226 [6]).
[4]
<http://www.bacula.org/git/cgit.cgi/bacula/commit/?h=Branch-5.0&id=967d4334e69d2ab739a2f270f2df1bcfb0ffc7d4&gt;
[5] <http://bugs.bacula.org/view.php?id=743&gt;
[6] <http://bugs.bacula.org/view.php?id=1226&gt;
However, it seems that the readline support does not need any TERM_LIB to be
built, given that according to build logs [7][8] /usr/sbin/bacula-console (this
is upstream bconsole, see [9] for the future fix in Debian) does not use any
symbols from libhistory.so.6, nor libtinfo.so.5 neither libncurses.so.5.
[7] <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=646730#31&gt;
[8]
<https://buildd.debian.org/status/fetch.php?pkg=bacula&arch=amd64&ver=5.0.3%2Bdfsg-0.1&stamp=1335779247&gt;
[9]
<http://anonscm.debian.org/gitweb/?p=pkg-bacula/bacula.git;a=commitdiff;h=2cdfcfec4764ae6cde77c44f474bd521f7562e1d&gt;
With the attached patch provided by Alexander Golovko [10] the Debian packages
are built with:
=====
$ cd /tmp/buildd/bacula-5.0.3+dfsgreal/debian/tmp-build-sqlite3 && \
QMAKE=/usr/bin/qmake-qt4 ./configure --config-cache \
--host=x86_64-linux-gnu --build=x86_64-linux-gnu \
--prefix=/usr \
--with-archivedir=/nonexistant/path/to/file/archive/dir \
--sysconfdir=/etc/bacula --with-scriptdir=/etc/bacula/scripts \
--sharedstatedir=/var/lib/bacula \
--localstatedir=/var/lib/bacula \
--with-pid-dir=/var/run/bacula --with-smtp-host=localhost \
--with-working-dir=/var/lib/bacula \
--with-subsys-dir=/var/lock \
--mandir=\${prefix}/share/man \
--infodir=\${prefix}/share/info \
--enable-smartalloc --with-python --with-tcp-wrappers
--with-openssl --with-libiconv-prefix=/usr/include
--with-readline=/usr/include/readline --disable-conio
--with-libintl-prefix=/usr/include --with-x
--docdir=\${prefix}/share/doc/bacula-common
--htmldir=\${prefix}/share/doc/bacula-common/html --libdir=\${prefix}/lib/bacula
--enable-batch-insert --disable-bwx-console --without-qwt --enable-ipv6
--with-dir-password=XXX_DIRPASSWORD_XXX --with-fd-password=XXX_FDPASSWORD_XXX
--with-sd-password=XXX_SDPASSWORD_XXX
--with-mon-dir-password=XXX_MONDIRPASSWORD_XXX
--with-mon-fd-password=XXX_MONFDPASSWORD_XXX
--with-mon-sd-password=XXX_MONSDPASSWORD_XXX --with-db-name=XXX_DBNAME_XXX
--with-db-user=XXX_DBUSER_XXX --with-db-password=XXX_DBPASSWORD_XXX
--with-sqlite3 --without-mysql --without-postgresql --without-sqlite
--enable-tray-monitor --enable-bat
[...]
Configuration on Fri May 18 19:54:40 UTC 2012:
Host: x86_64-pc-linux-gnu -- debian wheezy/sid
Bacula version: Bacula 5.0.3 (04 August 2010)
[...]
SQL binaries Directory /usr/bin
Large file support: yes
Bacula conio support: no -lreadline -lhistory
readline support: yes
TCP Wrappers support: yes -lwrap
TLS support: yes
Encryption support: yes
ZLIB support: yes
[...]
=====
[10]
<http://anonscm.debian.org/gitweb/?p=pkg-bacula/bacula.git;a=commitdiff;h=488241e3d2c90af4011597c652670d3c74e95cbe&gt;
dpkg-shlibdeps still complains about unused symbols, but this time only for
libhistory.so.6:
=====
dpkg-shlibdeps: warning: package could avoid a useless dependency if
debian/bacula-console/usr/sbin/bacula-console was not linked against libdl.so.2
(it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if
debian/bacula-console/usr/sbin/bacula-console was not linked against libz.so.1
(it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if
debian/bacula-console/usr/sbin/bacula-console was not linked against
libhistory.so.6 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if
debian/bacula-console/usr/sbin/bacula-console was not linked against
libcrypto.so.1.0.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if
debian/bacula-console/usr/sbin/bacula-console was not linked against
libwrap.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if
debian/bacula-console/usr/sbin/bacula-console was not linked against
libssl.so.1.0.0 (it uses none of the library's symbols).
=====
The resulting bacula-console binary works as expected (CTRL-UP/DOWN and
TAB-COMPLETION) against a 5.0.2 Director. Is there something I am missing?
======================================================================
----------------------------------------------------------------------
(0006326) kern (administrator) - 2012-05-24 07:34
http://bugs.bacula.org/view.php?id=1871#c6326
----------------------------------------------------------------------
Thanks for the patch.
This patch looks quite reasonable to me. We will test it and unless some
problems come up, highly unlikely given your testing, we will include it in the
next release. I will leave the bug report open until the patch is committed.
----------------------------------------------------------------------
(0006383) kern (administrator) - 2012-06-10 08:59
http://bugs.bacula.org/view.php?id=1871#c6383
----------------------------------------------------------------------
Thanks again for the patch. It is applied and will be available in version
5.2.8 to be released today or tomorrow.
----------------------------------------------------------------------
(0006385) marcovw (viewer) - 2012-06-11 09:46
http://bugs.bacula.org/view.php?id=1871#c6385
----------------------------------------------------------------------
Maybe this works on Linux but on Solaris its a disaster as readline uses a quite
some symbols from the termcap library
Undefined first referenced
symbol in file
tgoto /usr/lib/libreadline.so
tputs /usr/lib/libreadline.so
tgetent /usr/lib/libreadline.so
tgetnum /usr/lib/libreadline.so
tgetstr /usr/lib/libreadline.so
tgetflag /usr/lib/libreadline.so
I also see NO reason in changing this behaviour as on Linux this
also never been a problem (at least not on OpenSuse/Fedora and SLES/RHEL/CentOS)
----------------------------------------------------------------------
(0006387) slaanesh (reporter) - 2012-06-11 12:56
http://bugs.bacula.org/view.php?id=1871#c6387
----------------------------------------------------------------------
Hello,
this also breaks compilation with RHEL 5 libraries:
make[1]: Entering directory `/builddir/build/BUILD/bacula-5.2.8/src/console'
Compiling console.c
Compiling console_conf.c
Compiling authenticate.c
/builddir/build/BUILD/bacula-5.2.8/libtool --silent --tag=CXX --mode=link
/usr/bin/g++ -L../lib -L../cats -o bconsole console.o console_conf.o
authenticate.o \
-lreadline -lbaccfg -lbac -lm -lpthread -ldl -ldl \
-lssl -lcrypto
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `tgetnum'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `tgetent'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `tgetstr'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `tgoto'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `UP'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `BC'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `tputs'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `PC'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libreadline.so:
undefined reference to `tgetflag'
collect2: ld returned 1 exit status
make[1]: Leaving directory `/builddir/build/BUILD/bacula-5.2.8/src/console'
make[1]: *** [bconsole] Error 1
Works as expected in RHEL 6 and Fedora.
----------------------------------------------------------------------
(0006390) kern (administrator) - 2012-06-11 14:52
http://bugs.bacula.org/view.php?id=1871#c6390
----------------------------------------------------------------------
I guess I should have tried building it on a few more systems. I will back out
the patch and release a new version without it sometime tonight ...
----------------------------------------------------------------------
(0006398) kern (administrator) - 2012-06-17 12:44
http://bugs.bacula.org/view.php?id=1871#c6398
----------------------------------------------------------------------
Sorry but this patch is rejected because it is not portable (i.e. it breaks the
builds on a number of other systems).
----------------------------------------------------------------------
(0006716) kern (administrator) - 2013-05-23 15:50
http://bugs.bacula.org/view.php?id=1871#c6716
----------------------------------------------------------------------
Unfortunately, this patch got into a later version, and it also broke that
version causing some problems :-(
Issue History
Date Modified Username Field Change
======================================================================
2012-05-19 13:34 luca@... New Issue
2012-05-19 13:34 luca@... File Added: fix-readline-ncurses-depends.patch
2012-05-24 07:34 kern Note Added: 0006326
2012-05-24 07:34 kern Assigned To => kern
2012-05-24 07:34 kern Status new => acknowledged
2012-06-10 08:59 kern Note Added: 0006383
2012-06-10 08:59 kern Status acknowledged => closed
2012-06-10 08:59 kern Resolution open => fixed
2012-06-10 08:59 kern Fixed in Version => 5.2.8
2012-06-11 09:46 marcovw Note Added: 0006385
2012-06-11 09:46 marcovw Status closed => feedback
2012-06-11 09:46 marcovw Resolution fixed => reopened
2012-06-11 12:56 slaanesh Note Added: 0006387
2012-06-11 14:52 kern Note Added: 0006390
2012-06-17 12:44 kern Note Added: 0006398
2012-06-17 12:44 kern Status feedback => closed
2012-06-17 12:44 kern Resolution reopened => won't fix
2012-06-17 12:44 kern Fixed in Version 5.2.8 =>
2013-05-23 15:50 kern Note Added: 0006716
======================================================================

The following issue has been CLOSED
======================================================================
http://bugs.bacula.org/view.php?id=1995
======================================================================
Reported By: mkress
Assigned To:
======================================================================
Project: bacula
Issue ID: 1995
Category: Director
Reproducibility: random
Severity: minor
Priority: low
Status: closed
Resolution: unable to reproduce
Fixed in Version:
======================================================================
Date Submitted: 2013-04-12 08:02 BST
Last Modified: 2013-05-22 07:35 BST
======================================================================
Summary: Metadata of backuped file not stored in (postgresql)
database
Description:
Filedeamon on SLES8 and SLES10, Director on SLES11
Sometimes (every few weeks with a daily backup, 3 backups in parallel, all Linux
with SLES8, SLES10 and SLES11) a Verfiyjob (pins5) complains about a new file
(erverytime a different file):
12-Apr 02:02 iss-dir JobId 499: New file:
/usr/perl5/5.8.0/lib/unicore/lib/Ideograp.pl
This file was not changed/added in the filesystem during backup or verify, it
has no new timestamp or a new inode. The file was backuped/verified all days
before without any problems.
Metadata seems to be not stored in the bacula database (postgresql 9.2.3 on
SLES11), because I can't restore it using the restore command. Using bextract it
can restored. It seem it was not written to the database. The log of postgresql
shows no errors/warnings.
Steps to Reproduce:
Can't reproduce, because it happend only 3 times on different servers with
different files within of 8 weeks (120 backups).
Additional Information:
Using a older version of bacula 5.2.6 with postgresql 9.1.3 on SLES10 seems not
to have this problem. Using the same configuration for postgres. Bacula using a
small different configuration (no backups in parallel).
======================================================================
----------------------------------------------------------------------
(0006656) ebollengier (administrator) - 2013-04-18 08:35
http://bugs.bacula.org/view.php?id=1995#c6656
----------------------------------------------------------------------
Sorry you have problems with Bacula, however, it looks to be a support issue. To
check if the "Metadata" is stored in the catalog, just run a restore, access to
/usr/perl5/5.8.0/lib/unicore/lib, and type "dir".
It should display the file, with owner/group/rights, etc...
I advise you to contact the users list to get help with your problem.
----------------------------------------------------------------------
(0006662) mkress (reporter) - 2013-04-19 15:32
http://bugs.bacula.org/view.php?id=1995#c6662
----------------------------------------------------------------------
I already tried a restore ("because I can't restore it using the restore
command") and it was NOT in the database. So it still seems a bug in Bacula.
In the first and second time when the problem occured, it was in both times in a
directory which contains tech (not perl), but each time a different tech-file.
All three times, the server was a new installed (testing)-server without
productive users or productive files (only OS and some other static data). No
other user was on the server.
----------------------------------------------------------------------
(0006699) kern (administrator) - 2013-05-10 09:33
http://bugs.bacula.org/view.php?id=1995#c6699
----------------------------------------------------------------------
This is not something that we can reproduce here, and if we cannot reproduce it,
we will probably not be able to fix it. You should carefully examine your
PostgreSQL logs to see if some incompatibility has crept into it -- we have only
tested on 9.1. If you can show that the problem does not exist on Bacula 5.2.5,
with PostgreSQL 9.2x then we will take a careful look at the code changes.
You might also explain what is being backed up in parallel -- is the same
file/machine involved?
----------------------------------------------------------------------
(0006703) mkress (reporter) - 2013-05-10 10:50
http://bugs.bacula.org/view.php?id=1995#c6703
----------------------------------------------------------------------
I have made about 80 backups without any problems since I've created this bug
report. At the moment I don't decide to try a older Version of Bacula or
Postgres, because the problem occures very seldom (only 3 times within of
hundreds of backups).
We have more than hundred installations of Bacula 5.2.6 with Postgres 9.1.3
which each made every day 2 backups and we have had older Version of Bacula and
Postgres since 3 years and we had never a problem within thousends of backups.
Now we want to migrate servers in our locations to VMware ESXI and using the
same hardware (HP servers). I'm using the first time filedeamon-only systems and
the newst Version of Bacula and Postgres. This new configuration is not
productive until now. It's installed on test servers, without productive data
and without users and with a very low usage.
I made every weekday backups in parallel of 3 VM's on the same Hardware
(starting about one o'clock at night). One VM running director, sd and fd, the 2
other VM's only fd. The tapedrive is connected using a pci-passthough SAS-card.
The problem occours until now on the VM's which have only the fd installed.
Conspicuous of first 2 occurrences was, that the missing file in the database
was in the same directory. It were tech files which comes with the tech-package.
The 3nd time it was a perl file from the perl package (and in a different VM).
All three times, the missing file was a part of the system files. The files of
the system are backuped before any other data is backuped. I do not using
spooling.
1st time (on SLES 10):
/usr/local/texlive/2009/texmf-dist/doc/latex/upmethodology/Changelog
2nd time (on SLES 10):
/usr/local/texlive/2009/texmf-dist/doc/generic/pst-solides3d/doc-en/source/par-acknowledgements-en.tex
3rd time (on SLES 8):
/usr/perl5/5.8.0/lib/unicore/lib/Ideograp.pl
Between all 3 occurrences the VM ESXI and the VMs within were new installed.
with a complete unattendend (scripted) auto installation. The VM's have no
difference between the installations. Like I reported before, we have no Problem
with the postgres installation. Logs are all ok. We're using the same postgres
configuration (with a lower verion of postgres) on our hundered productive
servers, which running perfectly. Bacula configuration between the testing and
productive server differs only in the client configuration. Priorities were not
used on our productive bacula (concurrences = 1).
Only at the last occurrence with the perl-file I tried a restore using bextract.
I was able to restore it using bextract. With the the restore command in
bconsole, the file was not found.
My tests will be go on and it will go productive in some weeks on hundred of
servers. I hope to get more information I can provide here, but I sure, the
problem is sitting not before the keyboard.
----------------------------------------------------------------------
(0006714) mkress (reporter) - 2013-05-21 08:10
http://bugs.bacula.org/view.php?id=1995#c6714
----------------------------------------------------------------------
I tried in the last days 48 backups and verifies every 2h on vtapes (backup to
files). All backups were ok, but got 6 times a verify error. It's not the same
error as I got in my bug report. It seems not related to the problems in my bug
report, but it's also very strange. I got every 6 times a "Volume data error"
during verify, but the system have no Problems with the disks or any other
errors (during backup and verify). All Logs are ok, also the log of the Raid
controller. It's a HP server with 6 SAS Drives and Raid 5. It seems not to be a
hardware error. FD-Daemon runs on a SLES10 and SD/DIR on a SLES11. Both are
running on a VMWare ESXI-Server for HP Server with the newest Drivers. All
Software and Hardware are certified by Suse and VMWare. I put this into the bug
report for completeness.
18-May 11:15 iss-stor JobId 22: Forward spacing Volume "GLS_VTAPE-0036" to
file:block 0:209.
18-May 11:15 iss-stor JobId 22: Error: block.c:334 Volume data error at
0:115283150!
Block checksum mismatch in block=4776 len=64512: calc=c5a0196 blk=a9021e58
18-May 11:15 iss-stor JobId 22: Fatal error: fd_cmds.c:169 Command error with
FD, hanging up.
18-May 11:15 iss-dir JobId 22: Warning: The following files are in the Catalog
but not on the Volume(s):
----------------------------------------------------------------------
(0006715) kern (administrator) - 2013-05-22 07:35
http://bugs.bacula.org/view.php?id=1995#c6715
----------------------------------------------------------------------
The error messages you posted in your last message very clearly indicate a
hardware error. When Bacula reports a block checksum error as it did, you can
be 100% sure you have some error external to Bacula most likely a hardware
error: memory, OS drivers, disk drive, cables, controller, firmware, raid
hardware card, raid software (particularly known for problems). I am 100%
certain of the above, so please focus on that and not on other possibilities.
It is highly likely that if you look back at your kernel logs, you will find the
reason for the missing files.
I am going to close this bug now, because as long as you have hardware errors,
there is no use talking about Bacula problems. If you can fix your hardware
errors and there are still problems, my suggestion is to shorten your reports to
include the essential messages, from Bacula, the kernel logs and Postgres logs.
Exact copies of oututput such as what you showed in the last report is what is
most useful. For example, though you talk about Verify jobs, unless I missed
something, I have seen nothing that "shows" the Verify jobs problem.
Issue History
Date Modified Username Field Change
======================================================================
2013-04-12 08:02 mkress New Issue
2013-04-18 08:35 ebollengier Note Added: 0006656
2013-04-18 08:35 ebollengier Priority high => low
2013-04-18 08:35 ebollengier Severity major => minor
2013-04-19 15:32 mkress Note Added: 0006662
2013-05-10 09:33 kern Note Added: 0006699
2013-05-10 09:33 kern Status new => feedback
2013-05-10 10:50 mkress Note Added: 0006703
2013-05-10 10:50 mkress Status feedback => new
2013-05-21 08:10 mkress Note Added: 0006714
2013-05-22 07:35 kern Note Added: 0006715
2013-05-22 07:35 kern Status new => feedback
2013-05-22 07:35 kern Status feedback => closed
2013-05-22 07:35 kern Resolution open => unable to
reproduce
======================================================================

The following issue requires your FEEDBACK.
======================================================================
http://bugs.bacula.org/view.php?id=1995
======================================================================
Reported By: mkress
Assigned To:
======================================================================
Project: bacula
Issue ID: 1995
Category: Director
Reproducibility: random
Severity: minor
Priority: low
Status: feedback
======================================================================
Date Submitted: 2013-04-12 08:02 BST
Last Modified: 2013-05-22 07:35 BST
======================================================================
Summary: Metadata of backuped file not stored in (postgresql)
database
Description:
Filedeamon on SLES8 and SLES10, Director on SLES11
Sometimes (every few weeks with a daily backup, 3 backups in parallel, all Linux
with SLES8, SLES10 and SLES11) a Verfiyjob (pins5) complains about a new file
(erverytime a different file):
12-Apr 02:02 iss-dir JobId 499: New file:
/usr/perl5/5.8.0/lib/unicore/lib/Ideograp.pl
This file was not changed/added in the filesystem during backup or verify, it
has no new timestamp or a new inode. The file was backuped/verified all days
before without any problems.
Metadata seems to be not stored in the bacula database (postgresql 9.2.3 on
SLES11), because I can't restore it using the restore command. Using bextract it
can restored. It seem it was not written to the database. The log of postgresql
shows no errors/warnings.
Steps to Reproduce:
Can't reproduce, because it happend only 3 times on different servers with
different files within of 8 weeks (120 backups).
Additional Information:
Using a older version of bacula 5.2.6 with postgresql 9.1.3 on SLES10 seems not
to have this problem. Using the same configuration for postgres. Bacula using a
small different configuration (no backups in parallel).
======================================================================
----------------------------------------------------------------------
(0006656) ebollengier (administrator) - 2013-04-18 08:35
http://bugs.bacula.org/view.php?id=1995#c6656
----------------------------------------------------------------------
Sorry you have problems with Bacula, however, it looks to be a support issue. To
check if the "Metadata" is stored in the catalog, just run a restore, access to
/usr/perl5/5.8.0/lib/unicore/lib, and type "dir".
It should display the file, with owner/group/rights, etc...
I advise you to contact the users list to get help with your problem.
----------------------------------------------------------------------
(0006662) mkress (reporter) - 2013-04-19 15:32
http://bugs.bacula.org/view.php?id=1995#c6662
----------------------------------------------------------------------
I already tried a restore ("because I can't restore it using the restore
command") and it was NOT in the database. So it still seems a bug in Bacula.
In the first and second time when the problem occured, it was in both times in a
directory which contains tech (not perl), but each time a different tech-file.
All three times, the server was a new installed (testing)-server without
productive users or productive files (only OS and some other static data). No
other user was on the server.
----------------------------------------------------------------------
(0006699) kern (administrator) - 2013-05-10 09:33
http://bugs.bacula.org/view.php?id=1995#c6699
----------------------------------------------------------------------
This is not something that we can reproduce here, and if we cannot reproduce it,
we will probably not be able to fix it. You should carefully examine your
PostgreSQL logs to see if some incompatibility has crept into it -- we have only
tested on 9.1. If you can show that the problem does not exist on Bacula 5.2.5,
with PostgreSQL 9.2x then we will take a careful look at the code changes.
You might also explain what is being backed up in parallel -- is the same
file/machine involved?
----------------------------------------------------------------------
(0006703) mkress (reporter) - 2013-05-10 10:50
http://bugs.bacula.org/view.php?id=1995#c6703
----------------------------------------------------------------------
I have made about 80 backups without any problems since I've created this bug
report. At the moment I don't decide to try a older Version of Bacula or
Postgres, because the problem occures very seldom (only 3 times within of
hundreds of backups).
We have more than hundred installations of Bacula 5.2.6 with Postgres 9.1.3
which each made every day 2 backups and we have had older Version of Bacula and
Postgres since 3 years and we had never a problem within thousends of backups.
Now we want to migrate servers in our locations to VMware ESXI and using the
same hardware (HP servers). I'm using the first time filedeamon-only systems and
the newst Version of Bacula and Postgres. This new configuration is not
productive until now. It's installed on test servers, without productive data
and without users and with a very low usage.
I made every weekday backups in parallel of 3 VM's on the same Hardware
(starting about one o'clock at night). One VM running director, sd and fd, the 2
other VM's only fd. The tapedrive is connected using a pci-passthough SAS-card.
The problem occours until now on the VM's which have only the fd installed.
Conspicuous of first 2 occurrences was, that the missing file in the database
was in the same directory. It were tech files which comes with the tech-package.
The 3nd time it was a perl file from the perl package (and in a different VM).
All three times, the missing file was a part of the system files. The files of
the system are backuped before any other data is backuped. I do not using
spooling.
1st time (on SLES 10):
/usr/local/texlive/2009/texmf-dist/doc/latex/upmethodology/Changelog
2nd time (on SLES 10):
/usr/local/texlive/2009/texmf-dist/doc/generic/pst-solides3d/doc-en/source/par-acknowledgements-en.tex
3rd time (on SLES 8):
/usr/perl5/5.8.0/lib/unicore/lib/Ideograp.pl
Between all 3 occurrences the VM ESXI and the VMs within were new installed.
with a complete unattendend (scripted) auto installation. The VM's have no
difference between the installations. Like I reported before, we have no Problem
with the postgres installation. Logs are all ok. We're using the same postgres
configuration (with a lower verion of postgres) on our hundered productive
servers, which running perfectly. Bacula configuration between the testing and
productive server differs only in the client configuration. Priorities were not
used on our productive bacula (concurrences = 1).
Only at the last occurrence with the perl-file I tried a restore using bextract.
I was able to restore it using bextract. With the the restore command in
bconsole, the file was not found.
My tests will be go on and it will go productive in some weeks on hundred of
servers. I hope to get more information I can provide here, but I sure, the
problem is sitting not before the keyboard.
----------------------------------------------------------------------
(0006714) mkress (reporter) - 2013-05-21 08:10
http://bugs.bacula.org/view.php?id=1995#c6714
----------------------------------------------------------------------
I tried in the last days 48 backups and verifies every 2h on vtapes (backup to
files). All backups were ok, but got 6 times a verify error. It's not the same
error as I got in my bug report. It seems not related to the problems in my bug
report, but it's also very strange. I got every 6 times a "Volume data error"
during verify, but the system have no Problems with the disks or any other
errors (during backup and verify). All Logs are ok, also the log of the Raid
controller. It's a HP server with 6 SAS Drives and Raid 5. It seems not to be a
hardware error. FD-Daemon runs on a SLES10 and SD/DIR on a SLES11. Both are
running on a VMWare ESXI-Server for HP Server with the newest Drivers. All
Software and Hardware are certified by Suse and VMWare. I put this into the bug
report for completeness.
18-May 11:15 iss-stor JobId 22: Forward spacing Volume "GLS_VTAPE-0036" to
file:block 0:209.
18-May 11:15 iss-stor JobId 22: Error: block.c:334 Volume data error at
0:115283150!
Block checksum mismatch in block=4776 len=64512: calc=c5a0196 blk=a9021e58
18-May 11:15 iss-stor JobId 22: Fatal error: fd_cmds.c:169 Command error with
FD, hanging up.
18-May 11:15 iss-dir JobId 22: Warning: The following files are in the Catalog
but not on the Volume(s):
----------------------------------------------------------------------
(0006715) kern (administrator) - 2013-05-22 07:35
http://bugs.bacula.org/view.php?id=1995#c6715
----------------------------------------------------------------------
The error messages you posted in your last message very clearly indicate a
hardware error. When Bacula reports a block checksum error as it did, you can
be 100% sure you have some error external to Bacula most likely a hardware
error: memory, OS drivers, disk drive, cables, controller, firmware, raid
hardware card, raid software (particularly known for problems). I am 100%
certain of the above, so please focus on that and not on other possibilities.
It is highly likely that if you look back at your kernel logs, you will find the
reason for the missing files.
I am going to close this bug now, because as long as you have hardware errors,
there is no use talking about Bacula problems. If you can fix your hardware
errors and there are still problems, my suggestion is to shorten your reports to
include the essential messages, from Bacula, the kernel logs and Postgres logs.
Exact copies of oututput such as what you showed in the last report is what is
most useful. For example, though you talk about Verify jobs, unless I missed
something, I have seen nothing that "shows" the Verify jobs problem.
Issue History
Date Modified Username Field Change
======================================================================
2013-04-12 08:02 mkress New Issue
2013-04-18 08:35 ebollengier Note Added: 0006656
2013-04-18 08:35 ebollengier Priority high => low
2013-04-18 08:35 ebollengier Severity major => minor
2013-04-19 15:32 mkress Note Added: 0006662
2013-05-10 09:33 kern Note Added: 0006699
2013-05-10 09:33 kern Status new => feedback
2013-05-10 10:50 mkress Note Added: 0006703
2013-05-10 10:50 mkress Status feedback => new
2013-05-21 08:10 mkress Note Added: 0006714
2013-05-22 07:35 kern Note Added: 0006715
2013-05-22 07:35 kern Status new => feedback
======================================================================

A NOTE has been added to this issue.
======================================================================
http://bugs.bacula.org/view.php?id=1995
======================================================================
Reported By: mkress
Assigned To:
======================================================================
Project: bacula
Issue ID: 1995
Category: Director
Reproducibility: random
Severity: minor
Priority: low
Status: new
======================================================================
Date Submitted: 2013-04-12 08:02 BST
Last Modified: 2013-05-21 08:10 BST
======================================================================
Summary: Metadata of backuped file not stored in (postgresql)
database
Description:
Filedeamon on SLES8 and SLES10, Director on SLES11
Sometimes (every few weeks with a daily backup, 3 backups in parallel, all Linux
with SLES8, SLES10 and SLES11) a Verfiyjob (pins5) complains about a new file
(erverytime a different file):
12-Apr 02:02 iss-dir JobId 499: New file:
/usr/perl5/5.8.0/lib/unicore/lib/Ideograp.pl
This file was not changed/added in the filesystem during backup or verify, it
has no new timestamp or a new inode. The file was backuped/verified all days
before without any problems.
Metadata seems to be not stored in the bacula database (postgresql 9.2.3 on
SLES11), because I can't restore it using the restore command. Using bextract it
can restored. It seem it was not written to the database. The log of postgresql
shows no errors/warnings.
Steps to Reproduce:
Can't reproduce, because it happend only 3 times on different servers with
different files within of 8 weeks (120 backups).
Additional Information:
Using a older version of bacula 5.2.6 with postgresql 9.1.3 on SLES10 seems not
to have this problem. Using the same configuration for postgres. Bacula using a
small different configuration (no backups in parallel).
======================================================================
----------------------------------------------------------------------
(0006656) ebollengier (administrator) - 2013-04-18 08:35
http://bugs.bacula.org/view.php?id=1995#c6656
----------------------------------------------------------------------
Sorry you have problems with Bacula, however, it looks to be a support issue. To
check if the "Metadata" is stored in the catalog, just run a restore, access to
/usr/perl5/5.8.0/lib/unicore/lib, and type "dir".
It should display the file, with owner/group/rights, etc...
I advise you to contact the users list to get help with your problem.
----------------------------------------------------------------------
(0006662) mkress (reporter) - 2013-04-19 15:32
http://bugs.bacula.org/view.php?id=1995#c6662
----------------------------------------------------------------------
I already tried a restore ("because I can't restore it using the restore
command") and it was NOT in the database. So it still seems a bug in Bacula.
In the first and second time when the problem occured, it was in both times in a
directory which contains tech (not perl), but each time a different tech-file.
All three times, the server was a new installed (testing)-server without
productive users or productive files (only OS and some other static data). No
other user was on the server.
----------------------------------------------------------------------
(0006699) kern (administrator) - 2013-05-10 09:33
http://bugs.bacula.org/view.php?id=1995#c6699
----------------------------------------------------------------------
This is not something that we can reproduce here, and if we cannot reproduce it,
we will probably not be able to fix it. You should carefully examine your
PostgreSQL logs to see if some incompatibility has crept into it -- we have only
tested on 9.1. If you can show that the problem does not exist on Bacula 5.2.5,
with PostgreSQL 9.2x then we will take a careful look at the code changes.
You might also explain what is being backed up in parallel -- is the same
file/machine involved?
----------------------------------------------------------------------
(0006703) mkress (reporter) - 2013-05-10 10:50
http://bugs.bacula.org/view.php?id=1995#c6703
----------------------------------------------------------------------
I have made about 80 backups without any problems since I've created this bug
report. At the moment I don't decide to try a older Version of Bacula or
Postgres, because the problem occures very seldom (only 3 times within of
hundreds of backups).
We have more than hundred installations of Bacula 5.2.6 with Postgres 9.1.3
which each made every day 2 backups and we have had older Version of Bacula and
Postgres since 3 years and we had never a problem within thousends of backups.
Now we want to migrate servers in our locations to VMware ESXI and using the
same hardware (HP servers). I'm using the first time filedeamon-only systems and
the newst Version of Bacula and Postgres. This new configuration is not
productive until now. It's installed on test servers, without productive data
and without users and with a very low usage.
I made every weekday backups in parallel of 3 VM's on the same Hardware
(starting about one o'clock at night). One VM running director, sd and fd, the 2
other VM's only fd. The tapedrive is connected using a pci-passthough SAS-card.
The problem occours until now on the VM's which have only the fd installed.
Conspicuous of first 2 occurrences was, that the missing file in the database
was in the same directory. It were tech files which comes with the tech-package.
The 3nd time it was a perl file from the perl package (and in a different VM).
All three times, the missing file was a part of the system files. The files of
the system are backuped before any other data is backuped. I do not using
spooling.
1st time (on SLES 10):
/usr/local/texlive/2009/texmf-dist/doc/latex/upmethodology/Changelog
2nd time (on SLES 10):
/usr/local/texlive/2009/texmf-dist/doc/generic/pst-solides3d/doc-en/source/par-acknowledgements-en.tex
3rd time (on SLES 8):
/usr/perl5/5.8.0/lib/unicore/lib/Ideograp.pl
Between all 3 occurrences the VM ESXI and the VMs within were new installed.
with a complete unattendend (scripted) auto installation. The VM's have no
difference between the installations. Like I reported before, we have no Problem
with the postgres installation. Logs are all ok. We're using the same postgres
configuration (with a lower verion of postgres) on our hundered productive
servers, which running perfectly. Bacula configuration between the testing and
productive server differs only in the client configuration. Priorities were not
used on our productive bacula (concurrences = 1).
Only at the last occurrence with the perl-file I tried a restore using bextract.
I was able to restore it using bextract. With the the restore command in
bconsole, the file was not found.
My tests will be go on and it will go productive in some weeks on hundred of
servers. I hope to get more information I can provide here, but I sure, the
problem is sitting not before the keyboard.
----------------------------------------------------------------------
(0006714) mkress (reporter) - 2013-05-21 08:10
http://bugs.bacula.org/view.php?id=1995#c6714
----------------------------------------------------------------------
I tried in the last days 48 backups and verifies every 2h on vtapes (backup to
files). All backups were ok, but got 6 times a verify error. It's not the same
error as I got in my bug report. It seems not related to the problems in my bug
report, but it's also very strange. I got every 6 times a "Volume data error"
during verify, but the system have no Problems with the disks or any other
errors (during backup and verify). All Logs are ok, also the log of the Raid
controller. It's a HP server with 6 SAS Drives and Raid 5. It seems not to be a
hardware error. FD-Daemon runs on a SLES10 and SD/DIR on a SLES11. Both are
running on a VMWare ESXI-Server for HP Server with the newest Drivers. All
Software and Hardware are certified by Suse and VMWare. I put this into the bug
report for completeness.
18-May 11:15 iss-stor JobId 22: Forward spacing Volume "GLS_VTAPE-0036" to
file:block 0:209.
18-May 11:15 iss-stor JobId 22: Error: block.c:334 Volume data error at
0:115283150!
Block checksum mismatch in block=4776 len=64512: calc=c5a0196 blk=a9021e58
18-May 11:15 iss-stor JobId 22: Fatal error: fd_cmds.c:169 Command error with
FD, hanging up.
18-May 11:15 iss-dir JobId 22: Warning: The following files are in the Catalog
but not on the Volume(s):
Issue History
Date Modified Username Field Change
======================================================================
2013-04-12 08:02 mkress New Issue
2013-04-18 08:35 ebollengier Note Added: 0006656
2013-04-18 08:35 ebollengier Priority high => low
2013-04-18 08:35 ebollengier Severity major => minor
2013-04-19 15:32 mkress Note Added: 0006662
2013-05-10 09:33 kern Note Added: 0006699
2013-05-10 09:33 kern Status new => feedback
2013-05-10 10:50 mkress Note Added: 0006703
2013-05-10 10:50 mkress Status feedback => new
2013-05-21 08:10 mkress Note Added: 0006714
======================================================================

The following issue has been SUBMITTED.
======================================================================
http://bugs.bacula.org/view.php?id=2008
======================================================================
Reported By: atrox
Assigned To:
======================================================================
Project: bacula
Issue ID: 2008
Category: bat
Reproducibility: always
Severity: minor
Priority: normal
Status: new
======================================================================
Date Submitted: 2013-05-20 08:23 BST
Last Modified: 2013-05-20 08:23 BST
======================================================================
Summary: changing restore options has an unnecessary step
Description:
When restoring via Restore interface then after doing file selections and
pressing OK, a question pops up: "OK to run? (yes/mod/no)"
When there are some options to be changed (very typical would be 'where'), one
needs to write 'mod' in there and follow the questions. To my mind this question
is totally unnecessary - there already is the restore options form and OK/Cancel
buttons anyway.
When clicking Cancel to the question, the restore form stays on the active tab,
but changing anything in there and clicking OK results in an error.
Steps to Reproduce:
Open Restore
Select a job
Select files
Click OK
======================================================================
Issue History
Date Modified Username Field Change
======================================================================
2013-05-20 08:23 atrox New Issue
======================================================================

The following issue has been SUBMITTED.
======================================================================
http://bugs.bacula.org/view.php?id=2007
======================================================================
Reported By: slaanesh
Assigned To:
======================================================================
Project: bacula
Issue ID: 2007
Category: other
Reproducibility: always
Severity: major
Priority: normal
Status: new
======================================================================
Date Submitted: 2013-05-16 09:18 BST
Last Modified: 2013-05-16 09:18 BST
======================================================================
Summary: Configure does not support aarch64 [PATCH, 5.2.13]
Description:
Support for the ARM 64 bit CPU architecture (aarch64) was introduced in
autoconf 2.69.
bacula appears to use an earlier version of autoconf, preventing its being
built. The code is fine and builds / executes fine on aarch64; only configure
needs to be updated.
This can be fixed in two ways:
- Rerun autoconf or autoreconf with autoconf 2.69.
- Apply the linked patch which updates config.guess and config.sub to recognize
aarch64.
Steps to Reproduce:
Try to build on aarch64 (soon to be supported by all major Enterprise
distributions)
Additional Information:
http://ausil.fedorapeople.org/aarch64/bacula/bacula-aarch64.patch
======================================================================
Issue History
Date Modified Username Field Change
======================================================================
2013-05-16 09:18 slaanesh New Issue
======================================================================

The following issue has been CLOSED
======================================================================
http://bugs.bacula.org/view.php?id=1990
======================================================================
Reported By: hbrown
Assigned To:
======================================================================
Project: bacula
Issue ID: 1990
Category: Director
Reproducibility: N/A
Severity: minor
Priority: normal
Status: closed
Resolution: no change required
Fixed in Version:
======================================================================
Date Submitted: 2013-02-22 20:58 GMT
Last Modified: 2013-05-13 22:06 BST
======================================================================
Summary: 5.2.13 crashes when more than one Path
Description:
I've just upgraded from 5.2.12 to 5.2.13, and have found that the Bacula
director crashes if there is more than one Path. I've got a test job that
reproduces this crash perfectly in 5.2.13, and with 5.2.12 it just keeps on
going without problems.
First off, here's the error I see at the time of the crash:
22-Feb 09:47 cbs-01-dir JobId 227346: Warning: sql_create.c:590 More than one
Path!: 2 for path: /var/cfengine/.git/objects/37/
22-Feb 09:47 cbs-01-dir JobId 227346: Error: sql_create.c:596 error fetching
row:
22-Feb 09:47 cbs-01-dir: ERROR in sql_create.c:600 Failed ASSERT: ar->PathId
I've checked the database, and sure enough there *are* multiple rows returned
for this particular path:
mysql> SELECT * FROM Path WHERE Path='/var/cfengine/.git/objects/37/';
+---------+--------------------------------+
| PathId | Path |
+---------+--------------------------------+
| 5594948 | /var/cfengine/.git/objects/37/ |
| 5594949 | /var/cfengine/.git/objects/37/ |
+---------+--------------------------------+
2 rows in set (0.00 sec)
However, I have more than a few mentions of the "More than one Path" in logs
from previous jobs. Here's a sample of one, using Bacula 5.2.12, from December
2012, of the same fileset on the same host:
29-Dec 02:15 cbs-01-sd JobId 209631: Sending spooled attrs to the Director.
Despooling 1,940,283 bytes ...
[snip]
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:1016 More than one
Filename! 2 for file: a37007161975c06dd4c0a37b73b07bbdf29bc6
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:1016 More than one
Filename! 2 for file: f54f5cedb52bf7dcf7fe1fe4b2b797274bae67
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:590 More than one
Path!: 2 for path: /var/cfengine/.git/objects/37/
[snip]
Build OS: x86_64-unknown-linux-gnu redhat
JobId: 209631
Job: noc-var.2012-12-29_02.05.02_35
Backup Level: Differential, since=2012-12-01 02:09:22
Client: "noc-fd" 3.0.1 (30Apr09)
x86_64-redhat-linux-gnu,redhat,
FileSet: "var" 2009-07-09 12:17:34
Pool: "Daily" (From Run pool override)
Catalog: "MyCatalog" (From Client resource)
Storage: "tape" (From Job resource)
Scheduled time: 29-Dec-2012 02:05:02
Start time: 29-Dec-2012 02:06:03
End time: 29-Dec-2012 02:19:06
Elapsed time: 13 mins 3 secs
Priority: 10
FD Files Written: 6,248
SD Files Written: 6,248
FD Bytes Written: 2,145,282,121 (2.145 GB)
SD Bytes Written: 2,146,271,788 (2.146 GB)
Rate: 2739.8 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s): 000143
Volume Session Id: 3082
Volume Session Time: 1355945436
Last Volume Bytes: 611,754,200,064 (611.7 GB)
Non-fatal FD errors: 8
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK -- with warnings
(There were considerably more warnings about similar paths and filenames; let me
know if you'd like the whole thing.)
I've managed to get a backtrace (attached); I had to run it manually (run gdb as
bacula, apply commands from gdb_backtrace by hand), so let me know if I've left
anything out.
I've had a quick look at sql_create.c; I can't see any difference between the
two files, so I assume the problem is elsewhere. I've switched back to 5.2.12
and have successfully run this job, so it appears that *something* has broken in
5.2.13. The failure appears to be line 595:
593 /* Even if there are multiple paths, take the first one */
594 if (num_rows >= 1) {
595 if ((row = sql_fetch_row(mdb)) == NULL) {
596 Mmsg1(&mdb->errmsg, _("error fetching row: %s\n"),
sql_strerror(mdb));
597 Jmsg(jcr, M_ERROR, 0, "%s", mdb->errmsg);
I don't know why fetching the first row would fail.
I can test patches or switch back to 5.2.13 if needed. Thanks very much for
your help, and let me know if you need anything else from me.
Steps to Reproduce:
1. Have more than one entry for a particular Path in the Path table.
2. Run a job that includes that path in the files that are being backed up.
3. Crash.
Additional Information:
The "Product Version" field in Mantis did not include 5.2.13.
======================================================================
----------------------------------------------------------------------
(0006659) ebollengier (administrator) - 2013-04-18 08:54
http://bugs.bacula.org/view.php?id=1990#c6659
----------------------------------------------------------------------
Sorry you have problems with Bacula, however you are facing to a very strange
situation. Bacula ensures that only one Job can update the Path and the Filename
at the same time, and always inserts one copy of each filename/path, so for me,
this is impossible to get two filenames with the same value, unless you changed
something, or you played with the database by hand.
I'm dealing with billion of files, TB catalogs, and I never saw this problem
before. Something looks wrong with your MySQL catalog (perhaps some MyISAM
problem).
You must fix your catalog first, it is corrupted. dbcheck might have an option
to fix this situation, or you will have to do it yourself.
----------------------------------------------------------------------
(0006702) kern (administrator) - 2013-05-10 09:54
http://bugs.bacula.org/view.php?id=1990#c6702
----------------------------------------------------------------------
I believe that the failure is due to the ASSERT() at line 600 of sql_create.c
I am not 100% sure the ASSERT should be there, but it has been since 2009.
The difference in the Bacula versions you are seeing is very likely because in
the one failing, you have DEBUG turned on in <bacula>/src/version.h. Turn it
off and rebuild and Bacula will not crash.
Better yet, I recommend that you fix your database as it is broken. I think
dbcheck will clean it up, but I haven't looked at that program for a long time.
Also, you if you are using MySQL, you might wish to switch to PostgreSQL, which
is much more "ACID" than MySQL, and also performs better for large volumes,
though at first, PostgreSQL "seems" hard to understand compared to MySQL.
Please confirm my analysis.
----------------------------------------------------------------------
(0006711) hbrown (reporter) - 2013-05-13 17:59
http://bugs.bacula.org/view.php?id=1990#c6711
----------------------------------------------------------------------
Thanks for everyone's help. I'd forgotten entirely about the dbcheck utility;
I've run it and it has found a number of duplicate files. It's fixing them
right now; after that, I'll re-try the upgrade and verify that it works.
PostgreSQL is on the horizon, but I won't be switching right away. Thanks for
the tip about DEBUG being turned on.
Since the problem appears to be at my end, I've no objection to the ticket being
closed.
----------------------------------------------------------------------
(0006712) kern (administrator) - 2013-05-13 22:05
http://bugs.bacula.org/view.php?id=1990#c6712
----------------------------------------------------------------------
Problem resolved by cleaning the table with dbcheck, and possibly turning off
DEBUG mode.
Issue History
Date Modified Username Field Change
======================================================================
2013-02-22 20:58 hbrown New Issue
2013-02-22 21:00 hbrown File Added: rt_1503.gdb
2013-04-18 08:54 ebollengier Note Added: 0006659
2013-04-18 08:55 ebollengier Severity crash => minor
2013-04-18 08:55 ebollengier Reproducibility always => unable to
reproduce
2013-05-10 09:54 kern Note Added: 0006702
2013-05-10 09:54 kern Status new => feedback
2013-05-13 17:59 hbrown Note Added: 0006711
2013-05-13 17:59 hbrown Status feedback => new
2013-05-13 22:05 kern Note Added: 0006712
2013-05-13 22:06 kern Reproducibility unable to reproduce =>
N/A
2013-05-13 22:06 kern Status new => closed
2013-05-13 22:06 kern Resolution open => no change
required
======================================================================

A NOTE has been added to this issue.
======================================================================
http://bugs.bacula.org/view.php?id=1990
======================================================================
Reported By: hbrown
Assigned To:
======================================================================
Project: bacula
Issue ID: 1990
Category: Director
Reproducibility: unable to reproduce
Severity: minor
Priority: normal
Status: new
======================================================================
Date Submitted: 2013-02-22 20:58 GMT
Last Modified: 2013-05-13 22:05 BST
======================================================================
Summary: 5.2.13 crashes when more than one Path
Description:
I've just upgraded from 5.2.12 to 5.2.13, and have found that the Bacula
director crashes if there is more than one Path. I've got a test job that
reproduces this crash perfectly in 5.2.13, and with 5.2.12 it just keeps on
going without problems.
First off, here's the error I see at the time of the crash:
22-Feb 09:47 cbs-01-dir JobId 227346: Warning: sql_create.c:590 More than one
Path!: 2 for path: /var/cfengine/.git/objects/37/
22-Feb 09:47 cbs-01-dir JobId 227346: Error: sql_create.c:596 error fetching
row:
22-Feb 09:47 cbs-01-dir: ERROR in sql_create.c:600 Failed ASSERT: ar->PathId
I've checked the database, and sure enough there *are* multiple rows returned
for this particular path:
mysql> SELECT * FROM Path WHERE Path='/var/cfengine/.git/objects/37/';
+---------+--------------------------------+
| PathId | Path |
+---------+--------------------------------+
| 5594948 | /var/cfengine/.git/objects/37/ |
| 5594949 | /var/cfengine/.git/objects/37/ |
+---------+--------------------------------+
2 rows in set (0.00 sec)
However, I have more than a few mentions of the "More than one Path" in logs
from previous jobs. Here's a sample of one, using Bacula 5.2.12, from December
2012, of the same fileset on the same host:
29-Dec 02:15 cbs-01-sd JobId 209631: Sending spooled attrs to the Director.
Despooling 1,940,283 bytes ...
[snip]
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:1016 More than one
Filename! 2 for file: a37007161975c06dd4c0a37b73b07bbdf29bc6
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:1016 More than one
Filename! 2 for file: f54f5cedb52bf7dcf7fe1fe4b2b797274bae67
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:590 More than one
Path!: 2 for path: /var/cfengine/.git/objects/37/
[snip]
Build OS: x86_64-unknown-linux-gnu redhat
JobId: 209631
Job: noc-var.2012-12-29_02.05.02_35
Backup Level: Differential, since=2012-12-01 02:09:22
Client: "noc-fd" 3.0.1 (30Apr09)
x86_64-redhat-linux-gnu,redhat,
FileSet: "var" 2009-07-09 12:17:34
Pool: "Daily" (From Run pool override)
Catalog: "MyCatalog" (From Client resource)
Storage: "tape" (From Job resource)
Scheduled time: 29-Dec-2012 02:05:02
Start time: 29-Dec-2012 02:06:03
End time: 29-Dec-2012 02:19:06
Elapsed time: 13 mins 3 secs
Priority: 10
FD Files Written: 6,248
SD Files Written: 6,248
FD Bytes Written: 2,145,282,121 (2.145 GB)
SD Bytes Written: 2,146,271,788 (2.146 GB)
Rate: 2739.8 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s): 000143
Volume Session Id: 3082
Volume Session Time: 1355945436
Last Volume Bytes: 611,754,200,064 (611.7 GB)
Non-fatal FD errors: 8
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK -- with warnings
(There were considerably more warnings about similar paths and filenames; let me
know if you'd like the whole thing.)
I've managed to get a backtrace (attached); I had to run it manually (run gdb as
bacula, apply commands from gdb_backtrace by hand), so let me know if I've left
anything out.
I've had a quick look at sql_create.c; I can't see any difference between the
two files, so I assume the problem is elsewhere. I've switched back to 5.2.12
and have successfully run this job, so it appears that *something* has broken in
5.2.13. The failure appears to be line 595:
593 /* Even if there are multiple paths, take the first one */
594 if (num_rows >= 1) {
595 if ((row = sql_fetch_row(mdb)) == NULL) {
596 Mmsg1(&mdb->errmsg, _("error fetching row: %s\n"),
sql_strerror(mdb));
597 Jmsg(jcr, M_ERROR, 0, "%s", mdb->errmsg);
I don't know why fetching the first row would fail.
I can test patches or switch back to 5.2.13 if needed. Thanks very much for
your help, and let me know if you need anything else from me.
Steps to Reproduce:
1. Have more than one entry for a particular Path in the Path table.
2. Run a job that includes that path in the files that are being backed up.
3. Crash.
Additional Information:
The "Product Version" field in Mantis did not include 5.2.13.
======================================================================
----------------------------------------------------------------------
(0006659) ebollengier (administrator) - 2013-04-18 08:54
http://bugs.bacula.org/view.php?id=1990#c6659
----------------------------------------------------------------------
Sorry you have problems with Bacula, however you are facing to a very strange
situation. Bacula ensures that only one Job can update the Path and the Filename
at the same time, and always inserts one copy of each filename/path, so for me,
this is impossible to get two filenames with the same value, unless you changed
something, or you played with the database by hand.
I'm dealing with billion of files, TB catalogs, and I never saw this problem
before. Something looks wrong with your MySQL catalog (perhaps some MyISAM
problem).
You must fix your catalog first, it is corrupted. dbcheck might have an option
to fix this situation, or you will have to do it yourself.
----------------------------------------------------------------------
(0006702) kern (administrator) - 2013-05-10 09:54
http://bugs.bacula.org/view.php?id=1990#c6702
----------------------------------------------------------------------
I believe that the failure is due to the ASSERT() at line 600 of sql_create.c
I am not 100% sure the ASSERT should be there, but it has been since 2009.
The difference in the Bacula versions you are seeing is very likely because in
the one failing, you have DEBUG turned on in <bacula>/src/version.h. Turn it
off and rebuild and Bacula will not crash.
Better yet, I recommend that you fix your database as it is broken. I think
dbcheck will clean it up, but I haven't looked at that program for a long time.
Also, you if you are using MySQL, you might wish to switch to PostgreSQL, which
is much more "ACID" than MySQL, and also performs better for large volumes,
though at first, PostgreSQL "seems" hard to understand compared to MySQL.
Please confirm my analysis.
----------------------------------------------------------------------
(0006711) hbrown (reporter) - 2013-05-13 17:59
http://bugs.bacula.org/view.php?id=1990#c6711
----------------------------------------------------------------------
Thanks for everyone's help. I'd forgotten entirely about the dbcheck utility;
I've run it and it has found a number of duplicate files. It's fixing them
right now; after that, I'll re-try the upgrade and verify that it works.
PostgreSQL is on the horizon, but I won't be switching right away. Thanks for
the tip about DEBUG being turned on.
Since the problem appears to be at my end, I've no objection to the ticket being
closed.
----------------------------------------------------------------------
(0006712) kern (administrator) - 2013-05-13 22:05
http://bugs.bacula.org/view.php?id=1990#c6712
----------------------------------------------------------------------
Problem resolved by cleaning the table with dbcheck, and possibly turning off
DEBUG mode.
Issue History
Date Modified Username Field Change
======================================================================
2013-02-22 20:58 hbrown New Issue
2013-02-22 21:00 hbrown File Added: rt_1503.gdb
2013-04-18 08:54 ebollengier Note Added: 0006659
2013-04-18 08:55 ebollengier Severity crash => minor
2013-04-18 08:55 ebollengier Reproducibility always => unable to
reproduce
2013-05-10 09:54 kern Note Added: 0006702
2013-05-10 09:54 kern Status new => feedback
2013-05-13 17:59 hbrown Note Added: 0006711
2013-05-13 17:59 hbrown Status feedback => new
2013-05-13 22:05 kern Note Added: 0006712
======================================================================

A NOTE has been added to this issue.
======================================================================
http://bugs.bacula.org/view.php?id=1990
======================================================================
Reported By: hbrown
Assigned To:
======================================================================
Project: bacula
Issue ID: 1990
Category: Director
Reproducibility: unable to reproduce
Severity: minor
Priority: normal
Status: new
======================================================================
Date Submitted: 2013-02-22 20:58 GMT
Last Modified: 2013-05-13 17:59 BST
======================================================================
Summary: 5.2.13 crashes when more than one Path
Description:
I've just upgraded from 5.2.12 to 5.2.13, and have found that the Bacula
director crashes if there is more than one Path. I've got a test job that
reproduces this crash perfectly in 5.2.13, and with 5.2.12 it just keeps on
going without problems.
First off, here's the error I see at the time of the crash:
22-Feb 09:47 cbs-01-dir JobId 227346: Warning: sql_create.c:590 More than one
Path!: 2 for path: /var/cfengine/.git/objects/37/
22-Feb 09:47 cbs-01-dir JobId 227346: Error: sql_create.c:596 error fetching
row:
22-Feb 09:47 cbs-01-dir: ERROR in sql_create.c:600 Failed ASSERT: ar->PathId
I've checked the database, and sure enough there *are* multiple rows returned
for this particular path:
mysql> SELECT * FROM Path WHERE Path='/var/cfengine/.git/objects/37/';
+---------+--------------------------------+
| PathId | Path |
+---------+--------------------------------+
| 5594948 | /var/cfengine/.git/objects/37/ |
| 5594949 | /var/cfengine/.git/objects/37/ |
+---------+--------------------------------+
2 rows in set (0.00 sec)
However, I have more than a few mentions of the "More than one Path" in logs
from previous jobs. Here's a sample of one, using Bacula 5.2.12, from December
2012, of the same fileset on the same host:
29-Dec 02:15 cbs-01-sd JobId 209631: Sending spooled attrs to the Director.
Despooling 1,940,283 bytes ...
[snip]
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:1016 More than one
Filename! 2 for file: a37007161975c06dd4c0a37b73b07bbdf29bc6
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:1016 More than one
Filename! 2 for file: f54f5cedb52bf7dcf7fe1fe4b2b797274bae67
29-Dec 02:18 cbs-01-dir JobId 209631: Warning: sql_create.c:590 More than one
Path!: 2 for path: /var/cfengine/.git/objects/37/
[snip]
Build OS: x86_64-unknown-linux-gnu redhat
JobId: 209631
Job: noc-var.2012-12-29_02.05.02_35
Backup Level: Differential, since=2012-12-01 02:09:22
Client: "noc-fd" 3.0.1 (30Apr09)
x86_64-redhat-linux-gnu,redhat,
FileSet: "var" 2009-07-09 12:17:34
Pool: "Daily" (From Run pool override)
Catalog: "MyCatalog" (From Client resource)
Storage: "tape" (From Job resource)
Scheduled time: 29-Dec-2012 02:05:02
Start time: 29-Dec-2012 02:06:03
End time: 29-Dec-2012 02:19:06
Elapsed time: 13 mins 3 secs
Priority: 10
FD Files Written: 6,248
SD Files Written: 6,248
FD Bytes Written: 2,145,282,121 (2.145 GB)
SD Bytes Written: 2,146,271,788 (2.146 GB)
Rate: 2739.8 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s): 000143
Volume Session Id: 3082
Volume Session Time: 1355945436
Last Volume Bytes: 611,754,200,064 (611.7 GB)
Non-fatal FD errors: 8
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK -- with warnings
(There were considerably more warnings about similar paths and filenames; let me
know if you'd like the whole thing.)
I've managed to get a backtrace (attached); I had to run it manually (run gdb as
bacula, apply commands from gdb_backtrace by hand), so let me know if I've left
anything out.
I've had a quick look at sql_create.c; I can't see any difference between the
two files, so I assume the problem is elsewhere. I've switched back to 5.2.12
and have successfully run this job, so it appears that *something* has broken in
5.2.13. The failure appears to be line 595:
593 /* Even if there are multiple paths, take the first one */
594 if (num_rows >= 1) {
595 if ((row = sql_fetch_row(mdb)) == NULL) {
596 Mmsg1(&mdb->errmsg, _("error fetching row: %s\n"),
sql_strerror(mdb));
597 Jmsg(jcr, M_ERROR, 0, "%s", mdb->errmsg);
I don't know why fetching the first row would fail.
I can test patches or switch back to 5.2.13 if needed. Thanks very much for
your help, and let me know if you need anything else from me.
Steps to Reproduce:
1. Have more than one entry for a particular Path in the Path table.
2. Run a job that includes that path in the files that are being backed up.
3. Crash.
Additional Information:
The "Product Version" field in Mantis did not include 5.2.13.
======================================================================
----------------------------------------------------------------------
(0006659) ebollengier (administrator) - 2013-04-18 08:54
http://bugs.bacula.org/view.php?id=1990#c6659
----------------------------------------------------------------------
Sorry you have problems with Bacula, however you are facing to a very strange
situation. Bacula ensures that only one Job can update the Path and the Filename
at the same time, and always inserts one copy of each filename/path, so for me,
this is impossible to get two filenames with the same value, unless you changed
something, or you played with the database by hand.
I'm dealing with billion of files, TB catalogs, and I never saw this problem
before. Something looks wrong with your MySQL catalog (perhaps some MyISAM
problem).
You must fix your catalog first, it is corrupted. dbcheck might have an option
to fix this situation, or you will have to do it yourself.
----------------------------------------------------------------------
(0006702) kern (administrator) - 2013-05-10 09:54
http://bugs.bacula.org/view.php?id=1990#c6702
----------------------------------------------------------------------
I believe that the failure is due to the ASSERT() at line 600 of sql_create.c
I am not 100% sure the ASSERT should be there, but it has been since 2009.
The difference in the Bacula versions you are seeing is very likely because in
the one failing, you have DEBUG turned on in <bacula>/src/version.h. Turn it
off and rebuild and Bacula will not crash.
Better yet, I recommend that you fix your database as it is broken. I think
dbcheck will clean it up, but I haven't looked at that program for a long time.
Also, you if you are using MySQL, you might wish to switch to PostgreSQL, which
is much more "ACID" than MySQL, and also performs better for large volumes,
though at first, PostgreSQL "seems" hard to understand compared to MySQL.
Please confirm my analysis.
----------------------------------------------------------------------
(0006711) hbrown (reporter) - 2013-05-13 17:59
http://bugs.bacula.org/view.php?id=1990#c6711
----------------------------------------------------------------------
Thanks for everyone's help. I'd forgotten entirely about the dbcheck utility;
I've run it and it has found a number of duplicate files. It's fixing them
right now; after that, I'll re-try the upgrade and verify that it works.
PostgreSQL is on the horizon, but I won't be switching right away. Thanks for
the tip about DEBUG being turned on.
Since the problem appears to be at my end, I've no objection to the ticket being
closed.
Issue History
Date Modified Username Field Change
======================================================================
2013-02-22 20:58 hbrown New Issue
2013-02-22 21:00 hbrown File Added: rt_1503.gdb
2013-04-18 08:54 ebollengier Note Added: 0006659
2013-04-18 08:55 ebollengier Severity crash => minor
2013-04-18 08:55 ebollengier Reproducibility always => unable to
reproduce
2013-05-10 09:54 kern Note Added: 0006702
2013-05-10 09:54 kern Status new => feedback
2013-05-13 17:59 hbrown Note Added: 0006711
2013-05-13 17:59 hbrown Status feedback => new
======================================================================

A NOTE has been added to this issue.
======================================================================
http://bugs.bacula.org/view.php?id=1979
======================================================================
Reported By: angeloio
Assigned To:
======================================================================
Project: bacula
Issue ID: 1979
Category: File Daemon
Reproducibility: always
Severity: minor
Priority: normal
Status: feedback
======================================================================
Date Submitted: 2013-01-15 16:24 GMT
Last Modified: 2013-05-11 08:29 BST
======================================================================
Summary: bacula vss windows verify job file count mismatch
Description:
director and storage daemon on debian 6 (5.2.6+dfsg-1~b); file daemon on windows
2008 r2 standard 64 sp1.
if you enable VSS options for backup the verify job fails because the file
number verified by the verify job mismatch respect the numebre of file backed
up.
Steps to Reproduce:
a) enable Windows VSS for backup of e.g. 3 files on Windows daemon. Backup
complete succefully with backup of 4 file! I can see with trace enabled a GUID
added to backup for snapshot VSS pourposes.
b) launch verify on that job:
b1) the verify type volume-to-catalog fail cause number of files is mismatch,
(minus one): file expected 4, file verified 3.
b2) launch verify type disk-to-catalog end with warnings, but the number of file
verified is always zero!
Additional Information:
This issue does not occur with VSS disabled; with Windows 2008 R2 Enterprise.
In the catalog the informations about the file related to the guid are NOT
present.
This issue occur even with bacula director version on 5.0.2
This issue occur even with bacula windows 5.0.2, 5.2.5 and 5.2.5.
======================================================================
----------------------------------------------------------------------
(0006610) kern (administrator) - 2013-02-15 21:29
http://bugs.bacula.org/view.php?id=1979#c6610
----------------------------------------------------------------------
This appears to be a support problem, and nothing you report seems abnormal to
me.
So that I can be sure, please upload your FileSet resource, then after the
backup, run a restore menu item 5 and do "mark *" then "lsmark". I suspect
that you will find that four "files" were backed up. Please include the
complete
output from running those commands.
----------------------------------------------------------------------
(0006688) Steve Lee (reporter) - 2013-05-01 22:30
http://bugs.bacula.org/view.php?id=1979#c6688
----------------------------------------------------------------------
I have the same error
Running Bacula 5.2.6 on Linux version 3.5.0-22
Client running on bacula-win64-5.2.10 on Windows Server 2008 (SP1)
We recently added verify jobs to our schedule and have noticed that one of the
jobs always fails. it shows a mismatch between expected and examined files.
Files Expected: 2
Files Examined: 1
The backup job itself reports 2 files backed up
FD Files Written: 2
SD Files Written: 2
Running queries against the catalog...
Enter SQL query: select jobfiles from job where jobid = 988;
+----------+
| jobfiles |
+----------+
| 2 |
+----------+
Enter SQL query: select count(*) from file where jobid =988;
+-------+
| count |
+-------+
| 1 |
+-------+
The fileset contains a single file
FileSet {
Name = "zmail-delta"
Include {
Options {
signature = MD5
compression = GZIP
aclsupport = yes
}
File = "E:/WindowsImageBackup.delta"
}
}
----------------------------------------------------------------------
(0006689) Steve Lee (reporter) - 2013-05-01 22:34
http://bugs.bacula.org/view.php?id=1979#c6689
----------------------------------------------------------------------
Output from running lsmark
$ ls
E:/
$ mark *
2 files marked.
$ lsmark
*E:/
*WindowsImageBackup.delta
$
----------------------------------------------------------------------
(0006690) martin (reporter) - 2013-05-02 11:50
http://bugs.bacula.org/view.php?id=1979#c6690
----------------------------------------------------------------------
I think it is caused by the job_metadata.xml object, which is not the file
table.
----------------------------------------------------------------------
(0006691) Steve Lee (reporter) - 2013-05-05 08:06
http://bugs.bacula.org/view.php?id=1979#c6691
----------------------------------------------------------------------
I reran with Enable VSS = no and this time the verify job worked
FD Files Written: 1
SD Files Written: 1
Files Expected: 1
Files Examined: 1
Enter SQL query: select jobfiles from job where jobid = 1179;
+----------+
| jobfiles |
+----------+
| 1 |
+----------+
Enter SQL query: select count(*) from file where jobid =1179;
+-------+
| count |
+-------+
| 1 |
+-------+
$ mark *
2 files marked.
$ lsmark
*E:/
*WindowsImageBackup.delta
$
----------------------------------------------------------------------
(0006705) kern (administrator) - 2013-05-10 13:51
http://bugs.bacula.org/view.php?id=1979#c6705
----------------------------------------------------------------------
For Martin: what is the job_metadata.xml object? I never heard of that. The
only xml that I am aware of is what the Bacula Enterprise plugin produces when
it is doing application VSS backups, which are entirely different from what is
happening in the FileSet shown above.
For Steve: Your SQL queries don't mean a lot to me. What is more important is
what the output of the job log is. I don't really understand what you are doing,
that is when you are running with EnableVSS=yes/no and whether those are backup
jobs or verify jobs. Also, it would help if you would show the output of what
you say "it is failing". As I say at the current time, it is not clear exactly
what you are doing, and exactly what is failing or why.
Possibly what is happening is that the entry for the directory E: is being
ignored by the verify job, but then you haven't shown anything concerning it --
neither the FileSet your are using nor the Job resource, nor the output.
----------------------------------------------------------------------
(0006706) martin (reporter) - 2013-05-10 15:27
http://bugs.bacula.org/view.php?id=1979#c6706
----------------------------------------------------------------------
It is all in the source code.
$ grep -Ir 'job_metadata\.xml' src
src/filed/backup.c: ff_pkt->object_name = (char *)"job_metadata.xml";
src/filed/job.c: if (strcmp(rop.object_name, "job_metadata.xml") == 0) {
$
Strangely though, it doesn't seem to be used in the restore code.
----------------------------------------------------------------------
(0006707) kern (administrator) - 2013-05-10 16:45
http://bugs.bacula.org/view.php?id=1979#c6707
----------------------------------------------------------------------
Yes, you are right. The code that writes out the job_metadata.xml really should
do it only when the vss plugin is loaded and tells Bacula to save it. This is an
oversight on my part. The RestoreObjects are only used by the vss plugin during
application specific restores and not by Bacula core code. So saving this
metadata is a "minor" inefficiency that I will fix.
The RestoreObjects don't enter into any counting during restore or verifies so
the user's problem lies elsewhere.
----------------------------------------------------------------------
(0006709) martin (reporter) - 2013-05-10 18:08
http://bugs.bacula.org/view.php?id=1979#c6709
----------------------------------------------------------------------
You are right that RestoreObjects don't enter into any counting during restore
or verifies, but I suspect that they *do* enter into the counting during backup
(assuming encode_and_send_attributes is called and hence JobFiles is
incremented). That causes the mismatch of job.jobfiles (2) v.s. the number of
rows in the file table (1).
----------------------------------------------------------------------
(0006710) kern (administrator) - 2013-05-11 08:29
http://bugs.bacula.org/view.php?id=1979#c6710
----------------------------------------------------------------------
RestoreObject shouldn't be sent off to the SD. I would be surprised if they are
(though I have been surprised before).
They should and will someday be sent off to the SD, but obviously like any other
Bacula metadata they should not be counted as a file. RestoreObjects are just a
form of metadata that (for backup or tape unfriendly designs such as Microsoft
VSS) can be written at the end of a job but restored at the beginning of a
restore job.
Issue History
Date Modified Username Field Change
======================================================================
2013-01-15 16:24 angeloio New Issue
2013-02-15 21:29 kern Note Added: 0006610
2013-02-15 21:29 kern Status new => feedback
2013-05-01 22:30 Steve Lee Note Added: 0006688
2013-05-01 22:34 Steve Lee Note Added: 0006689
2013-05-02 11:50 martin Note Added: 0006690
2013-05-05 08:06 Steve Lee Note Added: 0006691
2013-05-07 15:37 ebollengier Severity major => minor
2013-05-10 13:51 kern Note Added: 0006705
2013-05-10 15:27 martin Note Added: 0006706
2013-05-10 16:45 kern Note Added: 0006707
2013-05-10 18:08 martin Note Added: 0006709
2013-05-11 08:29 kern Note Added: 0006710
======================================================================

A NOTE has been added to this issue.
======================================================================
http://bugs.bacula.org/view.php?id=1979
======================================================================
Reported By: angeloio
Assigned To:
======================================================================
Project: bacula
Issue ID: 1979
Category: File Daemon
Reproducibility: always
Severity: minor
Priority: normal
Status: feedback
======================================================================
Date Submitted: 2013-01-15 16:24 GMT
Last Modified: 2013-05-10 18:08 BST
======================================================================
Summary: bacula vss windows verify job file count mismatch
Description:
director and storage daemon on debian 6 (5.2.6+dfsg-1~b); file daemon on windows
2008 r2 standard 64 sp1.
if you enable VSS options for backup the verify job fails because the file
number verified by the verify job mismatch respect the numebre of file backed
up.
Steps to Reproduce:
a) enable Windows VSS for backup of e.g. 3 files on Windows daemon. Backup
complete succefully with backup of 4 file! I can see with trace enabled a GUID
added to backup for snapshot VSS pourposes.
b) launch verify on that job:
b1) the verify type volume-to-catalog fail cause number of files is mismatch,
(minus one): file expected 4, file verified 3.
b2) launch verify type disk-to-catalog end with warnings, but the number of file
verified is always zero!
Additional Information:
This issue does not occur with VSS disabled; with Windows 2008 R2 Enterprise.
In the catalog the informations about the file related to the guid are NOT
present.
This issue occur even with bacula director version on 5.0.2
This issue occur even with bacula windows 5.0.2, 5.2.5 and 5.2.5.
======================================================================
----------------------------------------------------------------------
(0006610) kern (administrator) - 2013-02-15 21:29
http://bugs.bacula.org/view.php?id=1979#c6610
----------------------------------------------------------------------
This appears to be a support problem, and nothing you report seems abnormal to
me.
So that I can be sure, please upload your FileSet resource, then after the
backup, run a restore menu item 5 and do "mark *" then "lsmark". I suspect
that you will find that four "files" were backed up. Please include the
complete
output from running those commands.
----------------------------------------------------------------------
(0006688) Steve Lee (reporter) - 2013-05-01 22:30
http://bugs.bacula.org/view.php?id=1979#c6688
----------------------------------------------------------------------
I have the same error
Running Bacula 5.2.6 on Linux version 3.5.0-22
Client running on bacula-win64-5.2.10 on Windows Server 2008 (SP1)
We recently added verify jobs to our schedule and have noticed that one of the
jobs always fails. it shows a mismatch between expected and examined files.
Files Expected: 2
Files Examined: 1
The backup job itself reports 2 files backed up
FD Files Written: 2
SD Files Written: 2
Running queries against the catalog...
Enter SQL query: select jobfiles from job where jobid = 988;
+----------+
| jobfiles |
+----------+
| 2 |
+----------+
Enter SQL query: select count(*) from file where jobid =988;
+-------+
| count |
+-------+
| 1 |
+-------+
The fileset contains a single file
FileSet {
Name = "zmail-delta"
Include {
Options {
signature = MD5
compression = GZIP
aclsupport = yes
}
File = "E:/WindowsImageBackup.delta"
}
}
----------------------------------------------------------------------
(0006689) Steve Lee (reporter) - 2013-05-01 22:34
http://bugs.bacula.org/view.php?id=1979#c6689
----------------------------------------------------------------------
Output from running lsmark
$ ls
E:/
$ mark *
2 files marked.
$ lsmark
*E:/
*WindowsImageBackup.delta
$
----------------------------------------------------------------------
(0006690) martin (reporter) - 2013-05-02 11:50
http://bugs.bacula.org/view.php?id=1979#c6690
----------------------------------------------------------------------
I think it is caused by the job_metadata.xml object, which is not the file
table.
----------------------------------------------------------------------
(0006691) Steve Lee (reporter) - 2013-05-05 08:06
http://bugs.bacula.org/view.php?id=1979#c6691
----------------------------------------------------------------------
I reran with Enable VSS = no and this time the verify job worked
FD Files Written: 1
SD Files Written: 1
Files Expected: 1
Files Examined: 1
Enter SQL query: select jobfiles from job where jobid = 1179;
+----------+
| jobfiles |
+----------+
| 1 |
+----------+
Enter SQL query: select count(*) from file where jobid =1179;
+-------+
| count |
+-------+
| 1 |
+-------+
$ mark *
2 files marked.
$ lsmark
*E:/
*WindowsImageBackup.delta
$
----------------------------------------------------------------------
(0006705) kern (administrator) - 2013-05-10 13:51
http://bugs.bacula.org/view.php?id=1979#c6705
----------------------------------------------------------------------
For Martin: what is the job_metadata.xml object? I never heard of that. The
only xml that I am aware of is what the Bacula Enterprise plugin produces when
it is doing application VSS backups, which are entirely different from what is
happening in the FileSet shown above.
For Steve: Your SQL queries don't mean a lot to me. What is more important is
what the output of the job log is. I don't really understand what you are doing,
that is when you are running with EnableVSS=yes/no and whether those are backup
jobs or verify jobs. Also, it would help if you would show the output of what
you say "it is failing". As I say at the current time, it is not clear exactly
what you are doing, and exactly what is failing or why.
Possibly what is happening is that the entry for the directory E: is being
ignored by the verify job, but then you haven't shown anything concerning it --
neither the FileSet your are using nor the Job resource, nor the output.
----------------------------------------------------------------------
(0006706) martin (reporter) - 2013-05-10 15:27
http://bugs.bacula.org/view.php?id=1979#c6706
----------------------------------------------------------------------
It is all in the source code.
$ grep -Ir 'job_metadata\.xml' src
src/filed/backup.c: ff_pkt->object_name = (char *)"job_metadata.xml";
src/filed/job.c: if (strcmp(rop.object_name, "job_metadata.xml") == 0) {
$
Strangely though, it doesn't seem to be used in the restore code.
----------------------------------------------------------------------
(0006707) kern (administrator) - 2013-05-10 16:45
http://bugs.bacula.org/view.php?id=1979#c6707
----------------------------------------------------------------------
Yes, you are right. The code that writes out the job_metadata.xml really should
do it only when the vss plugin is loaded and tells Bacula to save it. This is an
oversight on my part. The RestoreObjects are only used by the vss plugin during
application specific restores and not by Bacula core code. So saving this
metadata is a "minor" inefficiency that I will fix.
The RestoreObjects don't enter into any counting during restore or verifies so
the user's problem lies elsewhere.
----------------------------------------------------------------------
(0006709) martin (reporter) - 2013-05-10 18:08
http://bugs.bacula.org/view.php?id=1979#c6709
----------------------------------------------------------------------
You are right that RestoreObjects don't enter into any counting during restore
or verifies, but I suspect that they *do* enter into the counting during backup
(assuming encode_and_send_attributes is called and hence JobFiles is
incremented). That causes the mismatch of job.jobfiles (2) v.s. the number of
rows in the file table (1).
Issue History
Date Modified Username Field Change
======================================================================
2013-01-15 16:24 angeloio New Issue
2013-02-15 21:29 kern Note Added: 0006610
2013-02-15 21:29 kern Status new => feedback
2013-05-01 22:30 Steve Lee Note Added: 0006688
2013-05-01 22:34 Steve Lee Note Added: 0006689
2013-05-02 11:50 martin Note Added: 0006690
2013-05-05 08:06 Steve Lee Note Added: 0006691
2013-05-07 15:37 ebollengier Severity major => minor
2013-05-10 13:51 kern Note Added: 0006705
2013-05-10 15:27 martin Note Added: 0006706
2013-05-10 16:45 kern Note Added: 0006707
2013-05-10 18:08 martin Note Added: 0006709
======================================================================

The following issue requires your FEEDBACK.
======================================================================
http://bugs.bacula.org/view.php?id=1980
======================================================================
Reported By: johe.stephan
Assigned To: ebollengier
======================================================================
Project: bacula
Issue ID: 1980
Category: sql
Reproducibility: always
Severity: minor
Priority: normal
Status: feedback
======================================================================
Date Submitted: 2013-01-23 08:39 GMT
Last Modified: 2013-05-10 16:54 BST
======================================================================
Summary: SQL Failure while using MariaDb as backend
Description:
When using the MariaDB 5.5. as SQL backend the following error eccours
Jan 22 23:05:18 stor-01.infra.igb.gtld.key-systems.net mysqld: 130122 23:05:18
[Warning] Unsafe statement written to the binary log using statement format
since BINLOG_FORMAT = STATEMENT. Statements writing to a table with an
auto-increment column after selecting from another table are unsafe because the
order in which rows are retrieved determines what (if any) rows will be written.
This order cannot be predicted and may differ on master and the slave.
Statement: INSERT INTO File (FileIndex, JobId, PathId, FilenameId, LStat, MD5,
DeltaSeq) SELECT batch.FileIndex, batch.JobId, Path.PathId,
Filename.FilenameId,batch.LStat, batch.MD5, batch.DeltaSeq FROM batch JOIN Path
ON (batch.Path = Path.Path) JOIN Filename ON (batch.Name = Filename.Name)
Steps to Reproduce:
Just happens
======================================================================
----------------------------------------------------------------------
(0006593) ebollengier (administrator) - 2013-01-23 10:32
http://bugs.bacula.org/view.php?id=1980#c6593
----------------------------------------------------------------------
Sorry you have problems with Bacula, however this is a specific support issue
with MariaDB and how the replication is implemented in MySQL. Check the MariaDB
configuration to change the binlog_format to better support AutoIncrement fields
(something such as "Mixed", but I'm not an expert) or raise this problem to
MariaDB team.
A database replication scheme that is not using block replication is very likely
to have this kind of issues, for example, PostgreSQL built-in replication
doesn't have this kind of problem, this is clearly a database backend problem.
This is very likely that your slave is corrupted and you have to rebuilt it.
----------------------------------------------------------------------
(0006708) kern (administrator) - 2013-05-10 16:54
http://bugs.bacula.org/view.php?id=1980#c6708
----------------------------------------------------------------------
I would like to understand why you consider this an "error" or a "failure" (you
used both terms). As far as I can tell, and I am not an expert, this is a
warning message, as indicated in the message, that is produced in the MariaDB
log file.
It seems to me that it is relatively harmless and perhaps you could confirm that
it is harmless and maybe tell us how to turn off such harmless warning messages.
Issue History
Date Modified Username Field Change
======================================================================
2013-01-23 08:39 johe.stephan New Issue
2013-01-23 10:32 ebollengier Note Added: 0006593
2013-01-23 10:32 ebollengier Status new => closed
2013-01-23 10:32 ebollengier Assigned To => ebollengier
2013-01-23 10:32 ebollengier Resolution open => no change
required
2013-05-10 16:54 kern Note Added: 0006708
2013-05-10 16:54 kern Status closed => feedback
2013-05-10 16:54 kern Resolution no change required =>
reopened
======================================================================