No matter how much I set the HISTSIZE environment variable to be larger than 5000, when printing the history list with the history builtin, it prints only the last 5000 commands.
I need that because I often have a large .bash_history which exceeds 5000 lines, and sometimes one needs to address an early command by pressing Ctrl-R, but if that command is more than 5000 commands earlier, I can't access it using that mechanism. I know I can use grep on the .bash_history, but I think the Ctrl-R mechanism would be much more faster (and convenient). I use gnu bash version 4.1.

That is the full content of my .bashrc file:

#!/bin/bash
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples
# If not running interactively, don't do anything
[ -z "$PS1" ] && return
# don't put duplicate lines in the history. See bash(1) for more options
# ... or force ignoredups and ignorespace
#HISTCONTROL=ignoredups:ignorespace:erasedups
# append to the history file, don't overwrite it
shopt -s histappend
# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=50000
HISTFILESIZE=500000
# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize
# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"
# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "$debian_chroot" ] && [ -r /etc/debian_chroot ]; then
debian_chroot=$(cat /etc/debian_chroot)
fi
# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
xterm-color) color_prompt=yes;;
esac
# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes
if [ -n "$force_color_prompt" ]; then
if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
# We have color support; assume it's compliant with Ecma-48
# (ISO/IEC-6429). (Lack of such support is extremely rare, and such
# a case would tend to support setf rather than setaf.)
color_prompt=yes
else
color_prompt=
fi
fi
if [ "$color_prompt" = yes ]; then
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\ [\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
PS1='${debian_chroot:+($debian_chroot)}\@-\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt
# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
;;
*)
;;
esac
# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
alias ls='ls --color=auto'
#alias dir='dir --color=auto'
#alias vdir='vdir --color=auto'
alias grep='grep --color=auto'
alias fgrep='fgrep --color=auto'
alias egrep='egrep --color=auto'
fi
# some more ls aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'
# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if [ -f /etc/bash_completion ] && ! shopt -oq posix; then
. /etc/bash_completion
fi

This question came from our site for professional and enthusiast programmers.

I can't reproduce this with bash 4.1 or 4.2, HISTSIZE=9999 HISTFILESIZE=999 set in .bashrc, and a 6000-line .bash_history all of which show up in the output of history. Tell us your version of bash and where you got it from, and the full contents of your .bashrc.
–
GillesSep 16 '11 at 23:28

thanks for the reply, but how your .bash_history is 6000-line and meanwhile HISTFILESIZE=999 ? I use GNU bassh version 4.1
–
MarwanSep 17 '11 at 2:31

Sorry, that was a typo: I had HISTFILESIZE=9999. The .bash_history was artificially constructed for the test (I didn't want to type 6000 commands at the prompt), but bash does save it properly on exit. Please copy-paste your full .bashrc into the question.
–
GillesSep 17 '11 at 10:44

If you do a history | wc -l , how many lines are shown?
–
Tim Post♦Sep 17 '11 at 13:50

If the values of HISTSIZE and HISTFILESIZE are set, they will be used.

Readline, the library that actually handles input / line editing and history does offer facilities to put a cap on just how big the history buffer can grow. However, Bash does not place a hard ceiling on this where values any larger would be ignored, at least that I could find.

Edit

From comments, readline was indeed the culprit. I was looking (rather foolishly) at functional parameters:

there is a variable called history-size that can be read from the inputrc file. that variable sets the maximum number of history entries saved in the history list. I checked it's value in my local inputrc file to found it equal 5000. Setting it to a larger value solved the problem.

if it doesn't place a ceiling, then why setting HISTSIZE to any value greater than 5000 has no effect on the size of the history list after restarting the shell ? If you have a history file of size greater than 5000 lines try setting HISTSIZE in .bashrc to a value greater than 5000, then restart the shell and execute history | wc -l. You would see that the history list is less than or equal to 5000 regardless of HISTSIZE. However, setting HISTSIZE to any value less than 5000 would produce a visible effect using the same experiment.
–
MarwanSep 17 '11 at 17:15

3

Reading the GNU readline library documentatin, it turned out that you are right. there is a variable called history-size that can be read from the inputrc file. that variable sets the maximum number of history entries saved in the history list. I checked it's value in my local inputrc file to found it equal 5000. Setting it to a larger value solved the problem. Thanks for the insights :-)
–
MarwanSep 17 '11 at 18:14

@Marwan Awesome :) I thought that history-size was something that was passed (from the RL changelog) to the functions in readline tha t were ultimately called by bash. Looks like we figured it out together.
–
Tim Post♦Sep 17 '11 at 18:54

I set HISTFILESIZE to be 50000. The problem is with HISTSIZE which determines the last 'HISTSIZE' lines in .bash_history to be loaded into memory when starting bash and upon which you can use the ctrl-r mechanism.
–
MarwanSep 16 '11 at 17:33

Once you've logged in if you type echo "$HISTSIZE $HISTFILESIZE" what do you see?
–
ztank1013Sep 16 '11 at 18:41

I had the same (or a similar) problem, but inputrc was fine. In my case, the only thing that worked was commenting out HISTSIZE=1000 and HISTFILESIZE=2000 in my stock ~/.bashrc -- even though I was overriding those vars later in the same file!

I think you might be hitting a history ceiling on your OS for the HISTSIZE. From the man page for fc/history in Solaris 10 (running KSH):

[snip]

/usr/bin/fc

The fc utility lists or edits and reexecutes, commands pre-
viously entered to an interactive sh.

The command history list references commands by number. The
first number in the list is selected arbitrarily. The rela-
tionship of a number to its command will not change except
when the user logs in and no other process is accessing the
list, at which time the system may reset the numbering to
start the oldest retained command at another number (usually
1). When the number reaches the value in HISTSIZE or 32767
(whichever is greater), the shell may wrap the numbers,
starting the next command with a lower number (usually 1).
However, despite this optional wrapping of numbers, fc will
maintain the time-ordering sequence of the commands. For
example, if four commands in sequence are given the numbers
32 766, 32 767, 1 (wrapped), and 2 as they are executed,
command 32 767 is considered the command previous to 1, even
though its number is higher.

[snip]

which implies that the fc command can address up to 32767 entries in the history file, making it a hard ceiling for the number of commands kept in the history file. Of course YMMV, but I think you could consult your OS documentation/man pages on this matter.
My 0.02...