This token can be used in reads and when packing/unpacking to indicate that
you don’t care about the contents of these bits. Any padding bits will just
be skipped over when reading/unpacking or zero-filled when packing.

>>> a,b=s.readlist('pad:5, uint:3, pad:1, uint:3')

Here only two items are returned in the list - the padding bits are ignored.

If you ask for the hex, octal or binary representations of a bitstring then
they will no longer be prefixed with 0x, 0o or 0b. This was done as it
was noticed that the first thing a lot of user code does after getting these
representations was to cut off the first two characters before further
processing.

The step parameter in __getitem__, __setitem__ and __delitem__ used to act
as a multiplier for the start and stop parameters. No one seemed to use it
though and so it has now reverted to the convential meaning for containers.

If you are using step then recoding is simple: s[a:b:c] becomes s[a*c:b*c].

A number of methods take a bytealigned parameter to indicate that they
should only work on byte boundaries (e.g. find, replace, split). Previously
this parameter defaulted to False. Instead it now defaults to
bitstring.bytealigned, which itself defaults to False, but can be changed
to modify the default behaviour of the methods. For example:

Previously there were just two classes, the immutable Bits which was the base
class for the mutable BitString class. Both of these classes have the concept
of a bit position, from which reads etc. take place so that the bitstring could
be treated as if it were a file or stream.

Two simpler classes have now been added which are purely bit containers and
don’t have a bit position. These are called ConstBitArray and BitArray. As you
can guess the former is an immutable version of the latter.

The other classes have also been renamed to better reflect their capabilities.
Instead of BitString you should use BitStream, and instead of Bits you can use
ConstBitStream. The old names are kept as aliases for backward compatibility.

A lot of internal reorganisation has taken place since the previous version,
most of which won’t be noticed by the end user. Some things you might see are:

New package structure. Previous versions have been a single file for the
module and another for the unit tests. The module is now split into many
more files so it can’t be used just by copying bitstring.py any more.

To run the unit tests there is now a script called runtests.py in the test
directory.

File based bitstring are now implemented in terms of an mmap. This should
be just an implementation detail, but unfortunately for 32-bit versions of
Python this creates a limit of 4GB on the files that can be used. The work
around is either to get a 64-bit Python, or just stick with version 2.0.

The ConstBitArray and ConstBitStream classes no longer copy byte data when
a slice or a read takes place, they just take a reference. This is mostly
a very nice optimisation, but there are occassions where it could have an
adverse effect. For example if a very large bitstring is created, a small
slice taken and the original deleted. The byte data from the large
bitstring would still be retained in memory.

Optimisations. Once again this version should be faster than the last.
The module is still pure Python but some of the reorganisation was to make
it more feasible to put some of the code into Cython or similar, so
hopefully more speed will be on the way.

This is a major release, with a number of backwardly incompatible changes.
The main change is the removal of many methods, all of which have simple
alternatives. Other changes are quite minor but may need some recoding.

There are a few new features, most of which have been made to help the
stream-lining of the API. As always there are performance improvements and
some API changes were made purely with future performance in mind.

About half of the class methods have been removed from the API. They all have
simple alternatives, so what remains is more powerful and easier to remember.
The removed methods are listed here on the left, with their equivalent
replacements on the right:

Many of these methods have been deprecated for the last few releases, but
there are some new removals too. Any recoding needed should be quite
straightforward, so while I apologise for the hassle, I had to take the
opportunity to streamline and rationalise what was becoming a bit of an
overblown API.

The all and any methods (previously called allset, allunset, anyset and
anyunset) no longer accept a single bit position. The recommended way of
testing a single bit is just to index it, for example instead of:

>>> ifs.all(True,i):

just use

>>> ifs[i]:

If you really want to you can of course use an iterable with a single
element, such as s.any(False,[i]), but it’s clearer just to write
nots[i].

A single index slice (such as s[5]) will now return a bool (i.e. True or
False) rather than a single bit bitstring. This is partly to reflect the
style of the bytearray type, which returns an integer for single items, but
mostly to avoid common errors like:

>>> ifs[0]:... do_something()

While the intent of this code snippet is quite clear (i.e. do_something if
the first bit of s is set) under the old rules s[0] would be true as long
as s wasn’t empty. That’s because any one-bit bitstring was true as it was a
non-empty container. Under the new rule s[0] is True if s starts with a 1
bit and False if s starts with a 0 bit.

The change does not affect reads and peeks, so s.peek(1) will still return
a single bit bitstring, which leads on to the next item...

Empty bitstrings or bitstrings with only zero bits are considered False¶

Previously a bitstring was False if it had no elements, otherwise it was True.
This is standard behaviour for containers, but wasn’t very useful for a container
of just 0s and 1s. The new behaviour means that the bitstring is False if it
has no 1 bits. This means that code like this:

>>> ifs.peek(1):... do_something()

should work as you’d expect. It also means that Bits(1000), Bits(0x00) and
Bits('uint:12=0') are all also False. If you need to check for the emptiness of
a bitstring then instead check the len property:

You can no longer use True and False to initialise single bit bitstrings.
The reasoning behind this is that as bool is a subclass of int, it really is
bad practice to have Bits(False) be different to Bits(0) and to have Bits(True)
different to Bits(1).

If you have used bool auto-initialisation then you will have to be careful to
replace it as the bools will now be interpreted as ints, so Bits(False) will
be empty (a bitstring of length 0), and Bits(True) will be a single zero bit
(a bitstring of length 1). Sorry for the confusion, but I think this will
prevent bigger problems in the future.

There are a few alternatives for creating a single bit bitstring. My favourite
is to use a list with a single item:

Previously if you created a bitstring from a file, either by auto-initialising
with a file object or using the filename parameter, the file would not be read
into memory unless you tried to modify it, at which point the whole file would
be read.

The new behaviour depends on whether you create a Bits or a BitString from the
file. If you create a Bits (which is immutable) then the file will never be
read into memory. This allows very large files to be opened for examination
even if they could never fit in memory.

If however you create a BitString, the whole of the referenced file will be read
to store in memory. If the file is very big this could take a long time, or fail,
but the idea is that in saying you want the mutable BitString you are implicitly
saying that you want to make changes and so (for now) we need to load it into
memory.

The new strategy is a bit more predictable in terms of performance than the old.
The main point to remember is that if you want to open a file and don’t plan to
alter the bitstring then use the Bits class rather than BitString.

Just to be clear, in neither case will the contents of the file ever be changed -
if you want to output the modified BitString then use the tofile method, for
example.

If a find is unsuccessful then an empty tuple is returned (which is False in a
boolean sense) otherwise a single item tuple with the bit position is returned
(which is True in a boolean sense). You shouldn’t need to recode unless you
explicitly compared the result of a find to True or False, for example this
snippet doesn’t need to be altered:

>>> ifs.find('0x23'):... print(s.bitpos)

but you could now instead use

>>> found=s.find('0x23')>>> iffound:... print(found[0])

The reason for returning the bit position in a tuple is so that finding at
position zero can still be True - it’s the tuple (0,) - whereas not found can
be False - the empty tuple ().

The read, readlist, peek and peeklist methods now accept integers as parameters
to mean “read this many bits and return a bitstring”. This has allowed a number
of methods to be removed from this release, so for example instead of:

The byteswap method now allows a format specifier of 0 (the default) to signify
that all of the whole bytes should be reversed. This means that calling just
byteswap() is almost equivalent to the now removed bytereverse() method (a small
difference is that byteswap won’t raise an exception if the bitstring isn’t a
whole number of bytes long).

This also works for the bytes type, but only if you’re using Python 3.
For Python 2 it’s not possible to distinguish between a bytes object and a
str. For this reason this method should be used with some caution as it will
make you code behave differently with the different major Python versions.

Rather than a long list of initialisers the __init__ methods now use a
**kwargs dictionary for all initialisers except ‘auto’. This should have no
effect, except that this is a small backward incompatibility if you use
positional arguments when initialising with anything other than auto
(which would be rather unusual).

Introducing a brand new class, Bits, representing an immutable sequence of
bits.

The Bits class is the base class for the mutable BitString. The differences
between Bits and BitStrings are:

Bits are immutable, so once they have been created their value cannot change. This of course means that mutating methods (append, replace, del etc.) are not available for Bits.

Bits are hashable, so they can be used in sets and as keys in dictionaries.

Bits are potentially more efficient than BitStrings, both in terms of computation and memory. The current implementation is only marginally more efficient though - this should improve in future versions.

You can switch from Bits to a BitString or vice versa by constructing a new
object from the old.

A number of methods have been flagged for removal in version 2. Deprecation
warnings will now be given, which include an alternative way to do the same
thing. All of the deprecated methods have simpler equivalent alternatives.

Note that this version will not work for Python 2.4 or 2.5. There may be an
update for these Python versions some time next year, but it’s not a priorty
quite yet. Also note that only one version is now provided, which works for
Python 2.6 and 3.x (done with the minimum of hackery!)

Although these functions don’t do anything that couldn’t be done before, they
do make some common use cases much more efficient. If you need to set or check
single bits then these are the functions you need.

This is the first release not to carry the ‘beta’ tag. It contains a couple of
minor new features but is principally a release to fix the API. If you’ve been
using an older version then you almost certainly will have to recode a bit. If
you’re not ready to do that then you may wish to delay updating.

So the bad news is that there are lots of small changes to the API. The good
news is that all the changes are pretty trivial, the new API is cleaner and
more ‘Pythonic’, and that by making it version 1.0 I’m promising not to
tweak it again for some time.

To save some typing when using pack, unpack, read and peek, compact format
codes based on those used in the struct and array modules have been added.
These must start with a character indicating the endianness (>, < or @ for
big, little and native-endian), followed by characters giving the format:

The use of = is now mandatory in ‘auto’ initialisers. Tokens like uint12100 will
no longer work. Also the use of a : before the length is encouraged, but not yet
mandated. So the previous example should be written as uint:12=100.

The join function must now be called on a BitString object, which will be
used to join the list together. You may need to recode slightly:

>>> s=bitstring.join('0x34','0b1001','0b1')

becomes

>>> s=BitString().join('0x34','0b1001','0b1')

More than one value allowed in readbits, readbytes, peekbits and peekbytes¶

If you specify more than one bit or byte length then a list of BitStrings will
be returned.

>>> a,b,c=s.readbits(10,5,5)

is equivalent to

>>> a=readbits(10)>>> b=readbits(5)>>> c=readbits(5)

bytealigned defaults to False, and is at the end of the parameter list¶

Functions that have a bytealigned paramater have changed so that it now
defaults to False rather than True. Also its position in the parameter list
has changed to be at the end. You may need to recode slightly (sorry!)

The split method now has a ‘count’ parameter rather than ‘maxsplit’. This
makes the interface closer to that for cut, replace and findall. The final item
generated is now no longer the whole of the rest of the BitString.

The use of the step parameter (also known as the stride) in slices has been
added. Its use is a little non-standard as it effectively gives a multiplicative
factor to apply to the start and stop parameters, rather than skipping over
bits.

For example this makes it much more convenient if you want to give slices in
terms of bytes instead of bits. Instead of writing s[a*8:b*8] you can use
s[a:b:8].

When using a step the BitString is effectively truncated to a multiple of the
step, so s[::8] is equal to s if s is an integer number of bytes, otherwise it
is truncated by up to 7 bits. So the final seven complete 16-bit words could be
written as s[-7::16].

Negative slices are also allowed, and should do what you’d expect. So for
example s[::-1] returns a bit-reversed copy of s (which is similar to
s.reversebits(), which does the same operation on s in-place). As another
example, to get the first 10 bytes in reverse byte order you could use
s_bytereversed=s[0:10:-8].

You can now specify an offset of greater than 7 bits when creating a BitString,
and the use of offset is also now permitted when using the filename initialiser.
This is useful when you want to create a BitString from the middle of a file
without having to read the file into memory.

Some changes have been made to slicing so that less exceptions are raised,
bringing the interface closer to that for lists. So for example trying to delete
past the end of the BitString will now just delete to the end, rather than
raising a ValueError.

A new option for the auto initialiser is to pass it a list or tuple. The items
in the list or tuple are evaluated as booleans and the bits in the BitString are
set to 1 for True items and 0 for False items. This can be used anywhere the
auto initialiser can currently be used. For example:

This version is just a port of version 0.4.0 to Python 3. All the unit tests
pass, but beyond that only limited ad hoc testing has been done and so it
should be considered an experimental release. That said, the unit test
coverage is very good - I’m just not sure if anyone even wants a Python 3
version!

Previously a read or peek function that encountered the end of the BitString
would raise a ValueError. It will now instead return the remainder of the
BitString, which could be an empty BitString. This is closer to the file
object interface.

The offset property was previously read-only, and has now been removed from
public view altogether. As it is used internally for efficiency reasons you
shouldn’t really have needed to use it. If you do then use the _offset parameter
instead (with caution).

The methods append, deletebits, insert, overwrite, truncatestart and
truncateend now modify the BitString that they act upon. This allows for
cleaner and more efficient code, but you may need to rewrite slightly if you
depended upon the old behaviour:

The binary interpretation of a BitString is now prepended with ‘0b’. This is
in keeping with the Python 2.6 (and 3.0) bin function. The prefix is optional
when initialising using bin=.

Also, if you just print a BitString with no interpretation it will pick
something appropriate - hex if it is an integer number of bytes, otherwise
binary. If the BitString representation is very long it will be truncated
by ‘...’ so it is only an approximate interpretation.