Way back in the days of DOS (for you
younger people, I am referring to the 80's and early 90's),
Computers and the operating system (DOS) were both 16 bit. In
binary, this translates to 65,536 or 64 KB. Microsoft used a trick
to add 4 more bits (total of 20), by using an "offset" so they could
achieve a maximum number of 1,048,576 (1 MB), which is why DOS was
limited to 640K of RAM. Other tricks, such as "Expanded
Memory", and "Extended Memory" were added to increase this limit, by
using the memory betwen 640K and the 1M limit to swap in and out.
This required special "memory managers" to handle and was very
inefficient. The early versions of Windows (2, 3, 3.1) had a
built-in extended memory manager so you could go "all the way" up to
4 MB.

Starting with Windows NT, and later
into Windows 95 and so on, Windows became fully 32 bit. This
allowed memory addressing up to 4,294,967,296 or 4 GB or memory.
This limit persisted through Windows XP (there is a version of 64
bit XP, but nobody uses it; it is also used in the Windows
Server 2003; later server versions are all 64 bit).

As time went on, and the operating
system and applications became more hogging of memory, 64 bit is
becoming more common. Windows Vista became the first version
where 3-4 GB of memory was barely enough. Hence the move to 64
bit.

A computer running a 64 bit operating
system can handle numbers up to 18,446,744,073,709,551,616 or 16
exbibytes (EiB), which today can be considered unlimited (wait 'til
tomorrow). Practically, today's 64 bit computers handle up to
4 Terabytes of RAM which is 100 times more than a 32 bit computer.

This increase comes at a price.
Software has to be written for the operating system. Each time
the bitness of the OS increase, new hardware drivers are needed.
So, If you get a new 64 bit computer, you will need new drivers for
all of your hardware, such as mice, printers, scanners, etc.
Fortunately, most things are recognized by the OS and drivers are
automatically installed, but there are a few exceptions. I
have encountered printers and scanners in particular where there is
no 64 bit driver available, and none is planned. I recommend
that you check online if updated drivers are available for your
peripherals, or plan to replace them (or stick to 32 bit; not a
satisfying solution).

In addition, certain software may not
run at all. When the change to 32 bit came in the 90's,
Microsoft included a "16 bit subsystem" to maintain compatibility
with the older software, and almost everything ran just fine (except
for some utilities that directly addressed hardware). Now with
the change to 64 bit, Microsoft includes a 32 bit susbystem for
older software, but has removed the 16 subsystem altogether.
So, very old software will not run; in fact it won't even install.
You'll get a nice, polite pop-up telling you that the software is
not compatible.

So, what do you do if you need this
older software? Well, there are 3 options.

First, you can upgrade to a
later version if available. This is the best option.

Second, you can keep and old
computer running Windows XP just for that purpose.

And third, If you pay the extra
for the business or ultimate versions of Windows 7, there is
something called "XP mode" that can be installed and essentially
runs an Windows XP virtual computer, on which you can install
your older software, which should run just fine. I have
done this in cases where there was no newer version available.

One final note. Just because
you have a 64 bit version of Windows, does NOT mean you must buy 64
bit versions of everything (but if you have the 32 bit version of
Windows, you CANNOT run 64 bit software). As I mentioned before, the
32 bit versions of almost everything will run just fine. In
fact, you may prefer the 32 bit versions. I have heard some
complaints (which I have not verified or researched) that there are
issues with compatibility in the 64 bit version of Microsoft Office
2010, so perhaps you should stick to the 32 bit for now. For
some applications, like picture or video editing, the 64 bit
versions will give you a lot more leeway with the size of the
pictures or video you can handle.