ImageJ is an image processing program written in Java by the NIH. On Windows, by default, it will use Java 32 bits with 640 MiB of RAM. It is however possible to give it more RAM and to configure it to run in Java 64 bits (the 64 bits version of Java will allow you to assign more RAM).
To do so, simple edit the configuration file ImageJ.cfg located in ImageJ’s root folder:

The second line is the path to Java. It’s something like C:\Program Files (x86)\Java\jre6\bin\javaw.exe for Java 32 bits, and you just need to change it to C:\Program Files\Java\jre6\bin\javaw.exe for Java 64 bits (that is, if you installed your JREs in the default paths).

The third line sets some arguments. The RAM is set using “-Xmx640m” (at the beginning of the line), so obviously, to assign, say 1 GiB, you’ll replace that with “-Xmx1024m”.