One idea that shaped architect Alfred Mansfeld's architectural concept for the Israel Museum was that of a temple, situated at the highest point on the ground, which could be reached only by means of a gradual climb and a long journey. Only after a visitor to the Museum climbed the many steps from the entry to the campus to the entry to the main building of the Museum could he enter the "temple," the Museum itself, and approach the treasures preserved within. This was a controversial and problematic concept – thus one objective of the Israel Museum Renewal of 2007-2010 was to transform it and make access to the Museum and its treasures very convenient and user-friendly.

Yet if we remember that this concept exists, it is easy to understand the location of the library in the Museum building. It is situated at the highest point in the Museum. The library and its tall windows look beautiful from any vantage point in its surroundings. It is therefore possible to conclude that the library is the ultimate temple – it is indeed the "mind" and "heart" of the Museum; it is used by curators at every stage of mounting displays and exhibitions; and also used by visitors who come to read material related to the exhibitions that they have seen at the Museum and to survey new publications in the fields of art, archaeology, etc.

The Museum's architecture, based on a mushroom principle in which columns support the ceiling with no support from external walls, is also reflected in the library structure.

The library is constructed from four mushrooms which create a large, open space with arches that meet on top. A breathtaking view of Jerusalem is reflected in the large windows on the library's north wall. The interior space designed by Dora Gad, like that of the entire Museum, is quiet and simple. The columns and ceiling are made from concrete. Visible lines on the ceiling remain from the wooden beams used to pour the concrete, lending a sense of simplicity, modernity, and a certain coarse quality on one hand, and vitality and interest to a seemingly dull material like gray concrete on the other. The columns, in contrast, are smooth and serve as air-conditioning channels. At the top, immediately beneath the ceiling, a row of narrow windows encircles the building, bringing air into the library (their functional aspect) and granting the space a certain buoyancy and lightness. The interior is constructed from a ground floor and a gallery that surrounds the library, leaving the enormous space open to its height.

That produces a tall, expansive space full of natural light. The straight lines of the mushroom columns are repeated in the lines of library cabinets made of natural teak, which lend the space warmth and comfort. The same vertical lines are repeated again in the books that line the shelves in a queue of small, colorful stripes. The barely visible color of the books is enhanced by the design of the library's furniture: teak tables with white surfaces, flanked by lightweight chairs in a renowned design by Danish designer Arne Jacobsen, in red and yellow on the ground floor, and blue in the gallery.

The integration of ascending, vertical lines (four, massive concrete columns, wooden cabinets, and the books themselves inside the cabinets) creates the feeling of a lofty space filled with air. The horizontal lines of the gallery floor that encircles the library and the row of narrow windows under the ceiling which, like the gallery, surrounds the library except for the northern wall, which is almost entirely formed from windows, balances the lofty lines, creating a square space, like a cube, full of light, in which all the materials and all the trajectories meet in an engaging game and in harmony. Every participating element has a defined role in this game, and together these factors create a modern space by incorporating an unconventional combination of materials (concrete and wood) on one hand, and on the other hand, a classic library space thanks to its symmetry, large windows, and furnishings made from natural materials which provide readers with comfort and contribute to a nearly homey atmosphere.