Americans generally do not think of the United States as a colonial power. But other people often do, making it abundantly clear that, in their view, the US has indeed been colonialist - and continues to be so. While Marxist rhetoric and agitation is no doubt responsible in part for whipping up this chant, a good portion of it springs quite naturally from the experience of people who have come in contact with the US.

Panamanians, for example, long chafed over US control of the Panama Canal, which bisects their nation. Built by the US in that heady jingoistic era of American expansion after the turn of the century, the 51-mile-long waterway and the surrounding 500-square-mile zone were run by the US for 75 years. Under terms of a 1903 treaty, Panama gave the land in perpetuity to the US. Moreover, the US was allowed to act as if it were ''sovereign'' in running the canal and the zone.

Is it any wonder that Panamanians rankled over the arrangement? Even those Panamanians friendly with the US - and there have been many of them - knew that they were not quite complete masters of their own house. To be sure, the canal brought them a degree of economic prosperity that they would not have otherwise enjoyed. But the canal in their midst belonged to someone else.

This was precisely the point that Panamanian Gen. Omar Torrijos Herrera made repeatedly as he negotiated with Washington during the 1970s over the future of the waterway - negotiations that resulted in treaties that provide for the canal's eventual control by Panama.

William J. Jordan's closely knit account of those negotiations brings the whole question of colonialism into stark focus. As he graphically relates Panamanian-US relations over the past century, there can be little doubt that the US did act as a colonial power in its relations with Panama. That colonialism produced 75 years of tension. Although each president starting with Franklin D. Roosevelt recognized the need for a change in the relationship, US domestic politics derailed repeated efforts for the change. President Jimmy Carter finally brought negotiations to a climax.

As ambassador to Panama during the final portion of the treaties negotiations , Jordan played a key role in the whole process. His inside account is the best record to date of those critical years.

There is no doubt where Ambassador Jordan stood in this struggle. ''The elimination of the foreign enclave, the 'colony' if you will, in the very heart of a small and friendly country,'' he writes, was ''the obvious requirement.'' He has high praise for the US negotiators, Ambassadors Ellsworth Bunker and Sol Linowitz, and even more for the tough determination of President Carter to bring the goal to fruition. It was a long ''odyssey,'' but one that Ambassador Jordan feels ended an irritant to good relations not only with Panama, but also with Latin America. In his view, it represents the most significant foreign policy achievement of the Carter administration.

''Panama Odyssey'' is a vivid account of diplomacy in action. It reads well. It may be a bit too long for some readers (it could have been pruned of some of its detail), but it's a first-rate accounting of those stormy years of negotiation.