Significant data suggest that soluble Aβ oligomers play an important role in Alzheimer’s disease (AD), but there is great confusion over what exactly constitutes an Aβ oligomer and which oligomers are toxic. Most studies have utilized synthetic Aβ peptides, but the relevance of these test tube experiments to the conditions that prevail in AD is uncertain. A few groups have studied Aβ extracted from human brain, but they employed vigorous tissue homogenization which is likely to release insoluble Aβ that was sequestered in plaques during life. Several studies have found such extracts to possess disease-relevant activity and considerable efforts are being made to purify and better understand the forms of Aβ therein. Here, we compared the abundance of Aβ in AD extracts prepared by traditional homogenization versus using a far gentler extraction, and assessed their bioactivity via real-time imaging of iPSC-derived human neurons plus the sensitive functional assay of long-term potentiation. Surprisingly, the amount of Aβ retrieved by gentle extraction constituted only a small portion of that released by traditional homogenization, but this readily diffusible fraction retained all of the Aβ-dependent neurotoxic activity. Thus, the bulk of Aβ extractable from AD brain was innocuous, and only the small portion that was aqueously diffusible caused toxicity. This unexpected finding predicts that generic anti-oligomer therapies, including Aβ antibodies now in trials, may be bound up by the large pool of inactive oligomers, whereas agents that specifically target the small pool of diffusible, bioactive Aβ would be more useful. Furthermore, our results indicate that efforts to purify and target toxic Aβ must employ assays of disease-relevant activity. The approaches described here should enable these efforts, and may assist the study of other disease-associated aggregation-prone proteins.