The aim of this paper is to discriminate liver diseases from CT images automatically using a sigmoid radial basis function neural network with growing and pruning algorithm (SRBFNN-GAP). We develop a novel SRBFNN-GAP to discriminate cyst, hepatoma, cavernous hemangioma, and normal tissue using gray level and Gabor texture features. The proposed SRBFNN adopts sigmoid function as its kernel because the sigmoid function provides a more flexible shape than Gaussian. Furthermore, the GAP algorithm is used to adjust the network size dynamically according to the neuron’s significance. In the experiment, the SRBFNN-GAP classifies the features into four classes, and the receiver operating characteristic (ROC) curve is used to evaluate the diagnosis performance.