The inherit nonlinearity in analogue front-ends of transmitters and receivers have had primary impact on the overall performance of the wireless communication systems, as it gives arise of substantial distortion when transmitting and processing signals with such circuits. Therefore, the nonlinear compensation (linearization) techniques become essential to suppress the distortion to an acceptable extent in order to ensure sufficient low bit error rate. Furthermore, the increasing demands on higher data rate and ubiquitous interoperability between various multi-coverage protocols are two of the most important features of the contemporary communication system. The former demand pushes the communication system to use wider bandwidth and the latter one brings up severe coexistence problems. Having fully considered the problems raised above, the work in this Ph.D. thesis carries out extensive researches on the nonlinear compensations utilizing advanced digital signal processing techniques. The motivation behind this is to push more processing tasks to the digital domain, as it can potentially cut down the bill of materials (BOM) costs paid for the off-chip devices and reduce practical implementation difficulties. The work here is carried out using three approaches: numerical analysis & computer simulations; experimental tests using commercial instruments; actual implementation with FPGA. The primary contributions for this thesis are summarized as the following three points: 1) An adaptive digital predistortion (DPD) with fast convergence rate and low complexity for multi-carrier GSM system is presented. Albeit a legacy system, the GSM, however, has a very strict requirement on the out-of-band emission, thus it represents a much more difficult hurdle for DPD application. It is successfully implemented in an FPGA without using any other auxiliary processor. A simplified multiplier-free NLMS algorithm, especially suitable for FPGA implementation, for fast adapting the LUT is proposed. Many design methodologies and practical implementation issues are discussed in details. Experimental results have shown that the DPD performed robustly when it is involved in the multichannel transmitter. 2) The next generation system (5G) will unquestionably use wider bandwidth to support higher throughput, which poses stringent needs for using high-speed data converters. Herein the analog-to-digital converter (ADC) tends to be the most expensive single device in the whole transmitter/receiver systems. Therefore, conventional DPD utilizing high-speed ADC becomes unaffordable, especially for small base stations (micro, pico and femto). A digital predistortion technique utilizing spectral extrapolation is proposed in this thesis, wherein with band-limited feedback signal, the requirement on ADC speed can be significantly released. Experimental results have validated the feasibility of the proposed technique for coping with band-limited feedback signal. It has been shown that adequate linearization performance can be achieved even if the acquisition bandwidth is less than the original signal bandwidth. The experimental results obtained by using LTE-Advanced signal of 320 MHz bandwidth are quite satisfactory, and to the authors’ knowledge, this is the first high-performance wideband DPD ever been reported. 3) To address the predicament that mobile operators do not have enough contiguous usable bandwidth, carrier aggregation (CA) technique is developed and imported into 4G LTE-Advanced. This pushes the utilization of concurrent dual-band transmitter/receiver, which reduces the hardware expense by using a single front-end. Compensation techniques for the respective concurrent dual-band transmitter and receiver front-ends are proposed to combat the inter-band modulation distortion, and simultaneously reduce the distortion for the both lower-side band and upper-side band signals.