The club was founded in 1874 and, in 1881 it became one of the eleven founder clubs of the Welsh Rugby Union.[3][4]

In the early twentieth century Swansea RFC was an extremely successful club. For four consecutive seasons Swansea were the unofficial Welsh champions from the 1898/99 season through to 1901/02, coinciding with the heyday of Swansea's first star player Billy Bancroft. Under the captaincy of Frank Gordon the team would later go on a 22 month unbeaten run, from December 1903 through to October 1905. During this period Swansea appeared to be under-represented at international level. Gordon himself went uncapped throughout his entire career, and apart from Billy Trew, Dick Jones and Dicky Owen, the only other internationals in the senior team were forward Sid Bevan (1 cap),[5] wing Jowett (1 cap)[6] and outside-half Phil Hopkins (4 caps).[7] Trew (29 caps) was an outstanding centre who was accepted as one of the most important players in the evolution of Welsh rugby,[8] while Dicky Owen (35 caps), although only 5 foot 4 inches tall, was an incredible tactician.[9]

The immediate post war years brought only limited success, although a notable 6-6 draw was achieved against New Zealand in 1953 followed by a 9-8 victory against Australia in 1966. It was not until the club’s centenary season in 1973/74, however, that the club became Merit Table Champions. Swansea achieved further success as club champions in 1979/80, 1980/81, 1982/83 as well as Welsh cup winners in 1978.

Players during this period included Clem Thomas, Billy Williams, Dewi Bebb, Mervyn Davies, Geoff Wheel, David Richards and Mark Wyatt, the club's record points scorer with 2,740 points scored between 1976/77 and 1991/92.

The 1990s saw success for the club, including being league champions on 4 occasions (1991/92, 1993/94, 1997/98 and 2000/01) and Welsh cup winners in 1995 and 1999. A memorable 21-6 victory was recorded over then World champions Australia at St Helens on 4 November 1992. In season 1995/96 Swansea reached the semi-final stage of the European Cup. This period also included disputes with the Welsh Rugby Union over the way the league structure was being run in Wales following rugby union's move to being a professional game, which culminated in the club's boycott of the 1998/99 league season, in what was referred to as the rebel season.[10]

The 2003/04 season has seen a significant change with the introduction of regional rugby in Wales. Swansea Rugby Football Club Ltd, alongside Neath RFC are co-owners of the Ospreys. As a result Swansea RFC returned to being an amateur team. Since the change to regional rugby Several players have played for Swansea RFC, as well Ospreys and Wales including Alun Wyn Jones, Ryan Jones, Scott Baldwin, Nicky Smith, Matthew Morgan, Eli Walker, Gavin Henson and Dan Biggar.

In 2014 the All Whites were relegated from the Welsh Premier league on the final day of the season when despite beating Neath at St Helens, a bonus point for Aberavon sent Swansea into the SWALEC Championship. This provoked a complete revamp of the club with Stephen Hughes taking the position of Chairman, Keith Colclough as Managing Director and Richard Lancaster leading a coaching team of former players including Rhodri Jones, Chris Loader and Ben Lewis. In their first season Swansea missed out on promotion back to the Premiership, finishing in second place overall.