Xiao He was born in Fengyi County, Sishui Commandery (present-day Feng County, Jiangsu). In his early days, he served as a gongcao (功曹; magistrate's secretary) in the administrative office of Pei County. He was studious, tactful and amiable. He was well versed in law and had a wide network of friends and acquaintances, among whom include Liu Bang, Cao Shen, Fan Kuai, Xiahou Ying and Zhou Bo.

Once, Liu Bang released some prisoners he was escorting to Mount Li to be labourers, and became a fugitive. He hid in an outlaw stronghold on Mount Mangdang (in present-day Yongcheng, Henan) and maintained secret contact with Xiao He and Cao Shen. In 209 BC, Chen Sheng and Wu Guang started the Dazexiang Uprising to overthrow the Qin dynasty, and was followed by numerous other rebellions throughout China. The magistrate of Pei County considered rebelling as well, so he heeded Xiao He and Cao Shen's advice to invite Liu Bang back to support him. However, he changed his mind later and denied Liu Bang's party entry into the city. As he was afraid that Xiao He and Cao Shen might open the gates for Liu Bang, he intended to have them executed, but Xiao and Cao escaped and joined Liu. As suggested by Xiao He, Liu Bang had letters sent to the townsfolk, fired into the city on arrows, urging them to join him. The people responded to Liu Bang's call and killed the magistrate and welcomed him back into the city.

After returning to Pei County, the townsfolk wanted Liu Bang to lead them to overthrow the Qin dynasty. However, Liu Bang expressed reluctance in taking up the leadership position and the people agreed to decide their leader by luck. Xiao He was in charge of the process and he had the names of ten respectable men in town, including Liu Bang, written on separate pieces of paper and shuffled. Xiao He then asked Liu Bang to pick any piece from the pile, without seeing the name on it, and Liu picked his own name by chance. Liu Bang looked at Xiao He and wanted to decline again. Xiao He immediately grabbed the remaining pieces of paper, put them into his mouth and chewed them to shreds, after which he proclaimed Liu Bang as their leader. Liu Bang became known as the "Duke of Pei" and built his rebel army in Pei County, with Xiao He and his friends supporting him. Liu Bang learnt from Xiao He later that actually all the ten pieces of paper contained his name, and Xiao had deliberately tampered with the system to help him.

In 206 BC, the Qin dynasty collapsed after the last Qin ruler, Ziying, surrendered to Liu Bang. Liu Bang's army occupied the Qin capital Xianyang. Upon entering the Epang Palace, Liu Bang and most of his men seized treasures and women for themselves, while Xiao He rushed to the chancellor's office and ordered it to be heavily guarded. Xiao He then brought a few men with him to collect all official documents in the office and bring them out safely. Xiao He later explained that the documents were more valuable than riches and women, because they contained crucial information about the Qin Empire and would be useful in intelligence.

Liu Bang was forced to retreat from Xianyang later as Xiang Yu's army arrived and occupied the city. After plundering Xianyang and destroying the palace, Xiang Yu divided the former Qin Empire into the Eighteen Kingdoms. Liu Bang received the title of "King of Han" and was relocated to the remote Bashu region (in present-day Sichuan), with Xiao He and his followers accompanying him. Liu Bang was unhappy and wanted to attack Xiang Yu because the land of Guanzhong rightfully belonged to him, according to an earlier promise by Emperor Yi of Chu, but was granted by Xiang to three surrendered Qin generals instead. However, Xiao He and others managed to persuade Liu Bang to temporarily remain in Bashu and build up his forces in preparation for a battle for supremacy with Xiang Yu later. Xiao He was appointed by Liu Bang as the chancellor of the Han kingdom.

During that time, Xiao He met Han Xin, who had recently joined Liu Bang's army and was serving as a low-ranking soldier, and recognised Han Xin's brilliance in military strategy. Not long later, many of Liu Bang's men deserted as they were unable to cope with living in remote Bashu, and Han Xin also left as he was disappointed because he felt his talent would go to waste in Liu Bang's camp. When Xiao He learnt that Han Xin had left, he immediately rushed to find Han and bring him back, without managing to inform Liu Bang in time. Xiao He caught up with Han Xin after trailing him for a few days, even at night, and succeeded in persuading Han to return. Meanwhile, Liu Bang thought that Xiao He had also deserted him and was shocked, but was relieved when he saw Xiao returning with Han Xin days later. Liu Bang asked Xiao He, "Of all those who deserted, why did you choose to bring back Han Xin only?" Xiao He then strongly recommended Han Xin to Liu Bang, saying that Han's talent was unmatched. Liu Bang heeded Xiao He's suggestion and appointed Han Xin as a general.

In 206 BC, Liu Bang conquered the Three Qins and left Xiao He in charge of Guanzhong and Bashu, while he led his army eastward to attack Xiang Yu's Western Chu kingdom. Xiao He governed those territories effectively and provided support to Liu Bang's army at the warfront in the form of supplies, provisions and reinforcements. While in Guanzhong, Xiao He restored peace and order by implementing a new system of governance and reconstructing destroyed buildings. Under the new system, taxes were readjusted to reduce the burden on the poor; land that used to belong to the Qin aristocracy was redistributed to peasants, in order to increase agricultural output; capable and respectable men were selected from among commoners to assist government officials in administration.

Liu Bang eventually defeated Xiang Yu at the Battle of Gaixia in 202 BC and unified China under his rule. He was proclaimed "Emperor" and became historically known as "Emperor Gaozu of Han". The following year, when Gaozu rewarded his subjects according to their contributions, Xiao He received the title of "Marquis of Zan" (酇侯) in addition to his appointment as chancellor. Besides, Gaozu also granted Xiao He a rare special privilege: Xiao was allowed to enter the imperial court carrying a sword and with his shoes on, and not required to walk in briskly while others had to.

In 201 BC, Emperor Gaozu wanted to build his capital in Luoyang but Zhang Liang suggested Xianyang instead, because it was surrounded by natural defences such as mountain passes, and had fertile land for agriculture. Gaozu put Xiao He in charge of the constructions in Xianyang. Two years later, the project was completed and Xiao He took Gaozu on a tour of Xianyang. The new palace was called Weiyang Palace and served as the central imperial palace throughout the Western Han dynasty and Xin dynasty. Gaozu also renamed Xianyang to "Chang'an", which meant "long-lasting peace".

In 196 BC, while Gaozu was away on a military campaign to suppress a rebellion by Chen Xi, one of Han Xin's servants reported to Empress Lü Zhi that his master was involved in a conspiracy with Chen Xi and was planning a coup in Chang'an. Empress Lü approached Xiao He for help and they formulated a plan to lure Han Xin into a trap. Xiao He had an imperial edict sent to Han Xin, announcing news of Gaozu's victory over Chen Xi, asking Han to meet the emperor and offer his congratulations. As soon as Han Xin arrived in Changle Palace, he was arrested and executed in a torturous manner for alleged treason.

In 195 BC, Xiao He saw that Chang'an was becoming more congested and the imperial garden was full of uncultivated land, hence he suggested to Gaozu to carve out parts of the garden and grant them to peasants as farmland. Gaozu was unhappy and accused Xiao He of accepting bribes and dishonouring imperial authority. Xiao He was arrested and imprisoned, but was released later after Gaozu was convinced that Xiao He had the people's interests at heart. Gaozu summoned Xiao He to see him and he responded with jest and embarrassment, "The chancellor was asking for the fields in the imperial garden out of interest for the people, but I did not approve. This showed that I'm only like Jie and Zhou", and that you're the most understanding chancellor. Therefore, I intentionally imprisoned you to show the people that I'm a tyrant."

After the death of Gaozu, Xiao He continued serving Gaozu's successor Emperor Hui as chancellor until he died in 193 BC. Before his death, Xiao He recommended Cao Shen to succeed him. Cao Shen inherited the system of governance left behind by Xiao He and maintained it well but did not implement any new changes.

Some Chinese idioms and sayings originated from the events in Xiao He's life and are listed as follows:

An illustration tablet of Xiao He chasing Han Xin under the Moonlight.

Xiao He chases Han Xin under the moonlight (蕭何月下追韓信): Originated from the event where Xiao trailed Han Xin for a few days, even at night, in order to catch up with the latter and bring him back to Liu Bang. It is used to describe an urgent situation in which action has to be taken immediately without notification.

Success is due to Xiao He, downfall is also due to Xiao He (成也蕭何 敗也蕭何 Hangul: 성야소하 폐야소하): Xiao He helped Han Xin become a general, which enabled Han to put his talent to good use. However, Xiao also played an important role in Han's death. It is used to describe a situation where one's success and failure are both due to the same factor.

Cao following Xiao's rules (萧规曹随): Cao Shen, Xiao He's successor as chancellor, inherited the system of governance left behind by Xiao and maintained it well but did not implement any new changes. It is used to describe the continuation of the work of one's predecessor.