It was not until 1853, with the arrival of Commodore Matthew Perry following the Convention of Kanagawa ending Japan's "closed-door" foreign policy that Jewish families began to settle in Japan. The first recorded Jewish settlers arrived at Yokohama in 1861. By 1895 this community, which by then consisted of about 50 families, established the first synagogue in Japan.[2] Part of this community would later move to Kobe after the great Kanto earthquake of 1923.

Another early Jewish settlement was one established in the 1880s in Nagasaki, a large Japanese port city opened to foreign trade by the Portuguese. This community was larger than the one in Yokohama, consisting of more than 100 families. It was here that the Beth Israel Synagogue was created in 1894. The settlement would continually grow and remain active until it eventually declined by the Russo-Japanese War in the early 20th century. The community's Torah scroll would eventually be passed down to the Jews of Kobe, a group formed of freed Russian Jewish war prisoners that had participated in the Czar's army and the Russian Revolution of 1905.

From the mid 1920s until the 1950s, the Kobe Jewish community was the largest Jewish community in Japan, formed by hundreds of Jews arriving from Russia (originating from the Manchurian city of Harbin), the Middle East (mainly from Iraq and Syria), as well as from Central and Eastern European countries (primarily Germany). It had both an Ashkenazi and a Sephardic synagogue.[3] During this time Tokyo's Jewish community (now Japan's largest) was slowly growing with the arrival of Jews from the United States, Western Europe, and Russia.

Some Japanese leaders, such as Captain Inuzuka Koreshige (犬塚 惟重), Colonel Yasue Norihiro (安江 仙弘) and industrialist Aikawa Yoshisuke (鮎川 義介), came to believe that Jewish economic and political power could be harnessed by Japan through controlled immigration, and that such a policy would also ensure favor from the United States through the influence of American Jewry. Although efforts were made to attract Jewish investment and immigrants, the plan was limited by the government's desire not to interfere with its alliance with Nazi Germany. Ultimately it was left up to the world Jewish community to fund the settlements and to supply settlers, and the plan failed to attract a significant long-term population or create the strategic benefits for Japan that had been expected by its originators.

During World War II, Japan was regarded as a safe refuge from the Holocaust, despite being a part of the Axis and an ally of Germany. Jews trying to escape German-occupied Poland could not pass the blockades near the Soviet Union and the Mediterranean Sea and were forced to go through the neutral country of Lithuania (which was occupied by belligerents in June 1940, starting with the Soviet Union, then Germany, and then the Soviet Union again).

Of those who arrived, many (around 5,000) were sent to the Dutch West Indies with Japanese visas issued by Chiune Sugihara, the Japanese consul to Lithuania. Sugihara ignored his orders and gave thousands of Jews entry visas to Japan, risking his career and saving more than 6,000 lives. Sugihara is said to have cooperated with Polish intelligence, as part of a bigger Japanese-Polish cooperative plan.[6] They managed to flee across the vast territory of Russia by train to Vladivostok and then by boat to Kobe in Japan. The refugees, 2,185 in number, arrived in Japan from August 1940 to June 1941. Tadeusz Romer, the Polish ambassador in Tokyo, had managed to get transit visas in Japan, asylum visas to Canada, Australia, New Zealand and Burma, immigration certificates to Palestine, and immigrant visas to the United States and some Latin American countries. Most Jews were permitted and encouraged to move on from Japan to the Shanghai Ghetto, China, under Japanese occupation for the duration of World War II. Finally, Tadeusz Romer arrived in Shanghai on November 1, 1941, to continue the action for Jewish refugees.[7] Among those saved in the Shanghai Ghetto were leaders and students of Mir yeshiva, the only European yeshiva to survive the Holocaust. They, some 400 in number, fled from Mir to Vilna with the outbreak of World War II in 1939, and then to Keidan, Lithuania. In late 1940, they obtained visas from Chiune Sugihara, to travel from Keidan, then Lithuanian SSR, via Siberia and Vladivostok to Kobe, Japan.[8] By November 1941 the Japanese moved this group and most of others on to the Shanghai Ghetto in order to consolidate the Jews under their control.[9]

The secretary of the Manchurian Legation in Berlin Wang Tifu (王, 替夫. 1911-) also issued visas to 12,000 refugees including Jews from June 1939 to May 1940.[10][11]

Throughout the war, the Japanese government continually rejected requests from the German government to establish anti-Semitic policies. Towards the end, Nazi representatives pressured the Japanese army to devise a plan to exterminate Shanghai's Jewish population, and this pressure eventually became known to the Jewish community's leadership. However, the Japanese had no intention of further provoking the anger of the Allies, and thus delayed the German request for a time, eventually rejecting it entirely.

One famous Orthodox Jewish institution that was saved this way was the LithuanianHarediMir yeshiva. The Japanese government and people offered the Jews temporary shelter, medical services, food, transportation, and gifts, but preferred that they move on to reside in Japanese-occupied Shanghai.

At war's end, about half of the Jews who had been in Japanese-controlled territories later moved on to the Western hemisphere (such as the United States and Canada) and the remainder moved to other parts of the world, mainly to Israel.

After World War II, a large portion of the few Jews that were in Japan left, many going to what would become Israel. Some of those who remained married locals and were assimilated into Japanese society.

The Israeli Embassy and its staff is based in Tokyo. Presently, there are several hundred Jewish families living in Tokyo, and a small number of Jewish families in and around Kobe. A small number of Jewish expatriates of other countries live throughout Japan, temporarily, for business, research, a gap year, or a variety of other purposes. There are always Jewish members of the United States Armed Forces serving on Okinawa and in the other American military bases throughout Japan.

There are community centers serving Jewish communities in Tokyo[13] and Kobe.[14] The Chabad-Lubavitch organization has one official center in Tokyo,[15] and there is an additional Chabad house run by Rabbi Yehezkel Binyomin Edery.[16]