Have the Olympics Ever Been Canceled? Here's the History | Time

The postponement of Tokyo 2020 was not a decision taken lightly: it came after the pressure to cancel or delay the event due to the risks of the new virus.
The financial blow of canceling the games will be substantial for Japan, which has already invested at least $12.6 billion into the 2020 Olympics; by some estimates, it would reduce Japan’s annual GDP growth by 1.4%.

The Berlin Games were canceled due to the outbreak of World War I in July 1914.

Organizers initially thought that the war would be over by Christmas but an armistice was not reached until November 1918.

Tokyo and Sapporo, Japan, had been originally set to host the 1940 Olympic Games (summer and winter), making them the first non-Western cities selected to host. But the war between Japan and China started in July 1937 and made the Japanese government decide to lose their right to host the Games.
The new hosts: Helsinki, Finland (for summer) and Garmisch-Partenkirchen Germany (for winter). But the outbreak of World War II caused the events to be canceled entirely, after the Nazi invasion of Poland in September 1939. The same happened with the 1944’s Summer Olympics in London and Winter Games in Cortina d’Ampezzo, Italy.