Dr. Erich Paulun, a German who was serving as a naval surgeon in a warship, he first came to Shanghai in 1891. At that time, Shanghai was suffering from epidemics of cholera, typhoid and malaria, but there was lack of drug and money to save the lives of locals. Dr. Erich Paulun decided to leave the German navy and wanted to establish a clinic in Shanghai. To prepare for such a difficult task, he returned to Germany to have a further study in medicine and raising fund for his benevolent goal. Dr. Paulun visited Shanghai again in 1893, he established a clinic near Shanghai German Church and he teamed up with several local German doctors to set up the "German Doctors' Union" in the city. But the union's business was not in good shape by 1899, so Dr. Erich Paulun asked for donation from local Chinese enterprises and German companies. W. Knappa, the German Consulate General in Shanghai appointed his counselor P. D. Fischer, to whom talks had been held with Shanghai's officials. Eventually, an agreement had been reached, Shanghai government provided a land lot located at Zhang Jia Bang area of the city for Dr. Paulun to build a new hospital.

In 1900, Tongji Hospital has successfully been attained a practicing license. Most of the doctors were from the German Doctors' Union. They had to work in their own clinics during daytime, and took time to do treatments or surgeries for patients in Tongji Hospital. Dr. Erich Paulun knew that there was lack of medical resources and staff in the hospital, so he managed to set up a school to cultivate local physicians. His plan was greatly supported by German Consulate General and Higher Education Bureau of Germany. In 1906 a foundation had been set up to aid the founding of the medical school. Its fund had helped the school to purchase a lot of new medical textbooks and advanced surgical equipments. The local compradors had donated a lot of money to the school too.

On October 1, 1907, Tongji Deutsch Medical School inaugurated its opening and began its first academic term. "Tongji" is pronounced as "Dongji" in Shanghai dialect, which is a partial tone akin to "Deutsch", so Dr. Erich Paulun christened the school as Tongji Deutsch Medical School. The classes had been conducted in German only until 1950s.

Dr. Erich Paulun passed away on March 5, 1909. In 1912, Tongji Deutsch Medical School opened a department of engineering. It was approved to become a university in 1923 and renamed as National Tongji University in 1927. During the Anti-Japanese War (1937 - 1945), Tongji University had been retreated to Zhejiang, Jiangxi, Guangxi, Yunnan, and finally to Sichuan Province, but its development was still going on. At that time, the German chancellor of the University resigned, and the first Chinese chancellor acceded. It returned to Shanghai in 1946, and became a comprehensive university, which comprised faculties such as sciences, engineering, medicine, arts and law. The engineering students graduated in the University could attain the same qualifications as those who were studied in Germany before 1949.

In 1952, Shanghai Tongji University became the largest institution of China famous in construction engineering. The Chinese government approved to resume of academic exchanges between the University and Germany in 1978. The University has been upgraded to a comprehensive institution in various subjects. It also became a window for academic and cultural exchanges between China and the outside world.

In recent decades, following the trend of building bigger and bigger universities in China, Shanghai Tongji University has been expanded through annexation of various local higher educational institutions. The University became a brand-new and giant academic institution with nine faculties. It is one of the key universities approved by China's State Council as well.