Comparing Five Photo Restoration Software for Stellar Results
With the rapid development of big language models and various AI applications, I finally decided to digitize all the paper and film photos from the previous generation's physical albums and perform a thorough AI restoration before they become yellow, faded, sticky or damaged so that these precious memories can be preserved for a longer time.
The digitization is done using the Photoscan app by Google Albums, which automatically adjusts angles, crops edges, eliminates glare and shadows, and saves both the original and the automatically edited image in iCloud Albums (the scanner is recommended for maximum resolution if possible).
The next problem is how to pick a suitable AI restoration tool. Based on experience and recent hits, I picked five candidates. In order to get the best results, I tried almost every single one of the hundreds of old photos to be restored with all the tools, and I had an in-depth use of these tools, so I'll share my experience with you below.
Five Photo Restoration Quick Overview:
CodeFormer
by Shang-Chen Zhou et al. at Nanyang Technological University, based on the Codebook Lookup TransFormer model implementation.
Official website: https://shangchenzhou.com/projects/CodeFormer/
Github project address: https://github.com/sczhou/CodeFormer
Replicate online demo: https://replicate.com/sczhou/codeformer (shallow experience can be built without yourself, directly with the demo page is good, beyond the Replicate free quota, then bind a paid account)
I use the parameters: fidelity = 0.8, background_enhance = true, face_upsample = true, upscale = 2
RestorePhotos.io
A tool for restoring old photos based on Tencent ARC Lab GFP-GAN model.
Official website: https://www.restorephotos.io/ (you can use it 5 times a day for free, if you deploy it by yourself, for example, using Vercel one-click deployment and filling in your own Replicate API key, you are not limited to this)
Github project address: https://github.com/Nutlope/restorePhotos
The first group of test pictures: single and double (headshot)
Test 1-1 ( Teresa Teng )
This is a screenshot of Teresa Teng from a TV show. Almost all tools reproduce this picture well, including the hair as well as the earrings ("CodeFormer" has some flaws in the detailing of the teeth). Which image do you think is more like the real Teresa Teng?
Test 1-2 Jay Chou
The original picture of Jay Chou's childhood photo is blurry and has a lot of noise. In terms of restoration performance, "CodeFormer", "You and Me Back Then" and "Old Photo Restoration" are better at background noise reduction and clothing texture restoration, but only "jpgHD.com" accurately restores Zhou's single eyelid and inner eye corner details, which in my opinion is the most like the original picture. All other tools used "imagination" appropriately.
Test 1-3 Photo of Leslie Cheung and Cecilia Cheung
A blurry group photo, and the restoration results of the 5 tools can be said to be on the same track.
Test 1-4 Jacky Cheung and Joey Wong
With the exception of "jpgHD.com", which is rough on Joey Wong's skin, the tools are almost indistinguishable on this single and double headshot. Note that all tools have restored the earrings for restoration.
Test 1-5 Einstein
Here is an attempt at restoring the three greats Albert Einstein, Hendrik Lorenz, and Madame Curie from the Fifth Solvay Conference in 1927. The original image is in black and white and has been manually colored, but the details have not been restored by enlargement. It can be seen that "CodeFormer" is better at restoring faces and suits, and successfully identifies Arthur Compton, who has only half a face in the back.
Test 2-1: Slightly blurred group photo of multiple people
The author of the image is not available, the original title is "On Camera: Chinese Children in the 80s" [1].
Test 2-2: Moderately blurred group photo of four people
Source: A random photograph taken on campus by students of a double-degree course in international trade at the School of Economics and Management of Nanjing University of Technology (summer 1990, from left: Han Mengling, Zhou Shuanwen, Quartz, and Lv Hong) [2].
The processing of this image produced relatively obvious differences. "CodeFormer" had a more natural face, successfully restored the glasses of the third girl from the left (but produced burrs), and restored the hair, clothes and buildings more accurately; "You and Me Back Then" had a very natural face restoration, and "Old Photo Restoration" did a good job in building restoration. RestorePhotos.io", on the other hand, showed a greater degree of distortion in facial restoration, and the glasses of the third girl from the left were not successfully restored.
Test 2-3: Highly blurred group photo of multiple people
Photo credit: Group photo in front of the small hall of the West Building of the Shanghai Institute of Optical Precision Machinery, Chinese Academy of Sciences in the 1980s (from left, front row: Huimin Wang, Lianying Zhang, Houcai Yang, Abin Yu, Deying Gu, Ailian Li, back row: Yuming Lin, Zujie Fang, Bitzai Zhang, Zhiliang Jin, Zhiwu Liu, Yanzhi Hu) [3].
The human face in this figure appears to be blurred to a greater extent, and each tool has given full play to its imagination for brainstorming.
Regarding the natural harmony of the features, I would rank the restoration results like this: CodeFormer > Old Photo Restoration > jpgHD.com > You and Me Back in the Day > RestorePhotos.io.
Regarding the restoration of backgrounds and clothing, I would rank the restoration results as follows: CodeFormer = Old Photo Restoration > You and Me Back in the Day > RestorePhotos.io = jpgHD.com.
Test Conclusion
CodeFormer
- Image quality improvement: supported, with option to improve image quality (custom magnification)
- Clarity: very high
- Accuracy: high, with high accuracy compared to the original image, but a few unnatural details may appear
- Restoration speed: average, ranging from 6s to 60s
RestorePhotos.io
- Image quality improvement: not supported
- Clarity: low, the maximum resolution of the generated image is 1200px
- Accuracy: low, more "imaginary" than the original image, poor performance on small avatars, easily distorted
- Repair speed: fast, usually completed within 10s
jpgHD.com
- Image quality improvement: support, optional image quality improvement (2x, 4x)
- Clarity: medium
- Accuracy: very high, can restore the real details of the original photo
- Restoration speed: very slow, may take 3 to 5 minutes even for small images
你我当年
- Image quality improvement: supported, default 2x
- Clarity: high
- Accuracy: medium, may be distorted for small avatars
- Restoration speed: fast, usually around 10s
老照片修复
- Image quality improvement: supported, default 2x
- Clarity: high
- Accuracy: high
- Restoration speed: very fast, usually around 10s
If you are like me and have the need to repair hundreds of photos at the same time, you can build CodeFormer locally to save your effort. The performance of "You and Me Back in the Day" and "Old Photo Restoration" is quite stable, but it is not as good as "CodeFormer" in the performance of small avatars, and it is a paid software.
All the tools except "RestorePhotos.io" support old photo coloring, but this feature has not been tested in this article. "RestorePhotos.io" has some advantages for large headshots and difficult images with more frosted particles or interference stripes, but is not ideal for other cases.
Conclusion
Due to the limited energy, this article has only tested and compared some common situations, hoping to help you make a preliminary choice judgment. In fact, in the process of old photo restoration will also face many difficult problems, such as particles generated by frosting treatment, interference stripes generated by flipping the screen, as well as some smears, creases, tears, mold and other problems. When facing these problems, you can try different tools for a comprehensive comparison, and even more patient manual intervention is also needed to overlay the effects of different restoration tools.
After the initial repair of old photos, you can also try to use these tools for further non-destructive enlargement, as well as intelligent coloring processing. It is also recommended for those who have strong hands to do further post-processing such as manual color grading and mapping for those very important and precious photos, after all, although AI can already help us restore the original look of photos to a great extent, it cannot reproduce history 100% accurately.
References:
https://www.toutiao.com/article/6388468827609776386/
https://sem.njust.edu.cn/sem40/bd/aa/c13401a245162/page.htm
http://pic.siom.cas.cn/qttp/lzp/201503/t20150323_4325589.html