Reconstructing images of objects: method for reconstructing images from digital off-axis holograms based on a generative adversarial neural network

The reconstruction of object images that are located in 3D scene cross-sections using digital holography is described. The potential of generative adversarial networks for reconstructing cross-sections of 3D scenes composed of multiple layers of off-axis objects from holograms is investigated. Such scenes consist of a series of sections with objects that are not aligned with the camera’s axis. Digital holograms were used to reconstruct images of cross-sectional views of 3D scenes. It has been shown that the use of neural networks increases the speed and reconstruction quality, and reduces the image noise. A method for reconstructing images of objects using digital off-axis holograms and a generative adversarial neural network is proposed. The proposed method was tested on both numerically simulated and experimentally captured digital holograms. It was able to successfully reconstruct up to 8 cross-sections of a 3D scene from a single hologram. It was obtained that an average structural similarity index measure was equal to at least 0.73. Based on optically registered holograms, the method allowed us to reconstruct object image cross-sections of a 3D scene with a structural similarity index measure over cross-sections of a 3D scene of equal to 0.83. Therefore, the proposed technique provides the possibility for high-quality object image reconstruction and could be utilized in the analysis of micro- and macroobjects, including medical and biological applications, metrology, characterization of materials, surfaces, and volume media.

1. Schnars U., Jueptner W. Digital holography: Digital hologram recording, numerical reconstruction, and related techniques. Springer, Berlin Heidelberg (2005). https://doi.org/10.1007/b138284

2. Wang Z., Bianco V., Maffettone P. L., Ferraro P. Holographic flow scanning cytometry overcomes depth of focus limits and smartly adapts to microfluidic speed. Lab on a Chip, 23, 2316–2326 (2023). https://doi.org/10.1039/D3LC00063J

3. Singh V., Joshi R., Tayal S., Mehta D. S. Speckle-free common-path quantitative phase imaging with high temporal phase stability using a partially spatially coherent multi-spectral light source. Laser Physics Letters, 16, 025601 (2019). https://doi.org/10.1088/1612-202X/AAF179

4. Calore D., Fraticelli N. State of the art offshore in situ monitoring of microplastic. Microplastics, 1, 640–650 (2022). https://doi.org/10.3390/MICROPLASTICS1040044

5. Zhang W., Li B., Zhang X., Shi C. Off-axis digital holography based on the Sagnac interferometer. Laser Physics Letters, 18, 035202 (2021). https://doi.org/10.1088/1612-202X/ABDECB

6. Bondareva A. P., Cheremkhin P. A., Evtikhiev N. N., et al. Measurement of characteristics and phase modulation accuracy increase of LC SLM “HoloEye PLUTO VIS”. Journal of Physics: Conference Series, 536(1), 012011 (2014). https://doi.org/10.1088/1742-6596/536/1/012011

7. Upatnieks J., Leith E. N. Wavefront reconstruction with diffused illumination and three-dimensional objects. Journal of the Optical Society of America, 54, 1295–1301 (1964). https://doi.org/10.1364/JOSA.54.001295

8. Park J., Kang H., Stoykova E. Twin-image problem in digital holography – a survey. Chinese Optics Letters, 12, 060013 (2014). https://doi.org/10.3788/COL201412.060013

9. Yamaguchi I. Phase-shifting digital holography. Optics Letters, 22, 1268–1270 (1997). https://doi.org/10.1364/OL.22.001268

10. Olivier T., Momey F., Denis L., Fournier C. From Fienup’s phase retrieval techniques to regularized inversion for in-line holography: tutorial. Journal of the Optical Society of America A, 36, D62–D80 (2019). https://doi.org/10.1364/JOSAA.36.000D62

11. Zeng T., Zeng T., Zhu Y., et al. Deep learning for digital holography: a review. Optics Express, 29, 40572–40593 (2021). https://doi.org/10.1364/OE.443367

12. Cheremkhin P. A., Evtikhiev N. N., Krasnov V. V., et al. Machine learning methods for digital holography and diffractive optics. Procedia Computer Science, 169, 440–444 (2020). https://doi.org/10.1016/j.procs.2020.02.243

13. Situ G. Deep holography. Light Advanced Manufactoring, 3, 278–300 (2022). https://doi.org/10.37188/LAM.2022.013

14. Cheremkhin P. A., Rymov D. A., Svistunov A. S., Zlokazov E. Yu., Starikov R. S. Neural-network-based methods in digital and computer-generated holography. А review. Opticheskii Zhurnal, 91, 62–78 (2024). (In Russ.) http://doi.org/10.17586/1023-5086-2024-91-03-62-78

15. Rymov D. A., Cheremkhin P. A., Starikov R. S., Neural-network-enabled holographic image reconstruction via amplitude and phase extraction. Journal of Optical Technology, 89(9), 511–516 (2022). https://doi.org/10.1364/JOT.89.000511

16. Pirone D., Sirico D., Miccio L., et al. Speeding up reconstruction of 3D tomograms in holographic flow cytometry via deep learning. Lab on a Chip, 22, 793–804 (2022). https://doi.org/10.1039/D1LC01087E

17. Razi A., Chen X., Wang H., et al. DH-GAN: a physics-driven untrained generative adversarial network for holographic imaging. Optics Express, 31, 10114–10135 (2023). https://doi.org/10.1364/OE.480894

18. Svistunov A. S., Rymov D. A., Starikov R. S., Cheremkhin P. A. HoloForkNet: Digital Hologram Reconstruction via Multibranch Neural Network. Applied Sciences, 13 (10), 6125 (2023). https://doi.org/10.3390/app13106125

19. Goodfellow I., Pouget-Abadie J., Mirza M., et al. Generative adversarial networks. Communications of the ACM, 63, 139–144 (2014). https://doi.org/10.1145/3422622

20. Tang H., Liu H., Xu D., et al. AttentionGAN: unpaired image-to-image translation using attention-guided generative adversarial networks, IEEE Trans. Neural Networks Learning Systems, 34, 1972–1987 (2023). https://doi.org/10.1109/TNNLS.2021.3105725

21. Xia J., Zhang L., Zhai Y., Zhang Y. Reconstruction method of computational ghost imaging under atmospheric turbulence based on deep learning. Laser Physics, 34, 015202 (2023). https://doi.org/10.1088/1555-6611/AD0EBF

22. Leonov M. M., Soroka A. A., Trofimov A. G. Russian language speech generation from facial video recordings using variational autoencoder. Studies in Computational Intelligence, 1120, 489–498 (2023). https://doi.org/10.1007/978-3-031-44865-2_51

23. Fevens T., Jaferzadeh K. HoloPhaseNet: fully automated deep-learning-based hologram reconstruction using a conditional generative adversarial model. Biomedical Optics Express, 13, 4032–4046 (2022). https://doi.org/10.1364/BOE.452645

24. Ay B. Open-set learning-based hologram verification system using generative adversarial networks. IEEE Access, 10, 25114–25124 (2022). https://doi.org/10.1109/ACCESS.2022.3155870

25. Kiriy S. A., Rymov D. A., Svistunov A. S., et al. Generative adversarial neural network for 3D-hologram reconstruction. Laser Physics Letters, 21, 045201 (2024). https://doi.org/10.1088/1612-202X/ad26eb

26. Prilepsky J. E., Manuylovich E., Freire P., Turitsyn S. K. Artificial neural networks for photonic applications – from algorithms to implementation: tutorial. Advances in Optics and Photonics, 15, 739–834 (2023). https://doi.org/10.1364/AOP.484119

27. Genty G., Salmela L., Dudley J. M., et al. Machine learning and applications in ultrafast photonics. Nature Photonics, 15, 91–101 (2020). https://doi.org/10.1038/s41566-020-00716-4

28. Dmitriev E., Bukharskii N., Korneev P. Powerful elliptically polarized terahertz radiation from oscillating-laser-driven discharge surface currents. Photonics, 10(7), 803 (2023). https://doi.org/10.3390/photonics10070803

29. Yu Z., Li X., Bai X., et al. Computational ghost imaging through a dynamic scattering medium based on a convolutional neural network from simulation. Laser Physics Letters, 20, 055204 (2023). https://doi.org/10.1088/1612-202X/ACC245

30. Loskutov A. I., Yakimov V. L., Karpushev S. I., et al. Model for Monitoring the Technical Condition of Onboard Equipment of Space Vehicles Based on the Telemetry Parameters of Transient Processes. Measurement Techniques, 66(6), 384–391 (2023). https://doi.org/10.1007/s11018-023-02238-1

31. Zlokazov E. Yu., Minaeva E. D., Rodin V. G., Starikov R. S., Cheremkhin P. A., Shifrina A. V. Methods of diffractive optical element generation for rapid, high-quality 3D image formation of objects divided into a set of plane layers, Measurement Techniques, 66(11), (2024). https://doi.org/10.1007/s11018-024-02301-5

32. Ronneberger T. B. O., Fischer P. U-Net: convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science, 9351, 1–8 (2015). https://doi.org/10.1007/978-3-319-24574-4_28

33. Lecun Y. The MNIST database of handwritten digits, available at: http://Yann.Lecun.Com/Exdb/Mnist/ (accessed: 01 April 2024).

34. Wang Z., Bovik A. C., Sheikh H. R., Simoncelli E. P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13, 600–612 (2004). https://doi.org/10.1109/tip.2003.819861

35. Minikhanov T. Z., Zlokazov E.Yu., Starikov R. S., Cheremkhin P. A. Phase modulation time dynamics of the liquid-crystal spatial light modulator. Measurement Techniques, 66(12), (2024). https://doi.org/10.1007/s11018-024-02309-x