GPU based techniques for deep image merging

Computational Visual Media, Aug 2018

Deep images store multiple fragments perpixel, each of which includes colour and depth, unlike traditional 2D flat images which store only a single colour value and possibly a depth value. Recently, deep images have found use in an increasing number of applications, including ones using transparency and compositing. A step in compositing deep images requires merging per-pixel fragment lists in depth order; little work has so far been presented on fast approaches. This paper explores GPU based merging of deep images using different memory layouts for fragment lists: linked lists, linearised arrays, and interleaved arrays. We also report performance improvements using techniques which leverage GPU memory hierarchy by processing blocks of fragment data using fast registers, following similar techniques used to improve performance of transparency rendering.We report results for compositing from two deep images or saving the resulting deep image before compositing, as well as for an iterated pairwise merge of multiple deep images. Our results show a 2 to 6 fold improvement by combining efficient memory layout with fast register based merging.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://link.springer.com/content/pdf/10.1007%2Fs41095-018-0118-8.pdf

GPU based techniques for deep image merging

Computational Visual Media pp 1–9 | Cite as GPU based techniques for deep image merging AuthorsAuthors and affiliations Jesse ArcherGeoff LeachRon van Schyndel Open Access Research Article First Online: 04 August 2018 Received: 23 December 2017 Accepted: 02 May 2018 Abstract Deep images store multiple fragments perpixel, each of which includes colour and depth, unlike traditional 2D flat images which store only a single colour value and possibly a depth value. Recently, deep images have found use in an increasing number of applications, including ones using transparency and compositing. A step in compositing deep images requires merging per-pixel fragment lists in depth order; little work has so far been presented on fast approaches. This paper explores GPU based merging of deep images using different memory layouts for fragment lists: linked lists, linearised arrays, and interleaved arrays. We also report performance improvements using techniques which leverage GPU memory hierarchy by processing blocks of fragment data using fast registers, following similar techniques used to improve performance of transparency rendering.We report results for compositing from two deep images or saving the resulting deep image before compositing, as well as for an iterated pairwise merge of multiple deep images. Our results show a 2 to 6 fold improvement by combining efficient memory layout with fast register based merging. Keywordsdeep image composite GPU performance  Jesse Archer is a Ph.D. student at RMIT University, Melbourne. His research interests are in realtime computer graphics and GPU computing. He completed his bachelor of computer science in 2008, bachelor of IT (games and graphics programming) in 2010, and honours in computer science in 2015 at RMIT University. Geoff Leach is a lecturer in the School of Science at RMIT University. His major research interests include computer graphics, computational science, and GPU computing. He mostly teaches computer graphics, and has been using OpenGL since version 1.1. He holds a M.App.Sci. degree from RMIT University. Ron van Schyndel is a senior lecturer from School of Science (formerly School of Computer Science and IT) at RMIT University. He is and has been an active researcher in the domain of digital watermarking for more than 2 decades, and is co-author to some of the most cited papers in the field. He obtained his Ph.D. degree from Monash University on the then nascent topic of digital watermarking, and has obtained many industry grants on watermarking and other applications. His other research interests beyond digital watermarking include signal, image, and vision processing, as well as software infrastructure specifically as applied to mobile navigation for the blind and visually impaired. Download to read the full article text Notes Acknowledgements The authors would like to thank Pyar Knowles for his original deep image software on which this work is based. It is available at https://doi.org/github.com/pknowles/lfb. References [1] Heckenberg, D.; Saam, J.; Doncaster, C.; Cooper, C. Deep compositing. 2010. Available at https://doi.org/www.johannessaam.com/deepImage.pdf. Google Scholar [2] Duff, T. Deep compositing using lie algebras. ACM Transactions on Graphics Vol. 36, No. 3, Article No. 26, 2017.Google Scholar [3] Maule, M.; Comba, J. L. D.; Torchelsen, R.; Bastos, R. Memory-efficient order-independent transparency with dynamic fragment buffer. In: Proceedings of the 25th SIBGRAPI Conference on Graphics, Patterns and Images, 134–141, 2012.Google Scholar [4] Knowles, P.; Leach, G.; Zambetta, F. Efficient layered fragment buffer techniques. In: OpenGL Insights. Cozzi, P.; Riccio, C. Eds. CRC Press, 279–292, 2012.CrossRefGoogle Scholar [5] Knowles, P.; Leach, G.; Zambetta, F. Fast sorting for exact OIT of complex scenes. The Visual Computer Vol. 30, Nos. 6–8, 603–613, 2014.CrossRefGoogle Scholar [6] Porter, T.; Duff, T. Compositing digital images. In: Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques, 253–259, 1984.Google Scholar [7] Egstad, J.; Davis, M.; Lacewell, D. Improved deep image compositing using subpixel masks. In: Proceedings of the 2015 Symposium on Digital Production, 21–27, 2015.CrossRefGoogle Scholar [8] Hillman, P. The theory of OpenEXR deep samples. Technical Report. Weta Digital Ltd., 2013.Google Scholar [9] Knowles, P.; Leach, G.; Zambetta, F. Backwards memory allocation and improved OIT. In: Proceedings of the Pacific Graphics, 59–64, 2013.Google Scholar [10] McGuire, M. Computer graphics archive. 2017. Available at https://doi.org/casual-effects.com/data. Google Scholar Copyright information © The Author(s) 2018 Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (https://doi.org/creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Authors and Affiliations Jesse Archer1Email authorGeoff Leach1Ron van Schyndel11.School of ScienceRMIT UniversityMelbourneAustralia


This is a preview of a remote PDF: https://link.springer.com/content/pdf/10.1007%2Fs41095-018-0118-8.pdf

Jesse Archer, Geoff Leach, Ron van Schyndel. GPU based techniques for deep image merging, Computational Visual Media, 2018, 1-9, DOI: 10.1007/s41095-018-0118-8