Awards and Achievements
DAAD PhD Research Grant
Technical University Berlin, Germany (August 2017 - March 2022)
This prestigious grant, offered by the German Academic Exchange Service (DAAD), supported full-time PhD research, allowing advanced study in computer vision and machine learning at a leading German institution, fostering international academic collaboration.
Best Student Award
IIT Roorkee, India (Sept. 2016)
Recognized for outstanding academic performance in the M.Tech. Engg./M.Arch. programs (2014-2016), this award highlights excellence in coursework and research, reflecting dedication to engineering innovation.
Dr. Jai Krishna Medal
IIT Roorkee, India (Sept. 2016)
Awarded for achieving the highest CGPA across all M.Tech. courses, this medal honors academic rigor and is named after Dr. Jai Krishna, a pioneer in earthquake engineering, symbolizing excellence in technical education.
Academic Prize
IIT Roorkee, India (Sept. 2016)
Granted for the highest CGPA in the M.Tech. (Electrical Engineering) final year, this prize underscores exceptional mastery of electrical engineering concepts and their practical applications.
DAAD IIT Master Sandwich Scholarship
Technical University Berlin, Germany (Sept 2015 - March 2016)
This DAAD-funded scholarship enabled a final-year master’s student to conduct full-time dissertation research in Germany, enhancing skills in 3D reconstruction and fostering global research networks.
Doctoral Thesis
Deep Learning Mesh Parameterization of 3D Shapes: For 3D Reconstruction, Shape Generation, Noise Filtering, and Mobile Rendering
Deep learning has made remarkable progress in extracting meaningful information from visual sensory 2D data. The regularized grid representation of the 2D image data makes it convenient to apply efficient convolutional kernels. Comparatively, the 3D data, specifically the widely used surface meshes are less utilized in deep learning applications. Along with the added dimension, this is attributed to the irregular representation of the mesh structure. This thesis is motivated by the performance of 2D convolutional kernels on regular grid and tries to employ them for 3D surface meshes. The inherent irregular connectivity of surface mesh can be regularized by borrowing the concept of mesh parameterization from computer graphics domain. The regularized surface meshes are trained in a deep convolutional neural network for 3D reconstruction from a single image, generative icosahedral mesh convolutional network and icosahedral mesh denoising. The efficient networks for 3D surface meshes proposed in this thesis use 2D convolutional kernels, making it possible to be inferenced on low-compute devices. To exhibit this capability, an android application for the 3D reconstruction network from a single image is designed and validated.
Research Publications
IMD-Net: a Deep Learning-based Icosahedral Mesh Denoising Network
In this work, we propose a novel denoising technique, the icosahedral mesh denoising network (IMD-Net) for closed genus-0 meshes. A preprocessing step, exploiting the homeomorphism between genus-0 mesh and sphere, remeshes an irregular mesh using the regular mesh structure of a frequency subdivided icosahedron. Enabled by gauge equivariant convolutional layers arranged in a residual U-net, IMD-Net denoises the remeshing invariant to global mesh transformations as well as local transformations.
GenIcoNet: Generative Icosahedral Mesh Convolutional Network
In the past few decades, the computer vision domain has achieved outstanding success in learning 3D shapes for classification, segmentation and image-based reconstruction. However, deep networks are less explored for the generative task of obtaining new 3D shapes from the learned representation. This problem becomes more prominent for 3D shapes represented as surface meshes, mainly because the mesh structure lacks regularity, an essential property for training deep generative networks. In this work, we remedy this problem by proposing a generative icosahedral mesh convolutional network (GenIcoNet) that learns data distribution of surface meshes. Our end-to-end trainable network learns semantic representations using 2D convolutional filters on the regularized icosahedral meshes. Our experiments demonstrate that both the architectures of GenIcoNet are able to outperform existing networks trained on intermediate surface mesh representations.
Learning to Reconstruct Symmetric Shapes using Planar Parameterization of 3D Surface
In this work, we try to reconstruct 3D shape from images by using a parameterized representation of the shape. We perform iterative parameterization of the surface to obtain a planar representation. This representation is encoded with surface information to generate 2D geometry images, which can be conveniently learned using traditional deep neural networks without additional overhead. Our experiments demonstrate that the proposed network learns detailed features and is able to reconstruct geometrically accurate shapes from single image.
Improving 3D Face Geometry by Adapting Reconstruction From Stereo Image Pair to Generic Morphable Model
Stereo reconstruction from image pairs is a standard method for 3D acquisition of human faces. Depending on available imagery and accuracy requirements the resulting 3D reconstructions may have deficits. In this work we remedy such deficits combining the 3D stereo reconstruction with a generic Morphable Model. Prior shape information can be obtained by already developed methods, which uses landmarks to fit a morphable model to a single image resulting in a second 3D reconstruction. This alternative to the stereo reconstruction is combined with it, allowing to prefer information from the single image reconstruction whenever the stereo reconstruction shows untypical deviations from the expected 3D features of a human face.
Master Thesis
Using Morphable Face Model to Improve Stereo Reconstruction and Visualising the Model on a Smartphone
Stereo Reconstruction from image pair is a standard method for 3D acquisition of human faces. Stereo surface reconstruction has lots of holes, and limited texture information. In this work we remedy such deficits combining the 3D stereo reconstruction with a generic Morphable Face Model. A Major part of the thesis is devoted to improvement in stereo face reconstruction pipeline by allowing to prefer information from the single image reconstruction whenever the stereo reconstruction shows untypical deviations. From a pair of stereo images, one is used for single image reconstruction and the completion gives the stereo model. The two reconstruction are then combined to result in the deformed face model. The fusion outcome results in geometrically more accurate face reconstruction. Finally, the resultant deformed face model is visually presented on a smartphone using cardboard, which addresses the modern trend of low cost devices in virtual 3D visualization.
Publications from Accomplished Projects
Calibration and Registration Method for Tomography-Based Laser-Guided Surgical Interventions using a 4-DOF Navigation Robot
A method for co-registering pre-surgically acquired tomographic data with a patient at time of surgical intervention is introduced. The system using this method consists of a registration element fixed to the skin of the patient at scan and intervention time, a camera, and a laser pointing device movable on a bow rail circumferencing the body of the patient. Camera and laser bow are firmly mounted and calibrated to each other. Pre-operative planning is done using the tomographic data. After moving the patient out of the gantry, the laser ray is steered to indicate position and orientation of a linear instrument (e.g. a needle) on the patient’s skin. It is shown, that the achievable accuracy of the method presented in this paper is sufficient for periradicular therapy.
Laser-Guided CT Intervention using Flexible Laser Bow
Laser bow systems are used for intervention guidance by steering laser rays. Such systems have shown applications in biopsy, nasal surgeries, drainage, pain therapy. In this paper, we introduce an image-based laser guided CT intervention. Instead of relying on traditional laser bow rigidly mounted to the CT scanner, our approach performs registration making use of a patient tracker in image as well as CT data to find the orientation of laser bow with respect to the CT data. The image is acquired by a camera rigidly connected to the laser bow. The proposed approach allows movability to the laser bow system without compromising on accuracy. Using camera allows real-time view of the intervention and can be used for futuristic applications.
A Power Efficiency Enhancements of a Multi-Bit Accelerator for Memory Prohibitive Deep Neural Networks
State of art deep neural network (DNN) models are both memory prohibitive and computationally intensive with millions of connections. Employing these models for an embedded mobile application is resource limited with large power consumption and significant bandwidth requirement. In this paper, we propose a power efficient multi-bit neural network accelerator, where we employ the technique of truncating the partial sum (PSum) results from the previous layer before feeding it into the next layer.
Patient Motion Compensation for Photogrammetric Registration
In this work, we treat the task for multi-view monocular imagery acquiring both body surface as well as reference markers. To fulfill the high accuracy requirements the patient is not supposed to move while images are taken. An approach towards relaxing this demanding situation is to measure small movements of the patient, e.g. with help of an electromagnetic device, and to compensate for the measured motion prior to body surface triangulation. We present two approaches for motion compensation: disparity shift compensation, and moving cameras compensation - both capable of achieving patient registration qualitatively equivalent to motion-free registration.
Passive Classification of Source Printer using Text-line-level Geometric Distortion Signatures from Scanned Images of Printed Documents
In this digital era, one thing that still holds the convention is a printed archive. Printed documents find their use in many critical domains such as contract papers, legal tenders and proof of identity documents. As more advanced printing, scanning and image editing techniques are becoming available, forgeries on these legal tenders pose a serious threat. Ability to easily and reliably identify source printer of a printed document can help a lot in reducing this menace.
An Enhanced Statistical Approach for Median Filtering Detection using Difference Image
In image forensics, detection of image forgeries involving non-linear manipulations have received a great deal of interest in recent past. Median filtering (MF) is one such non-linear manipulation technique which is quite often used in number of applications such as to hide impulse noises. Unlike other linear filtering operations, non-linear characteristics of median filtering makes it harder to detect using traditional forensics methods designed for detecting linear operations.
3D Medical Data Registration
An online guest lecture on 3D Medical Data Registration for students of Healthcare Technology at School of E&TC Engineering, MIT Academy of Engineering, Alandi Pune, India (Oct 2023).
DAAD Scholarship Meeting
Michael Jordan © DAAD
Glimpse of the presentation (Immerse yourself in Virtual Reality) at invited talk during DAAD Scholarship Meeting Cottbus (July 2018). An article on the same would be published in online magazine "DAAD Aktuell".
Talk at my School
A talk with class 12th students of my school (Lions Convent School, Sendhwa) in March 2018. Main agenda of the talk was not any motivational speech but a discussion of mistakes I did in my educational career and how I overcame them.
Projects
CV VR
A virtual reality app premiered at Long Night of Science 2019, showcasing computer vision projects.
GRSS VR
Illustrates the work of GRSS technical committees with Google Cardboard compatibility.
Birthday VR
First human model-based VR app on Google Play, featuring interactive animations.