Robust Camera Self-Calibration

Project Supervisor: Prof. Dr.-Ing. R.-R. Grigat
Coworker: M.Sc. Luh Putu Ayu Prapitasari
Running time: 10.2012 - 09.2016
Financing: Sholarship from Indonesia
Publications: here
References: Kruppa's equation, absolute conic, calibration

The demands in camera self-calibration is very high but unfortunately, at the moment, there is no mature method has been found or developed so far.

In this research we aim to develop a robust camera self-calibration algorithm for general cameras, either using single or multiple view as the input. The camera model is restricted to be a pinhole camera which is supposed to be projective camera, including the smartphone cameras. As the first method in self-calibration was the Kruppa's equations which is worked based on the absolute conic, then the method that we are going to develop will first investigate the Kruppa's as the basic and for the better understanding of the problem.

The possibility of the algorithm to be used for stereo camera will also be investigated for the future use.

current Research   |    Home  |   completed Research


Edge-Based Surface Reconstruction and Object Detection

Project Supervisor: Prof. Dr.-Ing. R.-R. Grigat
Coworker: Dipl.-Ing. Gerhard Mentges
Running time: 2.2014 -
Financing: TUHH
Publications: here
References: 3D Reconstruction, Surface Reconstruction, Edge-SLAM, Object Detection

Camera-based surface reconstruction is a major problem in computer vision. Existing approaches have problems with scenes featuring only sparse or no texturing. For untextured scenes, edges can still offer enough information for a complete 3D reconstruction or object detection. Aim of this project is the development of algorithms coping well with untextured as well as textured environments. A key concept is the use frame-spanning correspondences between 2D and 3D edges obtained via Simultaneous Localization And Mapping. This enables us to perform a hybrid 2D/3D image analysis, striving after an imitation of natural image and scene understanding.

current Research   |    Home  |   completed Research


Dense 3D Reconstruction from Monocular Mini-Laparoscopic Sequences

Project Supervisor: Prof. Dr.-Ing. R.-R. Grigat
Coworker: M. Sc. Sven Painer
Running time: 1.2015 - 12.2018
Financing: TUHH
Publications: here
References: Dense 3D Reconstruction, Mini-Laparoscopic Sequences

3D reconstruction using a single moving camera is possible up to a global scale factor. For a stereo camera rig with overlapping field-of-view, it is possible to reconstruct any object inside the overlap including the scale and therefore, the whole scene can be reconstructed including global scale. There are situations where the camera setup is fixed without overlap or where there is a limit in the number of cameras and as much as possible of the field-of-view should be used. In such situations, well-known stereo algorithms can not be used, but it is still possible to reconstruct the scene including global scaling. In this project, such algorithms are developed, extended and evaluated.

current Research   |    Home  |   completed Research


Real-time data processing for serial crystallography experiments

Project Supervisor: Prof. Dr.-Ing. R.-R. Grigat
Coworker: M.Sc. Yaroslav Gevorkov
Running time:  04.2016 - 03.2018
Financing:  Deutsche Elektronen-Synchrotron DESY
Publikations:
References: Real-time capable algorithms,  Real-time data processing,  GPU computing (CUDA),  Pattern recognition, Crystallography, Femtosecond X-ray imaging

In many scientific and industrial applications, the structure of molecules is of high interest.
As an example, in pharmacy one is interested in the structure of viruses or amino acids.
To identify the structure, it is not possible to construct microscopes that use visible light, since the resolution is limited by the wavelength of the light.
The solution is to use light sources with higher energy, such as an X-ray source. We use X-ray diffraction to extract information abut molecules out of artificially grown micro crystals. The X-ray sources typically are X-ray Free Electron Lasers (XFELs) that can deliver a very high radiation dose, far in excess of what the crystal could normally tolerate, in a time span in the range of femtoseconds.
The crystal diffracts the X-rays before it is destructed, overcoming the effect of radiation damage [1]. Due to the limited availability of beam time, the optimization of the collection process is crucial for obtaining good results.
Therefore, the real-time analysis and monitoring of the collected data is of great interest. We develop algorithms and tools that are capable of meeting real-time constraints of current X-ray sources while retaining or improving the results compared to the conventional methods of choice.

[1] Chapman et al. J. "Femtosecond diffractive imaging with a soft-X-ray free-electron laser". Nature Physics 2006, 2 (12), 839-8

current Research   |    Home  |   completed Research


Sign Language and Mouth Gesture Recognition

Project Supervisor: Prof. Dr.-Ing. R.-R. Grigat
Coworker: M.Sc. Maren Brumm
Running time: 09.2016 -
Financing: Universität Hamburg
Publications: here
References: Neural networks, deep learning, machine learning, mouth gestures, automatic sign language recognition

In sign languages there are two different kinds of mouth patterns. Mouthing is derived from a spoken language and its detection is therefore very similar to lip reading. Mouth gestures on the other hand have formed from within the sign languages and bear no relation to spoken languages [1].

In this project we aim at automatic detection and classification of mouth gestures in German sign language to facilitate the research on usage of mouth gestures and advance on the open problem of reliable automatic sign language recognition.

To do so we are developing a deep neural network that can be trained to learn the spatiotemporal features of mouth gestures.

[1] Keller, J. (2001). Multimodal Representation and the Linguistic Status of Mouthings in German Sign Language (DGS). The Hands are the Head of the Mouth: The Mouth as Articulator in Sign Languages (S. 191-230). Hamburg: Signum Verlag.

current Research   |    Home  |   completed Research


Learning an invariant distance metric

Project Supervisor: Prof. Dr.-Ing. R.-R. Grigat
Visiting Researcher: M. Sc. Zahra Goudarzi
Running time: 02.2017 -
Financing:
Publications: hier
References: Metric learning, Mahalanobis Metric, Grassmann manifolds

The requirement for suitable ways to measure the distance or similarity between data is omnipresent in machine learning, pattern recognition and data mining, but extracting such good metrics for particular problems is in general challenging.

This has led to the emergence of metric learning ideas, which intend to automatically learn a distance function tuned to a specific task. In many tasks and data types, there are natural transformations to which the classification result should be invariant or insensitive. This demand and its implications are essential in many machine learning applications, and insensitivity to image transformations was in the first place achieved by using invariant feature vectors.

Aim of this project is to learn a metric which is invariant to the different transformations such as horizontal translation, vertical translation, global scale, rotation, line thickness, and shear and also illumination changes that might be applied on data. To do so, the first idea is taking the advantage of the Projection metric on the Grassmann manifolds.