Flamenco - ANR Project
Modelling of Dynamic scene
About / Introduction
What is the Flamenco Project?
Flamenco is a 3-year scientific project (2007-2010) granted by the (French) National Agency for Research (ANR) [Ref. ANR-06-MDCA-007].
Partners are the CERTIS Lab. at Ecole Nationale des Ponts et Chaussees and the Perception Lab. at INRIA.
The project coordinators are Emmanuel Prados (INRIA Rhone-Alpes) and Jean-Philippe Pons (ENPC).
Flamenco Project enjoys two PhD grants of a duration of 3 years and one post-doctoral grant of a duration of 2 years. The total grant amounts to 340 000 Euros.
Scientific Context and Motivation of the Project :
The generalization of digital cameras, both in the public environment and at home, creates new opportunities
and new needs in visualization and in communication. The creation of three-dimensional
dynamic models of our environment from these omnipresent and inexpensive sensors will become
ubiquitous in the next decade. These techniques will constitute a major tool for building multimodal
virtual worlds in augmented reality.
This proposal deals with the challenges of spatio-temporal scene reconstruction from several
video sequences, i.e. from images captured from different viewpoints and at different time instants.
This major problem in computer vision has been limited by three important factors so far:
For this project, the CERTIS laboratory (Ecole Nationale des Ponts et Chaussees) and the
PERCEPTION group (INRIA Rhone-Alpes) will collaborate. These two teams are both expert in
3D reconstruction, motion representation and estimation, but they have pursued quite different
goals so far. While the PERCEPTION group has mainly focused on real-time techniques for coarse
reconstruction, the CERTIS laboratory has focused on high-resolution reconstruction techniques
with a high computational cost.
The collaboration between CERTIS and PERCEPTION in the FLAMENCO project will allow to
take the best of each world, and to design novel algorithms for spatio-temporal scene modeling
with the smallest possible trade-off between accuracy and efficiency. We will validate our results
on newly acquired experimental datasets that we will distribute freely.
the computational time / the poor resolution of the models: the acquisition of video sequences from multiple cameras generates a very large amount of data, which makes the design of efficient algorithms very important. The high computational cost of existing methods has limited the spatial resolution of the reconstruction and has allowed to handle video sequences of a few seconds only, which is prohibitive in real applications.
the lack of spatio-temporal coherence: to our knowledge, none of the existing methods has been able to reconstruct coherent spatio-temporal models: Most methods build threedimensional models at each time step without taking advantage of the continuity of the motion and of the temporal coherence of the model. This issue requires elaborating new mathematical and algorithmic tools dedicated to four-dimensional representations (three space dimensions plus the time dimension).
the simplicity of the models: the information available in multiple video sequences of a scene are not restricted to geometry and motion. Most reconstruction methods disregard such information as the illumination of the scene, and the reflectance, the materials and the textures of the objects. Our goal is to build more exhaustive models, by automatically estimating these parameters concurrently to geometry and motion. For example, in augmented reality, reflectance properties allow to synthesize novel views with higher photo-realism.
Expected Scientific and Technical Contributions :
The collaboration between the CERTIS laboratory and the PERCEPTION group in the FLAMENCO projet is likely to bring many significantly methodological and algorithmic contributions in the fields of computer vision, computer graphics and computational geometry. We expect to advance the state of the art in the difficult problem of spatio-temporal multi-view stereo reconstruction in three directions: towards a more consistent spatio-temporal scene representation, towards a sustainable computation time, and towards the integration of appearance and radiosity attributes. Practically, we expect five to ten publications in the major computer vision and computer graphics conferences (ICCV, CVPR, ECCV, 3DPVT, SIGGRAPH, SGP). We will distribute the experimental data acquired in the GRIMAGE platform, in order to encourage other research groups in the world to tackle the problem of dynamic 3D modeling. In order to popularize our methodologies, we also plan to distribute some software components developed in the FLAMENCO project under free sotware licenses.