Stefano Pellegrini

Text-guided image editing can have a transformative impact in supporting creative applications. A key challenge is to generate edits that are faithful to input text prompts, while consistent with input images. We present Imagen Editor, a cascaded diffusion model built, by fine-tuning Imagen [36] on text-guided image inpainting. Imagen Editor's edits are faithful to the text prompts, which is accomplished by using object detectors to propose inpainting masks during training. In addition, Imagen Editor captures fine details in the input image by conditioning the cascaded pipeline on the original high resolution image. To improve qualitative and quantitative evaluation, we introduce EditBench, a systematic benchmark for text-guided image inpainting. EditBench evaluates inpainting edits on natural and generated images exploring objects, attributes, and scenes. Through extensive human evaluation on EditBench, we find that object-masking during training leads to across-the-board improvements in text-image alignment – such that Imagen Editor is preferred over DALL-E 2 [31] and Stable Diffusion [33] – and, as a cohort, these models are better at object-rendering than text-rendering, and handle material/color/size attributes better than count/shape attributes.

The ability of detecting human postures is particularly important in several fields like ambient intelligence, surveillance, elderly care, and human-machine interaction. This problem has been studied in recent years in the computer vision community, but the proposed solutions still suffer from some limitations due to the difficulty of dealing with complex scenes (e.g., occlusions, different view points, etc.). In this article, we present a system for posture tracking and classification based on a stereo vision sensor. The system provides both a robust way to segment and track people in the scene and 3D information about tracked people. The proposed method is based on matching 3D data with a 3D human body model. Relevant points in the model are then tracked over time with temporal filters and a classification method based on hidden Markov models is used to recognize principal postures. Experimental results show the effectiveness of the system in determining human postures with different orientations of the people with respect to the stereo sensor, in presence of partial occlusions and under different environmental conditions.

This paper addresses the use of social behavior models for the prediction of a pedestrian's future motion. Recently, such models have been shown to outperform simple constant velocity models in cases where data association becomes ambiguous, e.g. in case of occlusion, bad image quality, or low frame rates. However, to account for the multiple alternatives a pedestrian can choose from, one has to go beyond the currently available deterministic models. To this end, we propose a stochastic extension of a recently proposed simulation-based motion model. This new instantiation can cater for the possible behaviors in an entire scene in a multi-hypothesis approach, using a principled modeling of uncertainties. In a set of experiments for prediction and template-based tracking, we compare it to a deterministic instantiation and investigate the general value of using an advanced motion prior in tracking.