STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action RecognitionWe study the problem of human action recognition using motion capture (MoCap) sequences. Unlike existing techniques that take multiple manual steps to derive standard-ized skeleton representations as model input, we propose a novel Spatial-Temporal Mesh Transformer (STMT) to directly model the mesh sequences. The model uses a hierarchical transformer with intra-frame off-set attention and inter-frame self-attention. The attention mechanism allows the model to freely attend between any two vertex patches to learn nonlocal relationships in the spatial-temporal domain. Masked vertex modeling and future frame prediction are used as two self-supervised tasks to fully activate the bi-directional and auto-regressive attention in our hierarchical transformer. The proposed method achieves state-of-the-art performance compared to skeleton-based and point-cloud-based models on common MoCap benchmarks. Code is available at https://github.com/zgzxy001/STMT.
Weakly Supervised 3D Semantic Segmentation Using Cross-Image Consensus and Inter-Voxel Affinity RelationsX. L. Zhu, Jeffrey Chen, Xiangrui Zeng et al.|2021 IEEE/CVF International Conference on Computer Vision (ICCV)|2021 We propose a novel weakly supervised approach for 3D semantic segmentation on volumetric images. Unlike most existing methods that require voxel-wise densely labeled training data, our weakly-supervised CIVA-Net is the first model that only needs image-level class labels as guidance to learn accurate volumetric segmentation. Our model learns from cross-image co-occurrence for integral region generation, and explores inter-voxel affinity relations to predict segmentation with accurate boundaries. We empirically validate our model on both simulated and real cryo-ET datasets. Our experiments show that CIVA-Net achieves comparable performance to the state-of-the-art models trained with stronger supervision.
Floating in the air: forecasting allergenic pollen concentration for managing urban public healthX. L. Zhu, Xuanlong Ma, Zhengyang Zhang et al.|International Journal of Digital Earth|2024 The presence of airborne allergenic pollen causes a variety of immune reactions and respiratory diseases, threatening human life in severe cases. Climate change is exacerbating the allergenic pollen-induced health risks and adding a significant economic burden to societies. Despite the pressing threats, vital health-related information is not available to the public to date, and the reshaping of future geographic allergenic pollen patterns remains unknown. To help establish a critical allergenic pollen forecasting capacity, a systematic review was conducted and three promising future directions were identified: (1) resolving heterogeneous urban plant species distribution and phenology using fine-resolution satellite constellations; (2) acquiring ancillary information about allergenic pollen and patient symptoms from emerging geospatial big data, such as social media; (3) deciphering the coupled effect of climate change and urbanization on future geographic patterns and phenology of allergenic species. On this basis, we recommend an optimized workflow that combines real-time pollen monitoring networks with high-resolution vegetation information and weather forecast systems, comprehensively considering the production and diffusion process of pollen to establish advanced prediction models. By focusing on critical knowledge gaps, this review provides much needed insight to propel the allergenic pollen forecasting research and eventually benefit the management of urban public health.
Multi-source material image optimized selection based multi-option compositionHao Wu, Ding An, X. L. Zhu et al.|Image and Vision Computing|2021 Study of the Effects on the Strengthening Mechanism and Wear Behavior of Wear-Resistant Steel of Temperature Controlling in Heat TreatmentTo improve the wear resistance of the materials used for blades in engineering machinery, this study focused on the microstructural characteristics, mechanical properties, and wear behavior of HB500 grade wear-resistant steel developed using an optimized heat treatment system. To improve the temperature uniformity of the heat treatment furnace, the method of cyclic heating was used to heat the components. Carefully designing the quenching equipment, such as using a cross-shaped press, was employed to enhance the quenching effect and reduce the deformation of the steel plates. The crystal orientation analysis revealed a uniform and fine-grained microstructure, primarily characterized by plate-type tempered martensite, which indicated a good hardenability. The microstructure observations showed that the width of martensite is approximately 200 nm, with a significant presence of dislocations and carbides. Tensile tests and multi-temperature gradient impact tests indicated superior mechanical properties compared to similar grade wear-resistant steels, including a Rockwell hardness of 53, tensile strength of 1610 MPa, yield strength of 1404 MPa, and total elongation around 12.7%. The results of friction and wear experiments indicate that the wear rate decreases as the load increases from 100 N to 300 N, demonstrating an excellent wear resistance under a large load. Observations of the worn surfaces indicated that the wear mainly involved adhesive wear, fatigue wear, and oxidative wear. The properties' improvements were attributed to microstructure refinement and precipitation strengthening. This study indicates that designing a heat treatment system to control temperature uniformity and stability is feasible.