M2TR: Multi-modal Multi-scale Transformers for Deepfake DetectionThe widespread dissemination of Deepfakes demands effective approaches that can detect perceptually convincing forged images. In this paper, we aim to capture the subtle manipulation artifacts at different scales using transformer models. In particular, we introduce a Multi-modal Multi-scale TRansformer (M2TR), which operates on patches of different sizes to detect local inconsistencies in images at different spatial levels. M2TR further learns to detect forgery artifacts in the frequency domain to complement RGB information through a carefully designed cross modality fusion block. In addition, to stimulate Deepfake detection research, we introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods. We conduct extensive experiments to verify the effectiveness of the proposed method, which outperforms state-of-the-art Deepfake detection methods by clear margins.
Nanotechnology-enabled sonodynamic therapy against malignant tumorsYunxi Huang, Wenhao Ouyang, Zijia Lai et al.|Nanoscale Advances|2024 Sonodynamic therapy (SDT) is an emerging approach for malignant tumor treatment, offering high precision, deep tissue penetration, and minimal side effects. The rapid advancements in nanotechnology, particularly in cancer treatment, have enhanced the efficacy and targeting specificity of SDT. Combining sonodynamic therapy with nanotechnology offers a promising direction for future cancer treatments. In this review, we first systematically discussed the anti-tumor mechanism of SDT and then summarized the common nanotechnology-related sonosensitizers and their recent applications. Subsequently, nanotechnology-related therapies derived using the SDT mechanism were elaborated. Finally, the role of nanomaterials in SDT combined therapy was also introduced.
A Prognostic Risk Score Based on Hypoxia-, Immunity-, and Epithelialto-Mesenchymal Transition-Related Genes for the Prognosis and Immunotherapy Response of Lung AdenocarcinomaWenhao Ouyang, Yupeng Jiang, Shiyi Bu et al.|Frontiers in Cell and Developmental Biology|2022 Background: Lung adenocarcinoma (LUAD), the most common subtype of non-small cell lung cancer (NSCLC), is associated with poor prognosis. However, current stage-based clinical methods are insufficient for survival prediction and decision-making. This study aimed to establish a novel model for evaluating the risk of LUAD based on hypoxia, immunity, and epithelial-mesenchymal transition (EMT) gene signatures. Methods: In this study, we used data from TCGA-LUAD for the training cohort and GSE68465 and GSE72094 for the validation cohorts. Immunotherapy datasets GSE135222, GSE126044, and IMvigor210 were obtained from a previous study. Using bioinformatic and machine algorithms, we established a risk model based on hypoxia, immune, and EMT gene signatures, which was then used to divide patients into the high and low risk groups. We analyzed differences in enriched pathways between the two groups, following which we investigated whether the risk score was correlated with stemness scores, genes related to m 6 A, m 5 C, m 1 A and m 7 G modification, the immune microenvironment, immunotherapy response, and multiple anti-cancer drug sensitivity. Results: Overall survival differed significantly between the high-risk and low-risk groups (HR = 4.26). The AUCs for predicting 1-, 3-, and 5-year survival were 0.763, 0.766, and 0.728, respectively. In the GSE68465 dataset, the HR was 2.03, while the AUCs for predicting 1-, 3-, and 5-year survival were 0.69, 0.651, and 0.618, respectively. The corresponding values in the GSE72094 dataset were an HR of 2.36 and AUCs of 0.653, 0.662, and 0.749, respectively. The risk score model could independently predict OS in patients with LUAD, and highly correlated with stemness scores and numerous m 6 A, m 5 C, m 1 A and m 7 G modification-related genes. Furthermore, the risk model was significantly correlated with multiple immune microenvironment characteristics. In the GSE135222 dataset, the HR was 4.26 and the AUC was 0.702. Evaluation of the GSE126044 and IMvigor210 cohorts indicated that PD-1/PD-LI inhibitor treatment may be indicated in patients with low risk scores, while anti-cancer therapy with various drugs may be indicated in patients with high risk scores. Conclusion: Our novel risk model developed based on hypoxia, immune, and EMT gene signatures can aid in predicting clinical prognosis and guiding treatment in patients with LUAD.
Artificial intelligence-based multi-modal multi-tasks analysis reveals tumor molecular heterogeneity, predicts preoperative lymph node metastasis and prognosis in papillary thyroid carcinoma: a retrospective studyYunfang Yu, Wenhao Ouyang, Yunxi Huang et al.|International Journal of Surgery|2024 BACKGROUND: Papillary thyroid carcinoma (PTC) is the predominant form of thyroid cancer globally, especially when lymph node metastasis (LNM) occurs. Molecular heterogeneity, driven by genetic alterations and tumor microenvironment components, contributes to the complexity of PTC. Understanding these complexities is essential for precise risk stratification and therapeutic decisions. METHODS: This study involved a comprehensive analysis of 521 patients with PTC from our hospital and 499 patients from The Cancer Genome Atlas (TCGA). The real-world cohort 1 comprised 256 patients with stage I-III PTC. Tissues from 252 patients were analyzed by DNA-based next-generation sequencing, and tissues from four patients were analyzed by single-cell RNA sequencing (scRNA-seq). Additionally, 586 PTC pathological sections were collected from TCGA, and 275 PTC pathological sections were collected from the real-world cohort 2. A deep-learning multi-modal model was developed using matched histopathology images, genomic, transcriptomic, and immune cell data to predict LNM and disease-free survival (DFS). RESULTS: This study included a total of 1011 PTC patients, comprising 256 patients from cohort 1, 275 patients from cohort 2, and 499 patients from TCGA. In cohort 1, the authors categorized PTC into four molecular subtypes based on BRAF, RAS, RET, and other mutations. BRAF mutations were significantly associated with LNM and impacted DFS. ScRNA-seq identified distinct T-cell subtypes and reduced B-cell diversity in BRAF-mutated PTC with LNM. The study also explored cancer-associated fibroblasts and macrophages, highlighting their associations with LNM. The deep-learning model was trained using 405 pathology slides and RNA sequences from 328 PTC patients and validated with 181 slides and RNA sequences from 140 PTC patients in the TCGA cohort. It achieved high accuracy, with an area under the curve (AUC) of 0.86 in the training cohort, 0.84 in the validation cohort, and 0.83 in the real-world cohort 2. High-risk patients in the training cohort had significantly lower DFS rates ( P <0.001). Model AUCs were 0.91 at 1 year, 0.93 at 3 years, and 0.87 at 5 years. In the validation cohort, high-risk patients also had lower DFS ( P <0.001); the AUCs were 0.89, 0.87, and 0.80 at 1, 3, and 5 years. The authors utilized the GradCAM algorithm to generate heatmaps from pathology-based deep-learning models, which visually highlighted high-risk tumor areas in PTC patients. This enhanced clinicians' understanding of the model's predictions and improved diagnostic accuracy, especially in cases with lymph node metastasis. CONCLUSION: The artificial intelligence (AI)-based analysis uncovered vital insights into PTC molecular heterogeneity, emphasizing BRAF mutations' impact. The integrated deep-learning model shows promise in predicting metastasis, offering valuable contributions to improved diagnostic and therapeutic strategies.
Unraveling the role of M1 macrophage and CXCL9 in predicting immune checkpoint inhibitor efficacy through multicohort analysis and single‐cell RNA sequencingThe exact function of M1 macrophages and CXCL9 in forecasting the effectiveness of immune checkpoint inhibitors (ICIs) is still not thoroughly investigated. We investigated the potential of M1 macrophage and C-X-C Motif Chemokine Ligand 9 (CXCL9) as predictive markers for ICI efficacy, employing a comprehensive approach integrating multicohort analysis and single-cell RNA sequencing. A significant correlation between high M1 macrophage and improved overall survival (OS) and objective response rate (ORR) was found. M1 macrophage expression was most pronounced in the immune-inflamed phenotype, aligning with increased expression of immune checkpoints. Furthermore, CXCL9 was identified as a key marker gene that positively correlated with M1 macrophage and response to ICIs, while also exhibiting associations with immune-related pathways and immune cell infiltration. Additionally, through exploring RNA epigenetic modifications, we identified Apolipoprotein B MRNA Editing Enzyme Catalytic Subunit 3G (APOBEC3G) as linked to ICI response, with high expression correlating with improved OS and immune-related pathways. Moreover, a novel model based on M1 macrophage, CXCL9, and APOBEC3G-related genes was developed using multi-level attention graph neural network, which showed promising predictive ability for ORR. This study illuminates the pivotal contributions of M1 macrophages and CXCL9 in shaping an immune-active microenvironment, correlating with enhanced ICI efficacy. The combination of M1 macrophage, CXCL9, and APOBEC3G provides a novel model for predicting clinical outcomes of ICI therapy, facilitating personalized immunotherapy.