Evolving Domain Generalization via Latent Structure-Aware Sequential Autoencoder

Domain generalization (DG) refers to the problem of generalizing machine learning systems to out-of-distribution (OOD) data with knowledge learned from several provided source domains. Most prior works confine themselves to stationary and discrete environments to tackle such generalization issue arising from OOD data. However, in practice, many tasks in non-stationary environments (e.g., autonomous-driving car system, sensor measurement) involve more complex and continuously evolving domain drift, emerging new challenges for model deployment. In this paper, we first formulate this setting as the problem of evolving domain generalization. To deal with the continuously changing domains, we propose MMD-LSAE, a novel framework that learns to capture the evolving patterns among domains for better generalization. Specifically, MMD-LSAE characterizes OOD data in non-stationary environments with two types of distribution shifts: covariate shift and concept shift, and employs deep autoencoder modules to infer their dynamics in latent space separately. In these modules, the inferred posterior distributions of latent codes are optimized to align with their corresponding prior distributions via minimizing maximum mean discrepancy (MMD). We theoretically verify that MMD-LSAE has the inherent capability to implicitly facilitate mutual information maximization, which can promote superior representation learning and improved generalization of the model. Furthermore, the experimental results on both synthetic and real-world datasets show that our proposed approach can consistently achieve favorable performance based on the evolving domain generalization setting.