Self-Evolving Neural Networks A Meta-Learning Framework for Autonomous Architecture Optimization

Abstract

The rapid advancement of deep learning has led to increasingly complex neural network architectures, often requiring substantial human expertise, iterative experimentation, and domain-specific knowledge to achieve optimal performance. Traditional architecture design approaches, including manual tuning and grid- or random-search-based hyperparameter optimization, are computationally expensive and do not scale well with the increasing diversity of tasks and datasets. To address these limitations, this research introduces Self-Evolving Neural Networks (SENN), a meta-learning framework designed for autonomous architecture optimization that enables neural networks to self-improve, adapt, and evolve with minimal human intervention. SENN integrates principles of meta-learning, evolutionary computation, reinforcement learning, and differentiable neural architecture search (NAS) to create a unified, self-evolving system capable of discovering optimal architectures dynamically.

 The proposed framework operates at two hierarchical levels. At the meta-level, the system learns how to generate, mutate, and refine neural architectures through a policy-driven controller trained using reinforcement learning. This controller optimizes architecture configurations by continuously interacting with evaluation environments, receiving performance feedback, and updating its search strategies. At the base-level, candidate architectures—composed of layers, activation functions, connectivity patterns, and hyperparameters—are instantiated and trained on downstream tasks. Their performance metrics are collected to inform the meta-learner, enabling iterative self-evolution. SENN’s meta-learning component encodes knowledge about successful architectural patterns, learning to generalize across tasks and datasets, thereby reducing search redundancy and improving exploration efficiency.