RINGS: Resilient Edge Networks with Data-driven Model-based Learning

Description

Next-generation cellular wireless edge networks with embedded computational capabilities and services have become a critical infrastructure that enables secure, robust, and high-performance applications in many domains, including education, business, transportation, healthcare, and entertainment. Meanwhile, the availability, reliability, and resiliency of edge networks are constantly being challenged by both the expected resource and demand variations, e.g., diurnal application traffic patterns, user mobility, and random link/node failures, as well as the unexpected level-shifts of operating conditions, e.g., traffic flash-crowds triggered by emerging events, major infrastructure failures after coordinated malicious attacks and natural disasters.

The project is developing novel hybrid learning solutions for autonomous and resilient wireless edge networks from the angle of joint provisioning, allocation, and scheduling of communication and computation resources. It includes several research thrusts: 1) To achieve high efficiency and robustness in the face of expected demand/resource variations, the project team is investigating robust communication and computation resource provisioning at long time scales. At short time scales, the team is studying adaptive routing under the Model Predictive Control framework coupled with Adaptive Dynamic Programming (ADP). The researchers are also investigating adaptive scheduling of virtual middle-boxes using a domain-knowledge enriched reinforcement learning framework. 2) To recover from major disruptions, the project team is studying how to progressively bring network services back by strategically utilizing backup and external resources. Resilient progressive recovery is achieved through joint provisioning, routing, and scheduling based on hybrid online learning. The stability of the recovery process is analyzed under the Robust ADP framework. 3) Individual research components are being evaluated using mathematical analysis, trace-driven simulations, and experiments on the CloudLab testbed. The project is generating new theories and algorithms on hybrid learning that can be used to study complex systems in other domains. Valuable research opportunities are being created for graduate and undergraduate students, especially women and minority students.

Presentations

Project Overview

Publications

• "Online Restless Multi-Armed Bandits with Long-Term Fairness Constraints",
  Shufan Wang, Guojun Xiong and Jian Li,
  Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI), Vancouver, Canada, February 2024.
  paper
• Xiaotian Li and Yong Liu,
  “Fast Computation Flow Restoration with Path-based Two-stage Traffic Engineering”,
  ACM/IEEE Symposium on Edge Computing (SEC), Best Paper Shortlist, December, 2023,
  paper   
• "Finite-Time Analysis of Whittle Index based Q-Learning for Restless Multi-Armed Bandits with Neural Network Function Approximation",
  Guojun Xiong and Jian Li,
  Proceedings of the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), New Orleans, LA, December 2023.
  paper 

• U. Usubutun, M. Kodialam, T.V. Lakshman and S. Panwar,
  "Oblivious Routing Using Learning Methods”, 
  in IEEE Global Communications Conference (Globecom), December, 2023,
  paper, slides
• Tong Liu, Michelangelo Bin, Ivano Notarnicola, Thomas Parisini, Zhong-Ping Jiang,
   "Discrete-Time Distributed Optimization for Linear Uncertain Multi-Agent Systems", 
  submitted to the 62nd IEEE Conference on Decision and Control (CDC), Marina Bay Sands, Singapore, December 2023.

• "Reinforcement Learning for Dynamic Dimensioning of Cloud Caches: A Restless Bandit Approach",
  Guojun Xiong, Shufan Wang, Gang Yan and Jian LI, 
  IEEE/ACM Transactions on Networking (IEEE/ACM TON), vol. 31, no. 5, pp. 2147-2161, October 2023,
  paper

•  L. Cui, T. Basar and Z. P. Jiang,
  "A reinforcement learning look at risk-sensitive linear quadratic Gaussian control",
  Proceedings of Machine Learning Research, 2023, to appear.
• L. Cui, B. Pang and Z. P. Jiang,
  "Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approach", 
  Accepted for publication in IEEE Transactions on Automatic Control, 2023    
• "Learning Infinite-Horizon Average-Reward Restless Multi-Action Bandits via Index Awareness",
  Guojun Xiong, Shufan Wang and Jian Li,
  Proceedings of the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), New Orleans, LA, November 2022,
  paper 

• "Index-Aware Reinforcement Learning for Adaptive Video Streaming at the Wireless Edge",
  Guojun Xiong, Xudong Qin, Bin Li, Rahul Singh and Jian Li,
  Proceedings of ACM Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing (ACM MobiHoc), Seoul, South Korea, October 2022,
  paper

• Lifan Mei, Jinrui Gou, Jingrui Yang, Yujin Cai, and Yong Liu,
  "On Routing Optimization in Networks with Embedded Computational Services",
  arXiv:2210.03338

People

• Yong Liu (Principal Investigator, NYU)
  
• Shivendra Panwar (Co-Principal Investigator, NYU)
  
• Pei Liu (Co-Principal Investigator, NYU)
  
• Zhong-Ping Jiang (Co-Principal Investigator, NYU)
  
• Jian Li (Co-Principal Investigator, SUNY Binghamton)

Collaborators

Funding
      
This project is funded by the USA National Science Foundation as part of the RINGS program, under contract Award # 2148309