- this projects aims to simulate the data flow mechanism, explore performance + bottleneck in Infiniband network specifically for distributed machine-learning systems in data center using OMNeT++ (based on OMNeT++ Infiniband open source)
- The details of the network simulation model are as below:
- Networking Protocol:
Infiniband - Data Transmission Principle:
Credit-based Flow Control - Congestion Control Mechanism:
Congestion Control Table Indexing (BECN, FECN) - Virtual Lane Selection Algorithm:
Weighted Round Robin (WRR) - Data Transmission Framework:
Ring Allreduce - Data Granularity:
Flow Control Unit (FLIT)
- create AI training / computational tasks; initialize the relevant data information & size in every
flit, and forward it to CPU to assign the computing tasks among the nodes in the networking cluster
- the core component that manages the entire networking cluster
- break down the AI training tasks received from Task Generator ,and distributes it evenly among every nodes in the networking cluster
- generates the routing information for every
messages, and updates the relevant routing tables in switches
- responsible for generating computational tasks for GPU
- when it receives AI training tasks from Central Controller, it forwards them to GPU for computational training
- responsible for AI training + relevant computation
- when it receives the data required for AI training from CPU, it performs the required computaion (model training)
- whn the training has completed, it forwards the data to HCA and ready to be transmitted to the destination node
- a module that integrates App, Gen, Virtual Lane Arbiter, Output Buffer, Input Buffer & Sink
- similar to a
Network Interface Card (NIC), it encapsulates the trained data, and transmit them to the destination node through network link layer to update the AI model parameters
- a module that integrates Virtual Lane Arbiter, Output Buffer, Input Buffer & Packet Forwarder
- responsible for the information /
flitexchange between different nodes in the networking cluster
- component that connects the HCA and lower-level GPU, responsible for coordinating the information /
flitexchange between the 2 components - Forwards the
flitreceived from lower-level GPU to Generator - when HCA receives a complete
message, it notifies the lower-level GPU
- component in HCA that is responsible for the implementation of
Infinibandprotocol - breaks down the message received from App into finer
packets, flits, and analyze the main components inflitheaders - responsible for controlling the injection rate of the node, according to the selected congestion protocol, after receiving
Backward Explicit Congestion Notificaion (BECN)value
- responsible for the arbitration on
Virtual Lane (VL), and select the appropriate Virtual Lane for data transmission based on real-time conditions - temporarilt store the received data on a specific Virtual Lane, then check whether the node has enough credit to forward the
flit, if criterias are fulfilled check which Virtual Lane is suitable for data transmission (based onWeighted Round Robinalgorithm)
- a buffer that temporarily stored the
flitbefore they are transmitted to another node - updates the relevant credit values when the node successfuly processes some data
- when the destination node doesn't have enough buffers to receive the
flit, put theflitinside aqueue, else transmit the firstflitinside the queue
- a buffer that temporarily stores
flitafter receiving them - when it recieves
flit, encapsulates it into amessageby parsing the relevant content, and forward it to Output Buffer and Virtual Lane Arbiter to update the respective credits - if this component is being applied inside the Switch module, it triggers the Packet Forwarder component to determine the correct output port to transmit the data based on the destination node inside the
flitheader; if any congestion is detected, it triggers and set theForward Explicit Congestion Notification (FECN)field to be true
- component inside the server node that is responsible for collecting and processing the
flitreceived - responsible for checking whether the data arrives at the correct destination
- alert Input Buffer to update the relevant credit value everytime it successfully process a
packet - notifies and update Application after collecting a complete
message - if the
BECNorFECNfield of a receivedflitis TRUE, creates aCongestion Notification Packet (CNP)and forward it Generator to alleviate the congestion detected
- responsible for data or
flitrouting inside the Switch module - establish a routing table based on the routing information received from Central Controller
- determine the output port for data transmission based on the destination node field inside the
flitreceived