Network Working Group Ning So (UTD)
Internet Draft Young Lee (Huawei)
Intended status: Informational Dave McDysan (Verizon)
Greg Bernstein (Grotto)
Tae Yeon Kim (ETRI)
Kohei Shiomoto (NTT)
Oscar Gonzalez de Dios (Telefonica)
April 20, 2011
Problem Statement for Network Aware Application Resource Assignment
and Mobility (NA-ARAM) in Data Center Environments
draft-so-network-aware-application-problem-02.txt
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on October 20, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
So & Lee, et al. Expires October 20, 2011 [Page 1]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document.
Abstract
Data Centers offer various application services to end-users such as
video gaming, cloud computing and others. Since the data centers used
to provide application services are distributed geographically around
a network, many decisions made in the control and management of
application services, such as where to instantiate another service
instance or to which server a new client is assigned, should be made
with the knowledge of the underlying network resources availability
and the application resources conditions.
Currently such decisions are made with very little or no information
concerning the underlying network used to deliver those services.
Hence such decisions are inherently sub-optimal. In addition the lack
of network awareness may result in not meeting the end-user
application service objectives.
Table of Contents
1. Introduction........................................ 3
1.1. Terminology..................................... 3
2. Network and Application Contexts........................ 4
2.1. Server level load condition........................ 6
2.2. Intra-DC network condition......................... 6
2.3. Carrier MAN/WAN network condition (the networks that connect
the data centers and connect end users to the data centers).. 6
2.4. User condition................................... 7
3. Problem Statement .................................... 7
4. High-level requirements................................ 8
4.1. End-User to Application/DC Provider Communication...... 9
4.2. Inter DC communication............................ 9
4.3. Data Center-Network Stratum Communication (NS Query).... 9
4.3.1. Application Profile.......................... 10
4.3.2. Network Load Data to be queried................ 10
4.3.3. Responses to NS Query from network to application. 11
5. Security Considerations............................... 11
6. IANA Considerations.................................. 11
7. References......................................... 11
7.1. Informative References........................... 11
Author's Addresses..................................... 13
So & Lee Expires October 20, 2011[Page 2]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
Intellectual Property Statement .......................... 13
Disclaimer of Validity.................................. 14
1. Introduction
Data Centers offer application services to end-users such as video
gaming [WoWAct], [WoWHrs], [GameServ] and [GroupGame], cloud
computing [CostCloud], grid application [GFD-122] and others. Many
application services offered by Data Center to end-users make
significant use of the underlying networks resources in the form of
bandwidth consumption used to carry the actual traffic between a data
center and the end-users.
It is a common practice for the same application stratum service to
be offered by multiple geographically dispersed data centers. One of
the major drivers for operating multiple Data Centers is allowing the
application to be closer to the end-users, so that the overall
service performance and the user experience can be enhanced.
As the application servers are distributed geographically across many
Data Centers for various reasons (e.g., load balancing), the decision
as which server to select for an application request from end-users
can negatively affect the quality of experience (QoE) of the users if
not done correctly.
In addition, the internal infrastructure supporting the applications
(e.g., virtual machines) may be actively distributed within and/or
between data centers. This practice can improve the utilization of
the servers, and prevent service performance degradation during the
peak usage time period, and during equipment failures. When
instantiating new virtual machines or migrating existing virtual
machines across data centers, the underlying network loading
conditions within a data center (LAN) or between data centers
(MAN/WAN) need to be considered in the decision making process.
This document describes problems encountered by distributed
applications and provides the motivation for the coordinated resource
allocation and management across application stratum and network
stratum.
1.1. Terminology
This section describes key terminology used in this document.
Application Stratum -- The functional block which manages and
controls application resources and provides application resources to
a variety of clients/end-users.
So & Lee Expires October 20, 2011[Page 3]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
Application Profile -- The characteristics of the application from a
network traffic perspective and the QoS requirements that the
application service will require from the network.
Application Resources -- Non-network resources critical to achieving
the application service functionality. Examples include: caches,
mirrors, application specific servers, content, large data sets, and
computing power.
Application Service -- A networked application offered to a variety
of clients.
Application Controller - The central controller that makes assignment
the decision as which server to select for an application request
from end-users. This is also known as global load balancer.
Network Stratum -- The functional block which manages and controls
network resources and provides transport of data between clients/end-
users and application sources.
Network Resources -- Resources of any layer 3 or below network such
as bandwidth, links, paths, path processing (creation, deletion, and
management), network databases, path computation, admission control,
resource reservation, etc.
2. Network and Application Contexts
This section discusses the network and application contexts and the
key components that will help understand problems that will be
discussed in the subsequent section. Figure 1 below depicts
overarching network and application architecture in which to show the
key components.
So & Lee Expires October 20, 2011[Page 4]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
,-----. ---------------
---------- / App \ | DC 1 |
| End-user |. . .>( Control ) | o o o |
| | \ / | \|/ |
---------- `-----' | O |
| ----- --|------
| |
| |
| --------------------------|--
| / PE1 | \
| / ...................O \ --------------
| | . | | o o o DC 2 |
| | PE4 . PE2 | | \|/ |
----|---O.........................O---|---|---O |
| . | | |
| . PE3 | --------------
\ ..........O Carrier /
\ | Network /
---------------|-------------
|
--------|------
| O |
| /|\ |
| o o o |
| DC 3 |
---------------
Figure 1. Data Center Architecture
Figure 1 shows four distinct entities:
. End-users
. Application Controller (Global Load Balancer)
. Data Center Networks
. Carrier Network (access network is not shown for brevity's sake)
There are a number of factors that need be considered in choosing the
right server in the right data center for an application request or
in instantiating new VMs/applications or migrating existing
VMs/applications:
. Server level load condition in a data center
So & Lee Expires October 20, 2011[Page 5]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
. Intra data center network condition
. Carrier MAN/WAN network condition
. User Condition
Details of potential constraining factors for each of the list above
are as follows:
2.1. Server level load condition
o Virtual Machine (VM) supportability limitations
o CPU utilization;
o Memory segmentation and consumption;
o Application limitations such as max number of simultaneous
instances of the application supportability;
o Storage access speed (disk, RAM, etc.);
o Environment considerations such as server temperature,
power load, and electrical cost at the time;
o Operational and managerial considerations such as location
of peer servers and storages.
o VM to NIC switching in a virtual switch
2.2. Intra-DC network condition
o Server NIC to Top of Rack (ToR) Switch;
o TOR switch to Layer 2 Switch - link and node level;
o Between L2 Switches and L2 switch to L2 core/gateway
switch/router - link and node level;
o L2 gateway router to provider edge (PE) router - link and
node level.
2.3. Carrier MAN/WAN network condition (the networks that connect the
data centers and connect end users to the data centers)
o Type of networks and the technical capabilities of the
networks;
o Bandwidth capabilities and availability;
o latency;
o jitter;
o packet loss;
So & Lee Expires October 20, 2011[Page 6]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
o And other Network Performance Objective (NPO) as defined in
section 5 of [ITU-T Y.1541].
2.4. User condition
o User access capabilities and limitations (e.g., user
terminal information such as codec for video application);
o User location;
o Optional user preferences (for some application, user may
be able to specify its preferences. For example, the
preferred server location for gaming).
It is worth to note that the service performance objectives of many
new/emerging applications are real-time and interactive applications
that require bandwidth guarantee and/or dynamic bandwidth
allocation/modification with a strict latency requirement. Such
application examples are real-time video distribution services,
conferencing and gaming, grid computing, and etc.
The need for the coordinated resource allocation and management in
all the factors including server level load condition, intra DC
network condition, the underlying carrier network condition, and the
user condition is paramount in order to meet increasingly
sophisticated end-user service needs.
3. Problem Statement
In the current Intra-Data Center network, the server selection for an
application/VM is done by load-balancer. The load balancer is aware
of a certain level of server usage data (e.g., the number
simultaneous instances of the application usage) and distributes the
application requests based on that data.
However, the current load balancing technology is insufficient in
providing an optimal decision across multiple VLANs and multiple Data
Centers. This capability is often referred to as global load
balancing.
First of all, there is no standard solution for the communication
exchange among load balancers located in different Data Centers. This
implies that load balancers from different vendors cannot communicate
to each other.
So & Lee Expires October 20, 2011[Page 7]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
Secondly, load balancers know little about the underlying network
conditions listed in the previous section. Nor is it user condition
aware.
In some cases, application controllers can estimate network load
based on ping latency, and network topology based on trace routes in
the Internet, based on the assumption that the underlying transport
network is an IP network, and the routing is based on simple IP
forwarding.
In reality, the carrier's routing schemes are likely to include IP
tunneling or MPLS tunneling on top of or in place of IP forwarding.
In some cases, the actual network may be VPN, MPLS-TE or GMPLS-TE
networks where trace route does not work.
This implies that network status estimation technique made from
application stratum may have limited accuracy. Thus, application
resource allocation to end-users can suffer sub-optimality and fail
to meet performance objective for the application.
When migrating existing VMs/applications from one data center to
another, the underlying network load condition in LAN/MAN/WAN can be
constraining factors. Migration of VMs/applications, for instance,
typically requires a high-speed data transfer across LAN/MAN/WAN to
minimize service impact. Application controllers responsible for this
operation is not aware of LAN/MAN/WAN network conditions.
Another issue is that there is no standard way for an application
control gateway in Data Center to ask for network stratum in a way
suitable for a third party, i.e., an entity "outside the network".
Even if the network load information is available, few networks would
want to reveal the details of the information, such as topology, link
bandwidth availability, latency, packet loss, etc, to the outside
entity. This warrants some works on abstraction from network side to
preserve the privacy of network stratum details from the application
stratum.
This capability is referred to as Network Stratum Query (NS Query) in
this document.
4. High-level requirements
This section discusses high-level requirements to support network-
aware application resource assignment and mobility in the data center
environments.
So & Lee Expires October 20, 2011[Page 8]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
4.1. End-User to Application/DC Provider Communication
End-users are the entities that make requests to the Application
Controller (aka, global load balancer, or front-end server). As
Figure 1 shows, the Application Controller interfaces users and makes
server assignment decision for the user application. Once resources
are allocated or made available, they consume application resources
offered by application/Data Center provider.
End-users may communicate several things to the Application
Controller:
. Required application: this may be a simple URL request;
. Specific server identity or location. This is applicable to
gaming where gamers can access game server information and can
specify the user's preference for server location.
. Additional security requirements;
. Modified application specs. For example, a mobile user may
request a video download with reduced resolution to conserve
battery.
. Optional end-user terminal information, e.g., codec for video
application.
4.2. Inter DC communication
A couple of new communication protocols are required in order for the
optimal network-aware global load balancing to be achieved.
. Server/Application Status needs to be made available to the
Application Controller (Global Load Balancer) from each of the
Data Centers where the application servers are located (See
section 2.1 for details); and
. Intra-DC network conditions defined in Section 2.2 need to be
made available to the Application Controller (Global Load
Balancer) from each of the Data Centers where the application
servers are located.
4.3. Data Center-Network Stratum Communication (NS Query)
The Application Controller plays the key role in choosing the optimal
server for an application request. The Application Controller can
So & Lee Expires October 20, 2011[Page 9]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
interface with an application gateway that interfaces with network
and runs NS Query. The details of NS Query are addressed in a
separate draft [NS Query].
4.3.1. Application Profile
The application Stratum needs to provide the application profile to
network.
Example service profile information that can be useful to network to
understand is as follows:
. End user IP address;
. User access router IP address;
. Authentication Profile: Authentication Key;
. Bandwidth Profile: Minimum bandwidth required for the
application;
. Connectivity Profile: P-P, P-MP, Anycast (Multi-destination);
. Directionality of the connectivity: unidirectional, bi-
directional;
. Path Estimation Objective Function: Min latency, etc.
Additional profile information can be added depending on the network
capability.
4.3.2. Network Load Data to be queried
The query should be able to ask all the network condition information
listed in Section 1.
Note that this can be asked in a different way. For example, the
query can simply ask:
. Can you route x amount of b/w (in a particular point in network)
within y ms of latency?
. Can you route x amount of b/w (in a particular point in network)
with no packet loss?
So & Lee Expires October 20, 2011[Page 10]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
4.3.3. Responses to NS Query from network to application
Upon receiving the network query from application, the network should
be able to perform the following functions:
. Using the given location mapping information (e.g., the server
location and the end-user location or a set of server locations
in different data centers), the network should be able to derive
its network condition data.
Note: How to estimate and generate the network condition data by the
network is beyond the scope of this draft, and should be addressed in
other pertinent WGs.
. The network should be able to formulate the abstraction of the
requested network condition data in response to the NS Query
request from application.
5. Security Considerations
TBD
6. IANA Considerations
This informational document does not make any requests for IANA
action.
7. References
7.1. Informative References
[RFC2261] D. Harrington, et al., "An Architecture for Describing SNMP
Management Frameworks," January, 1998.
[RFC2265] B. Wijnen, et al., "View-based Access Control Model (VACM)
for the Simple Network Management Protocol (SNMP),"
January, 1998.
[Y.2011] General principles and general reference model for Next
Generation Networks, October, 2004.
[Y.2012] Functional Requirements and architecture of the NGN, April,
2010.
So & Lee Expires October 20, 2011[Page 11]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
[GameServ]P. Quax, J. Dierckx, B. Cornelissen, G. Vansichem, and W.
Lamotte, "Dynamic server allocation in a real-life
deployable communications architecture for networked
games," Proceedings of the 7th ACM SIGCOMM Workshop on
Network and System Support for Games, Worcester,
Massachusetts: ACM, 2008, pp. 66-71.
[GroupGame] K. Vik, C. Griwodz, and P. Halvorsen, "Applicability of
group communication for increased scalability in MMOGs,"
Proceedings of 5th ACM SIGCOMM workshop on Network and
system support for games, Singapore: ACM, 2006, p. 2.
[WoWHrs] P. Tarng, K. Chen, and P. Huang, "An analysis of WoW
players' game hours," Proceedings of the 7th ACM SIGCOMM
Workshop on Network and System Support for Games,
Worcester, Massachusetts: ACM, 2008, pp. 47-52.
[WoWAct] M. Suznjevic, M. Matijasevic, and O. Dobrijevic, "Action
specific Massive Multiplayer Online Role Playing Games
traffic analysis: case study of World of Warcraft,"
Proceedings of the 7th ACM SIGCOMM Workshop on Network and
System Support for Games, Worcester, Massachusetts: ACM,
2008, pp. 106-107.
[GFD-122] Tiziana Ferrari (editor), "Grid Network services Use Cases
from the e-Science Community", GFD-I-122, Open Grid Forum,
December 12, 2007.
[CostCloud] A. Greenberg, J. Hamilton, D. Maltz, and P. Patel, "The
cost of a cloud: research problems in data center
networks," ACM SIGCOMM, Vol. 39, Number1, January 2009.
[NS Query] Y. Lee, et. al., "Problem Statement for Network Stratum
Query," draft-lee-network-stratum-query-problem, work in
progress.
[Y.1541] Network performance objectives for IP-based services,
February, 2002.
So & Lee Expires October 20, 2011[Page 12]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
Author's Addresses
Ning So (Editor)
Univerity of Texas at Dallas
Email: [email protected]
Young Lee (Editor)
Huawei Technologies
1700 Alma Drive, Suite 500
Plano, TX 75075
USA
Phone: (972) 509-5599
Email: [email protected]
Dave McDysan
Verizon Business
Email: [email protected]
Greg M. Bernstein
Grotto Networking
Fremont California, USA
Phone: (510) 573-2237
Email: [email protected]
Tae Yeon Kim
ETRI
[email protected]
Kohei Shiomoto
NTT
Email : [email protected]
Oscar Gonzalez de Dios
Telefonica
Email : [email protected]
Intellectual Property Statement
The IETF Trust takes no position regarding the validity or scope of
any Intellectual Property Rights or other rights that might be
claimed to pertain to the implementation or use of the technology
described in any IETF Document or the extent to which any license
under such rights might or might not be available; nor does it
So & Lee Expires October 20, 2011[Page 13]
Internet-Draft Network Aware Application Resource Assignment & Mobility
April 2011
represent that it has made any independent effort to identify any
such rights.
Copies of Intellectual Property disclosures made to the IETF
Secretariat and any assurances of licenses to be made available, or
the result of an attempt made to obtain a general license or
permission for the use of such proprietary rights by implementers or
users of this specification can be obtained from the IETF on-line IPR
repository at http://www.ietf.org/ipr
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
any standard or specification contained in an IETF Document. Please
address the information to the IETF at [email protected].
Disclaimer of Validity
All IETF Documents and the information contained therein are provided
on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE
IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL
WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY
WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE
ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
So & Lee Expires October 20, 2011[Page 14]