Don't miss the NFV mind map !

Well! I have done all the efforts and made concepts simple through the Free NFV Mind Map. Get it now before I take it off. Plus get free updates to my blog.

The Marriage of IP with OTN/DWDM !

When revenue per bit starts falling and cost per bit starts increasing, it’s time to think about alternative strategies to expand Transmission/IP network.

As IP is predominant traffic in Transmission networks, therefore any expansion strategy on Transmission should be driven by IP demands. Indeed, both IP and Transmission departments need to work for a joint and synergetic expansion strategy, in this regard.

Hence, I intend to share with you some insights and tips that will enable you to plan the expansion of IP and Transmission network in a holistic and synergetic way. If you stay till the end, you will see how IP can live with lower layers of Transmission (DWDM, OTN, SDH, and L2) through a successful marriage.

Whether you belong to IP department (responsible for the IP/MPLS domain) or Transmission department (responsible to run SDH, DWDM, OTN and most recently Ethernet), there is some food for thought for you in this article. More specifically, I will show you how to put your traffic on the right layer that will reduce the TCO (Total Cost of Ownership) of your Core network.

So let’s see where is the problem?

Problem#1

The Problem#1 is that of treating both IP and Transmission as two different entities   in terms of any expansion strategy. There is a reason for that: classically, both IP and Transmission are run by separate departments.  Transmission (lower layer) is treated as a resource to serve IP layer (upper layer) without the need for Transmission to understand or help upper layer ( except connectivity).

I will give you one example:

Have you heard of the “Dumb Pipe” approach?

An approach, in which IP uses Transmission as transparent pipe. In other words, shifting complete aggregation, protection and intelligence to IP layer, thus keeping Transmission as dumb and non-intelligent pipe.

For obvious reasons, this strategy is simple and easy to implement. As Transmission does not need to provide any other service except connectivity, therefore there is little or no interaction needed between IP and Transmission folks.

But there is another side of this approach:

More CAPEX and OPEX spending!

How?

Such operators need to consistently build up two layers in parallel: the IP layer and the Transmission layer.

I call it “Grow-Routers-Fast, Grow-Lower-Layers-Fast” (GRF-GLF) approach.

In GRF-GLF approach, Transmission is participating neither in aggregation, nor in protection or any intelligence for IP packets. Therefore, IP layer takes complete burden to achieve these objectives. More specifically, the bits, after entering into IP core, need to touch every core router/IP layer on its way to destination. All this because, IP is the only layer that can aggregate, protect or route the traffic.

As, all bits have to pass through the transit routers as well, so we end up building interfaces for three layers:

  1.  Originating Core Router
  2. Terminating Core Router
  3. All Intermediate/Transit Core Routers

Isn’t it unfortunate that the intermediate router has to take the burden of the bits that are not dropped or added by it?

Worst still:

With every scale of routers, there would be a need to corresponding scale the Transmission resource. So indeed, we are constantly building routers and building Transmission with fast pace- a GRF-GLF Strategy!

Problem# 2

Secondly and in contrast to above, there is another approach that calls for “Router Offload”. The term “Router Offload” has been abused a lot by opponents and proponents on both sides. “Router offload” in its simplest definition mean that any transit traffic at any intermediate core router should be offloaded to lower layer like SDH/OTN/ DWDM.

Therefore, core routers at any intermediate site do not need to be scaled for transit traffic. This will result in huge cost savings.

However, terms like “Router offload” and “IP offload” are pitched in a sense that gives out a massage that core routers are not needed at all. They go to the extent of even calling it “hollow core” designs with no core router in a network at all. Indeed, a wrong perception. Later on, I can show you that “Router offload” for some traffic does make sense but not for all kind of traffic.

It will also become apparent that Router offload is part of a solution but not a complete solution, if applied to all traffic.

What is the solution? How can IP and Transmission live through a successful marriage?

Having presented the problem, let’s see how we can solve these issues.

I would call upon some statistics that will support me to solve these problems.

Some stats form Cisco VNI-2013: (Annual report on future global data; following excerpts are from the updated report of June 2014)

  1. Fixed internet data will rise at CAGR of 20% annually up to 2018
  2.  Mobile data (includes Mobile data and Internet traffic) will rise at CAGR of 61% annually up to 2018
  3. By 2018,Consumer Internet will represent two thirds of all IP traffic followed by managed IP at 20%

The key message here is that internet data will rise considerably (at CAGR 20%) more than any other data (e.g. enterprise data) in networks. And by 2018, consumer Internet will be two thirds of all data carried over networks.

I repeat the key message is, “Internet data” ( Also called HSI:” High Speed Internet”) is/will be the pre-dominant traffic in networks”.

The direct corollary is; “The nature of “Internet data/ HSI data” should drive any plan for network expansion”.

So what is the nature of the Internet data? The Internet data flows to fixed destination in network: towards internet gateways from originating routers. In fact this is a “point to point” data .This very nature is quite different than any other layer 3 VPN data  that has to follow any to any/ mesh connectivity and needs to stop at every core router.

In other words, for the Internet data, the destination is fixed and known to an operator.

This nature of Internet data, as probably you have guessed, is similar to the point to point behavior of lower layers of Transmission. Wouldn’t it be appropriate to push this date to lower layers and take advantage of the lower layers?

This works on the principle that ““Cost to carry data on the lower layers is less than carrying it over IP layer”

By intelligently designing core network, routers can take help of the lower layers (L2, SDH, OTN or DWDM) to carry the traffic to the Internet Gateway site.

This means, the data does not need to touch IP layer at every intermediate core router because of transit router bypass.

Wouldn’t it result in slow down of core routers build up?

In fact, core routers do not need to grow as fast as lower layers need to, thus bringing significant cost savings for Operators. This strategy would result in “Grow-Routers-Slowly, Grow-Lower-Layers-Fast” (let’s call it GRS-GLF strategy). For Obvious reasons this will bring the TCO of the network lower.

Going to the earlier debate I can now say that GRS-GLF strategy neither makes the Transmission dumb as lower layers are helping the routers in aggregation (in addition to protection also but that is a subject for another day), neither makes a complete offload of core routers (as we are talking about the Internet data only) as the proponents of ” Router Offload” would sometimes want you to believe.

Having explained why the GRS-GLF strategy makes sense, let me now share some guidelines on how this can be implemented in networks.

(PS: While the discussion would refer to “OTN”. The data applies equally to SDH. Note that OTN here means OTN switch, not the OTN mapper in DWDM)

TIP #1 Know the nature of your Traffic very well

Meaning you should know what kind of traffic is carried over routers. What is the mix of layer 3 VPN versus Internet data? In other words, what is the ratio of traffic for enterprise/business connectivity versus internet data? Perhaps the biggest mistake the planners do is designing the network for one pattern and one type of traffic.

Let’s say the Internet data is 60% (which is a decent estimate) compared to other data. This means you can start zooming on this 60% of data to be offloaded to lower layers.

TIP #2  Know the Origin and Destination of your Internet Data

An easy way to look is the IP destination of the traffic. The IP destination will tell you whether the traffic is destined for Internet Gateways or not. The best point would be the aggregation points in your network where you are collecting the Internet data.

Here you can ask this question. Can I send this traffic directly to the Internet Gateway bypassing the intermediate core routers?

If the answer is yes, you need to answer the question in TIP 3.

TIP #3  Select Suitable Lower Layer to take the traffic to Destination (DWDM or OTN or L2 Switch?)

You need to take important decision here. Which layer to offload to?

If the up-link pipe of router is congested and already filled with internet data and up-link rate is same as DWDM lambda rate, the best would be to go to DWDM layer directly as OTN is not adding any benefit of its aggregation capabilities. Let the DWDM take the traffic directly to Internet Gateway.

If the uplink pipe of the router is partially filled, there would be a need for a lower layer that has aggregation/statistical multiplexing capabilities, both low layers like Layer 2 Switch and OTN can help here. (See next point also)

Interestingly these days, layer 2 switch and OTN are packaged in the same switch thus operators can take advantage of such switches for grooming capabilities. Remember we are talking about GRS-GLF strategy, so we are building lower layers fast but not the IP layer at same pace.

TIP #4  Keep Router Up-Link rate to OTN, lower compared to DWDM lambda rate!

(I am assuming that you already own OTN switch, if not you can still offload traffic to lower layers like DWDM and L2 Switch as TIP 3 tells).

Why would use an OTN switch ( in between router and DWDM) and then keep the up-link of the router at same rate as DWDM rate. This is in fact more CAPEX. We are now growing higher layers and lower layers at the same time.

If you are using an OTN switch, it is recommended to keep the uplink of the routers low compared to DWDM lambda rate. For example if you are using 100G lambdas, recommended uplink rate for router to OTN should be 10G. This would enable you to use the grooming functionality of the OTN in the intermediate sites. One of the mistakes I see is that some service providers keep both rates same. This would deprive the operator of using the grooming functionality of the OTN layer.

TIP #5 Use Color/DWDM Interfaces on OTN

If you are using OTN switch and DWDM from same vendor, it would make sense to explore the use of color DWDM interfaces that can give traffic directly over the DWDM instead of using a lot of back to back grey interfaces between OTN and DWDM. This will result in a lot of CAPEX/ OPEX saving.

If you are using OTN switch and DWDM system from different vendors, you would need to run interoperability tests to use the color interface of the OTN of one vendor over the DWDM system of the other.

TIP #6  The IP and Transmission Departments need to Handshake

Last but not the least!

If the IP and Transmission departments are different ( as usually in Tier 1 Operators) , it would need a good coordination between the two in order understand from each other on how best to expand both IP and Transmission layer that can bring total cost savings for operators. They would need to plan together on how to use all layers intelligently bringing the Transmission cost per bit down.

If there is one department that plans both the above networks, this is ideal as there will be combined strategy for IP and Transmission expansion.

Conclusion:

As you can see that all the above tips apply to using lower layers of Transmission to support efficient and cheapest transport of IP data. These apply in particular to Internet/HSI data and I repeat by using selective offloading of IP traffic, an operator can get considerable cost benefits.

I can bet that by exploring these out of box approaches, the IP and OTN/DWDM can live happily together through a successful marriage.

While this article is mainly looking at how Transmission pipes can help the upper layers in terms of cheap transport of data by using aggregation intelligently, I can write at same later time on how the lower layers protection options can further help the MPLS layer in bringing down the cost of protection at upper layers.

Now it’s your turn!

I would love to listen to  what do you think about the arguments and Tips in this blog and further,how you are coping with the tremendous expansion in the IP data in your network.

 

Subscribe
Notify of
guest
40 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Shaheen Qamar
Shaheen Qamar
6 years ago

Faisal,

An excellent article and valid arguements in support of augmenting lower layers to support exponential growth in Internet/HSI traffic. The IP folk may raise some eyebrows; however, the fact is that lowers (DWDM, OTN, L2) are always considered dumb and impotent to make intelligent decisions at their own. Although WSON/GMPLS and ASON/GMPLS have added intelligence to the lower layers; however, the advantages related to aggregation, grooming and statistical multiplexing of the OTN switches have been largely ignored by the operators/carriers.

Your point of view is convincingly valid and shall be well taken by the readers regarding treatment of Internet/HSI traffic and your proposed marriage between layer-2 and layer-3 network planners can reduce double investments in these layers to the benefits both the network owners and users.

Your blog has cleared many misconceptions about the router offloading and and your tips has provided a way forward for the network designers, network managers and decision makers put the dollars where they are needed, ultimately avoiding double investment, reducing TCO and getting the best benefit of the available technology and assets.

Will be anxious to read the 2nd part of this blog where you will discuss how the lower layers protection mechanism can augment and add value to upper layers.

Shaheen Qamar
Executive Manager, Telecom

Syed Muhammad Ahsan
Syed Muhammad Ahsan
6 years ago

Dear Faisal,

I started reading this article with the intention of going through a text briefing the outlined concept of IP and OTN/DWDM as a single platform. However, this write-up ended up explaining the very details of how progressively the technology evolution and reducing guard bands in-between IP stack layers have made the choice of terminal equipment, access methods and ultimately traffic engineering, something to ponder upon more than ever needed before.

Additionally, this has led operators to generalize the Data and Transmission departments which alternatively means a merger, not yet an acceptable term in Telecommunication world.

This article on it’s own is one complete analysis and guideline for the vendors to present technology business to operators resulting in a lot of interest for appropriate business sales and for operators to understand their exact customer needs and propose best solutions for them.

The reference to revenue calculations in this article considering the targeted customer needs can be extremely helpful for solution developers and network planners to utilize the investments in a confident manner resulting in boosted revenue generation.

Excellent blog. In future, I’d surely be interested in probing more about operational side of OTN/DWDM networks merging with IP plane.

Syed Muhammad Ahsan
Technology Consultant, Telecom,
Middle East.

ALI ALGHAMDI
ALI ALGHAMDI
6 years ago

Dear Faisal
I enjoyed the article and believe that you have a lot to say regarding the handshake, strategic planning and offload decisions (possibilities and challenges) . this is also applied to Shaheen (greetings to him).

We’ve seen a soled model of this via merger rather than handshake or coordination in major technology leaders like Cisco when they introduced “HERO” High End Routing and Optical Group! as a new organization to refine the product/s roadmap and production plans.

I conclude here by saying that the action is for the operators top management towards complete merger in order for planners and designers to speak one language.

ALI ALGHAMDI

wellington guta
wellington guta
6 years ago

interesting and informative……Does this mean that the world of transmission is going back to layer 2? What technology will best suit this type of marriage…MPLS?

Su
Su
6 years ago

very good!

JIA
JIA
6 years ago

Your point is reasonable.

Bruno
Bruno
6 years ago

Congratulations Faisal,

With many technologies development going on (as the growth of carrier ethernet, MPLS-TP, SDN.. etc)

You’ve just pointed that there’s no “Best Technology”

But the best way to treat your network traffic as it is needed.

Great tips, great article!

Eric Wang
Eric Wang
6 years ago

This is a excellent article, introduce the current problem, GRF-GLF, and the advantage of use different lower aggregating technology (DWDM/OTN/) for different scenario.
But I think the fight between IP and TxM never stop in past, current and future, even the two departments were merged, one equipment could support L0/L1/L2/L3. As operator, firstly, need understand his traffic totally (you mentioned in TIP#1), include IP and TXM, select suitable technology for different traffic (FIX, Mobile, Enterprise, DATA Center…), make a strategy as a guidance and policy to planning and design traffic. Like VDF group has made a blueprint to regulate their subnet to plan traffic and establish network in IP and TXM, each subnet group need follow this policy.
I think the synergy IP and Optical is a big topic, I am anxious read next part article.

Thanks you for your share.

Eric.

janakiraman
6 years ago

Dear Mr.Faishal Khan,
Your articles are excellent and written with clarity. This shows your indepth practical knowledge. I have been working on a project of establishing a OTN integrated DWDM network with the IP overlay network of MPLS. I have understood the value of segregating the Internet traffic and Enterprise traffic for traffic engineering at L3 and L2 level. I am looking forward to your articles.

With best regards,

Janakiraman

RajeshKumar Babu
RajeshKumar Babu
4 years ago
Reply to  janakiraman

Hello Mr.Jankiraman

I would be interested to have the working model of your “OTN integrated DWDM network with the IP overlay network of MPLS”. Can you share some of the technical specs and logical diagram.

Regards
Rajesh

Braun Wu
Braun Wu
6 years ago

Fine essay as always from Faisal. I stand for your side for one point which we share same comments. That is the efficiency of bearer network. Considering the expensive invement for Router 40G/100G interfaces, it is really the last choice to peel off the traffic at most of Router node and then dress on again without any dropping or adding. It is not wise to pay for the curiosity. Statiscally, the majority of traffic should be pre-set to bypass through router for a point- point journey ultimately.

Qasim Alvi
6 years ago

Hi,
Nice to read your blog.
I always had wish foresee a time when we will have OTU\OMU\ODU type cards get embedded into routers. Its not far though one step is already taken by manufacturers …e.g. Cisco CSR and Juniper TX Series and other vendors too! calling it IPoDWDM which eliminated the need of separate DWDM transponder unit and only requires Mux\Demux + Amplification. I hope this gets integrated soon. And we will have a king device of bearer networks.

Regards,
Tech Manager ZTE – Packet Transport Networks

Shahid
Shahid
6 years ago

Nice Analysis

Raymond Macharia
Raymond Macharia
6 years ago

An excellent article. You have articulated very well thoughts I have had on how capacity should be addressed on networks of the future.
Thank you Faisal

Raymond Macharia

Ali
Ali
6 years ago

Hi Faisal

A well presented article with thought provoking tips & analysis for future IP & Transport network planners.

I do have one query though, when you say that HSI traffic will be the prevalant traffic in future (2018) networks, and then you talk of its nature being P2P and deterministic, with the analysis of offloading the intermediate core routers (since their job is just to pass on the IP traffic without any L3 processing required, which mimics the lower layers boxes functionality to some extent), IP/MPLS planners will see this as lower layer invasion, and thus their remit will be limited purely to the core edge routers. This surely will be alarming for them. The only planning / expansion they would come across would limited to the edge routers.

However, having said that, broadly speaking, a successful, and I repeat “successful” marriage between the IP & SDH/OTN/DWDM will definately bring massive cost savings and lower TCO for the operator as a whole. The decision makers will be bang on target in terms of putting their money in the right baskets and reaping the better ROI.

A few notes….

TIP#3

There are many Tier-1 operators in WEU, including UK, where no OTN switch has been deployed, as they do not see the requirement of statistical multiplexing / grooming, since they did not experience the transition from SDH->NG-SDH->Hybrid (that mobily did), and thus phased out completely the circuit swtiched and brought upon the packet switched technologies for metro networks. However, in your case, if an uplink pipe of the router is not at the same rate as DWDM lambda, then OTN would be a best fit…what in your view will be the solution to operators in the west that have don’t have OTN available prior to DWDM.

Ali

Dilip Gulati
Dilip Gulati
6 years ago

Dear Faisal,
Cloud Computing, Caching server would change the nature of Costumer Internet Traffic, and we would need Router Intervention many intermediate locations to improve customer experience.
What is your take on this…please reply

Abdul Rauf
Abdul Rauf
6 years ago

Great Stuff and explanation is very clear on Marriage of IP with lower layers like DWDM/OTN/SDH. I completely agree that IP designers/Planners think on thier perspective and Transmission planners/designers think on thier own perspectives. Now it is time for merger of two departments and work towards the success of organization. When I see service providers, there are seperate departments for Transsmission Engineering and IP Engineering. I would say that, single department design the network . Stratagic planning apporach is the best by forseeing future requirements.

Tariq
Tariq
5 years ago

Mr Faisal Khan
Through blog and Good explanation, Aggree with the merger of both departments

Haroon
Haroon
5 years ago

Hi Mr. Faisal,

Thanks very much for publishing such article which is blending two technologies IP & TXM plus enlightening hidden areas which always driven

by two different departments/authorities like IP planning (Enterprise/DC +MPLS) & Transmission.

It is found very interesting to see & start orchestrating bandwidth planning across these two Depts into one Box (Lamda & Gigabits)

Is it possible if you can share lamdas XY/Gigabits XY compatibility units/limits in tabular format ? as best deploying practice for IP Design Engineer.

Regards
Haroon Yousaf

RajeshKumar Babu
RajeshKumar Babu
4 years ago

Hello Mr.Faisal

Thanks for this article. I am in the process of designing a 40G network with 30 sites around the county. We are basically targeting to have the wholesale bandwidth model. Pls let me know if you have any RFPs for such project and what would be your proposal for this business model.

Regards
Rajesh

RajeshKumar Babu
RajeshKumar Babu
4 years ago
Reply to  Faisal Khan

Thanks Mr.Faisal for the update. What Model you would suggest for the wholesale bandwidth market. IPoDWDM or OTN?

Rajesh

joe
joe
4 years ago

Hi Faisal..you find that “Internet” traffic is very point to point in fashion therefore you can implement a lower layer technology to support..however it seems quite opposite to me with the substantial number of content end points and number of peering arrangements any one carrier may have. Isn’t IP a much better solution than treating this as point to point? Plus the end point of content on a network today may not be where it goes tomorrow.

40
0
Would love your thoughts, please comment.x
()
x