Industrial Ethernet: Unifying Networks for Real-time Performance and Enhanced Security, Exams of Technology

Download Industrial Ethernet: Unifying Networks for Real-time Performance and Enhanced Security and more Exams Technology in PDF only on Docsity! GUIDELINES FOR INDUSTRIAL ETHERNET INFRASTRUCTURE IMPLEMENTATION: A CONTROL ENGINEER’S GUIDE By Carlos Rojas Peter Morell Director Enterprise Sales Global Manager, Network Emerging Markets and Security Services Cisco Systems Rockwell Automation ABSTRACT As part of a continuing effort to make their organizations more efficient and flexible, manufacturers are rapidly migrating to Industrial Ethernet technology to network their industrial automation and control systems. The use of standard Ethernet technology enables organizations to control costs by moving from costly plant-optimized networks to a proven technology that is simpler to integrate, requires widely available skills, and is more secure and reliable while still meeting real-time traffic requirements. This white paper provides an overview of Ethernet technology and implementation guidelines to implement in the both control and information networking environment. It discusses the requirements and consideration in implementing a switched Ethernet architecture in industrial networking environments. Discussion will include: ● Real-time network performance (including low latency, low jitter, and minimal packet loss) to develop deterministic systems ● Security -Onsite and Remote Access ● Reliability ● Manageability and ease of use features Industrial Ethernet applies the Ethernet and IP suite of standards developed for data communication to manufacturing control networks. By implementing an intelligent Industrial Ethernet solution, organizations can build a manufacturing infrastructure that delivers the resiliency and network security of traditional fieldbus solutions, as well as the improved bandwidth, open connectivity, and standardization that Ethernet provides. Industrial Ethernet gives organizations substantially greater control over their networked manufacturing equipment. INTRODUCTION As manufacturers seek to improve processes, increase productivity, and integrate manufacturing and business networks, many are turning to Ethernet technology at their plant. This migration is rapidly gaining momentum. Once considered a solution that was limited to corporate network environments, Ethernet technology has proven to be a robust alternative that can meet the unique needs of the manufacturing environment. Industrial Ethernet networks that use intelligent switching technology can offer a variety of advantages compared to traditional industrial networks. The technology can be deployed using a switched Ethernet architecture and has proven successful in multiple critical applications in different markets. Because the technology is based on industry standards, Industrial Ethernet enables organizations to save money by moving away from expensive, closed, factory-floor optimized networks. Using standard Ethernet technologies also reduces overall risk and provides investment protection, as manufacturers and automation vendors can take advantage of continued industry By providing a scalable platform that can accommodate multiple applications, Ethernet-based automation systems can increase flexibility and accelerate deployment of new applications in the future. At the same time, Ethernet delivers the network security, performance, and availability required to support critical manufacturing applications. To deploy this technology, engineers on the manufacturing floor should be familiar with some of the important concepts behind Industrial Ethernet. This paper will provide a general overview of the most important traditional Ethernet technologies in use today. It will also discuss how Industrial Ethernet upgrades traditional, proprietary factory-floor networks to a low-cost, secure, high-performance, scalable architecture. Finally, this paper will review some of the intelligent features that make Industrial Ethernet an attractive choice for manufacturing organizations. TRADITIONALLY SEPARATE NETWORKS Today, many manufacturing companies maintain separate networks to support their factory floor operations and business operations (Figure 1). Over the years, these networks were developed to respond to the different information flows and control requirements involved with manufacturing processes. Figure 1: Traditional Proprietary Fieldbus Architecture The corporate IT network supports traditional administrative functions and corporate applications, such as human resources, accounting, and procurement, as well as WAN connections between sites and Internet connectivity. This network is usually based on the Ethernet and IP suite of standards. The control-level network connects control and monitoring devices, including programmable automation controllers, PC-based controllers, I/O racks, drives, and human-machine interfaces (HMIs). This network, which has not been based upon standard Ethernet and IP in the past, requires a router or, in most cases, a gateway to translate application-specific protocols to Ethernet-based protocols. This translation lets information pass between the control network on the factory floor and the corporate network infrastructure, but has limited functionality and bandwidth, and requires significant effort to keep up to date. The device-level network links the controllers with the plant floor’s I/O devices, including sensors such as transducers, photoeyes, and flowmeters, and other automation and motion equipment, which are used to network locally, within a LAN. IP has two primary responsibilities: providing connectionless, best-effort delivery of datagrams or packets through a network; and providing fragmentation and reassembly of datagrams to support data links with different maximum- transmission unit (MTU) sizes. . TCP provides reliable, in-order delivery of packets between two devices. It relies upon IP. TCP applications establish connections between one another over which they send packets. TCP is a stateful protocol; it maintains the state after the packet is sent. TCP checks whether all packets have arrived and can request re-transmission if a packet is dropped, lost, or corrupted during transmission. Due to this overhead though, TCP is not always ideal for real-time applications. UDP is often used for real-time communications such as voice and I/O traffic. UDP also relies on IP. UDP does not guarantee delivery or the order of the packets, thus simplifying the protocol. The applications would rather drop a packet than receive it late. UDP is considered a ―stateless‖ protocol. It is compatible with packet broadcast (sending to all on local network) and multicasting (sending to all subscribers). There are a whole host of other protocols designed for specific purposes in the networking world, but the key ones for an automation and control system are IP, TCP, and UDP WHAT IS INDUSTRIAL ETHERNET? Recognizing that Ethernet is the leading networking solution, many industry organizations are porting the traditional fieldbus architectures to Industrial Ethernet. Industrial Ethernet applies the Ethernet standards developed for data communication to manufacturing control networks (Figure 3). Using IEEE standards-based equipment, organizations can migrate all or part of their factory operations to an Ethernet environment at the pace they wish. Figure 3: Using Ethernet for Automation Control For example, Common Industrial Protocol (CIP) has implementations based upon Ethernet and the IP protocol suite (EtherNet/IP), DeviceNet, and ControlNet (among others). Most controllers (with appropriate network connections) can transfer data from one network type to the other, leveraging existing installations, yet taking advantage of Ethernet. The fieldbus data structure is applied to Layers 5, 6, and 7 of the OSI reference model over Ethernet, IP, and TCP/UDP in the transport layer (Layer 4). . The advantage of Industrial Ethernet is that organizations and devices can continue using their traditional tools and applications running over a much more efficient networking infrastructure. Industrial Ethernet not only gives manufacturing devices a much faster way to communicate, but also gives the users better connectivity and transparency, enabling users to connect to the devices they want without requiring separate gateways. Technology Tailored for Manufacturing Although Industrial Ethernet is based on the same industry standards as traditional Ethernet technology, the implementation of the two solutions is not always identical. Industrial Ethernet usually requires equipment that can handle more severe environmental conditions, flexible node counts, varieties of media, very predictable real-time data traffic performance, and increased levels of segmentation as compared to traditional Ethernet networks in a corporate data network. Wiring EMF protection The primary difference between Industrial Ethernet and traditional Ethernet is the type of hardware used. Industrial Ethernet equipment is designed to operate in harsh environments. It includes industrial-grade components, convection cooling, and relay output signaling. And it is designed to operate at extreme temperatures and under extreme vibration and shock. Power requirements for industrial environments differ from data networks, so the equipment runs using 24 volts of DC power. To maximize network availability, Industrial Ethernet equipment also includes fault-tolerant features such as redundant power supplies. Other differences are important as well. The Industrial Ethernet automation and control protocols themselves and their use of the available technologies within the standard Ethernet and IP protocol suite often differ significantly from standard Ethernet implementations. For example, in most automation and control applications, 80 percent of the network traffic is local – one local device talking to another local device often using multicast (one sender, many receivers) packets. In most IT installations, the reverse is true where 80 percent of the network traffic is routed to external locations (such as the data center or the Internet) using unicast (one sender, one receiver) packets. Automation and control systems also differ from other applications in their need for determinism and real-time network requirements – quick and consistent transmission of the data. The Industrial Ethernet network must be designed and implemented with these differences in mind. Ethernet and the IP protocol suite have developed a number of technologies and features that support these requirements. To help optimize synchronous data access, Industrial Ethernet equipment must include the intelligence to support features such as multicast control (IGMP Snooping), QoS, and virtual LANs (VLANs). Other high availability, security, and management functions should also be considered depending on the specific automation and control application. Benefits of a Switched Ethernet Architecture Organizations can choose from a variety of devices and architectures when building an Ethernet LAN. Devices range from simple hubs, to unmanaged switches, to intelligent, managed switches. The network components are important to the proper functioning of the automation network, and careful consideration should be given to selecting the appropriate device. Early Ethernet deployments often used hubs. Hubs act at the physical layer of the OSI model, and are essentially repeaters that connect multiple devices over the same shared medium. Because hubs use a shared medium, collisions can occur when multiple devices try to communicate at the same time. This may not be a significant concern in a small network without high-performance requirements, but is typically not acceptable in environments where real-time, predictable performance is important, such as automation networks. The collisions that resulted from the use of hubs contributed to the perception that Ethernet is not deterministic, even though hubs are rarely used anymore. Over the past 10 years, most Ethernet deployments have used full-duplex switched Ethernet switches. Switches make it possible for several users to send information over a network at the same time without slowing each other down. In a fully switched network, there are no hubs so each Ethernet network has a dedicated segment for every node. Because the only devices on each segment are the switch and the node, the switch picks up every transmission before it reaches another node. The switch then forwards the data over to the appropriate segment. In a fully switched network, nodes only communicate with the switch and never directly with each other. Fully switched networks employ either twisted pair or fiber-optic cabling, both of which use separate conductors for sending and receiving data. The use of dedicated communication channels allows nodes to transmit to the switch at the same time the switch transmits to them, eliminating the possibility of collisions. Transmitting in both directions also can effectively double the apparent speed of the network when two nodes are exchanging information. For example, if the speed of the network is 10 Mbps, each node can transmit at 10 Mbps at the same time. Switches usually work at Layer 2 (data link) of the OSI reference model using MAC addresses, and deliver a number of important advantages compared to hubs and other LAN devices. Some of these advantages include the following: ● Predictable performance: The ability to ensure that a packet is sent and received in a specific period of time is an important design goal for industrial networks. For the network to support predictable, real-time traffic, the design must be as simple and highly structured as possible. ● Latency: Switches normally have very low latencies, which refers to the time it takes for a network packet to transit between a source and a target. Most control operations in industrial applications can tolerate latencies of 10 to 50 milliseconds (ms). Because control traffic frames in industrial applications are usually below 500 bytes, the latency introduced by a switch at 100 Mbps is only about 30 microseconds with a worst-case scenario of close to 100 microseconds – well below the limit and 100 times faster than most applications require. ● Standardization: One of the main motives for using Industrial Ethernet is the need to standardize around a common infrastructure.. Managed switches provide performance, management, diagnostics, and security capabilities that are not supported on unmanaged switches. These types of features allow the network administrator to configure the switch to provide traffic prioritization, basic and advanced security capabilities, multicast traffic control, diagnostic capabilities, and a number of other features that are important for most industrial and office network environments. Given the critical nature and performance requirements of automation and control networks, a managed switched Ethernet architecture is the most appropriate choice for most industrial environments. Some of the most important features on intelligent managed switches in an industrial environment include: ● Packet loss under congestion: Today’s intelligent switches offer QoS features that make it possible to prioritize critical traffic so that it will be handled with priority and not be dropped due to congestion. By implementing simple QoS parameters in an intelligent switch, organizations can prioritize critical traffic over non-critical traffic at wire speed, helping to ensure consistent packet delivery and integrity for the control network. Even under heavy congestion, QoS features help ensure that important traffic will reach its destination quickly and consistently. ● Broadcasts and multicast: Industrial applications often rely on broadcast or multicast communication. Intelligent switching platforms can dynamically configure the interfaces so that traffic is forwarded only to ports associated with requested data. This feature reduces the load of traffic crossing the network and relieves the client devices from processing unneeded frames. ● Demilitarized zone (DMZ): A DMZ is a buffer zone between various areas of a network, and supports sharing of data and services between the network zones A DMZ supports exchange of data and services between the zones, yet enables strict control of the traffic from either zone. Cisco recommends that a manufacturing network supporting automation and control systems be segmented from the general enterprise network with a DMZ. ● Secure connectivity and management: To provide additional protection for manufacturing networks, organizations can take several approaches to authenticate and encrypt network traffic. Using VPN technology, Secure Sockets Layer (SSL) encryption can be applied to application-layer data in an IP network. Organizations can also use IP Security (IPsec) technology to encrypt and authenticate network packets to thwart network attacks such as sniffing and spoofing. Manufacturers can use all or a portion of these technologies to help ensure network availability, confidentiality, and integrity. Figure 5: Simplified Remote Access Architecture RELIABILTY Because factory-floor applications run in real time, the network must be available to users on a continuous basis, with little or no downtime. Manufacturers can help ensure network reliability using effective network design principles, as well as intelligent networking services. Reliability needs to be considered at each layer in the OSI model. Especially important for network design considerations are Layers 1–3 Table 1: Network Design Considerations Reliability exists at other layers, but is applied by the automation and control applications. For example, TCP (a Layer 4 protocol) is inherently reliable because dropped or corrupt packets are automatically resent when detected by the protocol. But the overhead and delay represented in the process have led automation and control vendors to use UDP for some types of traffic. UDP has less overhead, but no automated resending for particular information to communicate and therefore relies upon the application to detect and manage packet loss or delay Physical Layer Reliability At the physical layer, a number of techniques can be applied to help achieve a resilient, highly available network. First, the various components can be configured or purchased with resilient features such as redundant power supplies (or even UPS), and redundant components (such as fans, CPUs, network interface cards [NICs], etc.). Additionally, some devices may also support in- line upgradeability of components or software that allows for continued service while the device is being maintained or upgraded. These techniques will usually significantly improve the mean time to repair (MTTR) the device itself or ensure the device has network access in the case of media disruption or port failure (on either the end device or switch). Using redundant devices may also help maintain high network availability. For example, multiple switches or routers can be configured in a high-availability manner so that in the case of disruption of one device, the other device will take over the network services quickly and automatically. Data Link Reliability Manufacturers deploying an Ethernet solution should design networks with redundant paths to ensure that a single device or link outage does not take down the entire network. How the end devices and network devices are all inter-connected is a network topology. Two network topologies most often used to achieve higher availability are ring and redundant star. The topology chosen also has implications on wiring cost and complexity, performance, and installation and maintenance cost. Other topologies (such as bus or trunk-drop) may be cheaper to install and easier to maintain, but are more susceptible to outage and have a higher impact when a connection or device is lost. In redundant star designs (Figure 6), switches and routers are connected in a hierarchical fashion. The first layer where devices are connected to switches is often referred to as the access layer. These switches provide connections for endpoint devices such as PLCs, robots, and HMIs. Access-layer switches generally operate at Layer 2 (data link) of the OSI model. Above the access layer is another layer of switches referred to as the distribution layer. These switches interlink the various access layer switches. If they support multiple cell/area zones, they may need to operate at Layer 3 (network) of the OSI model, referred to as Layer 3 switches or routers, to support multiple VLANs. Figure 6: Redundant Star Network Topology In ring topologies (Figure 7), all devices are connected in a ring. Each device has a neighbor to its left and right. If a connection on one side of the device is broken, network connectivity can still be maintained over the ring via the opposite side of the device. In a typical topology, the ring is at the access layer, and connected up through the distribution and core layers using a redundant star topology. In this model, the distribution and core layers provide the same functionality as in the redundant star, with the distribution layer routing between cells, and the core connecting to higher-level or external networks. . Figure 7: Ring Topology IGMP Snooping Many manufacturing applications depend on multicast traffic, which can introduce performance problems in the network and in the end devices themselves. To address these challenges in an Industrial Ethernet environment, organizations can turn on IGMP ―snooping‖6 on their managed switch network. IGMP snooping limits the flooding of multicast traffic by dynamically configuring the interfaces so that multicast traffic is forwarded only to interfaces associated with IP multicast devices. In other words, when a multicast message is sent to the switch, the switch forwards the message only to the interfaces that are interested in the traffic. This is very important because it reduces the load of traffic traversing through the network. It also relieves the end devices from processing frames that are not needed. . In a producer-consumer model used by Industrial Ethernet protocols such as CIP, IGMP snooping can limit unnecessary traffic from the I/O device that is producing, so the traffic only reaches the device consuming that data. Messages delivered to a particular device that were intended for other devices consume resources and slow performance, so networks with many multicasting devices will suffer performance issues if IGMP snooping or other multicast limiting schemes are not implemented. Figure 8: The Importance of IGMP Snooping in Producer-Consumer Model The IGMP snooping feature allows Ethernet switches to ―listen‖ to the IGMP conversation between end devices. With IGMP snooping, the Ethernet switch examines the IGMP traffic coming to the switch and keeps track of multicast groups and member ports. When the switch receives an ―IGMP join‖ report from a host for a particular multicast group, the switch adds the host port number to the associated multicast forwarding table entry. When it receives an IGMP ―leave group‖ message from a host, it removes the host port from the table entry. After the switch relays the IGMP queries, it deletes entries periodically if it does not receive any IGMP membership reports from the multicast clients. Only devices that are part of the group receive the multicast messages sent to that group, thus reducing the amount of messages that the network must send and that the end devices need to process. QUALITY OF SERVICE An Industrial Ethernet network may transmit many different types of traffic, from routine data to critical control information (such as I/O traffic), or even bandwidth-intensive video or voice. The network must be able to distinguish among and give priority to different types of traffic. By giving priority to different types of traffic, the network can deliver real-time network services: low latency and jitter and minimal packet loss when the network infrastructure is under load. This capability to share the network with other applications, yet maintain the priority of the critical traffic, is a key differentiating factor for Industrial Ethernet versus existing industrial network protocols. Organizations can implement QoS using several techniques. QoS involves three important steps. First, different traffic types in the network need to be identified through classification techniques and then tagged. The classification and marking can occur either in the end device or the network infrastructure, depending on the capabilities of both. There are also various ways to tag and priority levels that can be applied. It is important to understand the type of traffic that will exist on the network and ensure that the QoS approach takes into consideration all of those traffic types and the capability of the network infrastructure. Second, advanced buffer-management techniques need to be implemented to prevent high-priority traffic from being dropped during congestion. Finally, scheduling techniques need to be incorporated to transmit high-priority traffic from queues as quickly as possible. All three steps need to be considered when developing a technique to implement QoS . Although the application of QoS should be designed and tested before implementation, it is relatively easy to deploy and maintain through the use of predefined port and switch configurations. As well, the QoS approach for the manufacturing zone may be different for the approach in the enterprise zone (where, for example, voice traffic gets the highest priority), thus reinforce the need to logically segment the manufacturing zone to set up boundaries between the different QoS implementations. Figure 9: Applying QoS to Industrial Applications In an Industrial Ethernet application, real-time I/O control traffic would share network resources with administrative data (such as explicit data), configuration files (FTP, for example) and data- collection flows, as well as other traffic, in the upper layers of the OSI reference model. By using QoS to give high priority to real-time UDP control traffic, organizations can realize the benefit of sharing resources yet maintain the real-time network characteristics required for I/O control traffic in industrial automation and control applications. Network Management and IP Addressing Network Management Network management is a set of tools, applications, and devices used monitor and maintain a network. Although a typical automation and control network does not drastically change after deployment, as with all other aspects of the system, the network does need to be maintained and managed. To a large extent, these functions have not been incorporated into the automation and control systems, but this is changing. Therefore, production control engineers designing and managing automation and control systems need to be aware of how the network will be managed and what tools, training, and resources are required to put those in place. The key functions of a network management as defined by the International Organization for Standards (ISO) are: ● Performance management is gathering, analyzing, and reporting on key network variables including device and link availability, throughput and utilization, and user response time. ● Configuration management is managing and updating network configurations including operating system versions, and network parameters (port, switch, and router settings). ● Accounting management is managing user and device accounts on the network. ● Fault management is detecting, logging, and notifying administrators of issues or faults within the network. ● Security management is controlling access to the network and monitoring the network traffic for security threats and breaches. Production control engineers should make sure tools are available to help personnel to perform these functions and they are trained in their use. This usually involves working with IT organizations that already have tools and expertise.. One key aspect of network management that does need particular attention is IP address allotment and allocation. IP Addresses Establishing policy and managing the IP addresses are relevant to a control engineer. Typically, any Industrial Ethernet device (new or replacement) needs an IP address assigned to it. Many production facilities use statically assigned addresses where someone has to decide the address and configure end devices with their IP addresses. As most automation and control applications use the IP address directly in their programs, this is a straightforward way to make sure they stay in-synch, although as a facility grows, it can become a maintenance burden. Therefore, some facilities use dynamically administered IP addresses, where every time the device starts, it gets its IP address from a network service, for example using the Dynamic Host Configuration Protocol (DHCP) Option 82. The network service can be configured to issue consistent IP addresses so automation and control programs do not have to be changed, provided the appropriate network design and configuration. Lastly, control engineers should also ensure they get enough IP addresses allocated and have an allocation method that allows factory floor devices to be easily recognized. IT is usually responsible for allocating enterprise IP addresses. Using DHCP Option 82 Ethernet switches provide excellent connectivity and performance; however, each switch is another device that must be managed on the factory floor. To make switched Ethernet networks easy to support and maintain, intelligent switches include built-in management capabilities. These intelligent features make it easy to connect manufacturing devices to the network, without creating additional configuration tasks. And they help minimize network downtime if part of the network should fail. One of the most useful intelligent features in a switched Ethernet network is Option 82.