Charles Leaver – Monitoring Of Activities In The Cloud Is Now Possible With Enhanced NetFlow

Written by Roark Pollock and Presented by Ziften CEO Charles Leaver


According to Gartner the public cloud services market exceeded $208 billion last year (2016). This represented about a 17% increase year over year. Pretty good considering the on-going issues most cloud consumers still have relating to data security. Another particularly intriguing Gartner finding is the typical practice by cloud consumers to contract services to several public cloud providers.

In accordance with Gartner “most businesses are currently utilizing a combination of cloud services from various cloud service providers”. While the business rationale for using several vendors is sound (e.g., preventing supplier lock in), the practice does develop extra complexity inmonitoring activity across an organization’s significantly fragmented IT landscape.

While some providers support more superior visibility than others (for example, AWS CloudTrail can monitor API calls throughout the AWS infrastructure) organizations have to comprehend and resolve the visibility issues associated with relocating to the cloud despite the cloud supplier or service providers they work with.

Unfortunately, the capability to track application and user activity, and networking communications from each VM or endpoint in the cloud is limited.

Irrespective of where computing resources live, companies must address the concerns of “Which users, machines, and applications are communicating with each other?” Organizations need visibility across the infrastructure so that they can:

  • Quickly recognize and prioritize problems
  • Speed root cause analysis and recognition
  • Lower the mean time to repair issues for end users
  • Quickly recognize and eliminate security risks, reducing general dwell times.

On the other hand, bad visibility or poor access to visibility data can decrease the effectiveness of existing management and security tools.

Companies that are used to the maturity, ease, and reasonably cheapness of monitoring physical data centers are apt to be dissatisfied with their public cloud options.

What has actually been missing is an easy, ubiquitous, and sophisticated solution like NetFlow for public cloud infrastructure.

NetFlow, of course, has had twenty years approximately to become a de facto standard for network visibility. A typical deployment involves the tracking of traffic and aggregation of flows where the network chokes, the collection and saving of flow data from multiple collection points, and the analysis of this flow data.

Flows include a basic set of source and destination IP addresses and port and protocol info that is typically gathered from a router or switch. Netflow data is relatively inexpensive and easy to gather and provides almost ubiquitous network visibility and enables actionable analysis for both network monitoring and performance management applications.

The majority of IT staffs, especially networking and some security groups are extremely comfy with the technology.

But NetFlow was developed for fixing what has become a rather restricted issue in the sense that it only gathers network information and does so at a minimal variety of possible locations.

To make better use of NetFlow, 2 essential modifications are essential.

NetFlow to the Edge: First, we have to expand the useful implementation scenarios for NetFlow. Instead of only gathering NetFlow at networking choke points, let’s broaden flow collection to the network edge (cloud, servers and clients). This would considerably expand the overall view that any NetFlow analytics supply.

This would permit organizations to augment and take advantage of existing NetFlow analytics tools to get rid of the ever increasing blind spot of visibility into public cloud activity.

Rich, contextual NetFlow: Second, we have to use NetFlow for more than basic network visibility.

Instead, let’s utilize an extended version of NetFlow and include data on the user, device,
application, and binary responsible for each monitored network connection. That would allow us to rapidly connect every network connection back to its source.

In fact, these two modifications to NetFlow, are precisely what Ziften has actually accomplished with ZFlow. ZFlow offers an expanded version of NetFlow that can be deployed at the network edge, also as part of a container or VM image, and the resulting info collection can be consumed and analyzed with existing NetFlow analysis tools. Over and above standard NetFlow Internet Protocol Flow Info eXport (IPFIX) visibility of the network, ZFlow provides greater visibility with the addition of info on user, device, application and binary for each network connection.

Eventually, this allows Ziften ZFlow to deliver end-to-end visibility in between any 2 endpoints, physical or virtual, removing standard blind spots like East West traffic in data centers and enterprise cloud deployments.