Observability may be the latest buzzword in an industry loaded with them, but Cisco will tell you the primary goal of the technology is to help enterprises get a handle on effectively managing distributed resources in ways that have not been possible in the past.

The idea of employing observability tools and applications is a hot idea. Gartner says that by 2024, 30% of enterprises implementing distributed system architectures will have adopted observability techniques to Excellerate digital-business service performance, up from less than 10% in 2020.

“Today’s operational teams have tools for network monitoring, application monitoring, infrastructure monitoring, call monitoring, and more, but they rarely intermingle to provide a cohesive view of what’s going on across the enterprise,” according to Carlos Pereira, Cisco Fellow and chief architect in its Strategy, Incubation & Applications group.

Observability looks to address real problems by gathering information across domains and using it to show how one domain influences another and to predict problem areas or trigger incident management, Pereira said.

“By using observability tools, the business is able to determine the state of its applications with a high degree of certainty and understand how their services impact business key performance indicators and customers’ digital experience,” Gartner wrote in a latest observability report. “Observability enables quick interrogation of a digital service to identify the underlying cause of a performance degradation, even when it has never occurred before.”

At the latest Cisco Live! event in Amsterdam, Pereira provided a preview of the underlying architecture for observability called the Cisco Full-Stack Observability Platform. It’s expected in June, though some details have already been announced.

For example, at the Live! event Cisco said it will be unveiling deeper, bi-directional integration between AppDynamics and Cisco’s ThousandEyes digital-experience monitoring software. The tighter integration will enable correlation of business issues across application transactions and their dependencies, end-user experiences, the network path, and internet routing.

Last year, Cisco announced AppDynamics Cloud will offer applications and services that correlate telemetry across multiple domains.

Telemetry use is at the core of observability. Pereira said Cisco’s Full-Stack Observability will bring together the widest set of telemetry data in the industry—

metrics, events, logs, and traces (MELT), driven by OpenTelemetry—and feature a broad portfolio of Cisco technologies as well as an ecosystem of partners and open-source tools.

OpenTelemetry, which is being developed under the Cloud Native Foundation, is a collection of tools, APIs, and SDKs used to instrument, generate, collect, and export telemetry data to analyze software performance and behavior. It’s being developed by contributors from AWS, Azure, Cisco, F5, Google Cloud, and VMware among others.

Cisco expects that the sheer volume of MELT from its ecosystem of enterprises, telcos, and cloud relationships combined with its family of industry partners will deliver it competitive advantage, including development of applications to Excellerate observability, Pereira said.

The Full-Stack Observability Platform will also take advantage of Cisco’ security portfolio and research to provide telemetry that can be included in new applications to control security across multiple domains.

“The ability to map and correlate security telemetry with all of our other information gives us a huge advantage as we look to help customers manage systems across multiple domains in multiple geographic regions,” Pereira said.

An example of this is the announcement last week of a Business Risk Observability application designed to gauge the seriousness of vulnerabilities and prioritize which are most pressing. It combines data from multiple sources to generate a business risk score for applications or services that have a high likelihood of exploitation and attacks. It gathers data from Cisco’s Kenna Risk Meter, business transaction details from Cisco AppDynamics, API details from its Panoptica software, and threat intelligence data from Talos, its security-research arm.

“If customers have 100 vulnerabilities across their network, they don’t want to hear that they need to patch them all because that just doesn’t happen. They need to know which ones to prioritize, and that’s what Business Risk does,” said Tom Gillis, senior vice president and general manager of security at Cisco. “It gives customers a better understanding of the risk of problems and the business impact of fixing them.”

In a highly distributed world, observability must stretch across all points on the enterprise network edge, whether that be user devices, campus and branch sites, multi-cloud endpoints, or IOT endpoints, said Brandon Butler, research manager with IDC’s Enterprise Networks practice speaking at a latest Cisco-sponsored webinar.

“It's important for the visibility platform to be able to enhance automation platforms, to be able to not just identify when there's a problem in the network, but actually either recommend ways to fix it or proactively mak that fix before it impacts users,” Butler said.

Add security to that, because you can't secure the network unless you know what's actually happening in the network, Butler said.

“So observability platforms provide a foundational layer for being able to enhance the security footprint of the enterprise network edge as well,” Butler said.

Taken together, components of the observability technology will drive use cases such as improving performance and optimization enterprise resources, as well as application security, Pereira said.

Inside the Latest Cisco Security Bugs

The two vulnerabilities are a byproduct of a shift in the nature of routing technologies, according to Trellix. Network administrators today have the ability to deploy application containers or even entire virtual machines on these miniature-server-routers. With this greater complexity comes both greater functionality, and a wider attack surface.

"Modern routers now function like high-powered servers," the authors of the report explained, "with many Ethernet ports running not only routing software but, in some cases, even multiple containers."

Both CSCwc67015 and CVE-2023-20076 arise from the router's advanced application hosting environment.

CSCwc67015 reflects how, in the hosting environment, "a maliciously packed application could bypass a vital security check while uncompressing the uploaded application." The check attempted to secure the system against a 15-year-old path traversal vulnerability in a Python module that Trellix itself had identified last September, CVE-2007-4559. With a "moderate" CVSS v3 score of 5.5, it allowed malicious actors to overwrite arbitrary files.

Meanwhile, the bug tracked as CVE-2023-20076 similarly takes advantage of the ability to deploy application containers and virtual machines to Cisco routers. In this case, it has to do with how admins pass commands to run their applications.

"The 'DHCP Client ID' option within the Interface Settings was not correctly being sanitized," the researchers discovered, which allowed them root-level access to the device, connoting "the ability to inject any OS command of our choosing."

A hacker who abused this power "could have a significant impact on the device's functionality and the overall security of the network," Quinn explains, including "modifying or disabling security features, exfiltrating data, disrupting network traffic, spreading malware, and running rogue processes."

The bad news doesn’t end there, though. The authors of the report highlighted how "Cisco heavily prioritizes security in a way that attempts to prevent an attack from remaining a problem through reboots and system resets." However, in a proof-of-concept video, they demonstrated how exploitation of the command-injection bug could lead to completely unfettered access, allowing a malicious container to persist through device reboots or firmware upgrades. This leaves only two possible solutions for removal: a full-on factory reset or manually identifying and removing the malicious code.

Cisco Industrial Gear: Potential Supply Chain Risk

If there's a silver lining to these bugs, it's that exploiting either would require admin-level access over a relevant Cisco device. A hurdle, granted, but hackers obtain administrative privileges all the time from their victims, through regular social engineering and escalation. The researchers also noted how, often, users don't bother to change the default username and password, leaving no protection whatsoever for this most sensitive account.

One must also consider the supply chain risk. The authors highlighted how many organizations purchase networking devices from third-party sellers, or use third-party service providers for their device configuration and network design. A malicious vendor could utilize a vulnerability like CVE-2023-20076 to do some very easy, subtle, and powerful tampering.

The sheer degree of access this hole provides "could allow for backdoors to be installed and hidden, making the tampering entirely transparent for the end user," the authors explained. Of course, the overwhelming majority of third-party service providers are perfectly honest businesses. But those businesses may themselves be compromised, making it a moot point.

In concluding their report, the Trellix researchers urged organizations to check for any abnormal containers installed on relevant Cisco devices, and recommended that organizations that don't run containers disable the IOx container framework entirely. Most important of all, they emphasized, was that "organizations with affected devices should update to the latest firmware immediately."

To protect themselves, users should apply the patch as soon as possible.

Cybersecurity

Networking

Photo: Cisco/Twitter

Show your support for our mission by joining our Cube Club and Cube Event Community of experts. Join the community that includes Amazon Web Services and Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many more luminaries and experts.