By Kelsey Hightower, July 14, 2015
Originally published on the CoreOS blog (archived copy) . Re-published with the authors' permission.
Today we congratulate Sysdig, the container visibility company, on its funding news and launch of its commercial offering, Sysdig Cloud. We interviewed Loris Degioanni, the creator and CEO of Sysdig, about the company, containers and how Sysdig works with CoreOS. He is a pioneer in the field of network analysis through his work on WinPcap and Wireshark, which are open source tools with millions of users worldwide.
Read on to dive in, and be sure to meet Sysdig and our team at our July 29 Meetup in San Francisco to learn more.
Q: In your own words, what is Sysdig? Why is it important in containerized environments?
Loris: Sysdig is an open source system visibility tool, designed to meet the needs of modern IT professionals. You can use it to monitor and troubleshoot things like system activity, network and file I/O, application requests and much more. Unique features include the ability to work with trace files (similar to tools such as Wireshark) and deep, native container support.
As for containerized environments: containers are an extremely interesting and powerful technology – I’m personally a big fan. But containers are also a relatively young technology (at least in their current form), and until now there has been a bit of catch 22 in terms of container visibility. Either you monitor your containers from the outside, with inherently limited visibility, given the opaque and self-contained nature of containers. Or you install extra monitoring software inside the container, which largely undermines the benefits of using a container in the first place – performance, deployability, portability, dependency simplification, security, etc.
Sysdig is the first visibility tool designed specifically to support containers. And in order to truly support containers, we knew we had to solve the issue above. Sysdig’s instrumentation is based on a little kernel module that can capture information like system calls from “underneath” containers. This makes it possible to explore anything that’s happening inside containers, while running sysdig entirely on the host machine or inside another container. There is no need to instrument your containers, or install any agent inside them. In other words, Sysdig provides full visibility into all your containers, from outside of the containers themselves.
This tends to be quite a radical departure from what people are used to, and is also the basis of our commercial product, Sysdig Cloud. Based on this same open source technology, Sysdig Cloud offers a container-native monitoring solution, with distributed collection, discovery, dashboarding, alerting, and topology mapping.
Q: What lessons from contributing to Wireshark influence what you are doing today?
Loris: I spent my Ph.D. and the first 10 years of my career working on network monitoring. The lessons I learned during that time have highly influenced the architecture and underlying philosophies of sysdig.
Network monitoring as a whole offers a pretty elegant set of workflows. First, there is the fundamental ability to capture the information you need into trace files. These trace files are not only easily shared, but maybe even more importantly, they decouple the troubleshooting process from the issue itself. No longer are you working inside of a broken system, trying to fix a problem, as the problem is bringing down the system around you. Network monitoring workflows also include the ability to filter information with well known languages, and visualize your data with industry standard tools like Wireshark.
I believe these workflows are not only relevant in the context of network monitoring. Trace files, decoupled troubleshooting, natural filters, standardized visualizations: these are widely applicable concepts. With our work on sysdig, we are trying to bring these well-proven approaches from the world of network monitoring into the world of system, container and application visibility.
Q: How does Sysdig work with CoreOS environments? What types of information can Sysdig pull from a CoreOS host?
Loris: Sysdig fully supports CoreOS environments, and offers the same 100% visibility you would find in a non-containerized environment. Sysdig works with CoreOS by installing the container we provide, which contains all the required dependencies and offers an isolated execution environment. Since we provide a precompile driver, installation is really easy – it is a single command line and takes 30 seconds.
Once installed, sysdig will be able to surface very rich information about your CoreOS environment: both the host OS and the containers you have running. This includes everything from top processes, to network connections, to top files and directories, to a list of executed commands for both the host OS and any of the running containers. And that’s just the tip of the iceberg. For some interesting use cases with sysdig running in CoreOS environments, you can refer to our two-part CoreOS blog series here and here.
Q: What is the memory and CPU overhead required by Sysdig?
Loris: Typically low, but it depends on what kind of activity is happening on the machine. Sysdig instruments the operating system’s kernel, and the overhead depends on how many events there are to be captured. On a machine with average load, the CPU occupation should be very low: a few percentage points. CPU occupation of sysdig can go higher on systems with a lot of I/O or network activity. The Sysdig Cloud agent, on the other hand, incorporates additional protective mechanisms, such as subsampling techniques, to ensure the CPU occupation always stays within an acceptable range of <5%.
Q: You presented the Dark Art of Container Monitoring at CoreOS Fest this year. Tell us more about what should be monitored.
Loris: In terms of what should be monitored, my answer is: everything! The really important question is: how should it be monitored? The same features that make containers so interesting and revolutionary (i.e. the fact that they are isolated, self-contained, simple and lightweight), make them a real challenge to monitor. In particular, the traditional approach of having an agent on any “entity” doesn’t work well with containers, because it’s too invasive and doesn’t scale.
This is the problem we’re trying to solve with sysdig and Sysdig Cloud. We’re excited about working on it because great visibility is a key requirement to adopt containers in production.
Q: Describe what Sysdig does with CoreOS Linux to help monitor system security.
Loris: Sysdig has powerful security-oriented features. Here are some examples of what CoreOS users can do with sysdig to monitor system security:
Now think about being able to obtain this information for any container running on a CoreOS host, but from outside the container, with no instrumentation and no dependencies.
If you are curious to try sysdig out, installation on CoreOS is super easy and instructions can be found here. And don’t forget to let us know what you think on twitter or at info@sysdig.com!
Attend this month’s CoreOS San Francisco Meetup that will feature the CoreOS team and Gianluca Borello, senior software engineer at Sysdig.
When: Wednesday, July 29, 2015 starting at 6 p.m. PT
Where: Okta, 301 Brannan Street, San Francisco, CA 94107
RSVP: http://www.meetup.com/coreos/events/223897172/