Is Docker Ready for Prime Time?

Docker - ready for prime time or not? It's a question that has been asked (and answered) hundreds if not thousands of times already. So - rather than repeat that long and somewhat tired conversation I want to focus on one piece of the debate - Security.

There's three issues that I see with Docker and Security

1. Because Docker is opensource it has been widely adopted and is almost certainly already deployed, whether you like it or not, inside your organization. This makes for all sorts of security nightmares that the CISO's and their teams are unable to control. What if an employee introduces uncontrolled code to a mission critical stack? What does that do for Compliance, internal audit and Corporate governance issues not to mention liability problems.
2. Many others have covered the point about large attack surface. Thousands of containers vs hundreds of apps, VMs etc. The larger the attack surface the more vulnerable your organization is to internal and external breaches.
3. The very flexibility that Docker and containerization in general provide gives it a massive security hole. What if a rogue employee or external intruder plants a container that launches and East-West attack? Good luck finding that single container in the thousands you have already deployed.

There is plenty of advice out there on how to implement Docker security effectively - this article from Amir Jerbi co-founder and CTO of Aqua Security, is a good basis.

In my discussions with customers about Docker it's clear that, at the Enterprise level, they are just not comfortable yet in adopting Docker. Typical responses include 'Maybe next year,' 'let's wait and see', 'who else is using Docker across their infrastructure?' All good points with limited answers. Look at the list of Docker customers at docker.com. Are these all in production? Let's hope so.

When customers ask me about Docker security I always tell them 'Be careful, move forward in a considered way and you might just end up where you expect to be. If you let it get out of control you will spend a lot of time and money getting Docker under control.' Full disclosure - we offer Docker/Containerization as a service from Alauda We do this because Containerization as a service is intrinsically more secure running on AWS or Azure than letting Docker loose in your Datacenter.

What do you think? Is Docker ready for Prime time?

Big Data and Security - the next big disruptor?

Last quarter I was invited to a Cloudera sales event in Las Vegas. Some impressive stats on last year's performance, a lot of enthusiasm and in particular a great session from Charles Zedlewski @zedlewski outlining some of the product and Apache initiatives coming soon.

Two in particular are now announced 

• the Open Network Insight (ONI) https://vision.cloudera.com/introducing-open-network-insight-accelerating-cybersecurity-analytics-solutions/ ONI is effectively Cloudera's move into core security functions such as threat detection.
• Apache Arrow https://arrow.apache.org/ Arrow leverages latest SIMD (Single input multiple data) operations optimization of analytical data processing. Arrow is Cloudera's attempt to gain control of in-memory columnar data processing.

So far, so good, but these two announcement will make a huge impact in the IT Security market. for sometime now there has been little innovation in Security. the main players are all offering incremental enhancements to technology that has been around for years.

Big Data and the Hadoop eco-system can (and already has) disrupt the ITSec market. Principally it's a cost/scale dynamic. SIEM's, Vulnerability Management, Configuration Management tools and others are essentially about reacting to events that have already happened. they also use Metadata structured repositories to normalize, correlate and report. Look at any SIEM vendors details and you will see this common theme. Detect and fix something that has already happened. 

With Hadoop and it's various components and, in particular, the continuing path to maturity in machine learning products, this old style architecture is going to disappear. Sometime between now and 2020 the Enterprise Security Warehouse concept will be widely adopted. All data from all sources poured into a massive data lake (in real-time of course), with an HDFS/Kudu style repository for persistence and machine learning algorithms constantly monitoring what is happening and taking appropriate action as the threats happen  not after they happen. Gartner predicted this back in 2014 so it must be true..... http://www.gartner.com/newsroom/id/2778417 

In our discussions with clients we see a gradual realization, usually in the biggest clients first, that the old style Security Architectures have failed to keep up and new architectures built on big Data eco-systems and machine learning in particular, offer the greatest potential for the next disruptor. Look at how Splunk has built a $600m business on just this premise but without the machine learning part.

For an alternative view of ML and Security read Matt Harrigan's post@mattharrigan at Tech Crunch. http://techcrunch.com/2016/02/29/machine-learning-is-not-the-answer-to-better-network-security/ 

What do you think, is Machine learning already the big disruptor in Cyber Security?

The Open Source Vendor game

The latest in a long line of struggling Data Warehouse companies announced their results  on 5th November. Teradata had flat revenues and flat to slightly declining revenues predicted for the year. Look at the Dell/EMC idea, HP''s continuing machinations and flat revenues, IBM's problems (including the strange decision to buy .... The Weather Channel , Oracle's well documented problems  and it's clear the seismic shift in Enterprise software that started with AWS and continues with Hadoop by way of open source continues to grind down the Proprietary Enterprise software vendors.

As Mike Olson put it over 2 years ago - the Proprietary vendors now have no hope of competing. They don't have the resources, the knowledge or the skills to compete with globalized software development teams running Apache projects or similar.

So when a client asks us - what does this all mean? Why is everything moving so fast? Why can't we get back on the old familiar proprietary software merry go round? All this uncertainty makes customers very nervous about what to do, and in some cases they therefore..... do nothing.

When a client asks this question we tell them three things :-

1. Look at what is happening in the critical Apache projects. Take Hadoop, or Spark, or Kudu or many others. Look at the trend in committers to see which projects have a solid bench of committers and is growing.
2. Secondly - look at where these committers are doing their day jobs. Here is a good example of Spark. Scroll down the list and you will see it's solidly dominated by Databricks, with a good mix of UC Berkeley (where Spark was developed), Intel (watching their investment in Cloudera) and, of course Cloudera.
3. Thirdly - if you are intent on looking at projects still in incubation - be careful. Hadoop Development tools  seemed like a good idea but failed to get any sort of traction with the community.

The last piece of advice we give clients is always look at what the market leaders are donating to Apache. Last week Cloudera announced they want to donate Impala and Kudu to Apache. Impala has been widely adopted and so Cloudera want the community to take over some of the development work. Kudu is a big complex project so, now that it's architecture has been broadly set, Cloudera want the bigger, more agile, more skilled worldwide community to shoulder some of the development load.  

For many clients, used to easy decision making provided by Gartner and others, Open Source adoption in the enterprise and in the data center in particular is changing everything. 5 years ago and certainly 10 years ago clients would insist on Roadmaps under NDA, long term support with access to source code if the vendor went out of business, long complex discussions about pricing and licensing and entry into elite customer user groups. Not one of those types of discussion happens any more in Open Source and Big Data. No roadmaps (it's up to the community to direct the roadmap so join in!), source code is already freely available, licensing is easy - it's free!, 

We believe clients need to adapt not just their technology options but also their vendor outlook to get the most out of the Open source world.

What do you think - are customers able to adapt to the changing market and technology landscape? Or they stuck in a time warp - working in a style and with a market view that no longer exists?

And take a look at the Platinum sponsors of ASF - they are sponsoring for a reason of course.