RafStack: Rafael's OpenStack Blog

Over the last few months I did a series of interview with OpenStack board members, representing various types of organizations. Hence, allow me to share my key takeaways from those interviews: Why does OpenStack matter? OpenStack matters, because:

Enterprises want Amazon and VMware alternatives
“There was a lot of pressure in the market to have an alternative ecosystem to Amazon as a public cloud and to VMware as a licensed internal cloud.” Rob Hirschfeld, Principal Cloud Architect at Dell. As for the public cloud, hosting companies such as Rackspace and Dreamhost want to create an alternative ecosystem to Amazon AWS in order to differentiate through service. Private cloud users on the other hand, are seeking for alternatives, where costs do not scale linearly as they grow their infrastructure.

Cloud is open by nature …
“Cloud in general is a baby of the open source culture.” Boris Renski, Co-Founder and EVP at Mirantis. Large consumer internet companies such as Amazon, Facebook and Google built their cloud infrastructure out of components they invented and then later on outsourced as well as on already existing open source solutions:  “These companies understood that if they took the traditional enterprise route, the price for licenses ultimately would be greater than the revenues they could ever achieve. So they built the superefficient infrastructure stack completely leveraging open components and paying licenses to nobody.”

… and so is OpenStack
“From day one it was not positioned central to any particular vendor but as a conglomerate of different independent organizations. Because cloud is about open, and OpenStack is THE thing in the open cloud, it is effectively going to be one of the most disruptive movements in infrastructure computing during the next 5 to 10 years. OpenStack is going to change the entire industry upside down.” Boris Renski.

OpenStack matures at rapid pace
“Two years ago OpenStack was more of a promise than a reality. We had a production grade object storage environment, but Nova, the compute project was at best a couple thousands lines of code. Here we are now 600,000 lines codes later with hundreds of contributors from nearly a hundred countries. It’s amazing to see the progress we made in maturing the product. At Rackspace, we’re using that code to power the world’s second largest public cloud … and there are a lot of diverse use cases such as MercadoLibre, eBay and PayPal to name a few.” Jim Curry, GM Private Cloud at Rackspace.

Its fundamental architecture is very sound
“If you want to build Infrastructure as a Service (IaaS) in a scale out manner, then you need an asynchronous, loosely coupled, message based type of solution, so that you can create a distributed software system.”Randy Bias, Co-Founder and CTO at Cloudscaling. OpenStack meets those expectations, unlike many other open source IaaS projects.

It’s driven by a diverse, huge community
“We have probably the most dynamic, engaged and diverse community ever. Linux had a perfect recipe of academic partners, enterprise partners and non-profits that had really moved the project forward. OpenStack has that same mix of academic users such as CERN and NeCTAR in Australia, and commercial users like eBay, Sony and PayPal, and a certain amount of non-profits like Wikimedia as well as big entities like IBM, HP and Intel.” Joshua McKenty, Co-Founder and CTO at Piston Cloud Computing.

OpenStack is ready to scale
“One of the developments we have been watching very closely has been the cells development.  Clearly a number of sites are pushing the thousand plus hypervisor scale at the moment, but the key break through will be with the OpenStack Grizzly release when the cells functionality is there, and this will allow us to construct hierarchies of cells of compute resources. This would remove one of the major limitations in terms of the total scalability.” Tim Bell, Infrastructure Manager at CERN.

If you want to learn about OpenStack, its components and capabilities please go to the OpenStack Foundation website. And if you have the opportunity to join the crowd in person, visit the upcoming OpenStack Summit in Portland (OR).

Can you explain briefly what Chef is?
Opscode Chef is an open-source systems integration framework built specifically for automating at scale. Using Ruby-based ‘recipes’ and ‘cookbooks’ of code commands, Chef makes it easy to deploy servers and scale applications throughout an entire infrastructure. Through a combination of configuration management and service-oriented architectures, Chef makes it easy to create fully automated infrastructure, while simplifying systems management. Chef is available as an open source download, a SaaS subscription, or as software installed behind the user’s firewall.

What’s different about Chef compared to other automation solutions such as Puppet?
Puppet provides a declarative model for systems administration. It works well in small, mostly static environments, with low complexity. Opscode Chef provides a more flexible automation framework that allows enterprises to model their current workflow, at any scale, from development through to deployment and operations. Chef is based on primitives that create patterns that can be bent to any workflow/environment. Puppet is based on declaratives that can’t be changed. 

You recently released Chef 11. What are the major enhancements compared to the previous version?
Chef 11 was re-written from the ground up and leverages best-of-breed infrastructure technologies including the Erlang programming language and PostgreSQL database, delivering a rock-solid automation platform that can easily scale up to 10,000 nodes from a single Chef server – which is far greater than any previous Chef generation. Opscode is also two tiers of commercial support for open source Chef users (who are running Chef 11) covering both live system support and cookbook code troubleshooting. Other enhancements include:

• Comprehensive Testing: Chef 11 features the Pedant Testing Suite, delivering robust testing capabilities that can be executed with a single command, automating more than 2,000 end-to-end tests that ensure the Chef server is installed and working properly.
  
  
• Easy Installation: Chef 11 comes packaged with a ‘one-click’ installer, enabling easy and rapid deployment of Chef regardless of IT environment.
  
  
• Enhanced Windows Support: With the Pedant Testing Suite, Chef 11 includes automated testing across seven different versions of Windows, improving functionality and integration within Windows environments.

What resources (blogs, webinars, events) do you provide to get a solid technical understanding of Chef?
The most visible resource is the open source Chef Community, which is an important, active and vibrant online community where users can find recipes and cookbooks for everything from Windows to Hadoop, as well as a wide range of best practices, instruction guides, and more. However, the most important and helpful resource is likely our Documents page, where users can find out everything they need to know about Chef, from getting started, to basic deployments, to advanced use cases, recipes, cookbooks, patches and more. We’re working hard to make our Documents page your one-stop-shop for all things Opscode Chef.

Who owns Chef?
Chef is an open source systems integration framework stewarded and licensed by Opscode.

Which language is Chef written in?
The back-end API is written in Erlang. The front-end is Ruby.

How is Chef licensed?
As a free, open source download, a SaaS solution, or as enterprise software installed behind the firewall. The latter two are commercial solutions sold through a subscription model.

How many contributors / commits do you count?
Over 1,000 individual contributors, 170+ corporate contributors and tens of thousands of registered users. Open Source Chef has been downloaded nearly 2 million times in less than four years of availability. 

Which functionalities is the community particularly focused on?
That’s a tough question to answer because the Chef Community is so diverse, active and large that there are few “core focus” points. That said, ensuring Chef works seamlessly with MySQL, Apache and the many public cloud providers are frequent topics of conversation and code contribution, as is Chef + Windows.

Can you give us examples of typical use cases for Chef?
We call it the “three C’s of Chef”: Configuration Management of servers in physical data centers, private and public clouds; Continuous Application Delivery in any environment; and Cloud Management for public, private and hybrid clouds. The vast majority of use cases for any of the three flavors of Chef fall into one or more of these categories. We have solution pages on each of these use case available here, as well as a wide range of customer success stories available here.

Which prerequisites should enterprises meet when interested in using Chef?
There are no specific prerequisites needed to use Chef. However, familiarity with Ruby within IT and your Dev teams will be helpful, as is a willingness to deploy infrastructure as code for greater agility and less risk. 

Tell us about your collaboration with Dell and Chef integration in Crowbar – Dell’s deployment mechanism for OpenStack.
We’re excited that Dell has embedded Opscode Chef in Crowbar and are very appreciative of the patches and cookbooks Dell has contributed back to the Chef Community. Dell has been a great partner in the OpenStack project and we look forward to more collaboration in the future.

Thank you, Lucas.
You’re welcome, Rafael.

Rafael: Can you give us an overview over your trainings? What’s your offering?

David: About a year ago, we announced our 2-day Bootcamps for OpenStack program, to get people deep fast on OpenStack. Since then, more than 250 engineers have been trained through our Boot Camps. We offer them in public monthly, so developers and engineers learn about the open source cloud operating system that has rocketed to popularity throughout the industry. We also offer them as private on-site training for companies that want a dedicated training.

David: Since there’s a lot of hype around OpenStack, we wanted to avoid something that was all theory. So our instructors are Mirantis engineers who are active code committers to the OpenStack project and also consult for some of the most notable companies using OpenStack today – including PayPal, The Gap, AT&T, WebEx, HP, Dell and NASA. We developed the course content based on our experiences of real-world implementations of OpenStack; we feature lectures, hands-on labs, and one-to-one coaching. Upon successful completion, students are able to stand up and trouble-shoot an OpenStack cloud.

David: The high demand for the Mirantis training is more evidence of the growing adoption of OpenStack and the need for skilled engineering talent and expertise in OpenStack. Typical students are developers and systems administrators and IT professionals from SaaS vendors, service providers, enterprise IT … everyone who needs to build practical skills to put harness OpenStack’s advantages to their cloud effort.

We expect attendees to be comfortable with Linux CLI, have a good understanding of virtualization and hypervisors, and have some experience with Linux networking.

Rafael: Which are the most popular topics around OpenStack?

David: Our goal with the Boot camp is to cover the information you need to get started on an OpenStack cloud deployment. It includes overviews of OpenStack and OpenStack Networking, use cases, basic operating and deployment principles, cloud usage patterns, Swift Object Storage, OpenStack in production, and advanced topics such as software defined networking, deployment and issues workshops, and comparing VMware and OpenStack. We provide each student with a high-powered laptop, and then network them all together as a ‘tabletop cloud’.  SDN – with a view of networking before and after it – is an especially popular topic.

By “cover”, I don’t mean just death by slideware; about 50% of the class is labs and discussion. This is stuff you have to learn by doing it, and we have designed our labs to put that understanding to the test. For example, we have a break/fix session where the instructor will introduce some kind of flaw into the cloud that students have built earlier in the day, and pit them against one another to see who can troubleshoot the problem first. Because our instructors learned the technology the hard way – deploying OpenStack in successive releases dating as far back as Cactus – they are also well versed in what works and what doesn’t work yet.

Rafael: How can people register for trainings / keep up to date on upcoming trainings?

David: Register for the February 21-22 and March 20-21 Bootcamps for OpenStack at http://www.mirantis.com/training/. There’s another session in March listed there as well; we typically hold them each month in Mountain View, CA. It’s important to note that these public Bootcamps are so popular that they usually sell out well in advance, The January Bootcamp was the tenth consecutive sold-out public session.

We also schedule special workshops during certain industry conferences. As I mentioned earlier, we’ve delivered the bootcamp format in on-site classes for more than a bunch of companies to train staff at their own location. Class descriptions and reviews from past class participants are at www.mirantis.com/training.

Rafael: Thank you, David!

David: You’re welcome.

Learn firsthand about OpenStack, its challenges and opportunities, market adoption and 99cloud’s engagement in the community. My goal is to interview all 24 members of the OpenStack board (including former members like Ben), and I will post these talks sequentially at Dell TechCenter. In order to make these interviews easier to read, I structured them into my (subjectively) most important takeaways. Enjoy reading!

Rafael: What are the key accomplishments in OpenStack so far?

Ben: The key accomplishments it made was changing the world of cloud computing.

As for the service providers, not only the big players can provide cloud services, even small teams have the opportunity to use related technologies to provide the services.

As for the users, if they don’t want to be ensnared by vendor lock-in, they can try OpenStack. Cloud is inherently about open, and OpenStack was born in an effort to make technology more collaborative, affordable and available to everybody.

And last but not the least, it is the OpenStack Foundation. I think we should be proud of setting up the OpenStack Foundation!

#1 Takeaway: It’s all about usability … listening to users and incorporating what they ask for

Rafael: What still needs to be worked on? What are the “child diseases” that OpenStack has to cope with?

Ben: Winning the trust of its users will be key to our success. The OpenStack Foundation must engage with users and listen to their needs while also investing in user integration into the community. It will provide the most support and contribute the greatest value to the platform that draws the most customers.

The community-driven development must offer more stability process through open Design Summits. The changing nature of open source web-based development processes should play a positive role.

Rafael: What is the vision for OpenStack? …or what should be the vision in your opinon?

Ben: It is really about openness and the open source cloud mission. Openness is not only about open source alone but also the way OpenStack is applying to create cloud infrastructure. The open model is on the verge of being extended the collaboration and design process where collaboration and knowledge sharing extend beyond software. Other projects have donated to the Apache Foundation or opened the source. But OpenStack does more than that, it has set setup a new FOSS (Free and Open Source Software) Foundation. It’s big news for the open source community.

Rafael: What are the major challenges ahead of the Board of Directors?

Ben: First, feedback is needed from users about what needs to be improved and which features should be added. If we can define a standard policy and handling exceptions, it will be easy for users to give clear feedback on potential impacts and discuss roadmaps for changes to limit the disturbance.

Second, in cloud open standards will play a critical role in how companies are architecting for the increased demand, scale and security required by cloud deployments. We look forward to collaborating with OpenStack Foundation and the community on the OpenStack project, as well as other open source standards bodies.

#2 Takeaway: The government wants open cloud, a broad open source community wants it and customers want it – OpenStack has definitely a stron momentum in China

Rafael: Can you please explain why OpenStack is so popular in China?

Ben: I couldn’t imagine the popularity in China today. When I organized the first events Shanghai OpenStack Conferece 2011, 400 participants came from across the country. After the conference we saw many companies joining China OpenStack User Group(COSUG). The number of members has increased from less than 100 to more than 1000. There are mainly two reasons for OpenStack’s popularity in China:

First, it’s good to see the government embrace open source software - though some members of the open source movement will feel a bit queasy about that. But the government is massively promoting open source, and their expectation is that within the next five years the development of cloud computing in China will further broaden.

Second, it is the power of the open source community in China. Through the power of community crowdsourcing, cloud computing software development accelerates and becomes more efficient. The cloud computing community is very active in China.

Third, using open source software significantly reduces deployment cost, and this is consistent with the direction of the development of cloud computing, and Chinese enterprises follow this pattern.

Fourth, most of the internationally leading enterprises use open source as their cloud platform, which immediately affects corporate decisions in China.

So, open source cloud platform will develop rapidly, and OpenStack will be widely used in China.

#3 Takeaway: Large Chinese internet companies deploy OpenStack on up to thousands of servers

Rafael: What are the adoption patterns for OpenStack in China?

Ben: Some people complain about OpenStack not being production ready, but I see some internet companies have already deployed OpenStack. Sina.com  has made some deployments, and we see other large internet companies deploying up to thousands of servers using OpenStack.

Rafael: What challenges does the OpenStack community face in China?

Ben: While the community is thriving a lack of sufficient support is limiting growth. Many enterprises adopted open source software, but they lack willingness to share more technical details. They are currently trying to tread a nice middle ground between completely embracing the open source community and keeping control over software it has developed.

#4 Takeaway: Challenge to future of OpenStack in China – growing number of contributors needed on the project

Rafael: What are the opportunities?

Ben: With the OpenStack Foundation actively working on open and transparent governance, the real challenge now is to grow more contributors and technical resources to fill out projects. We have a lot of good developers in China. And as mentioned above, given the vast growth potential of the market, it is assumed to be a pretty good prospect. Foreign companies operating in China have been quick to see this potential but they are largely unable to grasp it.

#5 Takeaway: 99cloud is contributing through code and evangelism

Rafael: How does 99cloud contribute to OpenStack?

Ben: I agree with your interview with Boris Renski. There are two sorts of contributions to OpenStack: writing code and evangelizing the project, with the latter being even the more important.

99cloud is pleased to participate as a member of the OpenStack community. At 99cloud, we were one of the original organizers of the OpenStack community in China. We initiated and lead a community project trystack.cn, and we donate to the Foundation.

99Cloud’s interest in OpenStack derives from community and customers who need local support. OpenStack should be sufficiently stable for customers running production clouds. 99cloud is focusing to provide cloud solutions and products using OpenStack for customers in China. We know from community events that most of the customers who want to use OpenStack not only need an open source project but they expect it to be production ready. So, our goal is to create value for the customers, especially for the Chinese users.

Furthermore we want to donate Trystack.cn to the OpenStack Foundation, but there are legal issues we have to clarify, we are working with OpenStack Foundation Community Manager Stefano Maffulli on that.

Rafael: How do you intend to monetize on OpenStack?

Ben: OpenStack is an open source technology, while cloud service is a business model. There will be a lot more “Powered by OpenStack” software and appliances in five years. Software and hardware vendors will treat OpenStack as a de facto open standard platform, and they will naturally develop and support drivers and applications for this stack. Those customers who are deeply involved in the community ecosystem want greater cloud choice/flexibility without vendor lock-in, and the ability to customize the solution to meet their customers’ needs.

The best thing about open source software systems has always been the fact that it is freely available and any programmer or company can use it to develop its own version of that software. The end users get exactly what they needed and are willing to pay for it. Companies that use OpenStack a lot and generate enough revenue from it  can afford to outsourcing service. And the specialists of OpenStack need to be trained, tested and licensed by a valid authority which will always need to route back to the service providers.

Rafael: What is your view on Hadoop in conjunction with OpenStack?

Ben: I have been reading some news on the web about elephants to join the OpenStack cloud. But to me, the business model in China is not very clear. From a Hadoop cluster to a Hadoop cloud … will it be a SaaS? In any case, 99cloud can provide the deployment service.

Rafael: Can you name resources (both in Chinese as well as in English) … such as blogs, community pages etc.?

Ben: First of all, I want to mention Trystack.cn. It’s a community project, the largest OS testing and Showcase Platform in China, built for the newest OpenStack Folsom release. It’s partnering with Intel and VMware for physical and marketing resources, it was announced during the the San Diego Design Summit 2012. They are working with the Cloud Foundry community in china to provide OpenStack deployment environments with the newest code now.

I also recommend http://openstack.csdn.net/.  There is a large number of Chinese articles about OpenStack. And also there are a lot of bloggers such as http://www.chenshake.com/and also good OpenStack manuals.

Also, it’s important to mention Ken Pepple, a good writer who wrote the book on OpenStack – “OpenStack Cloud Computing Cookbook.” The book is structured into questions on how to do things with OpenStack and the answers guide the readers through a series of repeatable steps. I am translating this book now, and it will be published this year for Chinese users.

Rafael: Ben, thank you very much for this interview.

Ben: You are welcome!

Resources

Trystack Google Group: https://groups.google.com/d/forum/trystack-china
Twitter: @OpenStackChina
Presentations: Slideshare
Weibo: @COSUG(China OpenStack User Group) or @trystack
LinkedIn Group: openstack-china

Feedback

Twitter: @RafaelKnuth
Email: rafael_knuth@dellteam.com

Learn firsthand about OpenStack, its challenges and opportunities, market adoption and Rackspace’s engagement in the community. My goal is to interview all 24 members of the OpenStack board, and I will post these talks sequentially at Dell TechCenter. In order to make these interviews easier to read, I structured them into my (subjectively) most important takeaways. Enjoy reading!

#1 Takeaway: The world wants an open cloud

Rafael: What are the key accomplishments of OpenStack so far?

Jim: First, the transition to the foundation. Setting up a foundation is a big move, and technically it removes OpenStack from Rackspace to the community, which attracts a large number of companies to the project. I think we should be proud of how the community comes together to work very well.

Second, two years ago OpenStack was more of a promise than a reality. We had a production grade object storage environment, but Nova, the compute project was at best a couple thousands lines of code. Here we are now 600,000 lines codes later with hundreds of contributors from nearly a hundred countries. It’s amazing to see the progress we made in maturing the product. At Rackspace, we’re using that code to power the world’s second largest public cloud … and there are a lot of diverse use cases such as MercadoLibre, eBay and PayPal to name a few.

Third, the world has decided that open matters in cloud. People are rejecting a closed cloud model, and that’s why we’ve seen so much traction on OpenStack.

#2 Takeaway: The OpenStack community needs to focus more on usability of OpenStack

Rafael: What needs to be worked on in OpenStack, Jim?

Jim: The first two years were a race for features. The project was dominated by developers and not as much by users … and that’s ok, that’s where we are with OpenStack are right now.  The tradeoff is that you get code that is not necessarily that stable, the product is not necessarily as usable and certainly one that requires you to be an expert. I believe slowing down the process of innovation and focussing more on stability and usability is a really critical goal at this point: making sure that the way upgrades are going to occur from one release to another is fully thought-out, enabling chargeback functionalities … these are things the community just starts to work on.

In terms of features … we now have block storage capabilities built into the latest release as a separate project. It’s hard to imagine building an enterprise-grade OpenStack deployment without these capabilities. Then, virtual networking capabilities are a huge step forward … especially for us as a service provider where you scale massively it’s extremely important.  I think most of the major components that are needed are built into OpenStack by now.

But we need to find an answer to the question: What is the definition of OpenStack? It’s a brand that encompasses a broad range of subprojects … what should be included in core? How should we think about incubation?  All these things are very important … not only in terms of what we are going to build with OpenStack. The community needs to understand what the scope of OpenStack is, so that they frankly can have the opportunity make money around it.

#3 Takeaway:  A broad ecosystem of OpenStack distribution tightly connected to Linux distros accelerates market adoption

Rafael: There are a lot of distributions popping up around OpenStack … what trends do you see in this area?

Jim: Some people call what Rackspace does a distro …  I would say it’s rather a packaging. We are trying to make OpenStack trunk easily consumable by a non-OpenStack epxert, by making cloud up and running very quickly. People can use it however they want without being tied to a license or support. At Rackspace we make money by providing services on top of that, and some companies are following a similar model.

Some folks are following the traditional Linux distro model by putting OpenStack distros together with Linux distros, which is a well known and established process. Almost every major Linux distro includes OpenStack at this point, which is great in terms of a broad market adoption.

Also, some companies are taking OpenStack and doing proprietary work around it to solve specific use cases or to provide differentiation.

At Rackspace, we firmly believe that one of the promises of OpenStack was to make it easy to consume. It’s an approach very similar to the way you consume Linux. You don’t consume the Linux project, you do so through one of its many distributions. I think with regards to OpenStack, we want people to get as close to consuming trunk as possible, thereby making OpenStack truly open and free.

#4 Takeaway: 25 % of all Fortune 100 companies in the US have downloaded Rackspace’s OpenStack Private Cloud software within the first 45 days after the release

Rafael: Let’s talk about market adoption, Jim. Do you see signs of OpenStack going mainstream?

Jim: Honestly …  we are still a bit away from mainstream, I think that also holds true for cloud in general. At Rackspace, we have two OpenStack products which I can speak of … public and private cloud. Public cloud is still a small share of IT spent and I think that the large enterprises, the traditional IT buyers are just starting to get their hands around and how to consume public cloud and how build on it. Previously it was mostly developers and startups, but it’s certainly going mainstream.

Private cloud is even a little bit further behind in terms of being adopted by companies. But at this point, what has happened is that mainstream CIOs decided that the architecture of the future is cloud. Forget whether it’s in a public cloud or in their datacenter … the concept of consuming physical resources via APIs and building those APIs into applications is the way people are thinking about IT in future. That’s now worked into almost every CIO’s plans.

Rafael: Who are the current early adopters of OpenStack?

Jim: -The most obvious early adopters in the industry are financial services, large enterprises that do experiment with those emerging technologies.

Besides that, at Rackspace we have a good mix of other businesses eager to try OpenStack because of its promise in terms of service model, cost and speed to market.

We just recently released our private cloud software to the market. Over the first 45 days since the release 25% of the Fortune 100 companies in US downloaded it. Over all, we had downloads from 125 countries from all continents. We see a very broad interest in OpenStack.

#5 Takeaway: China is leapfrogging into OpenStack (just as they did with cell phones by skipping landlines)

Rafael: What regions are adopting OpenStack? China seems to be very keen on OpenStack …?

Jim: For certain the biggest market right now is the United States, second biggest market is China … both in terms of contributions as well as commercial interest. Third would be other South and East Asian countries such as India and Japan. Europe is certainly a little bit less … but certainly I would say US, China and broader Asia are showing the most interest in OpenStack at this point.

Rafael: What’s the reason for China’s massive involvement in OpenStack?

Jim: I started working in China with Rackspace about 5 years ago. They didn’t know anything about hosting and the managed services space at all at that time. They didn’t know much about cloud, and the evolution of knowledge has been substantial over that period of time.

When you take a look for example at telecommunications in countries like China: they basically skipped landlines and went straight to cell phones. In part that is happening in the cloud. China didn’t have a large IT infrastructure five or ten years ago, and many enterprises are jumping right into the cloud.

Certainly in a market like China where cost is a concern, where access to technology is a concern, OpenStack is of interest with code for all to have. Actually a significant number of companies that are significantly contributing to OpenStack as well as early adopters and deployers are based in China.

#6 Takeaway: Dell’s great advantage is the broad customer base and the trust these customers put into Dell

Rafael: Jim, how do you view Dell in the OpenStack game?

Jim: Dell is one of the first companies I called when we started OpenStack. We invited Dell to take a look at what we were doing in spring of 2010, even before we announced it. Dell participated in the first OpenStack Design summit, which at that time wasn’t open to everybody … we invited jointly with NASA 25 companies to brainstorm about the project.

Dell from the very beginning brought in a very impressive group of people to work with us on OpenStack. Dell has abroad reach, they know enterprise customers very well, and they engaged with customers very early, helping them to understand and adopt OpenStack. They contributed around OpenStack with Dell Crowbar, and they are very good community participants.

Dell sells to a lot of companies, which trust Dell on how to design and build IT infrastructure for the future. Dell is using that opportunity to bet on OpenStack, and it makes me really happy to see that happen.

Rafael: Thank you very much, Jim. It was a pleasure talking to you!

Resources

Rackspace: http://www.rackspace.com/cloud/openstack/

Feedback

Twitter: @RafaelKnuth
Email: rafael_knuth@dellteam.com

Learn firsthand about OpenStack, its challenges and opportunities, market adoption and Cloudscaling’s engagement in the community. My goal is to interview all 24 members of the OpenStack board, and I will post these talks sequentially at Dell TechCenter. In order to make these interviews easier to read, I structured them into my (subjectively) most important takeaways. Enjoy reading!

#1 Takeaway: OpenStack fundamental architecture is very sound

Rafael: Let’s start with a very basic question, Randy. What is the value that OpenStack brings to the table?

Randy: The core value of OpenStack is that the fundamental architecture is very sound. If you want to build Infrastructure as a Service (IaaS) in a scale out manner, then you need an asynchronous, loosely coupled, message based type of solution, so that you can create a distributed software system.

We looked at a number of different technologies on the market place today including CloudStack and OpenNebula, and we missed a combination of both components. In case of CloudStack, where we did one of the three largest deployments, the code base is monolithic. All the code runs in one place. And in the case of OpenNebula you have distributed software, but it lacks an asynchronous message passing paradigm.

#2 Takeaway: OpenStack key accomplishments are: 1) high profile contributors, 2) regular release cycle, 3) governed by a foundation and 4) broad suit of services

Rafael: What are the key accomplishments of OpenStack so far?

Randy: First, a number of high profile appointments like AT&T, HP and Dell. Second, a regular cadence in the major release cycle, which is a key factor in any large open source project. OpenStack has a six-month cadence with six major releases so far, and it’s becoming better and better with every new version. Third, the formation of the OpenStack Foundation, a single neutral entity that can support OpenStack and its mission, rather than being owned by any one organization. Fourth, OpenStack branched out from the core of Nova and Swift into Nova, Cinder, Horizon, Glance, Swift, Quantum and Keystone in the current OpenStack Folsom release. With that, OpenStack has a full end-to-end suite of services you can pick from, whereas other IaaS open source projects are focusing on a single piece of the problem like compute or networking, for example.

#3 Takeaway: OpenStack needs a broader market adoption and a clear mission

Rafael:  … and what does still needs to be worked on in OpenStack, Randy?

Randy: We still need significantly more adoption in the marketplace, although I don’t know how to quantify that well. A lot of people are in the process of evaluating OpenStack, and I think that a broader market adoption is just a matter of time.

Also, since the formation of the OpenStack Foundation, there’s an opportunity to re-evaluate what the mission for OpenStack should be. Initially the mission for OpenStack was defined by Rackspace. Now that Rackspace doesn’t own OpenStack anymore, the project’s mission needs to be clarified.

#4 Takeaway: OpenStack Foundation has two options: 1) a hands off and 2) a hands on approach to govern the project

Rafael:  What are viable options for OpenStack?

Randy: One option would be a similar approach to how the Linux Foundation exists in the market place.

The Linux Foundation doesn’t really say what Linux should be. It allows the marketplace and the key contributors to make those decisions themselves. Linux has a reasonable amount of standardization across the distributions, but yet all of them are designed for completely different purposes. RedHat Enterprise is seen by many as designed for servers in the way that Ubuntu is for desktops. There are even more specialized Linux distributions for things like embedded systems.

That sort of hands off approach would mean that OpenStack becomes a framework that can be used by anybody in the ecosystem to build all sorts of clouds. In that case the market forces would determine which distributions rise to the top and which don’t … just like is the case with Linux.

Alternatively, the OpenStack Foundation could take a hands on approach and make top down determinations about OpenStack standards … what OpenStack supports, which other cloud software it’s compatible with etc. That approach might get us sooner to interoperability with other clouds such as Amazon Web Services or Google Compute Engine, but it may alienate those members of the community who don’t want to go in the direction which the OpenStack Foundation and Technical Committee may decide makes sense. These folks might then want to build their own system, and that might create a threat of forking OpenStack.

Rafael: Hands off or hands on: Which approach would you prefer for OpenStack?

Randy: I have seen in the past that standards in interoperability seem to bubble up through market adoption. Once customers decide what they want, developers can respond to that rather than trying to predict what the market might want. We all know: Human beings are the worst predictors of the future (laughs).

#5 Takeaway: Cloudscaling focusses on improving OpenStack’s computing capabilities and API compatibility with other public clouds

Rafael: Let’s talk a bit about Open Cloud System – your company’s OpenStack distribution. What makes it unique in the marketplace?

Randy: Before I answer that question let me explain briefly that we use 100% stock OpenStack in our product, Open Cloud System. We don’t modify it, we don’t fork it. Our team has a background in building large scale out, production grade systems that are compatible with other clouds, and that’s our primary focus with OpenStack. We’ve built an integrated system solution rather than a bunch of separate components.

OpenStack from our point of view is a great technology, but it’s a little bit like the Linux kernel. You probably wouldn’t take it and run it in production. Just as with the Linux kernel, you actually have to do a number of things in order to make it robust and production ready … and by production ready we mean that there is a focus on availability, security, performance and maintainability.

OpenStack has lot of configuration parameters, and we simply took advantage of that in order to provide functionality that isn’t available in default OpenStack. Let me give you one example.

Default OpenStack comes with Nova, the compute component. It has two deployment modes. The first deployment mode is a centralized service, that all of your network traffic goes through. It has VLANs behind it, and every tenant of your cloud is on a VLAN. The challenge is that you have a central choke point for all your traffic. Regardless of how fast your switch fabric is … you’re driving traffic for all your VMs through this central Linux box which is obviously not ideal.

The second deployment mode is distributed, where you are also using VLANs … with your Nova network controller running on all your hypervisors. Obviously there are security concerns around that. But taking that aside, you end up making some compromises, because of the Nova network controller architecture with both public and private IPs running on your internal switch fabric. A lot of people don’t like to do that, they prefer to keep their public IPs at the edge of the network. In addition to that you have these weird bugs that crop up, because the network address translations are happening on each of the hypervisor nodes, instead of at the edge of your network like you would normally expect. There is a bug right now in OpenStack which is not fixable because of the architecture and you cannot use floating IPs.

This is really clunky. Open Cloud System takes a very different approach. We don’t use any VLANs. We use Layer 3 network routing just like Amazon Web Services does. Our networking model looks exactly like Amazon Web Services Elastic Compute Cloud (EC2).

We have a separate NAT service that runs at the edge of the system. Your public and private IPs are not running together on the switch fabric. We have a distributed DHCP service that runs on all hypervisors; the Nova network controller is off to the side and no network traffic goes through it anymore. Because of that we get full end to end bandwidth between all the VMs and tenants, and we get full throughput to the internet ingress and egress.

Rafael: Can you give some more examples of what your distribution does differently than raw OpenStack?

Randy: Sure. Default OpenStack comes with RabbitMQ as the messaging broker, and we felt that created a single point of failure which didn’t make sense to us, especially in larger deployments. So we came up with an Alternative Approach to OpenStack Nova RPC Messaging.  

We also spent time on pushing code back into OpenStack for API compatibility; we do a lot of work on AWS EC2 API and we recently announced that we are providing a set of APIs that are compatible with Google Compute Engine. Now people have a choice when they use OpenStack compute project Nova, whether they want to use the OpenStack native API, the AWS API or the GCE API.

#6 Takeaway:  Early adopters such as the finance sector are embracing OpenStack, businesses with less sophisticated IT needs might jump in later

Rafael: Randy, we talked very briefly about market adoption earlier. Let’s dwell a bit more on that. Who are the early adopters and do you see signs of OpenStack going mainstream?

Randy: Cloud computing is a new paradigm in cloud computing which is driven by large scale web companies. When you look Google, Amazon, Facebook or Twitter data centers: They don’t build up systems that look like traditional enterprise data centers.

OpenStack will be broadly adopted by the enterprise over time. If you look at other open source projects such as Hadoop … it gives the average enterprise a competitiveness that Google previously had with MapReduce. Many companies are comfortable with Hadoop already, and I believe the same will happen with OpenStack over time.

In general, we see two sets of customers. You have companies that see IT as a competitive advantage. They will refresh their datacenter infrastructure over the next 5 to 10 years, and they will embrace models such as OpenStack and Hadoop. Financial services companies are already very much involved with both open source projects. Usually they are a good indicator for other industries to follow later.

And then you have companies for which IT is not a significant competitive advantage, such as shipping, and logistics … they are just tracking where their ships or containers are and they don’t have significant need for sophisticated IT solutions. I think that over time those companies might adopt public clouds designed to run their workloads.

#7 Takeaway: Getting the software & solutions provider DNA into Dell is crucial for the future

Rafael: Last question, Randy. How do you view Dell in the OpenStack game?

Randy: We are currently in the middle of a fundamental change in IT, which can be compared to the transition from mainframe to enterprise computing. A lot of the mainframe companies didn’t survive. Those who did like IBM took a very close look at emerging technologies, and they changed their business model.

Dell is making a lot of changes in order to become a software and solution company. I think being a solution provider is the only way for a company like Dell, which is traditionally a hardware supplier, to survive in the future.

All the right moves have been made at Dell, and it’s more a question of execution: How do you get the software DNA into Dell? How do you get system thinking DNA into the business? It’s all about finding answers to those questions.

Rafael: Thank you very much for this interview, Randy.

Randy: You’re welcome!

Resources

Cloudscaling

Company website: http://www.cloudscaling.com/

Company blog: http://www.cloudscaling.com/blog/

Twitter: https://twitter.com/randybias

Feedback

Twitter: @RafaelKnuth
Email: rafael_knuth@dellteam.com

Learn firsthand about OpenStack, its challenges and opportunities, market adoption and Mirantis’s engagement in the community. My goal is to interview all 24 members of the OpenStack board, and I will post these talks sequentially at Dell TechCenter. In order to make these interviews easier to read, I structured them into my (subjectively) most important takeaways. Enjoy reading!

#1 Takeaway: Cloud is a child of the open source culture and OpenStack as the largest open enterprise project might turn the entire cloud industry upside down

Rafael: Boris, how do you envision OpenStack two or three years from now?

Boris: There is a common vision out there of OpenStack as the Android of cloud computing. If you think of Amazon as the iOS of the cloud, OpenStack ultimately is the Android.  Cloud in general is a baby of the open source culture: Large consumer internet companies such as Amazon, Facebook and Google had to build out an extremely scalable infrastructure to run their application logic on top. They built that infrastructure either out of components that they invented and then later open sourced or open source components they pulled from the outside. The bottom line is: These companies understood that if they took the traditional enterprise route, the price for licenses ultimately would be greater than the revenues they could ever achieve. So they built the superefficient infrastructure stack completely leveraging open components and paying licenses to nobody.

Ultimately companies like Amazon decided to open up part of their infrastructure to customers and that’s how infrastructure cloud in its current form has been born: Amazon pioneered the world with the Elastic Compute Cloud EC2 offering in 2006.

Cloud is inherently about open, and OpenStack today is by far the largest infrastructure cloud open community. If you look at number of companies participating, the amount of code commits and the nature of the project itself: From day one it was not positioned central to any particular vendor but as a conglomerate of different independent organizations. Because cloud is about open, and OpenStack is THE thing in the open cloud, it is effectively going to be one of the most disruptive movements in infrastructure computing during the next 5 to 10 years. OpenStack is going to change the entire industry upside down.

#2 Takeaway: OpenStack is vendor independent, it’s not owned by one particular company or institution, and that makes it attractive for anyone to join

Rafael: What are the key accomplishments in the OpenStack project so far?

Boris: The key accomplishments are strategic in nature. OpenStack is the true incarnation of open source. It was originally designed to create a large, diverse ecosystem where nobody is a dominant player which ultimately attracted everybody. It started out with Rackspace, NASA, Dell, Cisco and a few others, and it was not vendor centric like for instance Eucalyptus or cloud.com. This is a key strategic component that OpenStack got right from the very beginning, and it has been largely the formula to its success.

From the tactical standpoint, people oftentimes criticize OpenStack for the lack of certain features … unlike in vendor-centric projects … but I think OpenStack’s openness is its strength, which will become evident as the project matures. OpenStack is not an out of the box product, but rather a fabric aimed to ultimately glue together in an eloquent way a very diverse set of application infrastructure components.  If you want NetApp storage …you can potentially do it with OpenStack. If you want Nicira networking … OpenStack will work.  The whole idea behind the OpenStack design is a consistent set of APIs on the front end, where tenants can interface with.  Effectively anyone can write drivers to plug their particular hardware component in the backend and make it work with OpenStack.

Rafael: Are there any gaps or missing links in OpenStack?

Boris:  OpenStack is a new project, but I don’t think there are any clear gaps. Some functionalities are missing compared to Amazon or VMware solutions and the community needs to put them in place. But at Mirantis we always looked at OpenStack long term. OpenStack is the only project that is doing open cloud infrastructure right and that is the winning strategy long term. The features and functions will be there over time.

#3 Takeaway: Mirantis is the largest OpenStack systems integrator monetizing through service

Rafael: How is Mirantis positioned in the OpenStack ecosystem?   

Boris: We are the largest systems integrator in entire OpenStack ecosystem with over 70 consultants exclusively dedicated to delivering OpenStack solutions. Mirantis is monetizing OpenStack through service. We figured out that there’s definitely going to be a lot of need for doing OpenStack implementation, providing professional services around it. Besides, it’s a great path to get exposed to a lot of use cases and to learn on the way. We very consciously decided to deliver OpenStack services, helping organizations extract value from the trunk version of OpenStack rather than pushing out our own distro.

#4 Takeaway: There are two sorts of contributions to OpenStack: writing code and evangelizing the project, with the latter being even the more important one

Rafael: How does Mirantis contribute to OpenStack?

Boris: OpenStack contributions can be bucketed into two general categories, which is true for any company in the OpenStack ecosystem, not only for Mirantis. The first category is evangelism, which is about creating awareness momentum. When it comes to open source, this component is just as important if not more important than code contribute: that’s what defines the project and gets people excited to come in and contribute and push forward.

At Mirantis, we were one of the original organizers of the OpenStack meet up in the Bay Area, which is by far the largest user group that OpenStack has in the world with close to 1,800 members. We do similar meet ups in Russia and Ukraine where a lot of our back office and engineering is located. Many of our people at Mirantis including myself blog about OpenStack, we do webinars, we write articles, we participate in different speaking engagements. All these are our contributions to evangelize OpenStack.

The other category besides evangelizing, are code contributions. At Mirantis, we have contributed with a bunch of small and a couple of big code contributions. As for the small things, we have helped build many drivers for different components. We lobby our clients to open source these components, and we did so with drivers for Nexenta, NetApp and Dell Equallogic for example. In addition to that, we contributed with a bunch of bug fixes in OpenStack.

As far as big contributions are concerned, we have spearheaded the elastic load balancer initiative. OpenStack had a project called Atlas, which was an early attempted to build a load balancer in demand for that functionality. Many customers asked for it and we decided to a take it upon ourselves to revive Atlas in a new incarnation. Right now we have three developers and one manager fully dedicated to the elastic load balancer project. It’s most likely going into core in OpenStack Grizzly release … and we are still deciding if it’s going to be a separate standalone project or an extension of Quantum. But it’s a significant part of functionality that allows you to dynamically spin up instances of load balancers through an OpenStack API. You can actually use different load balancers … software load balancers like HAProxy or physical load balancers like F5 as the backend for it.

#5 Takeaway: Adoption of OpenStack happens in three waves: 1) Hardware vendors, 2) SaaS and infrastructure providers (current wave) and 3) Enterprise customers (2-3 years from now)

Rafael:  Who are the early adopters and do you see OpenStack going mainstream?

Boris: We see adoption happening in three distinct stages. One of the early work being spent on OpenStack was not among customers but vendors building up the ecosystem. Companies such as Cisco, Dell, HP and NetApp bought into the long term vision, they understood that this is a disruptive movement and they wanted to early on figure out a way to intelligently integrate their solutions into the OpenStack mix. Being a services company we naturally benefited from that. We’ve interfaced with all the big guys in the industry that have an OpenStack strategy in some shape and form.

The second wave which we are in the middle of today, involves adoption by SaaS vendors and service providers. These companies understood that they have to build the Amazon like cloud in order to compete in the emerging cloud ecosystem, and OpenStack is the de facto standard platform at this point. So we are seeing a lot of use cases to build a standard based application infrastructure that’s based on OpenStack, to replace some of the components that they have haphazardly developed in house over a period of many years as they grew. Because nobody was as smart as Amazon or Google in building the underlying application infrastructure for their SaaS service, a lot of them have spaghetti like diverse structure that is very hard and expensive to maintain. OpenStack provided an opportunity to replace all of that with something that is fairly standard based. We see a lot of projects like that.

Finally the third wave of adoption is the enterprise customers. The use case there is displacing VMware … basically building an Amazon like cloud inside the firewall for the enterprise customers. I think we are at a tip here … it doesn’t just have to do with OpenStack but with cloud adoption by enterprise in general. We’re just starting to see little bit of happening now. We have a number of customers in that space, building an Amazon like cloud infrastructure using OpenStack and displacing VMware with it. But we won’t really see any real mass option of cloud in general or OpenStack specifically by the enterprise sector until maybe 2 years away from now.

#6 Takeaway: Dell is facing challenges in its core business, but it might become a leader in the OpenStack infrastructure cloud market

Rafael: How do you view Dell in OpenStack game?

Boris: We work with Dell hand in hand on multiple projects. Dell is definitely a company that has a lot of opportunity to benefit from what’s happening with OpenStack. Like many other vendors such as Cisco for instance and probably HP in some sense, Dell is facing the problem that the core company business of Dell is effectively getting commoditized … largely by this new cloud mentality where everything is becoming commodity. The value added hardware is no longer is hot thing on the block … Google and Amazon just by cheap ODM hardware, and it’s the software that has all the logic in it.

Dell, naturally being in the hardware business, as well as CISCO, as well as HP and many others are facing effectively the same problem that their core business is soon turning to be very low margin on interest, which ultimately could be very devastating for all those vendors.

OpenStack is an opportunity for Dell to transform the company from being largely a hardware vendor to a cloud solution vendor, and I think for that exact reason Dell has been extremely proactive in taking some concrete and interesting steps to make that happen. Dell is in an extremely good position to become one of the leaders, if not THE leader in the OpenStack solutions industry that is emerging.

Rafael: Thank you so much, Boris. It was a pleasure to talk with you!

Resources

Mirantis

Company website: http://www.mirantis.com/

Company blog: http://www.mirantis.com/blog/

Github: https://github.com/Mirantis

Github (Atlas – load balancer): https://github.com/Mirantis/openstack-lbaas

Twitter (Boris Renski): https://twitter.com/zer0tweets

Feedback

Twitter: @RafaelKnuth

Email: rafael_knuth@dellteam.com

Learn firsthand about OpenStack, its challenges and opportunities, market adoption and Dell’s engagement in the community. My goal is to interview all 24 members of the OpenStack board, and I will post these talks sequentially at Dell TechCenter. In order to make these interviews easier to read, I structured them into my (subjectively) most important takeaways. Enjoy reading!

#1 Takeaway: Dell’s interest in OpenStack derives from market pressure on finding open source alternatives to Amazon and VMware

Rafael: Dell is one of the very early contributors to OpenStack. Why is Dell engaging in this project?

Rob: We were involved with Rackspace and NASA before there was public interest in OpenStack. Our involvement in open source cloud goes back even much further than that, because we were part of market dynamics pre-OpenStack. We were struggling to find open source cloud alternatives for our customers with a common API, a community, support and with momentum.  There was a lot of pressure in the market to have an alternative ecosystem to Amazon as a public cloud and to VMware as a licensed internal cloud. So our team was looking for something in that space, and OpenStack appealed to us because it was open source, and it was a community driven effort with a group of people supporting it. So far the consortium of people involved in OpenStack has by far exceeded what I was expecting to be able to accomplish.

The open source clouds that were on the market at that time, Eucalyptus and Cloud.com, were both really interesting alternatives but neither of them had the type of collaborative community that was the goal of OpenStack. That was REALLY the difference. OpenStack started on day zero as a community of collaboration, and there wasn’t one company that was maintaining the code base. That really differentiates the project.

#2 Takeaway: Dell is focusing on operationability of OpenStack for customers rather than on code contributions to the community

Rafael: How does Dell contribute to OpenStack?

Rob: At Dell we knew from hard experience in field that the number one challenge for customers is going to be using OpenStack. Whether it worked or not, installing it, getting a repeatable build, testing and using was going to be a significant challenge that our customers would face.

We made a very deliberate choice, that rather than trying to contribute code to OpenStack, we wanted to tackle reference deployments. Our goal was to make OpenStack as deployable as possible. So, we have been mainly operationally focused on OpenStack in helping our customers and the community to get OpenStack installed.

Moving forward, we will foster OpenStack integration into the Dell portfolio and we will bring in more partners from the open source community. We are going to expand our footprint with Dell Crowbar around making OpenStack more deployable, upgradable and available.

In brief: Dell’s interest in OpenStack has been very pragmatic. OpenStack is something we really see a market need for.

#3 Takeaway: Dell Crowbar is an OpenStack deployment mechanism targeted at experienced high scale customers

Rafael: Let’s talk a bit about Dell Crowbar, your team’s deployment mechanism for OpenStack. I often times hear from the OpenStack community that it’s considered a solution targeted at the experienced admin, whereas most companies participating in the OpenStack game are building solutions that are supposed to replace that person. Why did Dell pick that route?

Rob: That’s an excellent question. Dell Crowbar is definitely a DevOps tool for customers who are planning to build the experienced run with OpenStack. The goal for Dell Crowbar is to get OpenStack running without having to read the manuals though, but it’s not a managed appliance. Dell has partners like Morphlabs who are selling a managed appliance. There are great companies out there like Piston Cloud and Nebula who are very focused on the managed appliance aspect. I believe there is plenty of room in the market for that type of solutions.

My team is focused on high scale customers. They don’t want somebody to take over their operations. They want somebody to accelerate their operations. That’s why we’re working with Chef, that’s why we are adding Puppet capabilities into Dell Crowbar.  Over time, Dell Crowbar is going to be appliance-like, more and more easy to use, but our strategy is to work with a very targeted market of early adopters, open source experienced customers and to learn jointly about OpenStack best practices.

This is what I truly enjoy about open source … those communities know SO MUCH and it’s fun to have a pure relationship with them, to trade information among equals. Some of those community members are even more in field than I am, and if I switch into the “I am doing it for you!” mode, it changes the relationship.

#4 Takeaway: OpenStack raw is used by sophisticated community members, whereas OpenStack distributions target less experienced customers seeking for a managed appliance  

Rafael: Let’s talk a bit about OpenStack raw vs. OpenStack distributions. What is Dell’s offering around both type of solutions?

Rob: OpenStack distributions are a bit tricky because of the way customers interpret this subject.

Dell actually supports a number of distributions and we will expand our offering in that area. For example, SUSE is using Dell Crowbar as the deployment mechanism for their OpenStack distribution. Currently our shipping products use Canonical’s distribution, our Hadoop solution uses Red Hat and CentOS.

To me, OpenStack distributions are about support. Who the customer can call to get a bug fixed?

As for the OpenStack raw part: We have a new feature in the open source release of Dell Crowbar that we call Pull From Source (PFS). I am really excited about that feature because it addresses a new market: That feature lets you pull OpenStack raw … not necessarily off trunk … we expect customers will set up their own clone off trunk and then pull from that. It allows a customer who is in dev mode and working on a new feature to test it against their own deployment, to create a real test infrastructure with multi-nodes and real workloads.

We consider this a very important use case for addressing deployability in pre-release. We expect our high scale customers  who are a little bit ahead of trunk … they fix bugs, tweak and tune things … to use techniques like PFS.

During the recent OpenStack Summit Rackspace made a big point that their cloud runs on OpenStack pretty much off trunk, not on a distribution. They are pulling source code in and people are testing it, using it, fixing bugs and pushing code back. That’s exactly the type of vibrant community we want to see.

At the same time, there is a growing community that wants to use OpenStack distributions with support, certifications and they are fine with being 6 months behind OpenStack off trunk. That’s good, and we want that shadow, we want that combination of pure minded early adopters and less sophisticated OpenStack users both working together.

#5 Takeaway: OpenStack upgradability is currently the biggest barrier to adoption

Rafael: What are the biggest barriers to OpenStack adoption as of now?

Rob: That’s an excellent question. The most significant barrier to adoption is the release cycle. It delays customers starting with OpenStack because the next release is always just round the corner. OpenStack has a very fast release train.

As a community, we haven’t invested enough yet in upgrade strategies. My team’s top priority is addressing this issue. That’s what we hear in every customer meeting: “You must give me a way … if I take OpenStack Essex today, to get to OpenStack Folsom and from there to OpenStack Grizzly.” Customers want the innovation, but they want to be able to transit easily from one release to another.

On the other hand, things that I thought might be difficult turned out to be easy. For example, Linux and KVM have not been a barrier for customers, just as switching to a new API. At Dell we hit a barrier for how fast we can support new hardware versions. A lot of our early adopters like to get new hardware as fast as they possible can, and sometimes hardware shows up in a customer site before it does in our lab for testing. (laughs)

Rafael: Let me dwell a bit more on upgrading OpenStack, Rob. What does a customer specifically need to do when moving from OpenStack Essex to Folsom for example? Does he need to deploy from scratch or is there an easy upgrade path? What are the issues that a customer might face in that process?

Rob: There is now guidance in the community around that process … but it is a manual process. Our advice so far has been to do a new deployment. We’ve worked hard to make OpenStack deployment with Dell Crowbar as fast as possible, but at the same time it’s not a very good answer. We’re working on it. What we found is that most of our customers who are deploying OpenStack Essex are doing it as pilots and proofs of concept and upgrading OpenStack hasn’t been that much of a burden.

We do have customers who did much more significant investments in production deployment around OpenStack Essex and even in the previous release Diablo. They are trying to figure out the best way to do migrations. My advice to customers is always to use DevOps tools for cloud deployments, to invest time into automation to migrate your applications more easily.

#6 Takeaway:  Most customers do OpenStack proof of concepts. Only very few use OpenStack in production

Rafael: Let’s talk about proof of concept versus production, Rob. How are customers using OpenStack and can you give examples for both scenarios?

Rob: Most of our customers do proof of concepts and pilots. They are putting OpenStack through dev test cycles, they are doing dev workloads. In some cases customers are going with OpenStack into production. In that case they are not using our tool, because Dell Crowbar doesn’t do automatic production deployment yet … that’s something we’re working towards. Anybody who is doing OpenStack in production is building up operational expertise. Most of those customers are OpenStack contributors, they are very active in the community. That’s what it takes today to run OpenStack in production: a certain level of commitment to the OpenStack community.

AT&T is on top of my mind as well as Dreamhost. Both of those are hosting examples. I just recently talked to a communications company that is doing a production deployment for one of their media apps, I know of a financial institution that is running a big data production cluster. I know of very few companies running real OpenStack production clusters, most of them are trying out things right now. Even Rackspace is operating OpenStack in dual mode. Customers can opt in and out. But we will get there.

#7 Takeaway: OpenStack is meant to be an Amazon and VMware alternative, for different reasons though

Rafael: I oftentimes hear two different statements: “Open Stack is an alternative to Amazon.” The other is: “OpenStack is an alternative to VMware … maybe, hopefully in two or three years from now.”  Which of both statements is true?

Rob: I think that both statements are accurate, but for different reasons. OpenStack is an alternative to Amazon for public cloud companies like Rackspace, Dreamhost and many others. They want to drive an ecosystem, where they can create a cloud market which is not Amazon’ objective. Their objective is to be THE cloud.  For customers who want choice and portability, OpenStack makes a lot of sense, as well as for providers who want to differentiate on service.

On the VMware side customers I talk to do want alternatives to licensed software, where costs do not scale linearly as they grow their infrastructure. They want the transparency of open source, they want flexibility.  Those are real concerns for our customers, and they are willing to trade license cost for needing more operational knowledge. Over time their need to know how to run OpenStack operationally will decrease whereas licensing costs would scale linearly.

Having that alternative with OpenStack will definitely help customers and it will drive response both from Amazon and VMware.

#8 Takeway: VMware joining OpenStack is beneficial for the community, rather than a threat

Rafael: How do you view VMware joining OpenStack. Is it a threat to OpenStack or does VMware add value to the project?

Rob: That’s a really loaded question. (laughs) You have to split that question into pieces. From the Nicira acquisition side, which helps VMware coming into OpenStack as a major contributor: Nicira is doing lot of very important work, they are helping drive Quantum forward.

I think VMware has lot to offer in the community and their presence is good. I am not afraid of them coming in and then trying to establish walls and make things proprietary and disrupt the community.  There is no indication right now that we need to protect OpenStack from VMware.  They are showing leadership in OpenStack and frankly they have a lot of open source projects, which they are managing very robustly. These fears are overblown, and if someone tries to jockey OpenStack, the community pushes back pretty effectively.

#9 Takeaway: OpenStack has a very broad range of early adopters. But mass market adoption won’t happen until upgradability issues get resolved

Rafael: Let us speak about market adoption. Who are the early adopters of OpenStack? And when do you expect OpenStack to hit the tipping point for mass market adoption?

Rob: The early adoption is much more distributed than we have thought. Hosting companies definitely jumped in fast, because they have the operational ability and the financial incentives are very well lined. But I have also seen educational groups coming in, government entities and financial institutions which are known for aggressively adopting new technologies. Some of them are already comfortable with Hadoop … and I believe there are going to be more synergies between OpenStack and Hadoop, we haven’t seen them emerge yet … although I am little biased because my team does both. (laughs) Definitely social media, online retail, gaming companies are amongst early adopters … every single of those segments is already operationally sophisticated. By and large those companies I talk with are already strongly in Linux, DevOps and they are not afraid to get involved in these technologies. Unfortunately many companies I talk with are not in the position where their corporate culture allows them to contribute back to open source. So there is a bit of a shadow effect of companies using open stack but they don’t show up in the community.

We are not going to cross the chasm into mainstream until we solve the upgrade problem. That’s not surprising, but we are working very hard to help customers make adoption easier.

#10 Takeaway: Dell is in the process of building an entire ecosystem of OpenStack solutions for various types of customers

Rafael Rob, for all those interested in Dell’s commercial offering around OpenStack … can you give a brief overview? We already discussed about Dell Crowbar, the deployment mechanism for OpenStack as well as about some OpenStack distributions Dell is supporting. What else do we have in our portfolio?    

Rob: We have a very clear vision of what we want to do in OpenStack, and it involves building up what we consider a full taxonomy. We believe there is a need for orchestration at the API level with partners like enStratus. There is need for automated deployment upgrade … what we want to build is a full experience.

We’ve done this to a certain extent with our solution offering and expanding with partners and different capabilities. One of the things we are trying to do is to start with a free open source option and then, as customers get into larger deployments, to bring in license capabilities as well as further free options with support capabilities around them. Dell has already a lot of assets around storage, networking and compute … software products, cloud services. We have a very broad view of enabling the ecosystem.

Rafael: I am in the process of setting up a wiki page at Dell TechCenter that provides customers an overview over our OpenStack offering: Dell Crowbar as our DevOps tool in its various shapes and forms, OpenStack distros we support … cloud services we build around OpenStack … hardware capabilities optimized for OpenStack.

Rob: You just summed up exactly what our challenge is at Dell. We are working with different partners to bring OpenStack to different customers in different ways. It is confusing. Your question about Dell Crowbar was right … it is targeted at a certain class of users, and I don’t want enterprise customers who expect a lot of shiny chrome and zero touch. That’s not the target by now for Dell Crowbar. We definitely need that sort of magic decoder page to help customers understand our commercial offering.

#11 Takeaway: The main challenge for the OpenStack board is defining OpenStack in detail

Rafael: My final question to you Rob is: What are the challenges for the OpenStack Board of Directors?

Rob: There are some really weighty issues in front of the board for this coming year. People should be aware of them because they really shape OpenStack.

We just formed up the Technical Board, we have got the User Committee, and we can actually start discussing the most significant bundle of issues: “What is the core of Open Stack? What defines OpenStack? What makes OpenStack … OpenStack?” For example, a project like Swift, which is an important, widely used part of OpenStack: Is it a required core component of OpenStack? Or is it an optional component?

That is a serious question because the way the OpenStack board defines what’s core and non-core, helps people bringing in new projects understand where they fit in. That also changes the incubation definition. If it’s core, it is going to have a certain incubation pattern. If it’s not, it might have a different incubation option.

That leads us straight into the question: “Is OpenStack an API? Or is OpenStack an implementation?” Today most people don’t realize that it’s really an implementation. If OpenStack is the API, then how do we provide a fitness test to make sure that someone’s implementation of OpenStack complies? We have a lot of these questions around Swift but also in the Nova, Cinder and Quantum pieces. If someone wants to replace the implementation of Swift, which is about how objects are actually stored: How do they know that they comply with the Swift API, so that they can be certified in OpenStack Swift? … even if they didn’t use the RSync implementation of Swift?

The same would be true if you reimplemented the Nova API but back ended it with VMware vCloud … is that OpenStack or is it not?

Defining API is really important. At the same time OpenStack progressed really quickly because we haven’t gone through committees and API use negotiations … and that worked. (laughs) But part of OpenStack’s maturity is to define API and the Board of Directors has to find the resources to back these decisions. It also has to find resources in the community to provide API testing.

The other thing on technical side is that we need to drive upgrade into the code.  It’s a really a significant challenge to make sure that upgradeability is a key factor in the development cycle. We need to drive much more operational testing of OpenStack releases earlier in the cycle instead of waiting until the release is done. That’s not a good pattern, we need to do it earlier.

It’s important to understand what very pragmatically the challenges around OpenStack are.

Rafael: Rob, thank you very much for taking time. Be sure I will come back soon with even more questions.

Rob: (laughs) You’re welcome!

Resources

Rob Hirschfeld

Personal Blog: http://robhirschfeld.com/ 

Personal Youtube Channel: http://www.youtube.com/user/rollingwoodcitizen

Twitter: https://twitter.com/zehicle

OpenStack

OpenStack Foundation: http://www.openstack.org/

OpenStack at Dell: http://dell.com/openstack

Dell Crowbar: https://github.com/dellcloudedge/crowbar

Feedback

Twitter: @RafaelKnuth
Email: rafael_knuth@dellteam.com