Five years open: the trials and tribulations of OpenStack
Five years on from its high-profile inception, OpenStack has enjoyed a lot of hype as the de facto open-source standard for cloud computing – but will it ever be the cloud giant that many claimed it would?
The de facto open-source standard for cloud computing, OpenStack, has almost doubled its membership in the past year – but has enterprise adoption caught up?
OpenStack was launched as a free open-source platform for developing cloud services in July 2010 by Rackspace and NASA, which developed their own infrastructure-as-a-service (IaaS) offering.
A new version of OpenStack is released twice a year, and companies such as Red Hat build their own OpenStack platforms based on the new release. NASA dropped out as an active developer in July 2013, citing a lack of technical progress.
The platform initially saw popularity as a viable alternative to either the high capex a business requires to build its own cloud infrastructure or the public cloud options from the likes of Google and Amazon Web Services (AWS), which many have perceived as inflexible and lacking data privacy and security.
In 2012, PayPal became a high-profile early adopter by beginning a three-year migration to implement its own OpenStack private cloud for close to 100% of its operations. It completed the project in 2015, making it one of the largest financial services OpenStack clouds in production – with 4,100 standardised x86 servers running under the platform to service PayPal’s 173 million customers.
OpenStack now has 28,000 members in 140 countries, and 1,500 contributors in 169 companies. According to 451 Research, the OpenStack technology market will grow to £3.3 billion in 2018, representing a compound annual growth rate of 40% for the six years from 2012 to 2018.
However, despite huge enthusiasm and contribution from the vendor community, enterprise adoption hasn’t taken off as much as was anticipated.
According to 451, this will only change when more OpenStack-trained developers enter the workforce and a simpler deployment and operations experience is realised that does not need an army of technical resources.
In December, Japanese tech giant Fujitsu revealed that it will migrate all of its internal systems to OpenStack over the next five years, but its CTO, Joseph Reger, already anticipates major challenges, noting the platform’s lack of enterprise features and scalability.
‘The negative reports you are reading about are customers who took the community release and thought they could do it themselves,’ he said. ‘[There is] quite a bit of complexity there, [so it’s] better to work for somebody who is a service provider for that kind of stuff.
‘I am not saying it will succeed for sure because nothing is certain in this business, but I am saying let’s give it some slack – it’s very, very young. We’ve done the math for our own IT operations, and with the introduction of this new A5 cloud platform we hope to take out several millions a year of IT budget.’
Jim Whitehurst, CEO of Red Hat, the largest enterprise provider of OpenStack and contributor to it, believes the biggest challenge the technology has faced is unrealistic market expectations.
‘Everybody was excited about it and wanted to be involved in it, so a
lot of vendors started saying they were going to put out an OpenStack product before the software was ready,’ Whitehurst says.
In a traditional scenario where a software company controls the release cycle of its own proprietary software, it would never put out a product before it was ready. However, an eagerness to be involved with OpenStack saw many vendors release software that was still in alpha or beta stage.
‘There was frankly an 18-month or two-year period when vendors went out trying to sell products ahead of when they were ready,’ says Whitehurst. ‘I think we’re really getting to the point now where we’re seeing enterprise adoption – it’s just two years after when people thought we’d see it.’
The main benefit of OpenStack is the ability to gain overall control of an organisation’s IT infrastructure. Rather than having disparate hardware spread out across the organisation, OpenStack users can enjoy visibility of exactly what is being run on every server.
The platform provides greater control and faster provisioning of servers when required, together with the ability to bring up and tear down the entire server environment in minutes, rather than in weeks and months, and test different environments without the extra cost of new hardware.
‘As it’s used to manage and automate pools of hardware resources, it can be done at a lower cost than having to provision kit specifically for a given purpose,’ says Christopher Brown, OpenStack engineer at OCF. ‘Ultimately, you are achieving better use and allocation of your resources, making your hardware work harder for the money you’ve invested in it, and avoiding any wasted resource.’
The sense of transparency that comes from OpenStack means that end-users can see the code, know exactly what it is doing and potentially influence its future development.
And according to Mark Smith, global product and solutions manager at SUSE, it also brings contributing organisations ‘a sense of community that can help give a competitive advantage by driving opportunities for partnerships and potential customers’.
Hardware and service providers need a flexible and open platform they can work with to be competitive, stay in control of their own business priorities, and respond to customer needs in today’s software-driven economy.
Linux is a good historical example of this need. Back then, in response to the increasingly proprietary and fractured server operating system market, infrastructure vendors rallied around the Linux platform to enable the enterprise.
Today, Linux is unquestionably the workhorse of the internet, underlying most web applications and cloud platforms. However, OpenStack vendors are now positioning the cloud platform as becoming that operating system for the data centre.
Using OpenStack, companies can develop and run cloud applications, and vendors can innovate to provide the underlying physical compute, network and storage infrastructure.
‘It is clear at this point that cloud is the future,’ says Kamesh Pemmaraju, VP of product marketing at Mirantis. ‘OpenStack lets vendors stay relevant in the industry, and gives them a cloud future that they can rally around.
‘Nobody wants AWS and other public cloud giants to win all of the market share and become a cloud oligopoly. If they do, IT’s future will be, as [Mirantis CEO] Alex Freedland put it, “a massive array of homogeneous computer farms”. Vendors would much prefer a competitive, vibrant ecosystem.’
However, the platform tends to work better for organisations, such as universities, with lots of different groups of people using the server capacity for many different tasks.
Unless it is a company like Google, an enterprise will tend to run a simpler
IT infrastructure, so there is a less compelling business case for OpenStack.
‘I’ve never been a believer of one technology having the magic bullet for everything, and OpenStack is no different, particularly in the enterprise market,’ says Brown. ‘OpenStack is a very low-level system and takes a lot of effort to move over to, so enterprises need to be absolutely sure before they move in that direction.’
A lack of clarity on exactly what OpenStack does has also contributed to market confusion around the technology, which has further held back enterprise adoption. Some people have mistaken OpenStack for another form of virtualisation, despite the platform allowing users to create and adapt infrastructure much more quickly than they could with just a virtual machine sitting on a server.
Others have missed the end goal of OpenStack, which is not just about machines running on virtual servers, but an orchestration engine that can start up a whole group of servers in minutes and automatically scale in and out in response to demand.
‘Market confusion leads to organisations not exploring OpenStack further,’ says Brown, ‘and a general unwarranted negativity can build up around its uptake.’
The OpenStack community is very fast moving – with more than 40 projects at various stages of development, adoption and maturity levels, it can
be difficult to see what is going on.
For a long time, the community selected key projects through a process of graduation into one integrated release every six months. But while this approach helped to some extent in defining what OpenStack was, it was too big for many people to deploy, and too small to fit the features that many people wanted.
In May 2015, the OpenStack Foundation restructured the governance of OpenStack projects into a ‘Big Tent’ model made up of five core projects that define the essential components of the platform: object and block storage; identity management; compute; networking; and image service.
‘Other projects can still become OpenStack projects under Big Tent,’ says Pemmaraju, ‘but not core projects. They are “add-on” components that can enhance a user’s OpenStack architecture.
‘Using the OpenStack project navigator, which was introduced at the OpenStack Summit in Tokyo, one can get a detailed understanding of the adoption, maturity and age of the core projects and optional projects.’
Like all new technologies, OpenStack is something that needs to be evaluated as to whether it is the right choice for any organisation. It will completely replace the existing infrastructure, so this is not a decision to be taken lightly.
Users can run it alongside an existing infrastructure and migrate across, but it is important to check that the applications they are currently running also run happily in a virtualised or OpenStack environment.
It is advisable for an organisation to run a proof of concept environment with the IT team in-house or an experienced OpenStack integrator to prove that it will solve the existing problems.
‘We have worked with various organisations, and none of them have dived in with OpenStack,’ says Brown. ‘They have been slowly migrating over to an infrastructure as a service.
‘At the other end of the scale, we have also seen companies so on board with OpenStack that you need to submit a written application to purchase a new server demonstrating that any additional activity cannot be done within the existing OpenStack environment.’