Menu

Tag: Lean

History

The explosion of Agile for software development was pretty cool right? Empowered development teams could now liaise directly with the ‘customer’, make decisions and ‘crack on’ with exciting new software deliveries by taking an iterative approach to deliver value rapidly and often! But, hang on, this isn’t an ‘Agile’ post… this is DevOps… so what gives?

Well, somehow this exciting new approach didn’t happen within the operations space’. So whilst development was agile and continuous, operations continued to utilise a more ‘weighty’ Waterfall style approach.

I remember in a previous life, as a Project Manager (locked in ‘Waterfall’), I’d engage with Subject Matter Experts across the business and Business Analysts to ensure user and business requirements were understood, and existing business processes were ‘mapped’. Once this exercise was complete I would be able to mediate effectively between all stakeholders to get the job done… That alone was ‘chewy’ with lots of meetings, lots of paperwork, demos, risk management exercises, sign-offs and reports however none of this compared to the lengthy processes surrounding transition to the live production environment and into support and maintenance… i.e. the Operations domain.

Enter DevOps!

As the word describes, DevOps is a coming together of two key business functions: Development and Operations for a more efficient implementation of software and systems.

A successful IT product or service should meet the needs of the users, yes, but it must also be efficient, serviceable and cost effective to deploy and maintain. It is unlikely to be all of these unless it is architected in such a way to work well with target IT platforms, processes, operational activities and so on. Additionally there is little point in having short ‘release’ cycles if the actual deployment process takes weeks (perhaps months). DevOps promotes a Dev and Ops inclusive approach to ensure teams, skills, processes are aligned across development and into operations – so they may work in symphony to deliver faster and more efficiently without compromise.

What does DevOps actually look like?

A fuzzy answer would be ‘A shared design, development and delivery experience between IT operations and development teams across the SDLC’. If you try to be more specific you’re probably going to put some people’s noses out of joint but there is general agreement that IT automation, agile development, systems virtualisation, collaborative working and improved processes are all important. The TechInsights Report: “What Smart Businesses Know About DevOps” lists the following:

How does DevOps look in practice? Let’s consider a couple of scenarios…

Firstly, let’s consider the ‘culture’ side of DevOps. If, like ours, your organisation is reasonably small and runs a ‘tight ship’ (i.e. lean on staff) then you’ll probably already be multi-disciplined and your departments may already liaise on a regular basis (e.g. through management meetings). Like many small businesses we outsource a lot but we retain control from within. I represent the IT function but I also approve and commission new development. I’m continually meeting with Operations, Sales and Accounts to ensure our goals are aligned and that we are delivering for business benefit and capability. I’m lucky enough (because I love it) to be at the heart of any system and software architecture decision. My background is in software development so I remain ‘hands on’ in the creation of design documentation and in the choice of technologies, development practices and processes. When we transition to the production environment and undertake maintenance I get to influence which tools are used and how we deploy and support. I’m not autocratic in my decisions I’m inclusive and fully engage with key functions within the organisation to ensure that their needs are met and they can utilise and benefit from the new software or systems. Like I said, we’re small, so our deployment is reasonably straightforward – we don’t need specific deployment tools, our test/staging/live environments are well understood and do us just fine. So, culturally I would suggest we already kind of do DevOps! Does that me me a DevOps engineer? I think not… We do have processes and controls around source code management and deployment but we’re not really automating to any great extent and the process is still pretty hands-on. In reality we’re not undertaking many of the typical activities you’ll find DevOps ‘Engineers’ claim are essential to carry the accolade. So you won’t find me applying for DevOps Engineer roles just yet…

In another scenario a larger business may specifically recruit DevOps engineers to ‘own’ the entire Dev to Ops process. These engineers may be responsible for influencing architectural decisions and defining processes that improve efficiency across the SDLC and into production. They do this through tools and scripting that ensure code quality and integrity and that automate the deployment process. Improvements in code integrity can be achieved through unit test – perhaps a Test-driven development approach and the benefits of automation are many. If deployments are scripted, they are 100% repeatable, eliminating human error and enabling the tasks to be run by lesser skilled individuals – so that’s faster, with less issues, at a lower cost to the company (you can understand why organisations are embracing DevOps). Done correctly, DevOps should mean the unexpected no longer happens, new deployments are rapid and, in the virtual space, new ‘clone’ environments can be initiated at speeds that no one could dream of in the physical space. Of course, to get to this stage is no small undertaking. As well as costly DevOps engineers, organisations would have invested money and time in tools (see list below, original source here) and environments that enable engineers to implement virtualisation, code version control, continuous integration and deployment automation tools. Generally these organisations would have also adopted lean and agile development principles and undertaken the culture, processes and practices that accompany the approach.

So efficient DevOps can help to ensure our deployment ‘keeps up’ with our short release cycles.

So you want to be a DevOps Engineer?

I don’t blame you. I see some serious career progression for these guys and gals and with a broad experience across multiple technologies the DevOps engineers of today are likely to be the CTOs of tomorrow. So what skills should you have or be looking to obtain?

Firstly, it’s not reasonably practical to assume you can complete a weekend training course to become a DevOps engineer. You have to earn that title with experience that comes from hard graft in a tough career, in a tough industry.

Secondly, you’ll need a mix of soft and hard skills:

You need to be a strong communicator both with techies and ‘normal’ folk. You should know when to talk tech and when to not. You should be the kind of person that ‘shares’ knowledge and enjoys collaboration (I’ll be blogging about some of my favourite collaboration tools including Trello and Slack in another blog). You should be curious, analytical and have a flexible mindset. You should always challenge the status quo and look for a better way…

From a technology perspective you should be familiar with modern and open source technologies within the development and IT operations arena. You should have some hands-on experience with current programming/development practices, popular and emerging technologies, client-side vs service-side, coding frameworks, software libraries, continuous integration/deployment, test-driven development, API development and utilisation and so on. Solid systems experience (you’d know your OSI from your TCP/IP reference models) including how stuff works (like data packets, routing etc.) is pretty fundamental. Systems admin experience is vital – some *NIX (e.g UNIX, Linux) would do you proud. IT security is an essential requirement (and also important in software design) as well as monitoring, reporting. You should have experience of Solution design and architecture and know the value of Data and data management. To be future proof you should be familiar with virtualisation and cloud computing (Azure, AWS, private cloud) and ideally have hands-on experience of SaaS and IaaS. Lastly, experience of IT configuration management and automation and and some of the emerging DevOps tools like Puppet and Chef is going to help you claim that title for sure!

Now, before we continue, read the paragraph above again… That’s a pretty hefty set of skills right? It is easy to see how DevOps evolved from Lean Start-up requirements where funding is tight and staff must undertake multiple roles in order to meet ambitious release cycles. There’s an alternative view to DevOps – taken by the developer. I’d encourage you to read this excellent article by Jeff Knupp who basically argues that just because developers have had to undertake the roles of DBA, QA, Test, Deployment Engineer etc. (because of constraints) and just because they don’t suck at it (many are actually pretty good at it) it shouldn’t mean this should become expected of them. Such multi-tasking may actually detract from the role of core development and talented employees can become overworked and under productive…

You can’t know everything and specialising is just fine but you should have enough knowledge in a chosen field to have an opinion on what to use, where and when to use it and how to do so. As with any role, demonstrable experience and exposure to a wide variety of situations will improve your desirability as a DevOps engineer.

Summary

So DevOps seeks to align and harmonize the development and operations functions so they can work together in delivering systems/solutions rapidly, reliably and safely. As a result the transition into production will be more agile and the products can be more efficiently operated and supported. DevOps combines development, systems and IT services and operations experience and this combined set of skills helps to meet the requirements and challenges of development, infrastructure, automation, testing, release and maintenance.

DevOps is a way of thinking and working… a culture of sorts. It involves people, process and technology. There are a number of skills and best practices associated with DevOps and an increasing array of tools (with some emerging leaders) to help its successful implementation. A dedicated individual can become a DevOps Ninja Warrior but don’t think it’s a walk in the park – there’s some serious knowledge required across a broad array of technologies.

Regardless of background or religious belief most would agree that, in our current physical existence, our capacity for knowledge is limited. So much to learn, so little time… If, like me, you have a desire for knowledge and love learning you soon realise that it’s impossible to absorb anywhere near the information you crave.

I have a coping strategy,

Know where I’m heading… long, mid and short-term. Knowing this will help me refine the scope of my learning to what’s important at the current time.

Ignore the ‘noise’ – that’s anything that’s useless to my progression as a human being (e.g. what Justin Bieber is up to) or in conflict to the above.

Ignore the above rule if something screams “I’m important, look me up!”. If my brain tells me it’s important to learn, it probably is and it’ll be absorbed quickly with minimal impact.

Learn just enough to achieve what it is you are trying to achieve – considering the constraints on time, anything more is wasteful.

Return to a subject and learn more about it only when absolutely necessary – i.e. work or life requires that specialism.

In doing so, I have a chance of being able to productively contribute and add value to that small portion of reality that I am currently interacting with.’

As you can see, I basically take an agile approach to my learning, well, actually my life! With a busy job, a demanding family of 5 and an MSc to study for it’s the only way I can stay on top of things! In fact, I could happily add another item to the list above…

7. Deliver ‘Just In Time’ (JIT) which is equally applicable to my life as it is the manufacturing industry!

This blog focuses its scope on the knowledge I require to deliver value in my work. As a CTO / Project Manager / Software Delivery Manager and Solutions Architect (wow, we have so many roles to cover these days…) I face a continual struggle to stay abreast of modern technologies and useful insights within the software solutions ‘world’. However, it’s vitally important to do so and I want to ‘share’ my lean learning experiences with anyone else in a similar position. In doing so I’m anticipating that some of you may wish to reciprocate and we’ll all benefit.

We’re going to keep the articles purposefully ‘lean’ with the goal to obtain just enough value to provide benefit. You’ll not find any lengthy tutorials or step-by-step guides but you may find the odd technical overview supported by links to useful resources enabling you to explore further should you feel it necessary.

So that’s the ‘Agile’ and the ‘Lean’ covered… What about the ‘Open’?

My first introduction to programming came in the form of PERL, I moved to PHP and MySQL and then found myself in corporate environments requiring Microsoft technologies ASP (VBScript and HTML) which evolved to .NET and C#. This resulted in some expertise around the Microsoft Solution Stack including IIS, SQL Server, MVC, WebAPI and so on… All of this time I kept a keen eye on Open Source, the increasing prevalence of JavaScript, the thriving community and the very ‘cool’ tools they created and shared.

In my current role, I’m lucky enough to be delivering a variety of software projects for web and mobile and some web service integrations with partners. As such I’m exposed to various different technologies and trends and occasionally get to ‘hang out’ with some clever coding teams.

The argument for and against Open Source rages on but over time, as open source becomes ever more pervasive, I believe the arguments against it are slowly becoming less valid – particularly in the hosted software solutions space – which is the area where I concentrate my efforts. Even Microsoft realises it has to open source certain tools and technologies to remain ‘in the game’. So, for a variety of reasons (we’ll probably go into some of these in another post), I believe it’s time to move more towards the Open Source arena.

Along the way I’ll share my experiences of re-engineering our solutions and re-architecting them for migration to the cloud. We’ll consider technologies, choice of database store, hosting, and a number of other things… I’m armed with a Linux VPS which will serve as a host to this blog and a ‘playground’ for a number of technologies. Again, I’ll be sharing any experiments, outputs and learning experiences with you.