putting the idiot in idiot savant

Menu

Category Archives: ECS

I’m not at VMworld US this year, but I had the opportunity to be briefed by Sam Grocott (Dell EMC Cloud Strategy) on some of Dell EMC‘s key announcements during the event, and thought I’d share some of my rough notes and links here. You can read the press release here.

TL;DR?

It is a multi-cloud world. Multi-cloud requires workload mobility. The market requires a consistent experience between on-premises and off-premises. Dell EMC are doing some more stuff around that.

Cloud Platforms

Dell EMC offer a number of engineered systems to run both IaaS and cloud native applications.

VxRail

Starting with vSphere 6.7, Dell EMC are saying they’re delivering “near” synchronous software releases between VMware and VxRail. In this case that translates to a less than 30 Day delta between releases. There’s also support for:

As would be expected from a company with a large portfolio of products, there’s quite a bit happening on the product enhancement front. Dell EMC are starting to get that they need to be on-board with those pesky cloud types, and they’re also doing a decent job of ensuring their private cloud customers have something to play with as well.

I’m always a little surprised by vendors offering “Cloud Editions” of key products, as it feels a lot like they’re bolting on something to the public cloud when the focus could perhaps be on helping customers get to a cloud-native position sooner. That said, there are good economic reasons to take this approach. By that I mean that there’s always going to be someone who thinks they can just lift and shift their workload to the public cloud, rather than re-factoring their applications. Dell EMC are providing a number of ways to make this a fairly safe undertaking, and products like Unity Cloud Edition provide some nice features such as increased resilience that would be otherwise lacking if the enterprise customer simply dumped its VMs in AWS as-is. I still have hope that we’ll stop doing this as an industry in the near future and embrace some smarter ways of working. But while enterprises are happy enough to spend their money on doing things like they always have, I can’t criticise Dell EMC for wanting a piece of the pie.

Disclaimer: I recently attended Dell EMC World 2017. My flights, accommodation and conference pass were paid for by Dell EMC via the Dell EMC Elect program. There is no requirement for me to blog about any of the content presented and I am not compensated in any way for my time at the event. Some materials presented were discussed under NDA and don’t form part of my blog posts, but could influence future discussions.

Isilon

The latest generation of Isilon (previewed at Dell EMC World in Austin) was announced today. It’s a modular, in-chassis, flexible platform capable of hosting a mix of all-flash, hybrid and archive nodes. The smaller nodes, with a single socket driving 15 or 20 drives (so they can granularly tune the socket:spindle ratio), come in a 4RU chassis. Each chassis can accommodate 4 “sub-nodes”. Dell EMC are claiming some big improvements over the previous generation of Isilon hardware, with 6X file IOPS, 11X throughput, and 2X the capacity. Density is in too, and you can have up to 80 drives in a single chassis (using SATA drives). Dell EMC notes that it’s NVMe ready, but the CPU power to drive that isn’t there just yet. At launch it supports up to 144 nodes (in 36 chassis) and they’re aiming to get to 400 later in the year. Interestingly, there are now dual modes of backend connectivity (InfiniBand and Ethernet) to accommodate this increased number of nodes.
From a compute perspective, you’ll see the following specs:

Intel Broadwell CPU (with optimised compute to drive ratios)

Up to 6TB cache per node (2 Flash cards)

No SPoF

Networking flexibility – Infiniband, 10GbE/40GbE

Nodes can borrow power from neighbours if required too.

Dell EMC tell me this provides the following benefits:

4:1 reduction in RU

Optimised IOPS and throughput

Future-proof, enduring design – snap in next-gen CPUs, networks

New levels of modular, hot-swappable serviceability

From a storage perspective, you’ll see a range of configurations:

From 72TB to 924TB in 4RU

5 drive sleds per node. 3-6 drives per sled.

Front aisle, hot swap sleds and drives

Media flexibility: Flash, SAS and SATA media

Dell EMC tell me this provides the following benefits:

Start small and scale

Breakthrough density

Simplified serviceability and upgrades

Future-proof storage

Other Benefits?

Well, you get access to OneFS 8.1. You also get OPEX reduction by occupying a lot less space in the DC, and having the ability to host a lot more diversity of workloads. Dell EMC are also claiming this release provides unmatched resilience, availability, and security.

Scale? They’ve got that too.

From a capacity standpoint, you can start as small as 72TB (in one chassis) and expand that to over 33PB in a single volume and file system. In terms of performance, Dell EMC are telling me they’re getting up to 250K IOPs, 15GB/s, which scales to 9M IOPs, 540GB/s (aggregate throughput). Your mileage might vary, of course.

Speeds and Feeds

So what do the new models look like? You can guess, but I’ll say it anyway. F nodes are all flash, H nodes are hybrid, and A nodes are archive nodes.

F800 (All Flash)

1.6TB, 3.2TB and 15.4TB Flash

60 drives, up to 924TB per chassis

250K IOPs per chassis

15GB/s throughput

H600 (Hybrid)

600GB and 1.2TB SAS drives

120 drives and up to 144TB per chassis

117K IOPs per chassis

H500, H400 (Hybrid)

2/4/8TB SATA drives

60 drives per chassis

Up to 480TB per chassis

A200 (Archive)

2/4/8TB SATA drives

60 drives per chassis

Up to 480TB per chassis

A2000 (Archive)

10TB SATA drives

80 drives per chassis

800TB per chassis

“No node left behind”

One of the great things about Isilon is that you can seamlessly add “Next Gen” nodes to existing clusters. You’ve been able to do this with Isilon clusters for a very long time, obviously, and it’s nice to see Dell EMC maintain that capability. The benefits of this approach are that you can:

Beef up your existing Isilon clusters with Isilon all flash nodes; and

Consolidate your DC footprint by retiring older nodes.

OneFS

OneFS has always been pretty cool and it’s now “optimised [for the] performance benefits of flash – without compromising enterprise features”. According to Dell EMC, flash wear is “yesterday’s problem”, and the F800 can sustain more writes per day than its total capacity every day for over 5 years before approaching limits. OneFS is now designed to go “From Edge to Core to Cloud” with IsilonSD Edge, the Next Generation Core and Cloud (with CloudPools -> AWS, Azure, Virtustream).

IsilonSD Edge

IsilonSD Edge has some new and improved features now too:

VMware ESXi Hypervisor

Full vCenter integration

Scale up to 36TB

Single server deployment

Back end SAN: ScaleIO, VSAN and VxRAIL

Dell PowerEdge 14G Support

ECS

Dell EMC also talked about their vision for ECS.Next, coming in the next year.

Data streaming

Enterprise Hardening

Certifications

Compliance

Economics

Hybrid Cloud

Big bets?

Hybrid Cloud

Batch and real-time analytics and stream processing

Massive scale @ low cost with new enterprise capabilities

Hybrid Cloud

Dell EMC are launching a ECS Dedicated Cloud (ECS DC) Service. This is on-demand ECS storage, managed by Dell EMC and running on dedicated, single-tenant servers hosted in a Virtustream DC. It’s available in hybrid and fully hosted multi-site configurations.

So what’s in the box?

You get some dedicated infrastructure

Customer owned ECS rack

Dedicated network / firewall / load balancer

You also get 24×7 support of hosted sites from a professional DevOps team

Strong expertise in operating ECS

Proactive monitoring and fast response

As well as broad Geo coverage

5 DCs available across US (Las Vegas, Virginia) and Europe (France, London, Netherlands)

Coming to APJ by end of 2017

It will run on a subscription model, with a 1 year or 3 year contract available.

Project Nautilus

The team also took us through “Project Nautilus”, a batch and real-time analytics and stream processing solution.

Streaming storage and analytics engine

Scale to manage 1000s of high-volume IoT data sources

Eliminate real-time and batch analytics silos

Tier inactive data seamlessly and cost effectively

I hope to cover more on this later. They’re also working on certifications in terms of Hadoop and Centera migrations too (!). I’m definitely interested in the Centera story.

Conclusion

I’ve been a fan of Isilon for some time. It does what it promises on the tin and does it well. The Nitro announcement last year left a few of us scratching our heads (myself included), but I’m on board with a number of the benefits from adopting this approach. Some people are just going to want to consume things in a certain way (VMAX AF is a good example of this), and Dell EMC have been pretty good at glomming onto those market opportunities. And, of course, in much the same way as we’re no longer running SCSI disks everywhere, Flash does seem to be the medium of the future. I’m looking forward to seeing ECS progress as well, given the large numbers of scale-out, object-based storage solutions on the market today. If you’d like to read more about the new Isilon platform, head over to Dell EMC’s blog to check it out.

Elastic Cloud Storage (ECS to its friends) has been around a little while. Today Dell EMC announced the release of version 3.0. I thought I’d cover off some of the reasons why ECS might be something you’d be interested in. I’ll then go through the new features with ECS 3.0. If you make it that far you’ll be treated to some light opinionalysis to finish off.

Why ECS?

Dell EMC provided me with a list of reasons why you might want to consider ECS.

Highly Efficient Data Protection

One of the problems we have is protecting unstructured data at scale. To this end, ECS uses a hybrid protection scheme comprised of triple mirroring, erasure coding and XOR algorithms. The key benefits of this approach are:

Lower storage overhead option for cold data scenarios

Enhanced data durability without the overhead of storing multiple copies

Efficient Large and Small File Storage

Small files stored in cache and written to a single disk through box-carting

Large files over 128MB in size are erasure coded immediately vs triple mirroring and erasure coded later

Provides up to 20% higher throughput for larger files

Fully Geo-distributed High Availability & Protection

A geographically distributed environment that acts as single logical resource

Active/Active platform with access to content through a single global namespace

System securely and automatically separates Namespaces, object buckets and users

Integration with LDAP and AD environments

Ensures the integrity of customers’ stored data

Built-in Metadata search

Integrated Metadata storage – store metadata using the same constructs as objects eliminating the need for a separate database and infrastructure to run it.

Metadata search via SQL construct

Enables applications and users to query metadata using SQL constructs. Supports several attribute and sort functions.

Global metadata search

Enables applications and users to search across the global namespace.

So what’s really new in 3.0?

So this ECS stuff is great, but what’s exciting about 3.0?

Advanced Retention Management

Event Based Retention – Enables application to specify retention period that will start when a specified event occurs

Litigation Hold – Enables application to temporarily prevent deletion of an object that is subject to an investigation or legal action

Min/Max Governor – Enables system administrator to specify a min and max value for the default retention period

This unblocks Centera customers using ARM from migration to ECS. I’m actually really excited about this, mainly because I was a big Centera fanboy and have found it difficult to put forward other EMC solutions to replace it for customers heavily leveraging ARM.

SNMP Traps Support

ECS 3.0 will support for SNMP Traps for ECS critical events

SNMP Traps is an optional feature, based on whether system admin configures SNMP information via UI/API

When configured, ECS sends a SNMP Trap to the configured server for any event that causes an alert on the management API

ECS supports the ability to configure up to 10 SNMP Trap Destination targets

SNMPv2 and SNMPv3 (USM mode) support

SNMP Query Service support (CPU & Memory)

Remote Syslog Support

Shipping ECS Monitoring & Diagnostics logs to a remote syslog server

Ability to forward all ECS Audit Logs and ECS Alerts to a centralized Syslog server

Forward OS syslog messages

Support for UDP and TCP based communication with syslog servers

Support for multiple redundant syslog servers , all active

Distributed service, resilient to node failures

Only System Admin can perform syslog management operations

Specify a severity threshold of logs to be forwarded

Ability to Add, Edit and Delete Syslog server configuration from the portal and REST API

Logs can be seen on ECS nodes in /var/log/<node IP>/syslog.log

Platform Lockdown

ECS will support a ability to do the following via the ECS RESTful management API

Lockdown an entire cluster

Lockdown a specific node

Unlock a locked node

A new management user role, the lock admin user, for locking is defined that will have the privilege of locking/unlocking the cluster.

In ECS 3.0 this will be a pre-provisioned local user ‘emcsecurity’.

The lock admin user i.e. ‘emcsecurity’ will have the ability to

Modify their password (forced during first login)

Lock the cluster

Lock a node

Unlock a node

System admin/monitor has the privilege to view the lock status of a node but NOT modify it

Thoughts and Further Reading

I’ve been talking to a lot of scale-out object storage folks lately. Given the amount of EMC stuff I’ve covered here previously, it’s a little surprising that this is the first time I’ve posted about ECS. That doesn’t mean it’s the first time I’ve looked, and I’ve had customers looking at it fairly seriously. In my opinion, the advanced retention management in 3.0 is really going to put a few customers over the line and finally give them the confidence to throw their Centera grids in the river (figuratively speaking).

The cool thing about ECS, like a lot of these types of solutions, is that you can consume it on your terms, via

Appliance;

Software defined;

Dedicated cloud; and

Multi-tenant Storage Cloud.

If you’re feeling keen on ECS, you can take it for a spin here. You can also download a version for free, non-production use here. Grab the datasheet from here.

working for minimum rage

taking the social out of social networking

buy me a pony

photos of food

disclaimer

The opinions expressed here are my personal opinions. Content published here is not read or approved in advance by my employer and does not necessarily reflect the views and opinions of my employers, previous or current. This is my blog.

Search

Search

Subscribe to PenguinPunk.net by email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.