Re: Want some examples why NSM is a piece of junk?

@crypto: You are right about the attitude of vendor sales in general. Even with the large variations in sales behavior within an organization, it is a truism that vendor sales is a lot more aggressive and bullish about a vendor's product than VAR sales. I have seen that with every vendor I work with.

And that's where organizations like yours and mine come in: VARs that do have a clue, that have the consultative discussion with the customer before bringing the vendor into the room. VARs that qualify an opportunity as a good fit and aren't afraid to say "we'd rather tackle this another way" or even "we don't think we have a product that will be a great fit for you here."

I could tilt at windmills and demand that vendor sales behave differently, but the reality is, there's a reason they behave that way. They have incentive to. It's a rational, if at times short-sighted, behavior.

I'd rather focus on what can be done: Know which vendor AEs in which geographic patch show the ability to truly partner with a VAR and to let us lead, and be that qualification resource for my own AEs so we have successful engagements.

Re: Want some examples why NSM is a piece of junk?

The current JTAC recommended release for branch SRX devices is 10.4R6.5. You likely needed a feature introduced in 11.1, and I'm sorry you ran into trouble with that release. "Solid" != bug-free and I apologize if I gave that impression.

[in the interest of transparency, blog-like detail follows]

There was a time when your firewall, router, switch, UTM (if available), web-filtering, DHCP server, etc were all seperate devices. Now...let's suppose there was one big bug in each one. Pretty annoying, but also pretty typical, and usually some way to work around in the end - and you had the option of substituting anything (including the vendor) that really got in the way -- probably 5 different vendors as well.

Now roll all those features into a single (SRX) device. 5 big bugs in one device is catastrophic - but lucky for you only 1 throat to choke . I bring that up not to be too much of an apologist -- we shouldn't build things we can't adequately test. But the bar is raised in both subjective experience and sheer technical complexity. There are literally more tests that *could* be done then there are hydrogen atoms in the universe - a consequence of combinatorial math. So we try to be smart about it, and hit the combinations we think are most likely. Not always successfully, and every technology vendor faces the same problem (please if you find one building bug-free products, let me know -- I want to buy their stock*). There are code analysis tools and other techniques - but bench-tests are still bread and butter for complex systems.

The DHCP problem you ran into is known as a test-escape and there is a process to analyze how we missed it, and to make sure that it's captured in the next build of the test-bench. [Maybe we don't test DHCP with IP spoofing enabled and so our model configurations need tweaking]. There's also a process to analyze how the bug got introduced in the first place [maybe DHCP code is not sufficiently abstracted from spoofing code - or something they both rely on does sloppy memory management - etc].

We were late releasing 11.2 precisely because we are doing more tests, and taking more time because of lessons learned on 10.4R1 - a release frankly not as solid as I expected it to be (11.1R1 was already out when 10.4 field data really came in). 11.2R1 has the benefit of those lessons plus a lot of new features -- and feedback so far is quite positive. [note again that doesn't mean bug-free]. And 10.4R6 is being rapidly and successfully deployed by a large number of customers.

There is absolutely no question that we can do better on initial quality. Our goal is that customers can ultimatey deploy (again) on R1 releases. Our largest customers spend a great deal of time "qualifying" a release before deploying it to production - they do this with every vendor. We know that most enterprise customers don't have that luxury and that's one of the main reasons for the JTAC Recommended Releases document. If you deviate from those recommendations, I can't stress strongly enough the need for extensive testing and as @tbehrens suggests - a cautious approach.

At the same time, we're working on improving how well and how quickly we communicate known issues to the field (mentioned earlier) and the tools we make available for doing upgrade and deployment analysis. These are large, complicated projects involving large numbers of people and process change. So no overnight magic, but many improvements are already in place and more to come.

We will also do better at documenting the "envelopes of coverage" and providing tools to compare your own installations against known best-practice examples.

Getting back to NSM - I've had a chance to see the preliminary list of fixes planned for the upcoming 2010.3 build and it's...huge. This is an enormous engineering investment and it's my sincere belief that you will all see a great improvement.

Re: Want some examples why NSM is a piece of junk?

Funny thing is (and I stated that previously) that if you ask any of the Juniper sales reps about the state of the SRX, they will tell you a completely different story than the one you impressively depicted above. And that's one of those points where I get upset.

Only if you directly pinpoint them to problems and show them that you've had your share of experience with them, they start to admit and come up with roadmaps and promises. However, if you have no clue about the situation and you invite Juniper to a first sales meeting, they will sell you NSM+SRX.

I share this sentiment, but I will also say that it's not just Juniper that does this.

The golden rule of IT: Vendors Lie.

It's a cynical and harsh stance to take, but I have found it to be critical to avoid being taken advantage of by sales weasels**. It's unfortunate that the only information [potential] customers have access to is public stuff -- data sheets, FAQs, product pages, etc. None of those go into the real nuts-and-bolts of a product's true internal workings, nor its deficiencies, nor its current state of feature completeness, etc. If vendors all started being honest about what their products do and do not do well with, it would be great, but hopefully nobody is naive enough to think that vendors would actually all be truly and completely honest. It is the world we live in. That is why I'm a vendor's worst nightmare. I dig and dig and dig and research the hell out of things before I bring a vendor on-site. I leverage their competitors to fill me in on where products don't deliver or severe deficiences. The quickest way to find out where Juniper products are lacking is to ask Cisco, SonicWALL, Fortinet, Palo Alto, etc., and the same is true in any combination (ask Juniper about Cisco, etc.) They'll exaggerate and lie, too, but you can filter that information and use it to fine-tune and hone the questions you ask to really discover some nitty-gritty about a product when you do meet with the sales team ("so I've heard that your management software still doesn't fully support the SRX products... can you elaborate on that?")

The other invaluable resource is to talk to other customers. The sales weasels will gladly hand you a list of references that they WANT you to call, because either those customers had a fantastic deployment experience, or frankly, some of them have flat-out been bribed to say nice things (unethical, but it happens. It's been offered to me before, the vendor who tried it shall remain nameless). The people you want to talk to are the ones who *aren't* on the list of references. You have to ask around, explore your social network connections, and maybe even try a few cold calls. It's a lot of work. It's a pain in the butt, but that's why the phrase is "caveat emptor." You've got to do with the work and the due diligence to protect yourself, it's your job to protect yourself, not the vendor's job to protect you. It's the vendor's job to make sales and generate revenue.

** Not all sales teams are sales weasels. I reserve that term for the lot who earn that designation.

Re: Want some examples why NSM is a piece of junk?

Getting back to NSM - I've had a chance to see the preliminary list of fixes planned for the upcoming 2010.3 build and it's...huge. This is an enormous engineering investment and it's my sincere belief that you will all see a great improvement.

2010.3?

2010.3 is over a year old now... 2010.4 has been available for nearly 10 months (Nov. 16, 2010) and 2011.1 has been available since Feb of this year (6 months).

I don't know of a recommended releases page for NSM, so I've not seen anything official that says to stay on 2010.3. I've been advised by JTAC to use 2011.1 due to some other issues we've had, and that's where I'm at. It will be disappointing if the bulk of the development efforts are focused on an old release. I'm sure I'm not alone in that I've moved on past 2010.3, I'm sure a lot of customers would prefer to see the 2011 train take a better shape.

Re: Want some examples why NSM is a piece of junk?

KB_Fan wrote:2010.3 is the first target. I'll pass this feedback along to the product team and update here as I learn more.

See Keith, this is just one of those moments. I as a customer and juniper partner don't get this. 2010.3 is such an old version, most people (I am guessing) who have to manage SRX with NSM are probably on 2011.1 or at least 2010.4. And what happens? Bug fixes flow into 2010.3.

I am eager to hear what NSM product team has to say about *current* versions.

Re: Want some examples why NSM is a piece of junk?

KB_Fan wrote:2010.3 is the first target. I'll pass this feedback along to the product team and update here as I learn more.

See Keith, this is just one of those moments. I as a customer and juniper partner don't get this. 2010.3 is such an old version, most people (I am guessing) who have to manage SRX with NSM are probably on 2011.1 or at least 2010.4. And what happens? Bug fixes flow into 2010.3.

I am eager to hear what NSM product team has to say about *current* versions.

I got a good explanation from JTAC - we're doing things I think for the right reason, but there are some obstacles to doing what might seem the "obvious" right thing.

2010.4 introduced some instability that is somewhat inherent in that release. For most customers, 2010.3 has proven to be a better release. So that's why we chose it for the first target. We do not recommend upgrading to 2010.4.

(Unfortunately you can't downgrade from 2010.4 to 2010.3)

We have a 2011.x stability release planned for later this year, per the product team statement. (we don't know what x is yet)

People already on 2010.4 or a 2011 release should wait for the 2011.x stability release later this year. If they can't wait contact JTAC for patch releases or other workarounds to any major issues being seen.

We will continue to fully support 2010.3, 2010.4, etc per those policies.

I didn't really make a statement regarding 2010.3 vs the 2011 train quality. We are putting all current effort into 2010.3 and then can move on to 2011.x train mentioned so this is really a resource allocation plan.

The upcoming 2010.3 release will likely be 2010.3R2. The other part of the numbering indicates changes in feature payload.

Re: Want some examples why NSM is a piece of junk?

I believe it will be 2011.1R2 (with the current version being an implicit R1). The year.number convention implies new features, and these are intended to be maintenance releases only.

So while they "repair" 2011.1 in form of 2011.1R2 they also work on 2011.2 to add new features I guess. What will 2011.2 be based on? On the fixed 2011.1R2? Or would the R2 fixes have to be applied to 2011.2.

I don't get all these different versions. Seems very complicated to keep all these different "trains" running.

Re: Want some examples why NSM is a piece of junk?

Can you specify the platform, what you were trying to do and the version of NSM? That will help in two areas - 1) other customers avoiding the issue, and 2) I can try to validate that this would be resolved in the upcoming stability release (do you already have a PR?)

Re: Want some examples why NSM is a piece of junk?

Can you specify the platform, what you were trying to do and the version of NSM? That will help in two areas - 1) other customers avoiding the issue, and 2) I can try to validate that this would be resolved in the upcoming stability release (do you already have a PR?)

I don't have a PR.

This is NSM 2011.1a10 patch release. We downgraded two SRX650 from Junos 11.1R3.5 to 10.4R6.5. After that NSM recognized the version change and asked us to change the version on NSM. We did that and ended up getting this error message.

Our NSM was using schema version 197 so I went ahead and upgraded the schema to the latest version (forgot the number, 20x) but that didn't help.

Opened a JTAC case before I left the office and hope to find some answers when I return there on monday.

Re: Want some examples why NSM is a piece of junk?

> Limitations> The following items are known limitations in this version of NSM • NSM does not support Junos OS downgrades. However, if you need to downgrade a> device, follow these steps 1. From the device, use the CLI command to downgrade the image. For example root> request system software add <package-name> reboot> 2. After the downgrade, from NSM, delete the device and then add it again.

from NSM release notes.

@Keith: Please press for an update on when the 2011.1 bug fix release will be available. This is just not bearable.

I had thought I was going to get some advance notice on this, but unfortunately did not, so apologies.

The 2011.x story is not one that is going to resolve soon - I think we are trying for something before the end of the year, but it is increasingly looking like it may slip into 2012 (and thus would be a 2012.x release)

This is disappointing for many, and those with specific high-priority issues I would recommend contacting JTAC as there may be current or upcoming patch releases to resolve those issues.