Welcome to Splunk Answers, a Q&A forum for users to find answers to questions about deploying, managing, and using Splunk products. Contributors of all backgrounds and levels of expertise come here to find solutions to their issues, and to help other users in the Splunk community with their own questions.

This quick tutorial will help you get started with key features to help you find the answers you need. You will receive 10 karma points upon successful completion!

Refine your search:

How much RAM do I need to avoid I/O problems?

0

I have noticed a correlation between RAM usage and I/O on my indexers. Whenever RAM usage goes around or above 50%, I/O usage goes crazy. My understanding is it's because spare RAM is used by the OS as I/O cache. We want to add more RAM to the indexers to help but I have two questions:

1) how much RAM is too much? We're thinking of doubling it, but would there be much value for performance and/or future-proofing in quadrupling it instead? (note: we're using Enterprise Security quite extensively)

2) can anybody give me recent official-looking references backing this up? This is required to justify the cost to people who might say "why do you need more memory, you're only using half at the moment?". All I have so far is this and that ancient articles, as well as somebody saying "I'm not from splunk so I can say it: you need as much RAM as you can afford" in the "It Seemed Like a Good Idea at the Time...Architectural Anti-Patterns" talk from conf 2016.

2 Answers

Take a look at the documentation, especially if you need a source to quote: Reference Hardware in the Capacity Planning manual is a good starting point. Reading (or at least browsing) the whole manual is a good idea...

In the recommendations for indexers, you will find the following memory sizes:

Basic: 12 GB RAM

Mid-range: 64 GB RAM

High performance: 128 GB RAM

Finally, I contend that caching is a perfectly valid use of memory if it improves overall performance!

You should contact your splunk customer service team, ie your sales rep. You aren't trying to buy anything right now, but they are there to make sure you love the product, and I would bet that they have MILES of use cases and Excel spreadsheets and whatnot to accurately answer your questions. More importantly, they will be able to ask YOU the questions that are significant to ensuring the answer be well-fit to your particular situation and projections.

(And, by the way, if they help you do this right, then your users will be served well, usage of splunk will rise in your business and the splunk company will eventually make more money: win win win.)

I would bet they would ask you a few usage questions, like, How many simultaneous users do you expect: total, how many on each search head? At what rate will this grow? How many servers, what architecture, clustered, replicated, etc? What version of splunk? What kind of data are you searching through? What percentage of the searches are of underlying detail? What percentage of the searches are accelerated?

And then there are more advanced nudges, like, How well have you reviewed your data models against the usage? How are you making sure that your users know they are not supposed to boil the ocean? How are you making sure they know HOW not to boil the ocean?

Anyway, call splunk for support, and they'll give you the ammo you need to make your decision right.

By the way, "using half your RAM" isn't a thing. It's all being used for something.

Also, "too much RAM" isn't a thing either. I remember back in the dark ages, when the Internet was made up of two tin cans and a piece of string, and a PC came out with 256K RAM. I wondered and joked with my programmer friends why the PC designers were being so ridiculous and how they could ever use up all that space...

However, with regard to right-sizing your RAM, chat with customer support and compare the cost/benefit of more RAM with the cost/benefit of more servers. I'd expect if there's any doubt which one to do, you should do the servers.