For a 100,000 document DB that I was trying to work on, I spent a
little time tweaking main.js o reduce object creation (especially
arrays) as well as toJSON which creates an inner closure object on
each invocation. I'm definitely getting some speed-ups here though I
haven't benchmarked to get exact numbers.
Some of the tweaks I did were:
- Cache all the regex's created in toJSON
- Create a const map object that toJSON uses to minimize object creation
- Instead of reinitializing map_results to [], just set the length to zero
- This last one, I looked at the spidermonkey's code to see what
happens when you do this and it seems to be pretty efficient in
truncating arrays.
- One another possibility that I've been thinking is to move sum into
couch_js.c so it's native C. This is a very commonly used function
that it deserves to be fast.
Intuitively, no matter what language, object creation has a price to
pay. Anything we can do to cache objects and not have to create them
repeatedly helps the GC tremendously (million documents * #objects
created to map/reduce them).
I'm attaching what I have (not a diff, just the final main.js) and I
didn't spend the time to try each change and measure the results.
YMMV and glad that one change helped.
K.
On Wed, Apr 1, 2009 at 12:30 PM, Paul Davis <paul.joseph.davis@gmail.com> wrote:
> On Wed, Apr 1, 2009 at 3:23 PM, Manjunath Somashekhar
> <manjunath_somashekhar@yahoo.com> wrote:
>>
>> hi All,
>>
>> Would like to thank you all for such fantastic participation. Got to learn a lot
apart from the question i asked :).
>>
>> I did make a change as kowsik suggested - although aware of the include_docs feature
nvr thought it would make such a big difference. The results are pretty amazing - the view
generation completes in about 25 mins - just one execution. Will need to perform a few more
runs before getting a good average time.
>>
>
> Wow, that's way beyond what I would've expected. Do please try and get
> some solid numbers on this. Perhaps we should start a wiki section to
> try and gather some number reports for reference?
>
>> Thanks a lot. Do let me know if any more optimizations can be done.
>>
>> Thanks
>> Manju
>>
>>
>>
>>
>