Graphite

multiple carbon instances on the same server

Asked by FRLinux

I have multiple setups of graphite on different sites.

I am currently looking at getting graphite to process data faster on a given server. For this, I am considering starting several carbon-cache.py instances on different ports and PIDs (2003,2004,etc...)

Is it likely to work as I expect it should or do you foresee some specific shortcomings with this approach?

Cheers,
Steph

Question information

Language:
English Edit question
Status:
Solved
For:
Graphite Edit question
Assignee:
No assignee Edit question
Solved by:
chrismd
Solved:
Last query:
Last reply:
Revision history for this message
Best chrismd (chrismd) said : 		#1

Running multiple carbon instances is a great way to add capacity as long as there isn't an underlying system resource bottleneck. That said, the way instancing works is about to change quite dramatically. I have a branch in progress that will be hitting trunk next week that introduces a new configuration model for carbon. Basically instead of there being three different types of carbon daemons with disparate features there is going to be a single carbon-daemon.py whose behavior is completely configurable, and there is a per-instance configuration directory. This reduces the number of daemons you need to run in some situations and thus reduces serialization overhead significantly.

So... if your need is immediate or you want to go with the tried and true method, go ahead and set it up with 0.9.9. You just have to use --instance=foo, and define a [cache:foo] section in carbon.conf for each instance, and each has to use different ports. The next release will include the new model so it may be good to wait a week or so until the new stuff lands on trunk. Also, the new changes include support for the new ceres database (webapp support is coming next) and a major cleanup of carbon's own metric names.

I'd love to have another tester so if you go the latter route feel free to ping me with questions/issues as much as you want :)

Revision history for this message
FRLinux (frlinux-frlinux) said : 		#2

Hello Chris,

Thanks for your reply, this is fairly exciting news. I will not be able to put that into production for now (the 1.1.x branch) as we heavily rely on graphite to be rock stable :) but will implement some more test servers to experiment with it. I was not aware that 0.9.9 was actually out. Will do some testing with it possibly and see if I can upgrade the current infrastructure.

Thanks for your reply.
Steph

Revision history for this message
FRLinux (frlinux-frlinux) said : 		#3

Thanks chrismd, that solved my question.