Postfix/Heavy_duty_local_delivery
From http://archives.neohapsis.com/archives/postfix/2009-11/0170.html
User wants to delivery a lot of stuff to a local pipe, but it's topping out around 200-300 per second. When the postfix queue starts to fill up, it all goes to shit.
Posters point out that his disk subsystem can only do a couple hundred fsync()s per second, which is probably what they're running into.
From Victor:
Stop right there. At 200 msgs/sec, fork/exec of scripts via local(8) is never going to give you the right performance. You need (in addition to the faster disks already mentioned), to use a resident (pre-forked) SMTP or LMTP server script to receive the mail in question. - Do not use local(8) - Do not start a new interpreter process for each message
And a saner solution:
Does this application really need Postfix and a queue? Why not just turn the script into an SMTP server that pre-forks a fixed number of copies and loops receiving/processing email? For 200-300 msgs/sec on an I/O constrained server with a single "mailbox", one really does not need Postfix, and can't afford the I/O cost of a local queue. You are not going to succeed without more spindles and likely more CPUs. Just launching a "Perl" hello-world script takes >10ms on a fast machine, do that 100 times a second and you've burned a CPU, and non-trivial scripts are slower still.
Also of note was that he could just throw a metric fucktonne of disk spindles at the problem.