High loads...

[Bill Davidsen]

Russ Allbery wrote:
> Bjorn Hellqvist <bjhell at telia.net> writes:
>
>   
>> Ok, I'll check that out next week when I have more time on my hands.
>>     
>
>   
>> But isnt forking a waste of resources? wouldnt it be better to do
>> something like apache does? fork a couple of nnrpd that then can do
>> threading? like only have 20 forks and each fork can thread 100
>> processes?
>>     
>
> Forking is extremely fast on Linux, and not as heavy as people think it is
> even on other operating systems.  However, nnrpd does also support running
> in that mode if you want.
>
> Remember, nnrpd processes are *much* longer-lived than Apache requests.
> Web requests frequently live on the order of seconds, whereas nnrpd
> processes frequently live on the order of hours.  Startup overhead for
> nnrpd is mostly noise.
>
>   
Depends on the client. Some misbegotten garbage software opens a socket 
for each subscribed group at startup, and retry as fast as they can 
until success. Even with the proper limitation of max sockets that can 
generate a visible load spike. Limiting the connect rate in iptables is 
possible, but only moderately successful, since that, too, takes some CPU.

I'd love to do that in threads, but to do it both with and without is 
more complex, and some people don't have them, or have crappy 
implementations, or refuse to use them on religious grounds. Same 
opinion with innd, one thread per connected IP address would be good, 
select() doesn't scale well to a well fed machine. At least on Linux and 
Solaris, by measurements I've seen floating around.

-- 
bill davidsen <davidsen at tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979