New plumbing connectors for Unix

[fanf]

So this weekend I was playing around with my log summarizer, which produces a line of output for each second with one character for each significant event coloured according to the machine on which it occurred. The output is all real-time, and quite mesmerizing to watch. Behind the cut is a static sample of the output.

All my email relays are logging via syslog to our admin box, which writes the output to a fifo. The summarizer can read the fifo directly, but that means only one person can view it at a time, and I can't do anything else with the log data feed while it is running.

So I wrote a little multiplexing program which will reads from a pipe or a fifo, and listens on a socket (unix domain or internet). Clients can connect to the socket and the multiplexer feeds them all a copy of the data it reads from the input. If the multiplexer is listening on a network socket then telnet is a suitable client, but I wrote a simpler one for use with unix domain sockets.

So the result of this is that I can write:

logmux -f /var/log/ppsw_fifo -s /home/fanf2/logs/test_sock -c &

so whenever I connect to test_sock I get a copy of the amalgamated ppsw log feed.

loguse -s /home/fanf2/logs/test_sock | logfilter | logmux -s :2000

and I can peel a feed off, pipe it through my shiny summarizer, and provide the output as a service on port 2000. I can then just telnet to it and get a feed.

It seems to me that logmux and loguse might turn out to be quite useful in the future.


A quick key to the below:

< is a message coming in.

> is a message going out, and - is used when the same copy is delivered to multiple recipients.

Lower-case j is a message that has been accepted and which SpamAssassin thinks is spam; similarly v is used for viruses (which are usually deleted).

O is for rejected relaying attempts, A is for attempted unauthorized use of our smarthost services, H is a rejection because of incorrect HELO domain, P is rejection because of SMTP protocol violation, D is rejection because of DNS blacklisting.

S is rejection due to invalid sender address, and lower-case s is temporary failure to validate the sender.

R is rejection because of invalid recipient adress, and the rare lower-case r is temporary failure to validate the recipient (which we are usually lenient about).

The first number on the right is the smoothed number of actions per second, the second number is the percentage of email that is rejected, and the third number is the percentage of accepted email that SpamAssassin thinks is junk.


Jul 26 14:24:13 D>HR>sH<<RR<RRD<HvvDDRPD<RD>Rj>DD                                          32 83 8

Jul 26 14:24:14 D<DRRD>RD>H<<RRHsR<RRDRRRD>RDD<                                            32 84 7

Jul 26 14:24:15 HD>SRHj<sD<RRDsHsR<sR                                                      31 84 8

Jul 26 14:24:16 <RRR>RR><R><<>s<RR>DDR>D<R                                                 30 84 7

Jul 26 14:24:17 >D>>>>HsR><H<<sHRDD<R>vv<>>Ds>DH                                           30 83 7

Jul 26 14:24:18 >--Ds<sjH<DDHR>D>>s>s<S<sR>HR>D                                            30 82 8

Jul 26 14:24:19 s<RssHRs<Ds<>RRHD><RDssj<>DRjR                                             30 83 11

Jul 26 14:24:20 D<R<HsH>>DH<>R>sDDS>R><>sRRSDj>D><                                         31 82 12

Jul 26 14:24:21 H<<<s>HDRR<>>>SH>HSR><H>H>P<                                               30 82 10

Jul 26 14:24:22 <R<<<DRDD>S>P><>RH>HRR                                                     29 82 9

Jul 26 14:24:23 <DH<>S>>Hss>H<>jj>DDHDD<>HRss<RR>P                                         30 81 12

Jul 26 14:24:24 DRSRD<Hj>>vv<RHPDS>>>D>s>D>DR<<DHSHHH>>>b><RPDDD                           32 81 13

Jul 26 14:24:25 D>RR<DS<>>RD<Ds>R>RRSDDD>>SD<RS<DH<R>>HRD                                  32 81 11

Jul 26 14:24:26 HD><HD>HR>bHDD<s>>R<>>>>--RD>-D>DD<                                        33 80 10

Jul 26 14:24:27 >RD<s>>>RsDRRHD>->R<H><DD>sD                                               32 79 10

Jul 26 14:24:28 sR<<jRP>>D>RDRDsHS<DD>D                                                    31 80 11

Jul 26 14:24:29 HDssRR<<DRD<>DDPDHRRR>>D><HsDD>                                            31 80 10

Jul 26 14:24:30 bRDDD>>AHDR>><>RDD>RRR><R<sR<H>Hs<                                         31 80 9

Jul 26 14:24:31 <HjDRD<DDRH<SRDDDDH<<<sHHD<jHjP<                                           31 82 14

Jul 26 14:24:32 DHHDHRRHPs<<DHR>Dss>--PRH>D>DDH>R><>><RDR>>>->>R>><>>><<R<<<S              34 80 11

Jul 26 14:24:33 >---jjR<><RHRRRRD<<><SRsH<sD<RsR<j>>R<>>H<D<<>><HsD<>-----R                37 79 13

Jul 26 14:24:34 >H<DD>RH<S<<>><DR>------<>--------------RD<HH                              37 75 12

Jul 26 14:24:35 D><PHR<>s<ss<<DDRHH<<>RRD                                                  36 76 10

Jul 26 14:24:36 DHRHDR>RDRRD<<SsDH<H>P>D>RDD>>                                             36 77 10

Jul 26 14:24:37 RDDDR<HDDDRDR>><HD>sHDD<<>>>>>R>D<>H>Ds                                    36 77 9

Jul 26 14:24:38 RDR<DDRR>jRjDDD<<>RHR>-<<jRR>>D>H>>>H                                      36 77 13

Jul 26 14:24:39 >R<>><HD<ss<<DR<HHssDHsssRRD>D>DP<                                         36 78 12

Jul 26 14:24:40 >SRDRsDDR<s<SHD<sDsHbRsR>RR<ssDDHDR                                        36 79 11

Jul 26 14:24:41 RS<H>DDHHDH>><>HsRPDDsAHb<>DDD>>DRR<>>                                     36 79 10

Jul 26 14:24:42 HHsDDR>>>>H<<>HRPj>RH>><>D>DD>HH<>><RH                                     36 79 11

Jul 26 14:24:43 s>DSR<>HRPSDHR<sssHHHR<<D>                                                 35 79 10

Jul 26 14:24:44 DRR><>j>--js>>R>><<HDR>R>><RRR>DR<<R<H                                     35 79 12

Jul 26 14:24:45 PH<<H<RRDH>HDDsD>H<>D                                                      34 79 11

Jul 26 14:24:46 HPH<sRRs<DPRRDDDsDRHD<R<RP                                                 33 80 10

Jul 26 14:24:47 DH>>R>R><s>DD>>jRs>->R>RD<PR>DD>>>Rs<SH                                    33 79 12

Jul 26 14:24:48 DDRAssR<R<>D<HRRsDHR>>HR>D>R<R>                                            33 80 11

Jul 26 14:24:49 D<RsHD>R<>HDRDDR>R><R><>><>Dj<HRH<><><DsDHH<<R                             34 80 10

Jul 26 14:24:50 jD<DRHDD>s<sP<jHD<Rs<SRDHH>SRR>R                                           34 81 13

Jul 26 14:24:51 <HDRR>H>R>Rss<>RH<H<DR<<PH><                                               33 81 11

Jul 26 14:24:52 RD<<<<Rj<HHHDvv>DH<HRHR>-<R<R<PDDvvRHHDHD                                  34 82 11

Jul 26 14:24:53 jRDRRj<RHS>Hsss>>DRDRR>>>sR><>>R>>>j                                       34 81 16

Jul 26 14:24:54 RDD>>DHHDPHDssH<P<H>HH><                                                   33 81 15

Jul 26 14:24:55 >SD<>PDDD>sDRDRH>>RDrHHs<DSRRRDR>>                                         33 81 15

Jul 26 14:24:56 DRsDsH<><>D<><<<<R<<<                                                      32 82 12

Jul 26 14:24:57 H<DRD<DD<DRH<R>HHR<HsD<>D                                                  31 83 11

Jul 26 14:24:58 <<RHRDDDD<>DDDR>RH><<R>Dj<                                                 31 83 11

Jul 26 14:24:59 <H<Ds>>>DDDRRDRDD<<<<RD>R                                                  30 83 10

Jul 26 14:25:00 D>ss>HD<vv>DRH<S>RD>R>>>>sDR<Rs>>DRRD                                      31 82 9

Jul 26 14:25:01 DD>sRA<DH>>><<D<H>s>><                                                     30 82 8

Jul 26 14:25:02 >R><>RR>R<D<RRDD<jsDH                                                      29 82 10



Comments

[sbp.livejournal.com]

That's pretty cool. And 80% attempted deliveries rejected.

[senji.livejournal.com]

What is lower case b ?