November 2007 Archives
No, it's not my birthday, and I'm still not even 40 yet... but it is official.
Yesterday in the mail I received a thing from the Smithsonian Institute, trying to get me to subscribe to their magazine. Normally I skim through said material and then file it in the RoundFile(tm). However, I was shocked to see that that this offer came from their "Senior Discount Services" group, and I was being offered their "Senior Discount Rate" of only $12/year.
*sigh*
One does have to figure that the Smithsonian folks *do* know what "old" is :)
Yesterday in the mail I received a thing from the Smithsonian Institute, trying to get me to subscribe to their magazine. Normally I skim through said material and then file it in the RoundFile(tm). However, I was shocked to see that that this offer came from their "Senior Discount Services" group, and I was being offered their "Senior Discount Rate" of only $12/year.
*sigh*
One does have to figure that the Smithsonian folks *do* know what "old" is :)
I hate moving. More than that, I hate having to find a new place to live. I'm not totally enthralled with the commute pattern to/from my place (I know it's relatively short, but the fact that it can vary from 12min to 60mins is really hard on things sometimes). I also have grown quite a certain disdain for STAIRS. Four flights of stairs here. Don't get me wrong, once you get into my current condo I'm renting, it's very nice. I think the layout is mis-used, and as such you lose precious living space to hallways and a breakfast area that really isn't well utilized. And couple that with the fact that I'm just paying too much per sqft for this place (in the current market?).
So... I'm hunting. I'll probably have to keep renting for now. I'm still, after 4 yrs in California, dealing with "sticker shock" of housing here. But I'm going to try to find a house to rent that I can afford. Hoping folks at work can help with thoughts, ideas, opinions. I don't mind another condo, necessarily-- but a house would be really nice.
I'll keep you updated & informed.
So... I'm hunting. I'll probably have to keep renting for now. I'm still, after 4 yrs in California, dealing with "sticker shock" of housing here. But I'm going to try to find a house to rent that I can afford. Hoping folks at work can help with thoughts, ideas, opinions. I don't mind another condo, necessarily-- but a house would be really nice.
I'll keep you updated & informed.
final uptime: 2:13PM up 879 days, 17:50, 3 users, load averages: 0.28, 0.11, 0.33
*** FINAL System shutdown message from kirk@shockwave.concernd.com ***
System going down IMMEDIATELY
Nov 22 14:13:42 shockwave shutdown: reboot by kirk:
Nov 22 14:13:42 shockwave /kernel: Nov 22 14:13:42 shockwave shutdown: reboot by kirk:
*** FINAL System shutdown message from kirk@shockwave.concernd.com ***
System going down IMMEDIATELY
Nov 22 14:13:42 shockwave shutdown: reboot by kirk:
Nov 22 14:13:42 shockwave /kernel: Nov 22 14:13:42 shockwave shutdown: reboot by kirk:
Yesterday I started into some web-programming at work; just a small project at work which none of my staff really had time to do, but was so simple. I made a huge amount of progress today on it (making it pretty, usable, etc)(it's for end-users). Felt good to get "dirty" in PHP again.
That said, I then dove back into my anti-spam project tonight. Instead of writing a daemon to process syslog in real-time, I'll start simple with batch processing. A couple times a day I can run a processor-script against my syslog and find the spam source-hosts and blacklist them (via ipfw). I may go to using a daemon later on, if it makes sense.
I'm running a test of my processing-script now against today's syslog (since midnight). I'm up to 6:30am in the log and already 1651 hosts have been logged. As I watch the log in real-time, I'm only seeing 1 to 5 new hosts per minute (think of that as spam-per-minute).
Today I've rejected over 64K spam messages (and received 100 valid emails on the server). SIXTY-FOUR THOUSAND! A few months ago it was 1/4 of that.
BTW, I did traffic analysis on why my server was seeing 13GB of traffic/month. The excess 10GB I could not account for is all spam/spam-related. Meaning - either SMTP traffic - or DNS traffic -- the vast majority of which are DNS look-ups on dns-blacklists. Basically any email coming in, getting thru the greylist, then gets 5-7 DNS queries made. So I've taken out those DNSBL's which are not significantly contributing. Maybe they block 50 spam a day. Compared to 64K, I save traffic (and money) by just rejecting it based on "unknown user" once, rather than doing all the DNS queries.
The processing script is doing it's job well. I'm now only seeing new hosts every couple of minutes in the logs (I'm at 10:45a now -- 2163 blocked hosts). I tell ya, it's like watching the Matrix sometimes -- seeing patterns in the data, and understanding the cause/effect of it all.
Processing complete. Not the fastest script in the world - must have taken 20 mins to run (and that wasn't even on a full 24-hr log file -- only 18hrs,49mins worth). But, after that, 2,447 hosts to be blocked. More trivia for you -- of those 2,447 hosts, 991 were only seen once-- 1,456 were seen at least twice.
Now I have to write the script which will handle the firewall processing :)
That said, I then dove back into my anti-spam project tonight. Instead of writing a daemon to process syslog in real-time, I'll start simple with batch processing. A couple times a day I can run a processor-script against my syslog and find the spam source-hosts and blacklist them (via ipfw). I may go to using a daemon later on, if it makes sense.
I'm running a test of my processing-script now against today's syslog (since midnight). I'm up to 6:30am in the log and already 1651 hosts have been logged. As I watch the log in real-time, I'm only seeing 1 to 5 new hosts per minute (think of that as spam-per-minute).
Today I've rejected over 64K spam messages (and received 100 valid emails on the server). SIXTY-FOUR THOUSAND! A few months ago it was 1/4 of that.
BTW, I did traffic analysis on why my server was seeing 13GB of traffic/month. The excess 10GB I could not account for is all spam/spam-related. Meaning - either SMTP traffic - or DNS traffic -- the vast majority of which are DNS look-ups on dns-blacklists. Basically any email coming in, getting thru the greylist, then gets 5-7 DNS queries made. So I've taken out those DNSBL's which are not significantly contributing. Maybe they block 50 spam a day. Compared to 64K, I save traffic (and money) by just rejecting it based on "unknown user" once, rather than doing all the DNS queries.
The processing script is doing it's job well. I'm now only seeing new hosts every couple of minutes in the logs (I'm at 10:45a now -- 2163 blocked hosts). I tell ya, it's like watching the Matrix sometimes -- seeing patterns in the data, and understanding the cause/effect of it all.
Processing complete. Not the fastest script in the world - must have taken 20 mins to run (and that wasn't even on a full 24-hr log file -- only 18hrs,49mins worth). But, after that, 2,447 hosts to be blocked. More trivia for you -- of those 2,447 hosts, 991 were only seen once-- 1,456 were seen at least twice.
Now I have to write the script which will handle the firewall processing :)
I found the end of my inbox! Ever since the fires, my inbox got really fubar'd. But today I found the end of it. I'm hopefully going to stay caught up now, too. Trust me, it's a big milestone.
It's been a while since I ranted about spam. I'll try not to rant today, just provide some things I've seen today WRT my server logs.
I've noticed a number of 419 spams actually getting THROUGH to in my inbox lately (the last 2-3 weeks, probably a dozen). Very odd.
A number of them are coming from cox.net in the 68.230.241.x/24 -server reporting running InterMail vM.7.08.02.01
A couple from adelphia.net - server reported running iPlanet Messaging Server 5.2
A few other random hosts
(going back to 1 Nov, anyway).
Here's what I did note of interest. The spammers are learning to cope with greylisting -- to some extent. They understand the value of hitting an open-relay server properly. A spam zombie client will try once & give up. But a real MTA server, it'll try again. I saw one of the spam's "greylisted" (per the header) for 20888 seconds. Meaning it got greylisted, bounced...and it was roughly 5.8 hrs before the MTA tried to re-deliver, and in this case since it did try again, postgrey let it through.
So more and more, open relay servers are being used to deliver spam. I was hoping sysadmins were learning. Maybe the "new generation" hasn't figured it out yet. That's sad.
Also very interesting...I found spam coming in from what one would guess is an IronPort appliance!
I only go on about spam because, in this case, I pay for the bandwidth to/from my main mail server. All this spam traffic I have to handle, I have to pay for.
I've noticed a number of 419 spams actually getting THROUGH to in my inbox lately (the last 2-3 weeks, probably a dozen). Very odd.
A number of them are coming from cox.net in the 68.230.241.x/24 -server reporting running InterMail vM.7.08.02.01
A couple from adelphia.net - server reported running iPlanet Messaging Server 5.2
A few other random hosts
(going back to 1 Nov, anyway).
Here's what I did note of interest. The spammers are learning to cope with greylisting -- to some extent. They understand the value of hitting an open-relay server properly. A spam zombie client will try once & give up. But a real MTA server, it'll try again. I saw one of the spam's "greylisted" (per the header) for 20888 seconds. Meaning it got greylisted, bounced...and it was roughly 5.8 hrs before the MTA tried to re-deliver, and in this case since it did try again, postgrey let it through.
So more and more, open relay servers are being used to deliver spam. I was hoping sysadmins were learning. Maybe the "new generation" hasn't figured it out yet. That's sad.
Also very interesting...I found spam coming in from what one would guess is an IronPort appliance!
Curious! I sent a copy to TEP at work, as he's always touting the abilities of the IronPort we have.Nov 18 13:28:09 shockwave postgrey[30587]: action=pass, reason=triplet
found, client_name=ironport2.fasken.com,
client_address=209.89.174.213, sender=, recipient=S.Milo@concernd.com
I only go on about spam because, in this case, I pay for the bandwidth to/from my main mail server. All this spam traffic I have to handle, I have to pay for.
The last Insomniac Games videogame which I "worked on" (credited in) was just released - Ratchet & Clank Future: Tools of Destruction. Just talking to one of my old co-workers (Hey Jim-Bob!), he gave me the link to the 30-second TV commercial. It's goooooooood! :) :)
