——

We live in a society that has mostly agreed on what is right and what is wrong.  We have coined the term, Common Values in order to build communities where we can agree on what is acceptable behaviour.  To enforce our safe communal living, we turned these common values into laws.  As a society, we have set up governments to enforce the laws (common values) and protect us from those who would harm us.

As members of society, I think we’re all in agreement that theft is a bad thing.  No matter what country you are from, it’s pretty much a given that stealing is frowned upon.  It’s a pretty common value, maybe even sacrosanct.

As an individual living in a society with common values, government is the enforcer of the laws and government prosecutes suspected thefts.  What is interesting is that if an individual is convicted of theft, they receive a punishment based on the severity of the crime, and that punishment is roughly agreed upon by the COMMUNITY.  As a defendant, you have rights of due process and would be able to defend yourself according to the law of the land.  Further, in this country a defendant is protected from the Government, as the Government must also follow the law of the land.

Let’s flip it around.  As a society, we have agreed that if you are the victim of a crime, you will take your grievance to the government and file a claim in court.  As a plaintiff, you also are protected by the laws of the land and due process.  This is how even a single individual can take on a corporation and win.  There is DUE PROCESS for each side, Plaintiff and Defendant that as a society, we have all agreed upon using.

The process of using the governmental courts to settle disputes is what our society has agreed upon for longer than I know without Googling it.   If you feel like an individual or a corporation or even the government has wronged you, you must take your claim to court.

So why are corporations looking to ISP’s to become the enforcers of copyright law? The Anti-Counterfeiting Trade Agreement is being manipulated into a Copyright enforcement act.  But isn’t the enforcement of violated law the job of Government?

If you feel your copyright has been violated, file a claim in court!

There is a push for ISP’s to become the monitor, the snitch, and the enforcer.  That’s not what we’re good at.  I hate to say this, but isn’t that the job of Government?

Lions and tigers and bears!  Oh My!

This is my 4th post to this blog and I am seeing a recurring theme.  I feel like the future of the Internet is at stake.  I don’t know what the outcome of this treaty will be, but I’m hoping that some smart people start looking at this issue RIGHT AWAY and make sure EVERYONE is required to follow due process.

Peace.

-Bil

RFCs are, in a nutshell, the description of how a program, or procedure should work. The history of RFC is long and boring, but basically, they’ve been around since the ARPANET Project began, as written or typed memo that were literally Requests for Comments, open ended questions that someone wanted to solicit answers to. As ARPANET grew, RFCs became the standard way to record procedure, and a way for people to implement the fundamental technologies that make up the Internet as it stands today. Today, RFCs are managed by the Internet Engineering Task Force.

RFCs are numbered in chronological order, and serve as sort of a timeline of the Internet and its protocols, and their modifications. Many a bar bet has been settled by referring to an RFC index.

RFCs are referred to by their number, and many of these numbers pop up, especially in error messages. For example, mail headers (the information that records how an email was processed) was originally covered by RFC 821, so you’ll often see errors in a mail log that references RFC 821. The same goes for HTML, USENET, DNS, etc… The errors are written that way because the creators want to emphasize that they follow the RFCs, and so should you.

Why are RFCs important? Well, it boils down to communication theory. The Internet at large is basically an anarchy. There are no overriding rules. It’s just data going back and forth. The only way that too entities can communicate with each other is if they agree to. RFCs are a way to manage these agreements. It’s a way to say: “I follow these rules, and if you don’t, don’t expect me to understand what you’re saying.” If you write a program that follows the RFCs properly, you can expect other correctly written programs to understand what’s going on.

Some companies don’t follow RFCs, they try to use their marketing positions and user base to, what one calls “Embrace and Extend” certain protocols. They more or less want to pollute the internet with their own way of doing things, so that they can control who talks to their users, and who their users talk to. Many others are very strict about their interpretation of the RFCs, causing users to get caught in the middle of Open Standards vs Commercial Protocols. This war is hardly limited to RFCs, there are all sorts of standards bodies that companies ignore. Furthermore, RFCs tend to be vague about specific actions. There’s a lot of “you can,” “you should,” and “it is recommended,” talk in most of them. This often leads to arguments about what is allowed and what is not “per an RFC.”

Like I said, anarchy.

Ultimately, RFCs are holy writ to some, and merely “guidelines” to others. Most UN*X Admins follow RFCs and “Best Practices” as best they can. Many others do not. How important is it to follow them? Well, most of the Internet is still run on programs that use open protocols. So far, most initiatives to commandeer them by commercial entities have failed.

I personally believe in open standards, so if you expect to talk to me, or any my systems, you better follow the <expletive deleted> RFC!

I hope that helps.

we’ll be using the prototype and script.aculo.us javascript libraries to do most of the heavy lifting. prototype provides all the DOM selection and introspection tools we’ll need, and better array management. builder, which is part of script.aculo.us, provides tools for creating new DOM nodes, which we’ll use to build our tab navigation and insert it into the document.

first, our tabbing code needs to look for any blocks of tabbed content within the web page. we’ll define these by enclosing them within a <div> block, with an appropriate marker class. however, it can’t do this until the web browser has loaded the document and parsed the HTML markup, so we use prototype to observe that event, and invoke our initialization function when it happens. if there are any tabbed blocks, then…

for each tabbed block, do a bit of setup, then get a list of the tab blocks within it. again, we’ll define these within <div> blocks, with appropriate marker classes. each tab <div> should also have a title, which will be used for the tab link.

for each tab, attach a unique identifier to the content, build a new link to activate it, and then hide it by giving it the ‘inactive‘ class. after we’ve built our list of tab links (tab_nav), insert it into the top of the tabbed block. finally, determine the first tab in the current tabbed block, and invoke lift_tab() to activate it.

// tabbed.js

document.observe('dom:loaded',init_tabs);

function init_tabs () {
  if (blocks = $$('div.tab_block')) {
    var B = {}; Builder.dump(B);   // for convenience

    blocks.each(function (block) {
      block_attr = { 'class': 'tab_nav' };
      items = [];

      if (tabs = block.select('div.tab')) {
        tabs.each(function (tab) {
          tbd_id = tab.identify();
          a_attr = { 'onclick': "lift_tab(this,'" + tbd_id + "')" };

          items.push(B.LI({},B.A(a_attr,tab.title)));
          tab.addClassName('inactive');
        });
        block.insert({ 'top': B.UL(block_attr,items) });
        lift_tab(block.down('ul').down('a'), tabs[0].identify());
      }
    });
  }
}

the lift_tab() function expects a tab link and content identifier. it deactivates all the tab links within the same list, then activates the specified tab link. in a similar fashion, it activates the corresponding content. note, however, the opposite sense of the classes involved. tab links are explicitly active, content is explicitly not inactive. this is done this way to make the CSS style sheets a little simpler.

// tabbed.js continued...

function lift_tab (link, content_id) {
  link.up('ul').select('a').each(function (a) {
    a.removeClassName('active');
  });
  link.addClassName('active');

  link.up('div').select('div.tab').each(function (d) {
    d.addClassName('inactive');
  });
  $(content_id).removeClassName('inactive');
}

next, we define the CSS style sheets which actually effect the changes which the javascript is making to the HTML elements. on-screen, we want our list of tabs to be displayed inline, in a single row, with appropriate margins and borders. inactive tab links are greyed out, whereas active tab links are given a white background and bottom border to mask the border which normally separates the navigation and content, visually linking that tab to the content. finally, inactive content is hidden by setting its display to ‘none’.

// tabbed-screen.css

ul.tab_nav { margin: 0px; padding: 2px 0px; border-bottom: 1px solid; }
ul.tab_nav li { display: inline; margin: 0px; padding: 0px;
  list-style: none; }
ul.tab_nav li a { margin: 0px; margin-left: 3px; padding: 3px 0.5em;
  border: 1px solid; border-bottom: none;
  color: #808080; font-weight: bold; text-decoration: none; }
ul.tab_nav li a.active { border-bottom: 1px solid #FFFFFF;
  background: #FFFFFF; color: #000000; }
div.inactive { display: none; }

styling for print is much simpler. hide the navigation, and all tab blocks are presented normally.

// tabbed-print.css

.tab_nav { display: none; }

finally, we include all our javascript libraries and style sheets into the page. since the code is contingent upon having tabbing <div> blocks defined in the HTML markup, we can simply add this to a page template. this makes it available wherever we want to use it, and it simply does nothing wherever we don’t.

<script type="text/javascript" src="/lib/prototype.js"></script>
<script type="text/javascript" src="/lib/builder.js"></script>
<script type="text/javascript" src="/lib/tabbed.js"></script>

<link rel="stylesheet" type="text/css" media="screen"
    href="/lib/tabbed-screen.css" />
<link rel="stylesheet" type="text/css" media="print"
    href="/lib/tabbed-print.css" />

to turn a block of content into tabbed content,
enclose the entire block in <div class="tab_block"> ... </div>
and each tabbed block in <div class="tab" title="Title"> ... <div&gt

<h1>tabbing content with prototype</h1>
<p>this paragraph is above the tabbed content.</p>

<div class="tab_block">
  <div class="tab" title="Overview">
    <p>an overview of what we're doing and how it works.</p>
    ...
  </div>
  <div class="tab" title="Javascript">
    <p>details on the javascript coding which makes this work.</p>
    ...
  </div>
  <div class="tab" title="CSS">
    <p>details on the CSS styling which makes this work.</p>
    ...
  </div>
  <div class="tab" title="Implementation">
    <p>finally, how to put everything together.</p>
    ...
  </div>
</div>

<p>this paragraph is below the tabbed content.</p>

This is the mail system at host smtpin-2.iphouse.net.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

<blogtest@iphouse.com>: permission denied. Command output: maildrop: maildir
    over quota.

Breaking this error down; the first part tells us what server issued the rejection. In this case it was the host “smtpin-2.iphouse.net” – the mail server responsible for receiving email for the email address blogtest@iphouse.com   The second part tells us who we were sending the email to, in this case it was sent to testblog@iphouse.com  The third part tells us the error type, as in a permanent or temporary error. A permanent error means that the email won’t be delivered. A temporary error means that the sending mail server ought to try again later.  In this case it says “Permission Denied” – that’s a permanent error. The fourth part of this error message tells us why the email was rejected, in this case it says “Command output: maildrop: maildir over quota”. What this means is that the service “maildrop” says that the Mail Directory (maildir) is over quota – the mailbox is full. What this means for us the sender, is that we shouldn’t bother trying to send that person anymore email until they clear out their overstuffed mailbox. There isn’t anything we can do about it other that calling that person up to tell them that their mailbox is full.

Here’s the next error bounceback:

Unable to deliver message to: <blogtest@iphouse.com>
Delivery failed for the following reason:
smtpin-2.iphouse.net[216.250.188.181] responded with failure: 550 5.7.1 black
listed URL host ***.********.net by .black.uribl.com

This has been a permanent failure.  No further delivery attempts will be made.

I did make one edit to this error message, I replaced the blacklisted URL with a bunch of asterisks (*). I don’t see any reason to give a spammy site any more traffic than it already has. OK, so let’s break this error down. The first part of this error message tells us who we were sending the email to, in this case blogtest@iphouse.com was the intended recipient. The second part tells us what server issued the rejection, in this case it was the server smtpin-2.iphouse.net – so it was the recipient’s mail server issuing the rejection, not the sender’s. The third part tells us what kind of rejection it was, and why it was rejected. In this case the error code was “550″ (I’ll go into error codes later), and that the email was rejected because it had a link to a website (the one I used ***** to hide), which happened to be on the blacklist “black.uribl.com”. The last part is just another note that this was a permanent error, in case you didn’t already know that that’s what a 550 is. What this bounce back message means for us the sender is that we had a link to a website in our email that a spam filter on the recipient’s mail server didn’t like. We’ve got a few options here. The simplest thing would be to send the person a new email, but this time not include the offending website link. If we think that the website in question should not be on this blacklist in the first place (mistakenly listed), then we can always go to uribl.com and request that the offending website be removed from their list. Not all blacklists let you do this, but plenty do – and as it so happens, uribl.com lets you request removals. Another option would be to contact the intended recipient to tell them that a spam filter of theirs is rejecting that link. The intended recipient could then follow up with their IT staff, or their ISP. I suppose the final option would be to try to “game the filter” by breaking up the link a bit. For example, let’s say that iphouse.com was blacklisted somehow. I could try sending the person a link to iphouse.com like this:  www (dot) iphouse (dot) com    A human reading this ought to be able to figure out your intent, whereas a spam filter might be tricked.

Here’s the next error bounce back:

<testblog@iphouse.com>: host smtpgrey-2.iphouse.net[216.250.190.161] said: 550
    5.1.1 <testblog@iphouse.com>: Recipient address rejected: User unknown (in
    reply to RCPT TO command)

The first part of this error message tells us who we were sending the email to, in this case testblog@iphouse.com was the intended recipient. The second part tells us what server issued the rejection, in this case it was the server smtpgrey-2.iphouse.net – so it was the recipient’s mail server issuing the rejection, not the sender’s. The third part tells us what kind of rejection it was, and why it was rejected. It’s another 550, so a permanent rejection. It was rejected because the User was Unknown. This means that the email address doesn’t exist. If you caught the mistake in the email address right off the bat, plus 10 points for you. I accidentally sent an email to “testblog@iphouse.com” instead of “blogtest@iphouse.com”. Oops. As a funny aside, I actually generated this bounce back message accidentally when I was aiming for another type of bounce back. It took me a moment to catch my mistake. I guess that’s what more coffee is for…..    Anyhoo, what this means for us the sender, is that we just need to send out a new email, but this time type in the correct address. If you’re absolutely positively 100% double plus certain that you didn’t make any typo in the email you sent out, go ahead and look for the typo anyhow. It’s really easy to miss certain things like a “,” instead of a “.”, or a letter out of place. If your double check doesn’t yield any typos, and you’re certain that the address is a valid one, you might want to contact the intended recipient to inform them that something is amiss regarding their email address.

At the opposite end of our anti-spam system is content filtering. We use a third party vendor for this, MailFoundry in the form of two appliances. An appliance is a machine that you plug in, and is suppose to work with minimal configuration.

Now the MailFoundry appliances are “black box” systems. We don’t know how they work exactly, but we’re pretty sure that one of the techniques they use is Bayesian spam filtering.

Bayesian spam filtering uses the concept of probability to evaluate each token in a message, assign a weight to each, give the overall message a rating based on this weight, and evaluate the message based on a preset threshold.

Ok, unless you’re up on your statistics or logic based calculus, or a computer nerd with Wikipedia handy, I know your eyes just glazed over. Rest assured, you are not alone.

Basically, what it boils down to is that every “token” is a series of characters separated by whitespace. During this discussion, most “tokens” are words. Certain tokens are negative, they tend to appear in spam messages. Others are positive, they tend to appear in good (or ham) messages. Each token has a value (or a “weight”). A Bayesian spam filtering system reads the message, adds up all of the negative and positive weights of the tokens which produces an overall probability rating. If the rating is too negative, it considers the message spam. If it’s positive, it doesn’t.

Now, how does the Bayesian filter know which tokens are bad or good? Well, you have to give it examples of each. If a message is spam, and it gets through the filter, you have to tell the filter that it’s spam. When a message is marked as spam, all of the tokens in the message have their ratings lowered in the filter’s database. Ideally, you’d also mark good messages as good, but most people don’t. Most Bayesian filtering schemes are configured to mark all delivered messages as good unless they are marked as bad. Over time, the good tokens get “gooder” and the bad tokens get “badder” and the system can determine what is spam and what is not.

Bayesian spam filtering works amazingly well on individual email accounts, as it will be able to determine an individual’s taste in what is spam and what is not. Unfortunately, it’s not as effective across hundreds or thousands of users, but it still helps. Your personal spam filter will usually outperform anything on our end, because you may have a different definition of what is spam than other users out there. On a large system like ours, tokens that would be marked as negative for you, are nullified by others marking them as positive. So, say mail sent from a list that you signed up for, but no longer want may be spam to you, but may not be to other people out there. That’s why it’s best to unsubscribe from those lists rather than try to get our system to recognize it as spam.

If you want to help feed our filters, feel free to send examples of any spam you receive via our system, as attachments, to spam at iphouse.com.

I hope that helps!

1) If I’ve sent and received email from my friend for years, it shouldn’t get flagged as spam.

2) If I have their email address in my address book, their email won’t get flagged as spam.

3) If I avoid using certain words, my email won’t get flagged as spam.

None of these things are true. To understand why this is a tricky question to answer, it’s helpful to know a bit about what ISP’s are doing to filter spam. Most ISP’s have their own “custom blend” of what they do to filter spam, but it more or less boils down to using a combination of one or more of the following: Blacklists, Greylisting,  enforcing RFC’s, and more traditional Content Filters.

Blacklists can be based on all kinds of things. They can be lists of IP addresses that have been reported as sources of spam, lists of mail servers that have been found to be capable of being used as open mail relays, lists of URL’s that have been “spamvertised”, or any number of other things. Not all blacklists are the same. Some are very aggressive in what they list, and some are very conservative. The aggressive lists might block a lot of spam, but they are also more likely to have “false positives” – as in they blocked something that the recipient really did want to receive. Whereas the conservative lists might not have many false positives, but they’re likely to let more spam through.

Greylisting is when a receiving mail server issues a temporary error, which causes the sending mail server to re-queue the email and send it once more. Being able to re-queue an email is something that any RFC compliant mail server ought to be able to do. Greylisting can drastically reduce spam sent through “spam zombies” – home computers compromised by viruses that send spam out directly from the PC instead of through a mail server capable of re-queuing email.

RFC’s are, in a nutshell, the basic minimum standards for anything Internet related. Enforcing RFC compliance for mail can cut down on mail sent out from compromised PC’s/servers, and cut down on spam sent out from “sketchy” mail servers.

And lastly, content filters are the more traditional form of analyzing the content of an email to determine the “spamyness” of the email. Each spam filter system has its own “custom blend” of techniques to identify spam. Some of these criteria include; spammy words/spelling (\/1agra), format of an email (lot’s of CAPITAL/BOLD/etc lettering), lists of “spamvertised” websites, know spammer addresses, etc. Some filters use a feedback system that allows end users to submit examples of spam to train the filter.

Because blacklists and content filters are dynamic in nature, it can be very difficult to determine what it was at that exact moment that caused a particular email to be tagged as spam.

It’s also our first line of defense against spam.

Greylisting is a very simple technique. It is a daemon attached to database that keeps track of who externally sent mail to whom internally, including from what IP address. When a new sender/recipient/IP-address (or triplet as it is called) combination pops up, it bounces the transaction with a temporary, 450/451 response code. This is per the RFC and any properly implemented SMTP server should adhere to it, re-queue the message, and send it again later. If the server sends it before a specified “too early” window (in my case on my personal server, 2 mins, but that’s fairly aggressive) it’s temp-failed (tech term for try again later) again. If the message comes back after this “too early” window, but before a 24 hour expiration window, the message is passed through, and an entry is made in the database allowing that triplet to send mail unhindered for a few days (depending on configuration). If enough messages come from the same ip address and domain pass Greylisting, that whole domain can be automatically white-listed through the check.

The goal of greylisting is not to penalize legitimate mail servers but only to stop non-compliant botnets from getting through.

Greylisting is very effective because it keeps non-compliant SMTP servers from sending mail to our (or even your) servers. Most virus infected computers that send or relay spam won’t re-queue messages, or will re-queue them for only the briefest amount of time. Why? Their goal is to blast as much email/virus payload as possible, and any slowdown or long retry time is very counterintuitive to this goal.

Problems with greylisting are legitimate, by mis-configured SMTP servers either not re-queuing the messages because they are set to treat 400 series bounces as 500 series (permanent) bounces. Or they re-queue the messages, but report to the original sender that the message bounced.

Yahoo implements a more esoteric set up, where they have 4 servers listed in the MX record, and at any time, any of them will bounce messages. This is another way to test for non RFC compliant servers, as a server is supposed to try all of the MX entries in turn, by weight value. Most virus infected computers won’t do that. At least that is what it looks like from the outside.

Because some of our users may have problems with receiving mail, our web-based interface, ipMom, gives you the option to disable greylisting. If you log into ipMom with your email address and password, you’ll notice a “Greylist” option . Set it to off, and greylisting is no longer affecting your mail. Keep in mind that this does let more spam into the system, although our other anti-spam protections may still catch them.

I hope that helps!

The first thing to understand is the Registrar. A registrar is a company that is accredited, and allowed to work with ICANN ( The Internet Corporation for Names and Numbers). ICANN actually maintains domain names and their information. You pay a registrar for your domain registration, and they, in turn, pay ICANN and provide them with the domain’s information. There are other parties and services involved, but in the interest of keeping it simple, that’s how it works.

Once the registrar has the domain name registered and reserved, actual Domain Name Servers need to be assigned to it. DNS is the service that turns the domain name, say, example.com, into its associated IP address. Humans can remember words far better than numbers. And computers deal with numbers.  This is why we have DNS.

It also tells servers where to send email and what IP addresses various services may use. Each service normally has an A (or address) record. For example, an ftp server, ftp.example.com, may be on one IP address while the website, let’s say www.example.com, may be on another. Mail routing is controlled through the MX (mail exchange) record, which must point to a host name, like mail.example.com instead of an IP address. Canonical Name records and TXT record are used for more specialized purposes.

Usually, a registrar defaults to using their in-house nameservers. You can use these servers, putting in the information from your hosting company, usually by using a web based interface. However, we feel it is a better idea to switch the nameserver to your hosting company’s. That way, if your hosting company makes a change then they can update your DNS automatically alleviating your need to worry about these technical details.

DNS can be tricky and it is absolutely critical that your information is correct. One small mistake can cause you to not receive email, or not be able to view your own website. Many outages are associated with DNS problems and can easily be avoided by making sure the right information is in the right place.

Now, there are PTR (or rDNS) records that often are confusing. Most DNS queries are like looking up a name in a phone book directory, but you look up a server name to get the IP address (instead of a phone number). There are a few instances where you want to check the IP address and see who it belongs to, kind of like a reverse directory search of a phone number to see who’s calling. This is called a reverse DNS check.

This most often come into play when mail servers want to check the legitimacy of the SMTP servers that are sending them messages. If they query the IP address and get mail.example.com (or some other, similar Fully Qualified Domain Name) they let it pass. If they get something like 192-168-123-45-adsl-dynamic-customer.isp.net, they may reject or quarantine messages. These records are maintained by the ISP who provides the IP address, so if you run a server out of colocation space or off your Internet connection, you’ll want to contact the ISP to update that information.

Well, this was a very basic summary, hopefully it helps!

That phrase has a hidden implication trailing after it “…so what did you do?”

Well, usually, we didn’t do anything. People can accept that a car, or a furnace, or a TV can break someday out of the blue. We all know that those are machines, and we have some idea of how they work, and that they do break. The problem is, not a lot of people realize that computers are machines too. Sure, If you ask someone if a computer is a machine, they’ll say yes. However, if you show them the inside of a computer tower, they’ll usually shake their head and say “Is that it?”

It’s been clear, over my many years of tech support, that people think their computer is a magical device. And since it’s magical, it should work a certain way. Forever. And when it stops working, some mysterious outside forces conspired to break it.

My brother thinks this way. To him, a computer does what it’s supposed to, and then it stops. Then he calls me, and to him, I cast a few magic spells, prod at it chanting while waving a censor of incense in the air, and it magically stops working. They I mumble in a strange, arcane language about what went wrong. He smiles and nods, and the magic box runs again. For a while.

But when it breaks, it’s the ISPs fault, or his wife “touched something” or a computer virus caused his machine not to power on.

I tell him, it’s not any of those things, it’s him. It’s us.

Computers are beholden to humanity. They suffer from our lazy programming, our dusty houses, our animals that chew on their wires, our neurotic ticks causing our legs to kick them. Computers suffer this day in and day out while we curse them, berate them, and blame them for everything we all ultimately did or failed to do. Did install an anti-spam program. Is your computer is slow now? Did you visit customcusors.biz.co.tg? Is your hard drive always making noise? Did you forget to update your operating system? Is your email program acting erratically?

Do yourself a favor. Update your operating system. Blow out your computer’s fans and vents once and a while. Don’t install every security program ever made. Don’t install trinkets that were programmed by the lowest bidder. Treat you computer well, and it’ll treat you well.

Well, until it magically breaks one day.

At some of these events we have held a drawing for a free Amazon Kindle. Personally, I have a Kindle and I love it.  Do you folks feel the same way? Do you want a chance to win a free Kindle? Should we be giving something else away? Please send us your opinions and ideas.

Our next event is the 55th Annual Minnesota CPA Tax Conference on November 16th and 17th at the Minneapolis Convention Center.  This is a well-organized and educational event put on by the Minnesota Society of Certified Public Accountants. We look forward to seeing you there.