Having been a fan of the television show Magnum P.I., I always loved the expression "by the bye" (which the character Higgins seemingly uttered at least once an episode), in fact I used it in a post a week or so ago. At any rate, I thought I would see what I could find out about the expression. For those who are not familiar with it, "By the bye" is an idiomatic expression meaning by the way or incidentally. "By the bye" has the alternative spelling "by the by."
According to the Online Etymology Dictionary the word by is an "Elliptical use for "secondary course" (opposed to main) was in O.E. This also is the sense of the second by in the phrase by the by (1615).