It has two main features: the ability to identify related/similar links and suggestions/recommendations that actually work.

The basis for all of the algorithms is a document similarity metric presented in Chapter 3: Discovering Groups. Basically, to compare document A with document B, we calculate the Pearson correlation coefficient between the word frequencies of document A and the word counts of document B. (You can imagine this as plotting a series of points of a graph: each point’s x coordinate is its frequency in document A and each point’s Y coordinate is its frequency in document B. The Pearson correlation coefficient is a measure of how well the line-of-best-fit fits the points.)

Using this similarity metric, links can be clustered together using K-means clustering. This is what you get when you click on “related” at the bottom of each link. Clicking on “similar” gives the results of running K-NN. (“related” doesn’t work as well as it could be right now because there are too few links for a link to be similar with, but this is an example of where it does work: http://fyynd.com/links/197/related/ “similar” usually works better right now.)

There are two algorithms for giving recommendations, “Suggested” and “Recommended”. “Recommended” generally works better than Suggested when you haven’t yet made many votes but Suggested should be more in tune to your preferences in the long run.

In layman’s terms, the Recommendation algorithm works by “averaging” together the links that you liked and then find links that are similar to that while the Suggestion algorithm tries to determine whether you will like a particular link by seeing whether it is similar to any page that you have already rated highly. As a result, “Recommended” will list pages in your general interest area, but insensitive to any “niche” interest that you might have. The “Suggested” page will be sensitive to “niche” interests but will requires more votes to train. For example, if most of the link you rate highly are about computer science, with a only a few links about biology, when the recommendation algorithm averages them together, the biology links would count for very little. As a result, you wouldn’t see much on biology. On the other hand, the suggestion algorithm will not be hindered by this, though it will have trouble if you don’t vote much.

By now, I hope to have convinced you that I know what I am doing and that my algorithms do work!

Please note that because predictions are so computationally intensive, they are not updated in real-time but on a hourly basis. Thus, you have to wait a bit before they come out. Please be patient!

Please check it out and tell me what you think! Any questions/comments/suggestions are more than welcome! You can email me at comments@fyynd.com