What is PubSub?
"PubSub" is short for "publish and subscribe". It refers to a centralized information-processing system in which material fed into the system from a variety of sources (that's the "publish" part) is sorted, matched, reformatted, and sent along to "subscribers" according to their particular interests. The heart of the system is our blazingly fast Internet-scale "matching engine", which allows us to process new information and get it out to all of our users as soon as it appears on the Web.
Do I have to pay for the PubSub service?
No, you don't! The PubSub service is offered free of charge.
I don't get it — if you don't charge for your service, how are you going to make any money?
Our business model is built upon the fact that advertisers and content providers see PubSub as a new and exciting way for people to find and view online content as soon as it appears. The fact that everybody can use this powerful tool without having to pay for it is the key to the whole idea.
How is PubSub different from search engines like Google or Yahoo?
The biggest difference is that PubSub isn't a search engine — it's a matching engine, the first one in the world to work at Internet scale, and to be made available for public use. Search engines operate by "crawling" the Web and indexing the pages they find: they visit a Web page, make a note of what's there, and then follow whatever links the page may contain. By repeating this process at high speeds they create, over time, an enormous database of existing pages, which can then be searched very rapidly when users submit a query. This works wonderfully well for finding documents that have already been published, but the drawback is that once the crawler has found and indexed a page, it moves on, and if the page is later modified, or linked to new pages, it might be quite some time (in some cases, weeks or months) before the crawler revisits it and updates the search engine's database with the changes. This is a retrospective search – which is to say "searching the past": the user submits a one-shot query, and the search engine uses the query to retrieve documents stored in its database. The PubSub matching engine, however, does the reverse: rather than maintaining a collection of Web pages, we maintain a collection of our users' queries, and match them in real time against new documents as they appear on the Web. In other words, we offer a way to do a prospective search, or as we say, to "search the future". This important difference — the ability to deliver relevant information within minutes, or even seconds, of its coming into existence — makes the PubSub system the perfect complement to powerful search engines such as Google.
How does PubSub work?
PubSub uses a sophisticated "matching engine" to analyze incoming data from an enormous pool of sources and determine which items should be sent on to which subscribers, according to each user's individual interests. Because the high efficiency of our system allows us to do this matching very, very quickly, we are easily able to keep up with the tremendous pace at which new information appears on the Internet.
What is a "matching engine"?
A search engine, typically, searches an existing database of stored pages for occurrences of keywords submitted in a one-shot query, whereas a matching engine compares stored ("persistent") queries against newly-discovered pages on an ongoing basis, in real time. See question 4, above.
What is the difference between subscribing to Weblogs and/or Newsgroups?
The difference is in the sources that we examine for items that match your subscription criteria. Weblogs are online journals usually written by a single person (though there may be areas for readers to leave comments), whereas newsgroups are moderated discussion forums.
What is a weblog?
A Weblog ("blog") is a publicly available online journal, usually the work of just one person.
What is RSS?
RSS is an acronym for Really Simple Syndication — a machine-readable (XML) text format for providing brief summaries of, and links to, online content.
What is NNTP?
NNTP, which is short for Network News Transfer Protocol, refers to those online discussion forums commonly known as "newsgroups".
Can I use the PubSub system to distribute my own information?
If you have an existing RSS feed you can add it to the system at our Add Feed page. If you have custom information needs or wish to leverage the PubSub system for your company please contact us.
How do I get a PubSub account?
PubSub accounts can be created at https://www.pubsub.com/login.php. We require an email address in order to refer to your customized subscriptions.
Why do I have to give out my Email to subscribe?
We ask for an email address solely for technical purposes of user authentication. We have a strict privacy policy, and do not make these addresses available to anyone else, for any purpose.
What will my email address be used for?
PubSub will only use your email address as a unique identifier for your PubSub account. As our privacy policy indicates, we do not share this information with any third parties.
What are the system requirements for using PubSub?
The website requires that you have cookies and Javascript enabled, while the Internet Explorer Sidebar requires that the Microsoft .Net Framework be present on the user's computer. The .Net framework is available as a free download from the Windows Update website; to get there, open Internet Explorer, and on the menu bar follow the path Tools -> Windows Update.
What browsers can I use the PubSub Sidebar with?
The Microsoft Internet Explorer Sidebar works with IE version 4.0 and above. We also offer a Sidebar for Mozilla Firefox version 0.7 to 1.0.
What is PubSub's policy regarding "robots.txt"?
Web Robots, also known as "spiders", are programs that discover Web pages by recursively following hyperlinks. Such agents visit a Web page, make a note of what is there, then follow all the links on that page, follow all the links on the linked pages, and so on. These programs are used by search engines to index and catalog the ever-changing face of the Web; in fact they are what make conventional search engines possible in the first place. There are some Web hosts, however, that wish to restrict or prohibit such behavior, and in order to offer these hosts a way to inform spidering programs of their wishes, in 1994 a group of authors of such programs arrived at a consensus for a mechanism called The Robots Exclusion Protocol.
Under this protocol a small record, called /robots.txt, is placed in a host's filesystem. The robots.txt file lists robotic "user agents" by name or by wildcard, and specifies where they are or are not welcome to go. "Spiders" crawling a Web site should look for this file, and should abide by its instructions.
PubSub does indeed use an automated agent, which identifies itself as "PubSub-RSS-Reader/1.1 (https://www.pubsub.com/)", to fetch syndicated content. But it differs from the robotic crawlers described above in two important ways:
- First, in almost all cases, PubSub only visits a site if:
- We have received a request through our Add Your Feed page;
- We have received a notification, called a "ping", that tells us that there is new syndicated content available at that site. In other words, we never go where we haven't been told to go.
- Second, and this is most important of all, we never "crawl". Once we have received a "ping", we go to the URL specified in the ping, pick up the RSS or similar feed, and depart. Once we have found the feed, we follow no other links, and visit no other parts of the Web site.
For these reasons, and in the interests both of minimizing the traffic on the servers whose feeds we read and maximizing the speed and performance of our system, PubSub does not perform the extra HTTP request that is required to read "robots.txt" files.
If your feed is currently on the list of URLs that we fetch, and you would like to be removed from the list, you can visit our Remove Your Feed page. Note that if you subsequently send us another ping, we will once again fetch your feed.