The future of Web 3.0

People say that the future of the Internet are mashups. Mashups are applications that are made using different web applications. Mashups make use of the Internet as a big personal database. Using mashups, developers can come up with more robust, more personalized and more effective applications.

There are several mashup makers available like Kapow, Dapper, Yahoo! Pipes, Convertigo, Intel Mash Maker and Microsoft Popfly. These applications are unique to their own respects. I did some research about these applications and here is a little review to help you guys ( especially those who are just starting to play with mashups ):

OpenKapow
Prerequisites:
1. OpenKapow username and password.
2. Kapow IDE (RoboMaker) installer. RoboMaker is just one integral part of the Kapow RoboSuite.

Features/Strengths:
1. Desktop-based application for creating robots ( Kapow RoboMaker ). IDE is easy to work with.
2. Implementation of programming concepts like condition statements, looping and error catching.
3. Can expose output in different formats. ( JSON, HTML, XHTML, XML, CSV and RSS feeds )
4. Can execute javascript, may it be from the source or customized one.
5. Free to use.
6. Can be used in conjunction with a scheduler ( like Unix Crontab ).
7. Accepts/Does all kind of user inputs i.e filling out forms, clicking on buttons, etc.
8. Works with web pages like a real person does. Full automation to its essence.
9. Is more like a content scrapper than a mashup maker. Can serve as a powerful backbone (content provider/aggregator) for mashups.
10. Nice extraction method using tag finders ( path ).

Weaknesses:
1. Kapow servers ( they call it RoboServer ) host the robots. So if x number of clients send robot execution request simultaneously ( given x is a big number ), not all requests can be serviced.
2. Kapow servers are frequently down.
3. We can’t send our own headers for http requests. (unless you buy the whole RoboSuite)
4. Jobstreet and JobsDB blocked Kapow.
5. Limited resource utilization.

Proprietary version:
Includes:
RoboMaker for writing robots.
ModelMaker for creating and maintaining domain models.
RoboManager for managing robots.
RoboRunner for running robots in batch,
RoboServer for running robots in realtime (own RoboServer, saved on your own server, thus will let you use multiple proxy servers )
Control Center for managing RoboServers.

Yahoo! Pipes
Prerequisite:
1. Yahoo username and password.

Features/Strengths:
1. Web-based drag-and-drop interactive user interface.
2. Competitive list of modules.
3. You can get the output (feeds) as RSS, JSON, serialized PHP object and CSV.
4. Free to use. No proprietary version.
5. More of an aggregator than a scrapper. Personalizes feeds from different sources.
6. Uses regular expressions to parse HTML pages (to extract data), not so neat really.

Weaknesses:
1. Still in beta.
2. Output feeds format only. Can’t have customizable output unlike Kapow objects.
3. Can’t accept user inputs i.e filling out forms, clicking on buttons etc.
4. Cannot perform extraction method using xpath or the like.
5. Can’t execute javascript.

DAPPER
Prerequisite:
1. Dapper username and password

Features/Strengths:
1. Web-based interactive user interface.
2. Nice algorithm to select contents you want to scrape. (point-and-click) It’s nice but annoying because sometimes the algorithm can’t recognize the elements I want to pull out from the page. No xpath/tag path-way of doing it.
3. Expose output as RSS feeds, flash widgets, xsl, csv, json, php, xml, netvibes module, google gadgets, google map, imageloop, iCalendar, email and yaml.
4. Can link output of one dapp to another.
5. More of a personalized searching than a contentscrapper.
6. Free to use. No proprietary version.
7. Kapow-like output objects.

Weaknesses:
1. No programming logic.
2. Cant handle user inputs except search strings and url.
3. Can only use/open one webpage at each run (of dapp).
4. Cant handle/execute javascript.
5. Cannot perform extraction method using xpath or the like.

Google Mashup Editor
Wasn’t able to play around with GME because access is limited only to invited developers.

“Google Mashup Editor
The Google Mashup Editor is currently in beta and access is limited to small number of developers during this testing period. If you would like to be notified when spots are available to participate in the beta please sign up here.”

Features/Strengths
1. Online editor. No UI. No drag and drop. No interactivity. Pure coding.
2. Comes with an integrated feed browser so we can preview the content of any feeds without loading the feeds in new window.
3. Full power to manipulate feeds/contents. You can query data in any fashion you want, and you can structure it any way that fits your needs.
4. You can preview the mashup before you publish it.
5. Can create AJAX-based applications.
6. Codes you can use with GME: HTML, CSS, Javascript and declarative XML tags.
7. Free to use. Web apps/mashups are hosted on their servers.
8. Availability: “The Google Mashup Editor is currently in beta and access is limited to a small number of developers during this testing period. We hope to make it widely available in the near future, though we can’t comment on the timing of its availability.”

Weaknesses:
1. Not yet available.
2. Input is limited to RSS and Atom feeds.

Windows Popfly
Prerequisite:
1. Windows Live ID.

Features/Strengths:
1. More of a mashup maker than a content scrapper.
2. No programming involve. Pure drag-and-drop.
3. Create mashups without writing a single line of code.
4. Combines applications (limited only) to produce new ones.
5. Can run Dapps from Dapper.

Weaknesses:
1. Uses only well-known applications like Twitter, Facebook, Carousel, Flickr etc.
2. Can’t open webpage.
3. Modules are limited.
4. Looks beta.
5. Not user-freindly.

Twinsoft (Convertigo)
1. Trial version and demo installer is corrupted.
2. Not a single good review about it.

Still, nothing compares to Kapow. So far, it is the best application  to use for scrapping web contents.